Repository navigation
fix: Read the modules the standard set is actually written in - #18
Conversation
Reported from a real load: "IEEE8021-SECY-MIB: 1 definition could not be
placed: no node named iso(1) was found". The warning was right that something
was wrong and wrong about the scale - none of that module's 185 definitions
were in the tree, because they all hang off the one anchor that failed.
Six faults, each reproduced against the real file before it was fixed, and the
set of 23 standard and IEEE modules now resolves whole: 2048 nodes, nothing
unplaced, nothing excluded.
**An anchor in the named-number form.** Every IEEE module anchors itself at
`::= { iso(1) std(0) iso8802(8802) ... }`. The first element was taken as a
name in full, so the parent looked up was `iso(1)`, which nothing defines.
**Hyphens.** A descriptor may contain them, and the standard tree is full of
them: matching a name with `\w+` stops at the hyphen, so `member-body` was
registered as `body` and `mib-2` as `2`, and everything hanging off either
became an orphan. IEEE802dot11-MIB lost 218 definitions this way.
**Definitions quoted in prose.** IF-MIB explains `ifTestType` with
`noTest OBJECT IDENTIFIER ::= { 0 0 }` in the middle of its DESCRIPTION. That
example was read as a definition of its own, and - being before the object's
real assignment - it was also taken as that object's parent. Definitions are
now searched for in a copy of the file with the contents of string literals
blanked out, while descriptions are still read from the real text: the two are
the same length, so a match maps back by index.
**Macro definitions.** The SMI modules define the macros the others are written
in. In SNMPv2-SMI the `END` of one macro sits above `OBJECT-IDENTITY MACRO ::=`
and was read as an object called END whose assignment was the `::= { 0 0 }` of
`zeroDotZero` thirty lines below - inventing a node and swallowing the real
one. Macro bodies are blanked with the same machinery.
**Root arcs.** `zeroDotZero` lives at 0.0, and nothing was seeded for the arc,
so it and the modules pointing at it resolved to nothing. `ccitt` and
`joint-iso-ccitt` are seeded alongside `iso`; a root with no children is not
rendered, so an ordinary MIB set still shows one tree.
**Clause boundaries.** `\b` was applied to the whole terminator group, and
there is no word boundary between `=` and a space, so `::=` never ended a
SYNTAX clause: the last TEXTUAL-CONVENTION of a module ran on into whatever
followed it - SNMP-FRAMEWORK-MIB's SnmpAdminString swallowed four assignments.
Bounding the convention body then hid five of SNMPv2-TC's conventions, whose
own `SYNTAX OBJECT IDENTIFIER` line read as the start of the next definition,
so clause keywords are excluded from that boundary. All 102 conventions across
the 23 modules now parse, none of them polluted.
The reporting was undercounting too: an anchor that fails takes its whole
subtree with it, so the count is the subtree, not the one node that failed to
link. IEEE8021-SECY-MIB would have said 185, not 1.
Masking costs a pass over the text. Doing it once rather than once per
OBJECT-TYPE block, and blanking with runs of spaces rather than a
per-character regex, keeps that to about a tenth of the parse: 3.3 MB parses
in 177 ms against 159 ms before any of this existed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018qyesKa9si8szMUe7w8HNK
Found by parsing 99 real modules with the branch and with main and diffing the two, which is the only way this would have surfaced: the clause keywords are excluded from the boundary that ends a TEXTUAL-CONVENTION body, the match is case-insensitive, and OSPF-MIB has a convention actually called `Status` - so the convention above it swallowed it and the definition was lost. They are now excluded only where they are used as clauses, which is to say where they are not followed by `::=`. The same differential says the rest holds: across those 99 modules the merged tree gains 607 nodes and loses four, and those four are `and`, `In`, `2` and `body` - the phantoms this branch set out to stop inventing. No node that resolved before resolves differently now. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018qyesKa9si8szMUe7w8HNK
|
Pushed a second commit after checking this branch the way it should have been checked in the first place: parsing 99 real modules (the standard set plus Cisco, Juniper, Arista, IEEE, Fortinet, APC and others) with this branch and with It found a regression I had introduced, which the unit tests did not: the clause keywords are excluded from the boundary that ends a The same differential is the evidence for everything else:
The four nodes that Nothing that resolved correctly before resolves differently now — that is what the zero in the OID-changed row means, and it is the number I would look at first. 47 conventions changed, all of them from prose to a type: Also driven in a browser against the real files: Generated by Claude Code |
2 participants
Reported from a real load:
The warning was right that something was wrong, and wrong about the scale: none of that module's 185 definitions were in the tree, because they all hang off the one anchor that failed.
Six faults, each reproduced against the real file before it was fixed. The set of 23 standard and IEEE modules (SNMPv2-SMI/TC/CONF, RFC1155/1212/1215/1213, IF-MIB, BRIDGE-MIB, ENTITY-MIB, RMON-MIB, HOST-RESOURCES-MIB, UCD-SNMP-MIB, SNMP-FRAMEWORK-MIB, IEEE8021-SECY/TC/CFM/BRIDGE, IEEE8023-LAG, IEEE802dot11, …) now resolves whole: 2048 nodes, nothing unplaced, nothing excluded.
What was wrong
An anchor in the named-number form. Every IEEE module anchors itself at
::= { iso(1) std(0) iso8802(8802) ... }. The first element was taken as a name in full, so the parent looked up wasiso(1), which nothing defines.Hyphens. A descriptor may contain them, and the standard tree is full of them. Matching a name with
\w+stops at the hyphen, somember-bodywas registered as a node calledbodyandmib-2as2, and everything hanging off either became an orphan. IEEE802dot11-MIB lost 218 definitions this way.Definitions quoted in prose. IF-MIB explains
ifTestTypewithnoTest OBJECT IDENTIFIER ::= { 0 0 }in the middle of its DESCRIPTION. That example was read as a definition of its own — and, sitting before the object's real assignment, was also taken as that object's parent. Definitions are now searched for in a copy of the file with the contents of string literals blanked out, while descriptions are still read from the real text: the two are the same length, so a match maps back by index.Macro definitions. The SMI modules define the macros the others are written in. In SNMPv2-SMI the
ENDof one macro sits directly aboveOBJECT-IDENTITY MACRO ::=, and was read as an object calledENDwhose assignment was the::= { 0 0 }ofzeroDotZerothirty lines below — inventing a node and swallowing the real one. Macro bodies are blanked with the same machinery.Root arcs.
zeroDotZerolives at 0.0 and nothing was seeded for that arc, so it and the modules pointing at it resolved to nothing.ccittandjoint-iso-ccittare seeded alongsideiso; a root with no children is not rendered, so an ordinary MIB set still shows the singleisotree.Clause boundaries.
\bwas applied to the whole terminator group, and there is no word boundary between=and a space, so::=never ended aSYNTAXclause: the lastTEXTUAL-CONVENTIONof a module ran on into whatever followed it — SNMP-FRAMEWORK-MIB'sSnmpAdminStringswallowed four assignments. Bounding the convention body then hid five of SNMPv2-TC's conventions, whose ownSYNTAX OBJECT IDENTIFIERline read as the start of the next definition, so clause keywords are excluded from that boundary. All 102 conventions across the 23 modules now parse, none of them polluted.The report was undercounting
An anchor that fails takes its whole subtree with it, so the count is now the subtree rather than the one node that failed to link. The message that started this would have read 185 definitions could not be placed, not 1.
Cost
Masking is a pass over the text. Doing it once rather than once per OBJECT-TYPE block, and blanking with runs of spaces rather than a per-character regex, keeps it to about a tenth of the parse: 3.3 MB parses in 177 ms, against 159 ms before any of this existed (272 ms before those two optimisations).
Tests
193 pass (was 176).
TEST-IEEE-ANCHOR-MIB.txtjoins the fixtures — it carries all three of the shapes above at once — with its expectations intests/fixtures.test.tsandtest-data/README.md. The rest are unit cases intests/mib-parser.test.tsandtests/mib-tree-builder.test.ts, one per fault, each built from the real module that exposed it.Typecheck and build clean. Docs updated: the construction doc gains sections on what the definition scans are allowed to see, on names, and on root arcs; the performance doc records the masking cost.
🤖 Generated with Claude Code
https://claude.ai/code/session_018qyesKa9si8szMUe7w8HNK
Generated by Claude Code