Skip to content

Optimize MetaDataProtoEditor.renameRecordTypes() - #4536

Merged
alecgrieser merged 14 commits into
mainfrom
apple/robert-brunel/renameRecordTypes-2
Oct 7, 2026
Merged

alecgrieser merged 14 commits into
mainfrom
apple/robert-brunel/renameRecordTypes-2

Conversation

@robert-brunel

@robert-brunel robert-brunel commented Aug 28, 2026 •

Copy link
Copy Markdown
Contributor
  • Replace the naive O(N²) implementation of renameRecordTypes(), which renames the types one by one via renameRecordType(), with an efficient batched version that applies all renames in one go.
  • Cleanly split the per-rename validation from the actual mutation throughout, so that the whole batch gets validated before anything is mutated.
  • Add a RecordTypeRenames data structure that indexes renames by fully qualified name to avoid having to scan all renames per item. This brings what would otherwise still be an O(N²) algorithm down to O(N).
  • Leave renameRecordType() itself untouched for the time being. It will be simplified in a subsequent change by “rebasing” it on the utility methods introduced here.

The batched implementation performs the costly compilation of the FileDescriptor exactly once (via Descriptors.FileDescriptor.buildFrom(), from the original proto) and applies the name mapping in a single walk of the builder. The trivial implementation, by contrast, performs a recompilation of the entire records FileDescriptor plus a full records.toBuilder() deep-copy in each iteration of the loop over the record types. In the common use case where all top-level record types are to be renamed, that means O(N) descriptor compilations for N record types.

Applying multiple renames in the batched manner is equivalent to applying them one-by-one, provided that the name mapping defined by the given renamer is collision-free. The batched implementation validates the mapping upfront. This allows it to provide a stronger exception safety guarantee. If the mapping is invalid, an exception will be thrown before any of the renames is performed. The one-by-one implementation, by contrast, may raise an exception halfway through; and that may depend on the order in which the renames are applied.

Testing

  • Cross-check renameRecordType() and renameRecordTypes() against each other throughout the existing test suite.
  • Add dedicated tests for every MetaDataException reachable from renameRecordTypes(), plus a couple of previously unexercised edge cases.
  • Fix an unrelated, pre-existing bug in the Joined.json test fixture (a join referenced a nonexistent constituent), uncovered along the way.
  • Add a performance test that renames a large number of record types. In a local ad-hoc experiment, measured after warmup as the median of 11 runs, the batched implementation was about 15× faster than an equivalent one-by-one sequence of renameRecordType() calls with 50 record types (0.12 ms versus 1.7 ms), and about 76× faster with 250 record types (0.46 ms versus 35.0 ms), which is consistent with the O(N) vs. O(N²) difference.

@robert-brunel robert-brunel added performance Performance issues enhancement New feature or request labels Aug 28, 2026
@robert-brunel robert-brunel changed the title apple/robert brunel/renameRecordTypes 2 Optimize MetaDataProtoEditor.renameRecordTypes() Aug 28, 2026
@robert-brunel robert-brunel self-assigned this Aug 28, 2026
@robert-brunel
robert-brunel marked this pull request as ready for review August 28, 2026 19:06
* Replace the naive O(N²) implementation of `renameRecordTypes()`, which renames the types one by one via `renameRecordType()`, with an efficient batched version that applies all renames in one go.
* Cleanly split the per-rename validation from the actual mutation throughout, so that the whole batch gets validated _before_ anything is mutated.
* Add a `RecordTypeRenames` data structure that indexes renames by fully qualified name to avoid having to scan all renames per item. This brings what would otherwise still be an O(N²) algorithm down to O(N).
* Leave `renameRecordType()` itself untouched for the time being. It will be simplified in a subsequent change by “rebasing” it on the utility methods introduced here.

The batched implementation performs the costly compilation of the `FileDescriptor` exactly _once_ (via `Descriptors.FileDescriptor.buildFrom()`, from the original proto) and applies the name mapping in a single walk of the builder. The trivial implementation, by contrast, performs a recompilation of the entire records `FileDescriptor` plus a full `records.toBuilder()` deep-copy _in each iteration_ of the loop over the record types. In the common use case where _all_ top-level record types are to be renamed, that means O(N) descriptor compilations for N record types.

Applying multiple renames in the batched manner is equivalent to applying them one-by-one, _provided that_ the name mapping defined by the given `renamer` is collision-free. The batched implementation validates the mapping upfront. This allows it to provide a stronger exception safety guarantee. If the mapping is invalid, an exception will be thrown before _any_ of the renames is performed. The one-by-one implementation, by contrast, may raise an exception halfway through; and that may depend on the order in which the renames are applied.

### Testing

* Cross-check `renameRecordType()` and `renameRecordTypes()` against each other throughout the existing test suite.
* Add dedicated tests for every `MetaDataException` reachable from `renameRecordTypes()`, plus a couple of previously unexercised edge cases.
* Fix an unrelated, pre-existing bug in the `Joined.json` test fixture (a join referenced a nonexistent constituent), uncovered along the way.
* Add a performance test that renames a large number of record types. In a local ad-hoc experiment with 50 record types, the batched implementation measured about 17× faster than an equivalent one-by-one sequence of `renameRecordType()` calls (0.1 ms versus 1.5 ms), which is consistent with the O(N) vs. O(N²) difference.
@robert-brunel
robert-brunel force-pushed the apple/robert-brunel/renameRecordTypes-2 branch from efc6211 to 7678c8e Compare September 7, 2026 12:20
Cache the canonical union field names, old and new, on `RecordTypeRename`, along with a flag for whether the union field is to be renamed at all. This way, `validateUnionFieldRenames()` can index the retained field names upfront and check each rename against them in constant time (instead of re-scanning the union’s fields per rename).
Joined and unnested record types cannot be renamed by this method but share the same record type namespace. So a rename that targets one of their names is a collision.
Views and stored queries hold a SQL string that would need parsing to find the record types it references, exactly like `user_defined_functions` (which the rename already rejects right now).
Gate the `record.usage` fixup on the determined `Usage` of the rename rather than on the type name, and reject the case where the two disagree. Renaming a `RECORD`-usage type that happens to be named `RecordTypeUnion` would otherwise mark it as the union.
* Route the two single-rename tests through a `singleRename()` helper that cross-checks the batched implementation, leaving `renameRecordTypesOneByOne()` as the only caller of `renameRecordType()`.
* Run `MetaDataEvolutionValidator` over every rename result.
* Add the missing scenarios: Renaming onto a top-level `NESTED` type; renaming onto the name of a nested message; swapping two names without canonical union fields; a union that is not default-named but marked `UNION`; several union fields for the renamed type; and a union field referencing a nested type whose parent is renamed.
* Add an `UnnestedShadowedName` variant whose constituent names the top-level `T1` rather than the shadowing `T2.T1`.
* Drop the catch-assertion from `conflictingName`, which now succeeds in both orderings.
Determine which record types the metadata defines locally by resolving the union’s field types, rather than by matching record type names against the local top-level message types. That way, a record type name that denotes an imported type is now skipped even in the corner case where `MetaData.records` happens to declare an unrelated message type of that name. Previously the renamer would get applied to such a name and would rename the local message type.
@robert-brunel
robert-brunel force-pushed the apple/robert-brunel/renameRecordTypes-2 branch from 97c8091 to b413105 Compare September 22, 2026 16:43
Determine the record types to rename from the fields of the union message type, the same way `RecordMetaDataBuilder` does, rather than from `MetaData.record_types`. Previously, record types without a `record_types` entry, such as those whose primary key comes from the `(field).primary_key` extension, were never renamed. Also reject a rename onto the name of any other record type, including one backed by a nested message type.

@alecgrieser alecgrieser left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall, this looks good. I mainly focused on the documentation (and what it means from an API perspective) and the main implementation, so I didn't really look at the tests. I have a few concerns about drift between renameRecordType and renameRecordTypes, and I'd rather we aligned the methods, in general, rather than marking the difference. I guess I could see doing that as a follow-up, though.

@robert-brunel
robert-brunel force-pushed the apple/robert-brunel/renameRecordTypes-2 branch from 850ba50 to 38261e1 Compare October 2, 2026 17:27
@robert-brunel
robert-brunel force-pushed the apple/robert-brunel/renameRecordTypes-2 branch from 38261e1 to f7eb8d5 Compare October 2, 2026 18:03
Reject any rename upfront if the union references a record type whose message type is defined in a dependency file, instead of skipping that record type.

@ScottDugas ScottDugas left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I few more things that still seem a bit messy about the api of renameRecordTypes. I am honestly not sure whether it makes sense to continue improving this API as part of this PR, or add them separately to the stack.

… by a nested message type

Reject any rename upfront if the union references a record type whose message type is nested within another one, the same way as for imported record types, instead of skipping that record type.
Every record type is now backed by a top-level message type in `MetaData.records`, which the collision check already covers, and an entry in `MetaData.record_types` without a corresponding union field is invalid metadata. Drop the tests built on such entries.

@alecgrieser alecgrieser left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@alecgrieser
alecgrieser merged commit 8bf1b5f into main Oct 7, 2026
21 of 22 checks passed
@alecgrieser
alecgrieser deleted the apple/robert-brunel/renameRecordTypes-2 branch October 7, 2026 15:51
robert-brunel added a commit that referenced this pull request Nov 9, 2026
…field

When several union fields reference a renamed record type, `renameRecordType()` and `renameRecordTypes()` now always rename the one that is canonically named. Previously, a field with a higher number could “displace” it, which would result in the canonical field keeping its old name. This could also cause a batch rename that swaps two type names to be spuriously rejected with “Cannot rename union field because a field of the new name already exists”.

`RecordMetaDataBuilder` writes the records of a type under its canonically named union field if there is one, and otherwise under the one with the highest number. So the old behavior also effectively moved the records of the renamed type to the higher-numbered field. With this change, the editor now selects the union field by the same rule as `RecordMetaDataBuilder`, so that records keep getting written under the original field.

Follow-up to #4536. Resolves #4732.
robert-brunel added a commit that referenced this pull request Nov 9, 2026
…field

When several union fields reference a renamed record type, `renameRecordType()` and `renameRecordTypes()` now always rename the one that is canonically named. Previously, a field with a higher number could “displace” it, which would result in the canonical field keeping its old name. This could also cause a batch rename that swaps two type names to be spuriously rejected with “Cannot rename union field because a field of the new name already exists”.

`RecordMetaDataBuilder` writes the records of a type under its canonically named union field if there is one, and otherwise under the one with the highest number. So the old behavior also effectively moved the records of the renamed type to the higher-numbered field. With this change, the editor now selects the union field by the same rule as `RecordMetaDataBuilder`, so that records keep getting written under the original field.

Follow-up to #4536. Resolves #4732.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request performance Performance issues

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants