Skip to content

feat(schema): split schema sources so unreadable statements fail alone - #64

Merged
omer-cengel merged 1 commit into
masterfrom
feature/split-schema-sources-into-statements
Oct 10, 2026
Merged

omer-cengel merged 1 commit into
masterfrom
feature/split-schema-sources-into-statements

Conversation

@omer-cengel

Copy link
Copy Markdown
Member

Summary

Schema files are now split into statements using JSqlParser's own lexer, and each statement is parsed on its own. A statement the parser cannot read no longer fails the whole file. Real migration histories and pg_dump --schema-only snapshots that contain DO blocks, DROP TYPE, COMMENT ON TYPE, SET search_path TO a, b, identity columns or NOT VALID constraints now load.

Changes

  • Schema sources are split at the statement separators the SQL lexer reports. Separators inside strings, quoted identifiers, comments and $$ ... $$ bodies do not split. A tagged dollar-quoted body such as $body$ ... $body$ or $fn$BEGIN ... END$fn$ ends at the next occurrence of its own delimiter. A body that is never closed fails with its line and column.
  • psql meta-command lines, such as the \restrict and \unrestrict lines that recent pg_dump releases write, are skipped, and every reported line and column stays a position in the original file.
  • A statement the parser cannot read is ignored unless it opens as CREATE/ALTER/DROP TABLE or CREATE/ALTER TYPE. Such an unreadable table or type statement is rejected with the parser's reason at the file line and column of the unexpected token, for example Encountered unexpected token: "DATA" at line 3, column 41.
  • An unreadable ALTER TABLE is ignored, without resolving its table, when each of its top-level actions is ADD CONSTRAINT <name>, ADD CHECK/UNIQUE/EXCLUDE/PRIMARY KEY/FOREIGN KEY, or ALTER [COLUMN] <name> ADD GENERATED. This covers the identity form pg_dump writes, as well as EXCLUDE, DEFERRABLE INITIALLY DEFERRED, USING INDEX, NOT VALID and unnamed CHECK constraints. Any other unreadable ALTER TABLE, such as ALTER COLUMN ... SET DATA TYPE, is still rejected.
  • Functions and procedures whose body is single-quoted or written with RETURN are now ignored like dollar-quoted ones, and the statements after them are applied.
  • DROP TYPE is ignored, so a modeled enum and its columns stay modeled.
  • The PostgreSQL guide documents the splitting rules, the ignored and rejected unreadable forms, and the known mis-lex of a backslash-escaped quote in E'...' strings.

Scope and non-goals

  • Statements the parser does read are handled exactly as before, including the ALTER TABLE and ALTER TYPE actions that are still rejected, such as SET DEFAULT and OWNER TO.
  • DDL inside DO blocks and function bodies is still not modeled, and DROP TYPE ... CASCADE does not remove columns.
  • Query parsing, schema namespaces, LIKE and PARTITION OF tables, and COPY data are unchanged.
  • The E'it\'s' mis-lex is documented, not fixed.

Split each schema source at the statement separators JSqlParser's own
token manager reports and parse every statement alone. A tagged
dollar-quoted body ends at the next occurrence of its delimiter, and psql
meta-command lines such as \restrict are blanked before lexing.

A statement the parser cannot read is ignored unless it opens as table or
type DDL. An unreadable ALTER TABLE whose every top-level action adds a
constraint or an identity is ignored without resolving its table, which
lets pg_dump identity columns load. Any other unreadable table or type
statement is rejected with the parser's reason at its file line and
column. Functions and procedures with bodies that are not dollar-quoted
are now ignored.
@omer-cengel
omer-cengel merged commit 296f4f8 into master Oct 10, 2026
7 checks passed
@omer-cengel
omer-cengel deleted the feature/split-schema-sources-into-statements branch October 10, 2026 11:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant