Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
54 changes: 43 additions & 11 deletions docs/postgresql.md
Original file line number Diff line number Diff line change
Expand Up @@ -123,7 +123,10 @@ accepted and map exactly like their unparameterized spellings.
| `DECIMAL`, `NUMERIC` | `java.math.BigDecimal` | |
| `UUID` | `java.util.UUID` | |

Any spelling that is not listed above is rejected.
Any spelling that is not listed above, and any array of any element type, has no
Java mapping. Such a column is recorded with its declared type instead of
failing the schema, and fails only a query that uses it; see
[Unsupported Types and DDL](#unsupported-types-and-ddl).

The generated repository imports `java.time.LocalDate`, `java.time.LocalDateTime`,
`java.time.OffsetDateTime`, `java.math.BigDecimal`, and `java.util.UUID` as
Expand Down Expand Up @@ -200,9 +203,8 @@ columns of the integration schema snapshot, which cover `SMALLINT`, `VARCHAR`,

## Unsupported Types and DDL

The following type families are not supported at all, because only the
spellings listed in [Supported Column Types](#supported-column-types) are
accepted:
The following type families have no Java mapping, because only the spellings
listed in [Supported Column Types](#supported-column-types) are mapped:

- floating point, such as `REAL` and `DOUBLE PRECISION`,
- binary, such as `BYTEA`,
Expand All @@ -214,24 +216,43 @@ accepted:
- composite types,
- spatial types.

A column of such a type does not fail the schema. It is recorded with its
declared type, written as the canonical spelling of that type — upper case, with
parenthesized type arguments removed — followed by `[]` for each declared array
dimension, so `jsonb` is recorded as `JSONB`, `varchar(20)[]` as `VARCHAR[]`,
and `integer[][]` as `INTEGER[][]`.

A recorded column fails only the analysis of a query that

- reads it, as a `SELECT` item or a `RETURNING` item,
- binds it, meaning a placeholder takes its type in a comparison, an `IN` list,
a range bound, a `LIKE`/`ILIKE` pattern, an `INSERT` column, or an `UPDATE`
assignment, or
- expands it, through `SELECT *`, `SELECT qualifier.*`, or `RETURNING *`.

Every other query over the same table compiles, including one that references
the column without using its type, such as `WHERE tags IS NULL`.

### Failure Behavior

An unsupported type, an unsupported statement, or a schema sqlcj cannot parse
stops compilation. `sqlcj generate` prints a single message on standard error
An unsupported statement, a schema sqlcj cannot parse, or a query that uses a
column of an unmapped type stops compilation. `sqlcj generate` prints a single
message on standard error
and exits with status `1`. Because every configured source is analyzed and
generated before the run writes its first file, such a failure writes no
generated file, and output
written by an earlier successful run is left unchanged. The writing step itself
is sequential rather than atomic; see
[Generated Output and Failures](configuration.md#generated-output-and-failures).

The message names the schema source and the offending type or statement, or the
syntax error with the line and column it was found at:
A schema message names the schema source and the offending statement, or the
syntax error with the line and column it was found at. A query message names the
query, its source, its header line, and the offending column and recorded type:

```text
sqlcj: Invalid schema source /home/dev/project/schema.sql: Unsupported SQL column type: JSONB
sqlcj: Invalid schema source /home/dev/project/schema.sql: Unsupported schema statement: Alter
sqlcj: Invalid schema source /home/dev/project/schema.sql: Encountered unexpected token: ";" <ST_SEMICOLON> at line 4, column 1
sqlcj: Invalid query 'ListTags' in /home/dev/project/queries.sql at line 5: Column 'tags' has unsupported type VARCHAR[]
```

## Verified by
Expand All @@ -248,8 +269,19 @@ Type table:
and `PostgresIntegrationTest.shouldRoundTripPostgresTypeSpellingValues`
execute the Java mappings, including the blank-padded `CHAR` values, against
PostgreSQL 16.
- `DefaultSchemaParserTest.shouldRejectUnsupportedColumnType` covers the
rejected spellings.
- `DefaultSchemaParserTest.shouldRecordUnsupportedColumnType` and
`DefaultSchemaParserTest.shouldParseNullabilityOfUnsupportedColumnTypes` cover
the recorded type of an unmapped spelling and of an array, beside the mapped
columns of the same table.
- `QueryAnalyzerTest.shouldRejectQueryThatUsesAnUnsupportedTypeColumn` covers
each reading, binding, and expanding query, and
`QueryAnalyzerTest.shouldAnalyzeQueryBesideAnUnsupportedTypeColumn` and
`QueryAnalyzerTest.shouldAnalyzeIsNullOnAnUnsupportedTypeColumn` cover the
queries over the same table that still compile.
- `SqlcjCompilerIntegrationTest.shouldReportTheQueryThatReadsAnUnsupportedTypeColumn`
covers the query diagnostic and that no file is written, and
`SqlcjCompilerIntegrationTest.shouldGenerateCompilableRepositoryBesideUnsupportedTypeColumns`
compiles a repository generated beside an array and a `JSONB` column.

`CREATE TABLE` table:

Expand Down
30 changes: 25 additions & 5 deletions sqlcj-cli/src/main/java/dev/sqlcj/analysis/QueryAnalyzer.java
Original file line number Diff line number Diff line change
Expand Up @@ -336,7 +336,7 @@ private List<QueryColumn> resolveReturningItem(Expression expression, Source sou
return List.of(
new QueryColumn(
schemaColumn.name(),
schemaColumn.type(),
requireSupportedType(schemaColumn),
schemaColumn.nullable()
)
);
Expand Down Expand Up @@ -1029,9 +1029,13 @@ private void resolveLikeExpression(
return;
}

dev.sqlcj.schema.Column schemaColumn = resolveColumn(column, sources).column();

requireSupportedType(schemaColumn);

addParameter(
parameter,
requireTextColumn(resolveColumn(column, sources).column()),
requireTextColumn(schemaColumn),
parameters
);
}
Expand Down Expand Up @@ -1182,7 +1186,7 @@ private void addParameter(
addParameter(
parameter,
column.name(),
column.type(),
requireSupportedType(column),
parameters
);
}
Expand Down Expand Up @@ -1272,7 +1276,7 @@ private List<QueryColumn> resolveColumns(PlainSelect plainSelect, List<Source> s
columns.add(
new QueryColumn(
selectedColumnName(selectItem.getAlias(), schemaColumn),
schemaColumn.type(),
requireSupportedType(schemaColumn),
schemaColumn.nullable() || resolved.source().leftJoined()
)
);
Expand Down Expand Up @@ -1457,13 +1461,29 @@ private List<QueryColumn> resolveAllColumns(Source source) {
.map(
column -> new QueryColumn(
column.name(),
column.type(),
requireSupportedType(column),
column.nullable() || source.leftJoined()
)
)
.toList();
}

/**
* Reports the analyzed type of a schema column. A column the schema
* recorded without a mapped type fails here, in the query that reads,
* binds, or expands it, rather than when the schema is parsed.
*/
private ColumnType requireSupportedType(dev.sqlcj.schema.Column column) {
if (column.type() == null) {
throw new UnsupportedOperationException(
"Column '%s' has unsupported type %s"
.formatted(column.name(), column.unsupportedType())
);
}

return column.type();
}

private dev.sqlcj.schema.Column findColumn(dev.sqlcj.schema.Table table, String columnName) {
return lookupColumn(table, columnName)
.orElseThrow(
Expand Down
16 changes: 15 additions & 1 deletion sqlcj-cli/src/main/java/dev/sqlcj/schema/Column.java
Original file line number Diff line number Diff line change
@@ -1,8 +1,22 @@
package dev.sqlcj.schema;

/**
* One column of a schema table. A column whose declared SQL type sqlcj maps
* carries that {@link ColumnType}; a column of any other type, including every
* array column, is recorded with its declared type text in
* {@code unsupportedType} instead, so the schema loads and only a query that
* uses the column fails. Exactly one of {@code type} and
* {@code unsupportedType} is non-null.
*/
public record Column(
String name,
ColumnType type,
boolean nullable
boolean nullable,
String unsupportedType
) {

/** A column of a mapped type. */
public Column(String name, ColumnType type, boolean nullable) {
this(name, type, nullable, null);
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -72,16 +72,45 @@ private Table parseTable(CreateTable createTable) {
);
}

/**
* Parses one column, recording a column sqlcj cannot map with its declared
* type text instead of failing the schema. An array column is such a column
* regardless of its element type, because the mapped types are all scalar.
*/
private Column parseColumn(ColumnDefinition definition) {
String typeName = typeName(definition);
int arrayDimensions = arrayDimensions(definition);
boolean nullable = !isSerial(typeName) && isNullable(definition);

ColumnType type = arrayDimensions == 0
? columnType(typeName)
: null;

if (type != null) {
return new Column(columnName(definition), type, nullable);
}

return new Column(
columnName(definition),
columnType(typeName),
!isSerial(typeName) && isNullable(definition)
null,
nullable,
typeName + "[]".repeat(arrayDimensions)
);
}

/**
* Reports how many array dimensions a column declares. The parser reports
* an array's dimensions separately from its element type, so a column with
* any dimension is an array of the reported type.
*/
private int arrayDimensions(ColumnDefinition definition) {
List<Integer> arrayData = definition.getColDataType().getArrayData();

return arrayData == null
? 0
: arrayData.size();
}

/**
* Returns the canonical column name without SQL identifier delimiters.
*/
Expand All @@ -106,6 +135,7 @@ private String typeName(ColumnDefinition definition) {
.trim();
}

/** The mapped type of declared spelling, or {@code null} if unmapped. */
private ColumnType columnType(String typeName) {
return switch (typeName) {
case "INTEGER", "INT", "INT4", "SERIAL", "SERIAL4" -> ColumnType.INTEGER;
Expand All @@ -119,9 +149,7 @@ private ColumnType columnType(String typeName) {
case "TIMESTAMP WITH TIME ZONE", "TIMESTAMPTZ" -> ColumnType.TIMESTAMP_WITH_TIME_ZONE;
case "DECIMAL", "NUMERIC" -> ColumnType.DECIMAL;
case "UUID" -> ColumnType.UUID;
default -> throw new UnsupportedOperationException(
"Unsupported SQL column type: " + typeName
);
default -> null;
};
}

Expand Down
102 changes: 102 additions & 0 deletions sqlcj-cli/src/test/java/dev/sqlcj/analysis/QueryAnalyzerTest.java
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,24 @@ class QueryAnalyzerTest {
)
);

/**
* A schema whose {@code tags} column the parser recorded without a mapped
* type, beside the supported columns of the same table.
*/
private static final Schema unsupportedTypeSchema = new Schema(
List.of(
new Table(
"users",
List.of(
new Column("id", ColumnType.BIGINT, false),
new Column("name", ColumnType.VARCHAR, true),
new Column("tags", null, true, "VARCHAR[]")
),
List.of()
)
)
);

private static final Schema joinSchema = new Schema(
List.of(
new Table(
Expand Down Expand Up @@ -2837,4 +2855,88 @@ void shouldKeepPlaceholderTextThatIsNotAParameter() {
model.parameters()
);
}

/**
* A column the schema recorded without a mapped type fails the query that
* reads, binds, or expands it, naming the column and the recorded type.
*/
@ParameterizedTest
@CsvSource(
delimiter = '|',
value = {
"SELECT tags FROM users|MANY",
"SELECT * FROM users|MANY",
"SELECT b.* FROM users b|MANY",
"SELECT id FROM users WHERE tags = $1|MANY",
"SELECT id FROM users WHERE tags IN ($1)|MANY",
"SELECT id FROM users WHERE tags LIKE $1|MANY",
"INSERT INTO users (tags) VALUES ($1)|EXEC",
"UPDATE users SET tags = $1|EXEC",
"DELETE FROM users WHERE id = $1 RETURNING tags|MANY",
"DELETE FROM users WHERE id = $1 RETURNING *|MANY"
}
)
void shouldRejectQueryThatUsesAnUnsupportedTypeColumn(String sql, QueryType type) {
Query query = new Query("UseTags", type, sql);
ParsedSql parsedSql = parser.parse(sql);

UnsupportedOperationException exception = assertThrows(
UnsupportedOperationException.class,
() -> analyzer.analyze(query, parsedSql, unsupportedTypeSchema)
);

assertEquals(
"Column 'tags' has unsupported type VARCHAR[]",
exception.getMessage()
);
}

/**
* A query that neither reads, binds, nor expands the recorded column is
* analyzed exactly as it is over a table without that column. Both schemas
* declare the same {@code id} and {@code name} columns, and neither query
* references any other column.
*/
@ParameterizedTest
@ValueSource(
strings = {
"SELECT id, name FROM users WHERE id = $1",
"SELECT COUNT(*) AS total FROM users"
}
)
void shouldAnalyzeQueryBesideAnUnsupportedTypeColumn(String sql) {
Query query = new Query("ReadUsers", QueryType.MANY, sql);

assertEquals(
analyzer.analyze(query, parser.parse(sql), schema),
analyzer.analyze(query, parser.parse(sql), unsupportedTypeSchema)
);
}

/**
* {@code IS NULL} consumes no column type, so it resolves the recorded
* column without failing.
*/
@Test
void shouldAnalyzeIsNullOnAnUnsupportedTypeColumn() {
String sql = "SELECT id FROM users WHERE tags IS NULL";

QueryModel model = analyzer.analyze(
new Query("ListUntagged", QueryType.MANY, sql),
parser.parse(sql),
unsupportedTypeSchema
);

assertEquals(
List.of(new QueryColumn("id", ColumnType.BIGINT, false)),
model.columns()
);

assertTrue(model.parameters().isEmpty());

assertEquals(
"SELECT id FROM users WHERE tags IS NULL",
model.executableSql()
);
}
}
Loading
Loading