Skip to content

Commit a997211

Browse files
HaydenOrzLuckyFBBCythia828
authored
feat: collect entity (#265)
* feat: add text and word utils * feat: add entity collector class * refactor: rename SyntaxContextType to EntityContextType * refactor: improve EntityCollector * feat: improve mysql parser grammar * feat: add mysql entity collector * test: mysql entity collector tests * feat: remove useless method * feat: improve spark grammar file * feat: add spark entity collector * test: spark entity collector unit tests * feat: remove useless code * feat: add queryStatement label * feat: add crateDatabaseStmt * feat: add trino entity collector * feat: rename trinosql to trino * test: trino collect entity unit tests * test: fix spark test * feat(impala): support impale entity collector (#256) * Feat/collect entity hive (#263) * feat(hive): support hive collect entity * feat(hive): update tableAllColumns * feat: replace antlr4ts with antlr4ng * feat(pgsql): pgsql collect entity (#268) * feat(pgsql): pgsql collect entity * feat(pgsql): optimize some name --------- Co-authored-by: zhaoge <> * feat: get word text by token.text * feat: supprt collect db/function and add splitListener (#270) * feat: supprt collect db/function and add splitListner * feat: remove SplitListener interface in baseParser to use SplitListener in root * fix(mysql): fix show create xxx not celloct as createXXXEntity type * test: fix pgsql unit tests * Feat/error recover predicate (#274) * feat: optimize pgsql grammar * feat: add sql parser base * feat: apply SQLParserBase * feat: add geAllEntities method * test: test collect table when missing column * feat: compose collect and suggestion (#276) * feat: mark stmt which contain caret * test: correct name of getAllEntities * test: remove misscolumn unit tests * test: add suggestionWithEntity tests * feat: flink collect entity (#277) * feat: improve flink sql parser * feat: support flink entity collector * test: flink entity collect unit test * feat: move combine entities to parent class --------- Co-authored-by: 霜序 <976060700@qq.com> Co-authored-by: XCynthia <942884029@qq.com>
1 parent 3f62ad0 commit a997211

230 files changed

Lines changed: 60670 additions & 50434 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎src/grammar/flinksql/FlinkSqlParser.g4‎

Lines changed: 8 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,11 @@ parser grammar FlinkSqlParser;
77
options {
88
tokenVocab=FlinkSqlLexer;
99
caseInsensitive= true;
10+
superClass=SQLParserBase;
11+
}
12+
13+
@header {
14+
import SQLParserBase from '../SQLParserBase';
1015
}
1116

1217
program
@@ -180,7 +185,7 @@ columnNameCreate
180185

181186
columnName
182187
: uid
183-
| expression
188+
| {this.shouldMatchEmpty()}?
184189
;
185190

186191
columnNameList
@@ -289,7 +294,6 @@ transformList
289294

290295
transform
291296
: columnName # identityTransform
292-
| qualifiedName # columnTransform
293297
| LR_BRACKET transformArgument (COMMA transformArgument)* RR_BRACKET # applyTransform
294298
;
295299

@@ -484,6 +488,7 @@ selectClause
484488
projectItemDefinition
485489
: overWindowItem
486490
| columnName (KW_AS? expression)?
491+
| expression (KW_AS? columnName)?
487492
;
488493

489494
overWindowItem
@@ -583,6 +588,7 @@ groupItemDefinition
583588
| LR_BRACKET expression (COMMA expression)* RR_BRACKET
584589
| groupingSetsNotaionName LR_BRACKET expression (COMMA expression)* RR_BRACKET
585590
| groupingSets LR_BRACKET groupItemDefinition (COMMA groupItemDefinition)* RR_BRACKET
591+
| expression
586592
;
587593

588594
groupingSets

‎src/grammar/hive/HiveSqlParser.g4‎

Lines changed: 29 additions & 29 deletions
Original file line numberDiff line numberDiff line change
@@ -1,23 +1,21 @@
11
/**
2-
Licensed to the Apache Software Foundation (ASF) under one or more
3-
contributor license agreements. See the NOTICE file distributed with
4-
this work for additional information regarding copyright ownership.
5-
The ASF licenses this file to You under the Apache License, Version 2.0
6-
(the "License"); you may not use this file except in compliance with
7-
the License. You may obtain a copy of the License at
8-
9-
http://www.apache.org/licenses/LICENSE-2.0
10-
11-
Unless required by applicable law or agreed to in writing, software
12-
distributed under the License is distributed on an "AS IS" BASIS,
13-
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14-
See the License for the specific language governing permissions and
15-
limitations under the License.
16-
*/
2+
Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements.
3+
See the NOTICE file distributed with this work for additional information regarding copyright
4+
ownership. The ASF licenses this file to You under the Apache License, Version 2.0 (the "License");
5+
you may not use this file except in compliance with the License. You may obtain a copy of the
6+
License at
7+
8+
http://www.apache.org/licenses/LICENSE-2.0
9+
10+
Unless required by applicable law or agreed to in writing, software distributed under the License
11+
is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
12+
implied. See the License for the specific language governing permissions and limitations under the
13+
License.
14+
*/
1715

1816
/**
19-
* This file is an adaptation of antlr/grammars-v4's sql/hive/v4/HiveParser.g4 grammar.
20-
* Reference: https://github.com/antlr/grammars-v4/blob/master/sql/hive/v4/HiveParser.g4
17+
* This file is an adaptation of antlr/grammars-v4's sql/hive/v4/HiveParser.g4 grammar. Reference:
18+
* https://github.com/antlr/grammars-v4/blob/master/sql/hive/v4/HiveParser.g4
2119
*/
2220

2321
// $antlr-format alignTrailingComments true, columnLimit 150, minEmptyLines 1, maxEmptyLinesToKeep 1, reflowComments false, useTab false
@@ -30,6 +28,11 @@ options
3028
{
3129
tokenVocab=HiveSqlLexer;
3230
caseInsensitive= true;
31+
superClass=SQLParserBase;
32+
}
33+
34+
@header {
35+
import SQLParserBase from '../SQLParserBase';
3336
}
3437

3538
program
@@ -804,6 +807,7 @@ columnNameList
804807

805808
columnName
806809
: id_ (DOT id_)*
810+
| {this.shouldMatchEmpty()}?
807811
;
808812

809813
columnNameCreate
@@ -1096,7 +1100,10 @@ fromStatement
10961100
;
10971101

10981102
singleFromStatement
1099-
: fromClause b+=body+
1103+
: fromClause insertClause selectClause lateralView? whereClause? groupByClause? havingClause? window_clause? qualifyClause? orderByClause?
1104+
clusterByClause? distributeByClause? sortByClause? limitClause? # fromInsertStmt
1105+
| fromClause selectClause lateralView? whereClause? groupByClause? havingClause? window_clause? qualifyClause? orderByClause? clusterByClause?
1106+
distributeByClause? sortByClause? limitClause? # fromSelectStmt
11001107
;
11011108

11021109
/*
@@ -1106,8 +1113,8 @@ The valuesClause rule below ensures that the parse tree for
11061113
very similar to the tree for "insert into table FOO select a,b from BAR".
11071114
*/
11081115
regularBody
1109-
: i=insertClause s=selectStatement
1110-
| selectStatement
1116+
: i=insertClause s=selectStatement # insertStmt
1117+
| selectStatement # selectStmt
11111118
;
11121119

11131120
atomSelectStatement
@@ -1128,13 +1135,6 @@ selectStatementWithCTE
11281135
: w=withClause? selectStatement
11291136
;
11301137

1131-
body
1132-
: insertClause selectClause lateralView? whereClause? groupByClause? havingClause? window_clause? qualifyClause? orderByClause? clusterByClause?
1133-
distributeByClause? sortByClause? limitClause?
1134-
| selectClause lateralView? whereClause? groupByClause? havingClause? window_clause? qualifyClause? orderByClause? clusterByClause?
1135-
distributeByClause? sortByClause? limitClause?
1136-
;
1137-
11381138
insertClause
11391139
: KW_INSERT (
11401140
KW_OVERWRITE destination ifNotExists?
@@ -1667,8 +1667,7 @@ dropDataConnectorStatement
16671667
;
16681668

16691669
tableAllColumns
1670-
: STAR
1671-
| tableOrView DOT STAR
1670+
: (id_ DOT)* STAR
16721671
;
16731672

16741673
defaultValue
@@ -1866,6 +1865,7 @@ VALUES(1),(2) means 2 rows, 1 column each.
18661865
VALUES(1,2),(3,4) means 2 rows, 2 columns each.
18671866
VALUES(1,2,3) means 1 row, 3 columns
18681867
*/
1868+
18691869
valuesClause
18701870
: KW_VALUES valuesTableConstructor
18711871
;

‎src/grammar/impala/ImpalaSqlParser.g4‎

Lines changed: 15 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,11 @@ options
2222
{
2323
tokenVocab=ImpalaSqlLexer;
2424
caseInsensitive= true;
25+
superClass=SQLParserBase;
26+
}
27+
28+
@header {
29+
import SQLParserBase from '../SQLParserBase';
2530
}
2631

2732
program
@@ -75,7 +80,7 @@ createStatement
7580
createTableSelect
7681
: KW_CREATE KW_EXTERNAL? KW_TABLE ifNotExists? tableNameCreate (
7782
LPAREN columnDefinition (COMMA columnDefinition)* (COMMA constraintSpecification)? RPAREN
78-
)? (KW_PARTITIONED KW_BY (partitionedBy | createColumnAliases))? createCommonItem (
83+
)? (KW_PARTITIONED KW_BY (columnAliases | partitionedBy))? createCommonItem (
7984
KW_AS queryStatement
8085
)?
8186
;
@@ -555,6 +560,7 @@ functionNamePath
555560

556561
columnNamePath
557562
: qualifiedName
563+
| {this.shouldMatchEmpty()}?
558564
;
559565

560566
tableOrViewPath
@@ -582,8 +588,8 @@ assignmentItem
582588
;
583589

584590
viewColumns
585-
: LPAREN columnNamePath (KW_COMMENT stringLiteral)? (
586-
COMMA identifier (KW_COMMENT stringLiteral)?
591+
: LPAREN columnNamePathCreate (KW_COMMENT stringLiteral)? (
592+
COMMA columnNamePathCreate (KW_COMMENT stringLiteral)?
587593
)* RPAREN
588594
;
589595

@@ -610,6 +616,10 @@ foreignKeySpecification
610616
)? (KW_RELY)?
611617
;
612618

619+
columnSpec
620+
: columnNamePath type (KW_COMMENT stringLiteral)?
621+
;
622+
613623
columnDefinition
614624
: columnNamePathCreate type (KW_COMMENT stringLiteral)?
615625
;
@@ -625,7 +635,7 @@ kuduColumnDefinition
625635
;
626636

627637
columnSpecWithKudu
628-
: columnNamePath type (KW_COMMENT stringLiteral)? (kuduAttributes kuduAttributes*?)?
638+
: columnSpec? (kuduAttributes kuduAttributes*?)?
629639
;
630640

631641
createColumnSpecWithKudu
@@ -712,7 +722,7 @@ properties
712722
;
713723

714724
partitionedBy
715-
: LPAREN columnDefinition (COMMA columnDefinition)*? RPAREN
725+
: LPAREN columnSpec (COMMA columnSpec)*? RPAREN
716726
;
717727

718728
sortedBy
@@ -835,10 +845,6 @@ columnAliases
835845
: LPAREN columnNamePath (COMMA columnNamePath)* RPAREN
836846
;
837847

838-
createColumnAliases
839-
: LPAREN columnNamePathCreate (COMMA columnNamePathCreate)* RPAREN
840-
;
841-
842848
relationPrimary
843849
: tableOrViewPath
844850
| KW_LATERAL? subQueryRelation

0 commit comments

Comments
 (0)