oushu/lava-oushudb-dt-sql-parser
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
395 Commits 1 Branch 0 Tags 47 MiB
bb0fad1dbe
Commit Graph

62 Commits

Author SHA1 Message Date
Hayden
bb0fad1dbe
refactor: standard naming (#278) 
* refactor: rename flinksql to flink

* refactor: rename pgsql to postgresql

* refactor: rename trinosql to trino

* refactor: replace all default exports with named export

* refactor: rename basicParser to basicSQL

* refactor: rename basic-parser-types to types

* refactor: replace arrow func with plain func
 2024-03-27 10:33:25 +08:00
Hayden
a99721162b
feat: collect entity (#265) 
* feat: add text and word utils

* feat: add entity collector class

* refactor: rename SyntaxContextType to EntityContextType

* refactor: improve EntityCollector

* feat: improve mysql parser grammar

* feat: add mysql entity collector

* test: mysql entity collector tests

* feat: remove useless method

* feat: improve spark grammar file

* feat: add spark entity collector

* test: spark entity collector unit tests

* feat: remove useless code

* feat: add queryStatement label

* feat: add crateDatabaseStmt

* feat: add trino entity collector

* feat: rename trinosql to trino

* test: trino collect entity unit tests

* test: fix spark test

* feat(impala): support impale entity collector (#256)

* Feat/collect entity hive (#263)

* feat(hive): support hive collect entity

* feat(hive): update tableAllColumns

* feat: replace antlr4ts with antlr4ng

* feat(pgsql): pgsql collect entity (#268)

* feat(pgsql): pgsql collect entity

* feat(pgsql): optimize some name

---------

Co-authored-by: zhaoge <>

* feat: get word text by token.text

* feat: supprt collect db/function and add splitListener (#270)

* feat: supprt collect db/function and add splitListner

* feat: remove SplitListener interface in baseParser to use SplitListener in root

* fix(mysql): fix show create xxx not celloct as createXXXEntity type

* test: fix pgsql unit tests

* Feat/error recover predicate (#274)

* feat: optimize pgsql grammar

* feat: add sql parser base

* feat: apply SQLParserBase

* feat: add geAllEntities method

* test: test collect table when missing column

* feat: compose collect and suggestion (#276)

* feat: mark stmt which contain caret

* test: correct name of getAllEntities

* test: remove misscolumn unit tests

* test: add suggestionWithEntity tests

* feat: flink collect entity (#277)

* feat: improve flink sql parser

* feat: support flink entity collector

* test: flink entity collect unit test

* feat: move combine entities to parent class

---------

Co-authored-by: 霜序 <976060700@qq.com>
Co-authored-by: XCynthia <942884029@qq.com>
 2024-03-26 14:28:27 +08:00
Hayden
3f62ad0d32
feat: use SLL mode (#269) 
* feat: use SLL(*) PredictionMode for better performance

* feat: optimize mysql grammar to fit SLL mode

* feat: optimize postgre grammmar to fit SLL mode

* feat: optimize spark grammar to fit SLL mode

* test: correct unit tests

* feat: optimize pgsql grammar
 2024-03-19 11:27:23 +08:00
Hayden
195878da9b
feat: migrate to antlr4ng (#267) 
* feat: replace antlr4ts with antlr4ng

* feat: switch caseInsensitive option on

* feat: recompile all g4 file

* feat:  update parser to fit antlr4ng

* test: update test to fit antlr4ng
 2024-02-26 20:25:09 +08:00
霜序
4d19f05890 fix: rename createLexerFormCharStream to createLexerFromCharStream (#243) 2024-01-04 11:47:19 +08:00
Hayden
9c52542ec7
Feat/range boundary (#241) 
* feat: unify the index, line and column in all APIs

* docs: describe index, line and column
 2023-12-22 19:12:29 +08:00
Hayden
3c7c59fb70
feat: optimize suggestion (#231) 
* feat: optimize the strategy of finding the right range

* test: apply commentOtherLine util to all suggestion tests

* test: decomment suggestion test cases

* test: add suggestion test cases in multiple statements

* chore: improve comments

* test: update log info in test
 2023-12-13 11:33:47 +08:00
hayden
17eba1f2aa feat: add readonly modifier 2023-12-12 20:04:18 +08:00
hayden
eb5b0a5550 refactor: move findCaretTokenIndex file to parser/common/utils 2023-12-12 20:04:18 +08:00
hayden
96b0376510 fix: correct import path 2023-12-12 20:04:18 +08:00
Hayden
ee8b468778
feat: add ErrorStrategy(#230) 
* refactor: rename errorHandler to errorListener

* feat: add ErrorStrategy to mark context exceptions

* test: errorStrategy unit tests
 2023-12-11 17:34:49 +08:00
Hayden
23f5aac113
Fix/split listener (#228) 
* feat: improve FlinkSqlSplitListener

* feat: improve ImpalaSqlSplitListener

* feat: improve MysqlSplitListener

* fix: correct PgSqlSplitListener

* feat: improve TrinoSqlSplitListener

* test: add split listener unit test

* chore: ignore iml file

* feat: add pgsql missing rules

* test: fix pgsql unit tests
 2023-12-08 18:33:16 +08:00
霜序
484c881583
fix: add missing expression predicate to impala parser(#225) 
* fix(impala): add select supprt is true/false/unknown and support more SEMICOLON

* fix(impala): not use emptyStatement replace SEMICOLON

* test(impala): add select unit test sql and test splitSQLByStatement
 2023-11-30 19:56:07 +08:00
Hayden
31a811d1bb
Feat column codeCompletion (#218) 
* feat: flink support column suggestion

* feat: flink column suggestion unit test

* feat: hive support column suggestion

* feat: hive column suggestion unit test

* feat: trinosql support column suggestion

* feat: trino column suggest unit test

* feat: sparksql collect column suggesstion

* feat: spark column suggestion unit test

* feat: rebuild flinksql lib
 2023-11-28 21:18:44 +08:00
琉易
3dadc0c4b5
feat: mysql auto complete (#219) 
* refactor: mysql auto complete

* test: mysql auto complete

---------

Co-authored-by: liuyi <liuyi@dtstack.com>
 2023-11-28 21:17:18 +08:00
霜序
e203f1a48a
feat: support impala (#184) 
* feat(impala): add impala sqlLexer

* feat(impala): add impala grammar

* feat(impala): add alter table sql

* feat(impala): update alter table sql

* feat(impala): add alter db sql

* feat(impala): add alter view sql

* feat(impala): add compute stats/comment statement and update partition_desc for alter table

* feat(impala): add drop statement sql

* feat(impala): add revoke and grant sql

* feat(impala): add create db/function/role/view sql

* feat(impala): add describe/explain/invalidata_metadata/load_data sql

* feat(impala): add refresh/set/shutdown sql

* feat(impala): add truncate_table/use/values sql

* fix(impala): update shutdown and invaliddate_metadata

* feat(impala): add show/update/upsert sql

* feat(impala): add create/insert sql

* feat(impala): add select and delete sql

* feat(impala): add impala tokens and fix todo

* feat(impala): update impalaparser and some test unit

* feat(impala): add syntax suggestion

* feat(impala): add syntax suggestion

* feat(impala): update test unit

* feat(impala): remove reference

* fix(impala): add statement for sqlname and collect tableName

* fix(impala): fix syntax suggestion unit test

* fix(impala): update syntax suggestion and collect column

* feat(impala): add collect column create
 2023-11-28 21:11:07 +08:00
XCynthia
db05cb3e4f
feat: pgsql collect column name (#216) 
* feat(pgsql: collect column name): pgsql: collect column name

* feat(pgsql): repair column name's unit test and sql

* feat(pgsql): pgsql: collect expression's column

* feat(update column's collect of some expression): pgsql: update column's collect of some expression

---------

Co-authored-by: zhaoge <>
 2023-11-28 10:07:29 +08:00
琉易
428d851913
feat: #190 improve mysql grammer (#196) 
* refactor: generic rename to mysql

* refactor: g4 with mysql syntax

* test: mysql syntax

* refactor: remove useless keywords

* refactor: remove nonReserved keywords

* refactor: lint specificFunction

---------

Co-authored-by: liuyi <liuyi@dtstack.com>
 2023-11-27 15:25:40 +08:00
hayden
00b66ccf44 feat: add column to syntaxContextType 2023-11-20 19:45:24 +08:00
zhaoge
4736a65e0c feat(pgsql: upgrade rules' name): pgsql: upgrade rules' name 2023-11-20 09:41:12 +08:00
XCynthia
cbb1279f9e
feat: support pgsql code completion (#205) 
* feat: pgsql: complete sql, upgrade syntax file and complete autoImprove

* feat: pgsql: complete unit test

* feat: pgsql: update unit test for autocomplete

* feat(pgsql: update split's context type ): pgsql: update split's context type

* feat(pgsql: update schema, table, procedure, tablespace, function, view,database's rule name): pgsql

* feat: pgsql: update usualName's rule realize

* feat: pgsql:update funcName's using

---------

Co-authored-by: zhaoge <>
 2023-11-15 15:34:42 +08:00
Hayden
91b7fd24c0
Feat/trino sql (#188) 
* refactor: prefix trino lexer rule name with KW_

* test: add commentOtherLine function

* feat: optimize trino antlr grammar to adapt to c3

* feat: trinosqlParser supports codeCompletion and spliting

* test: trinoSql codeCompletion unit tests
 2023-10-24 14:37:27 +08:00
hayden
917cb988f2 feat: export trino sql 2023-10-20 17:48:29 +08:00
Hayden
f9dbd9fc23
Refactor/basic parser (#182) 
* feat: replace errorCollector with ParserErrorListener to collect lexer error

* refactor: remove useless method in basicParser

* feat: correct splitSQLByStatement method

* feat: rename parse to parseWithCache and add new parse method

* refactor: rename parserTree to parseTree

* test: rename parserTree to parseTree

* refactor: rename parserError to parseError

* feat: export ErrorHandler type

* feat: use errorhandler form params

* test: basic parser unit tests

* style: lint via prettier
 2023-10-16 17:59:28 +08:00
Hayden
7de192d486
chroe: devops (#180) 
* ci: add dependencies about lint tool

* ci: replace eslint with prettier

* ci: add husky, cz and commitlint

* style: lint fix via prettier

* ci: add prettier and check-types to github workflow

'
 2023-10-13 11:16:36 +08:00
琉易
4d1dfa676f
feat: support spark sql auto complete (#179) 
* refactor: spark sql g4

* feat: support spark sql suggestion

* test: spark sql suggestion unit test

* test: hive spell check

* feat: spark sql keyword has multiple values

* test: KW_NOT KW_RLIKE split into two value

---------

Co-authored-by: liuyi <liuyi@dtstack.com>
 2023-10-12 17:08:21 +08:00
Hayden
53ead45ff5
Optimize/auto complete (#178) 
* feat: optimize hive function name auto complete

* feat: optimize flink rules that c3 prefer to

* feat: optimize flink autoComplete

* test: flink auto complete unit tests
 2023-10-11 17:15:06 +08:00
Hayden
c4030929b2
Feat/auto complete (#175) 
* feat: update hive grammar to adapt to c3

* feat: support viewName, dbName, fnName autoComplete to hive

* test: add hive suggestion unit test

* test: optimze flink suggestion unit tests
 2023-10-10 16:37:49 +08:00
Frank
d1c2920f80
Feat/spark sql auxiliary syntax (#165) 
* feat: supplement SparkSQL add syntax unit test

* chore: recompile spark listener & visitor

---------

Co-authored-by: dilu <dilu@dtstack.com>
 2023-10-09 09:49:22 +08:00
XCynthia
0a9a7d15d3
test: hive complete data types' check and unit tests (#160) 
* feat: add Authorization SQL and update syntax file

* test: hive complete data types' check and unit tests

* feat: hive add syntax complete automaticlly

* feat: update hive's syntax complete

feat:update hive's syntax complete

---------

Co-authored-by: zhaoge <>
 2023-10-08 14:14:03 +08:00
hayden
e97e0b2ef6 feat: recomplie hive sql grammar and fix parser about 2023-07-07 14:48:44 +08:00
Yu Chen
f413a29fac fix: flinkSqlParser should not import src 2023-07-07 14:47:36 +08:00
Hayden
9cf79064de
refactor: optimize basic parser interface (#121) 
* feat: set all internal method to protected

* feat: public createLexer and createParser method

* feat: remove error listeners of parser when get suggestion
 2023-06-16 16:14:53 +08:00
Hayden
e34a9f6128
Fix/basic suggestion (#119) 
* fix: correct suggetion logic in multiple sql case

* test: add multiple sql test case of suggestion

* feat: export SyntaxContextType as enum
 2023-06-12 15:21:27 +08:00
Hayden
1b02ff5d75
feat: FlinkSQL supports auto complete (#115) 
* feat: add antlr4-c3 dependencies

* feat: distinguish table, catalog and database from uid

* feat: move semicolon from sqlStatements to sqlStatement

* chore: move antlr4ts-cli to devDependencies

* feat: improve basic parser and support suggestions of token and syntax

* feat: implement suggest method in sql parsers

* test: flink sql suggestion test cases

* feat: optimize ts defination of suggestion

* feat: add split listener and optimize performance of auto-completion

* test: supplementary flink suggestion unit tests
 2023-06-09 11:22:53 +08:00
Ziv
34f64e6bea
refactor: migrate antlr4 v4.12.0 to antlr4ts(4.9.0) (#106) 
* build: ignore gen folder

* refactor: remove useless code

* fix: correct the Javascript usage in grammar

* refactor: move to antlr4ts

* fix: remove useless

* fix: update grammars for javascript target

* refactor: migrate to antlr4ts

* refactor: migrate to antlr4ts

* refactor: implements ParserErrorListener

* fix: rename the  start reserved word

* refactor: remove unused import

* refactor: migrate to antlr4ts

* test: update the expects of test cases

* refactor: migrate hive to antlr4ts

* refactor: update the incompatible syntax for antlr4ts

* refactor: migrate pgsql grammar to antlr4ts, increasing tests

* refactor: migrate the plsql to antlr4ts

* build: remove unused config

* build: migrate to antlr4ts

* build: migrate ts-jest to @swc/jest

* refactor: migrate to anltr4ts

* build: migrate ts-jest to @swc/jest
 2023-05-30 14:44:03 +08:00
wewoor
8084c7919a fix: rename TrinoSqlParser to TrinoSql 2023-05-25 10:53:03 +08:00
mumiao
9c82a5d248
feat: support trino(presto) sql language (#105) 
* feat(trino): intergrate prestoGrammar to dt-sql-parser

* feat(trino): add trinoSQl test framework

* feat(trino): intergrate test files for trinoSQL

* test(trino): support alterStatement test

* test(trino): support alter table set authorization statement

* feat(trino): complete trinosql alter unit tests

* test(trino): complete dropStatement unit cases for trinosql

* test(trino): complete create statement unit cases for trinoSQL

* test(trino): complete insertStatement unit cases for trinoSQl

* test(trino): dropStatement test files changed to short line split

* test(trino): complete selectStatement unit cases and grammar check

* test(trino): complete commentStatement unit case for trinoSQL

* test(trino): complete analyze commit and call clause unit case

* test(trino): complete delete deny and describe statement unit case

* test(trino): complete explain execute and grant statement unit case

* feat(trino): improve GRANT Role grammar

* test(trino): complete show statement unit case

* test(trino): complete truncateTable startTransaction update and values  statement unit case

* test(trino): improve update statement test cases

* test(trino): complete revoke revoke roles  and rollback statement unit case

* test(trino): add set statement test case

* feat: generator new trino parser and lexer file

* feat(trino): improve alter statement grammar

* test(trino): complete alter statement unit cases

* feat(trino): support case-insensitive lexers

* fix(trino): rm unless gen files

* test(trino): complete merge and reset session statement unit cases

* test(trino): complete merge anduse statement unit cases

* test(trino): complete prepare and refresh materialized view statement unit cases

* test(trino): improve statement unit cases

* test(trino): complete match recognize statement unit cases

* test(trino): complete window with row pattern recognition statement unit cases
 2023-05-24 15:07:02 +08:00
mumiao
dc417e1381
feat: improve flinksql insertStatement 
* feat: feat: improve flinksql insertStatement

* feat: complete insertFromSelectQueries syntax unit tests

* feat: complete insertValuesIntoTable syntax unit tests

* feat: improve insertValuesIntoTable sql case data

* feat: complete insertMultipeTables unit cases

* feat: depracate insertMulStatement grammar for 1.1.4

* fix: depracate unless files

* test: recover jest.config info

* feat:  improve  insertMultipleTable.sql  test case

* feat: build new flinksql parser and lexer

* feat: generator new flink parser and lexer file

* test: improve readSQL logic

* test: fix insertMultipleTable sql error

* test:  recover jest.config.js info
 2023-05-12 09:31:54 +08:00
Ziv
c1c72def30
feat: upgrade antlr4 to 4.12.0 (#88) 2023-05-04 10:13:05 +08:00
Salvatore Ravidà
ea0f061ff9 Added tests - restore target es6 - transpile base into es5 2022-12-16 15:13:56 +08:00
Salvatore Ravidà
f79202a5b7 Rename root to program 2022-12-16 15:13:56 +08:00
Salvatore Ravidà
a15da8e275 Added pgSQL support 2022-12-16 15:13:56 +08:00
Ziv
ca85be9696
build: add the basic type declaration (#30) 2021-05-11 14:20:07 +08:00
xiaowei
dc9974bbff refactor(spark): extract spark export into src index 2021-01-05 17:57:33 +08:00
xiaowei
221482cc7c refactor(plsql): extract plsql export into src index 2021-01-05 17:49:34 +08:00
xiaowei
57aba357ab refactor(hive): extract hive export into src index 2021-01-05 17:45:54 +08:00
xiaowei
ca44962acc refactor: update flinkSQL and genericSQL exporting location 2021-01-05 17:40:34 +08:00
xiaowei
1ac760e01a refactor: rename BasicParser.ts to basicParser.ts 2021-01-05 15:27:43 +08:00
xigua
1c8d020826 modify parser export 2020-12-17 17:43:12 +08:00