docs: update docs (#224)
* docs: change authours field * docs: upgrade docs
This commit is contained in:
Hayden
GitHub
parent
329f161d2d
commit
7c75164449
155
README-zh_CN.md
155
README-zh_CN.md
@ -1,42 +1,48 @@
|
||||
# dt-sql-parser
|
||||
|
||||
[![NPM version][npm-image]][npm-url]
|
||||
[![NPM version][npm-image]][npm-url] [![NPM downloads][download-img]][download-url] [![Chat][online-chat-img]][online-chat-url]
|
||||
|
||||
[npm-image]: https://img.shields.io/npm/v/dt-sql-parser.svg?style=flat-square
|
||||
[npm-url]: https://www.npmjs.com/package/dt-sql-parser
|
||||
|
||||
[download-img]: https://img.shields.io/npm/dm/dt-sql-parser.svg?style=flat
|
||||
[download-url]: https://www.npmjs.com/package/dt-sql-parser
|
||||
|
||||
[online-chat-img]: https://img.shields.io/discord/920616811261743104?logo=Molecule
|
||||
[online-chat-url]: https://discord.gg/uVvq6mfPfa
|
||||
|
||||
[English](./README.md) | 简体中文
|
||||
|
||||
dt-sql-parser 是一个基于 [ANTLR4](https://github.com/antlr/antlr4) 开发的， 针对大数据领域的 **SQL Parser** 项目。通过[ANTLR4](https://github.com/antlr/antlr4) 默认生成的 Parser、Visitor 和 Listener 对象，我们可以轻松的做到对 SQL 语句的**语法检查**（Syntax Validation）、**词法分析**（Tokenizer)、 **遍历 AST** 节点等功能。此外，还提供了一些辅助方法, 例如 **SQL 切割（Split）**、**自动补全**等。
|
||||
dt-sql-parser 是一个基于 [ANTLR4](https://github.com/antlr/antlr4) 开发的， 针对大数据领域的 **SQL Parser** 项目。通过[ANTLR4](https://github.com/antlr/antlr4) 生成的 Parser、Visitor 和 Listener，我们可以轻松的做到对 SQL 语句的**语法检查**（Syntax Validation）、**词法分析**（Tokenizer)、 **遍历 AST** 节点等功能。此外，还提供了一些辅助方法, 例如 **SQL 切割（Split）**、**自动补全**等。
|
||||
|
||||
**已支持的 SQL 类型：**
|
||||
|
||||
- Generic SQL (MySQL)
|
||||
- MySQL
|
||||
- Flink SQL
|
||||
- Spark SQL
|
||||
- Hive SQL
|
||||
- PL/SQL
|
||||
- PostgreSQL
|
||||
- Trino SQL
|
||||
- Impala SQL
|
||||
|
||||
**SQL 辅助方法支持**
|
||||
|
||||
| SQL 类型 | SQL 切割 | 自动补全 |
|
||||
| SQL 类型 | SQL 切割 | 自动补全 |
|
||||
| ----------- | -------- | -------- |
|
||||
| Generic SQL | WIP | WIP |
|
||||
| Flink SQL | ✅ | ✅ |
|
||||
| Spark SQL | ✅ | ✅ |
|
||||
| Hive SQL | ✅ | ✅ |
|
||||
| PL/SQL | WIP | WIP |
|
||||
| PostgreSQL | ✅ | ✅ |
|
||||
| Trino SQL | ✅ | ✅ |
|
||||
| MySQL | ✅ | ✅ |
|
||||
| Flink SQL | ✅ | ✅ |
|
||||
| Spark SQL | ✅ | ✅ |
|
||||
| Hive SQL | ✅ | ✅ |
|
||||
| PostgreSQL | ✅ | ✅ |
|
||||
| Trino SQL | ✅ | ✅ |
|
||||
| Impala SQL | ✅ | ✅ |
|
||||
|
||||
> 提示：当前的 Parser 是 `Javascript` 语言版本，如果有必要，可以尝试编译 Grammar 文件到其他目标语言
|
||||
> 提示：当前的 Parser 是 `Javascript` 语言版本，如果有必要，可以尝试编译 Grammar 文件到其他目标语言。
|
||||
|
||||
<br/>
|
||||
|
||||
## 与 MonacoEditor 集成
|
||||
我们提供了一个[monaco-sql-languages](https://github.com/DTStack/monaco-sql-languages)包，你可以轻易的将`dt-sql-parser`与`monaco-editor`集成。
|
||||
我们提供了[monaco-sql-languages](https://github.com/DTStack/monaco-sql-languages)，通过它你可以轻易的将`dt-sql-parser`与`monaco-editor`集成。
|
||||
|
||||
<br/>
|
||||
|
||||
@ -55,28 +61,28 @@ yarn add dt-sql-parser
|
||||
## 使用
|
||||
在开始使用前，需要先了解基本用法。`dt-sql-parser` 为不同类型的 SQL 分别提供相应的 SQL Parser 类：
|
||||
```javascript
|
||||
import { GenericSQL, FlinkSQL, SparkSQL, HiveSQL, PLSQL, PostgresSQL, TrinoSQL } from 'dt-sql-parser';
|
||||
import { MySQL, FlinkSQL, SparkSQL, HiveSQL, PostgresSQL, TrinoSQL, ImpalaSQL } from 'dt-sql-parser';
|
||||
```
|
||||
|
||||
在使用语法校验，自动补全等功能之前，需要先实例化对应 SQL 类型的 Parser，以 `GenericSQL` 为例：
|
||||
在使用语法校验，自动补全等功能之前，需要先实例化对应 SQL 类型的 Parser，以 `MySQL` 为例：
|
||||
```javascript
|
||||
const parser = new GenericSQL();
|
||||
const parser = new MySQL();
|
||||
```
|
||||
|
||||
下文中的使用示例将使用 `GenericSQL`，其他 SQL 类型的 Parser 使用方式与`GenericSQL` 相同。
|
||||
下文中的使用示例将使用 `MySQL`，其他 SQL 类型的 Parser 使用方式与`MySQL` 相同。
|
||||
|
||||
### 语法校验（Syntax Validation）
|
||||
```javascript
|
||||
import { GenericSQL } from 'dt-sql-parser';
|
||||
import { MySQL } from 'dt-sql-parser';
|
||||
|
||||
const parser = new GenericSQL();
|
||||
const parser = new MySQL();
|
||||
|
||||
const correctSql = 'select id,name from user1;';
|
||||
const errors = parser.validate(correctSql);
|
||||
console.log(errors);
|
||||
```
|
||||
|
||||
输出：
|
||||
*输出：*
|
||||
|
||||
```javascript
|
||||
/*
|
||||
@ -84,7 +90,7 @@ console.log(errors);
|
||||
*/
|
||||
```
|
||||
|
||||
校验失败示例：
|
||||
**校验失败示例：**
|
||||
|
||||
```javascript
|
||||
const incorrectSql = 'selec id,name from user1;'
|
||||
@ -92,18 +98,18 @@ const errors = parser.validate(incorrectSql);
|
||||
console.log(errors);
|
||||
```
|
||||
|
||||
输出：
|
||||
*输出：*
|
||||
|
||||
```javascript
|
||||
/*
|
||||
[
|
||||
{
|
||||
endCol: 5,
|
||||
endLine: 1,
|
||||
startCol: 0,
|
||||
startLine: 1,
|
||||
message: "mismatched input 'SELEC' expecting {<EOF>, 'ALTER', 'ANALYZE', 'CALL', 'CHANGE', 'CHECK', 'CREATE', 'DELETE', 'DESC', 'DESCRIBE', 'DROP', 'EXPLAIN', 'GET', 'GRANT', 'INSERT', 'KILL', 'LOAD', 'LOCK', 'OPTIMIZE', 'PURGE', 'RELEASE', 'RENAME', 'REPLACE', 'RESIGNAL', 'REVOKE', 'SELECT', 'SET', 'SHOW', 'SIGNAL', 'UNLOCK', 'UPDATE', 'USE', 'BEGIN', 'BINLOG', 'CACHE', 'CHECKSUM', 'COMMIT', 'DEALLOCATE', 'DO', 'FLUSH', 'HANDLER', 'HELP', 'INSTALL', 'PREPARE', 'REPAIR', 'RESET', 'ROLLBACK', 'SAVEPOINT', 'START', 'STOP', 'TRUNCATE', 'UNINSTALL', 'XA', 'EXECUTE', 'SHUTDOWN', '--', '(', ';'}"
|
||||
}
|
||||
{
|
||||
endCol: 5,
|
||||
endLine: 1,
|
||||
startCol: 0,
|
||||
startLine: 1,
|
||||
message: "mismatched input 'SELEC' expecting {<EOF>, 'ALTER', 'ANALYZE', 'CALL', 'CHANGE', 'CHECK', 'CREATE', 'DELETE', 'DESC', 'DESCRIBE', 'DROP', 'EXPLAIN', 'GET', 'GRANT', 'INSERT', 'KILL', 'LOAD', 'LOCK', 'OPTIMIZE', 'PURGE', 'RELEASE', 'RENAME', 'REPLACE', 'RESIGNAL', 'REVOKE', 'SELECT', 'SET', 'SHOW', 'SIGNAL', 'UNLOCK', 'UPDATE', 'USE', 'BEGIN', 'BINLOG', 'CACHE', 'CHECKSUM', 'COMMIT', 'DEALLOCATE', 'DO', 'FLUSH', 'HANDLER', 'HELP', 'INSTALL', 'PREPARE', 'REPAIR', 'RESET', 'ROLLBACK', 'SAVEPOINT', 'START', 'STOP', 'TRUNCATE', 'UNINSTALL', 'XA', 'EXECUTE', 'SHUTDOWN', '--', '(', ';'}"
|
||||
}
|
||||
]
|
||||
*/
|
||||
```
|
||||
@ -112,29 +118,34 @@ console.log(errors);
|
||||
|
||||
### 词法分析（Tokenizer）
|
||||
|
||||
必要场景下，可单独对 SQL 语句进行词法分析，获取所有的 Tokens 对象：
|
||||
部分场景下，可以通过 `getAllTokens` 单独对 SQL 语句进行词法分析，获取所有的 Tokens 对象：
|
||||
|
||||
```javascript
|
||||
import { GenericSQL } from 'dt-sql-parser';
|
||||
import { MySQL } from 'dt-sql-parser';
|
||||
|
||||
const parser = new GenericSQL()
|
||||
const parser = new MySQL()
|
||||
const sql = 'select id,name,sex from user1;'
|
||||
const tokens = parser.getAllTokens(sql)
|
||||
console.log(tokens)
|
||||
```
|
||||
|
||||
*输出：*
|
||||
|
||||
```javascript
|
||||
/*
|
||||
[
|
||||
{
|
||||
channel: 0
|
||||
column: 0
|
||||
line: 1
|
||||
source: [SqlLexer, InputStream]
|
||||
start: 0
|
||||
stop: 5
|
||||
tokenIndex: -1
|
||||
type: 137
|
||||
_text: null
|
||||
},
|
||||
...
|
||||
{
|
||||
channel: 0
|
||||
column: 0
|
||||
line: 1
|
||||
source: [SqlLexer, InputStream]
|
||||
start: 0
|
||||
stop: 5
|
||||
tokenIndex: -1
|
||||
type: 137
|
||||
_text: null
|
||||
},
|
||||
...
|
||||
]
|
||||
*/
|
||||
```
|
||||
@ -144,13 +155,13 @@ console.log(tokens)
|
||||
使用 Visitor 模式访问 AST 中的指定节点
|
||||
|
||||
```javascript
|
||||
import { GenericSQL, SqlParserVisitor } from 'dt-sql-parser';
|
||||
import { MySQL, MySqlParserVisitor } from 'dt-sql-parser';
|
||||
|
||||
const parser = new GenericSQL()
|
||||
const parser = new MySQL()
|
||||
const sql = `select id,name from user1;`
|
||||
// parseTree
|
||||
const tree = parser.parse(sql)
|
||||
class MyVisitor extends SqlParserVisitor {
|
||||
class MyVisitor extends MySqlParserVisitor {
|
||||
// 重写 visitTableName 方法
|
||||
visitTableName(ctx) {
|
||||
let tableName = ctx.getText().toLowerCase()
|
||||
@ -164,12 +175,15 @@ class MyVisitor extends SqlParserVisitor {
|
||||
}
|
||||
const visitor = new MyVisitor()
|
||||
visitor.visit(tree)
|
||||
```
|
||||
|
||||
*输出：*
|
||||
|
||||
```javascript
|
||||
/*
|
||||
SelectElements id,name
|
||||
TableName user1
|
||||
*/
|
||||
|
||||
```
|
||||
|
||||
> 提示：使用 Visitor 模式时，节点的方法名称可以在对应 SQL 目录下的 Visitor 文件中查找
|
||||
@ -179,13 +193,13 @@ TableName user1
|
||||
Listener 模式，利用 [ANTLR4](https://github.com/antlr/antlr4) 提供的 `ParseTreeWalker` 对象遍历 AST，进入各个节点时调用对应的方法。
|
||||
|
||||
```javascript
|
||||
import { GenericSQL, SqlParserListener } from 'dt-sql-parser';
|
||||
import { MySQL, MySqlParserListener } from 'dt-sql-parser';
|
||||
|
||||
const parser = new GenericSQL();
|
||||
const parser = new MySQL();
|
||||
const sql = 'select id,name from user1;'
|
||||
// parseTree
|
||||
const tree = parser.parse(sql)
|
||||
class MyListener extends SqlParserListener {
|
||||
class MyListener extends MySqlParserListener {
|
||||
enterTableName(ctx) {
|
||||
let tableName = ctx.getText().toLowerCase()
|
||||
console.log('TableName', tableName)
|
||||
@ -197,12 +211,15 @@ class MyListener extends SqlParserListener {
|
||||
}
|
||||
const listenTableName = new MyListener();
|
||||
parser.listen(listenTableName, tree);
|
||||
```
|
||||
|
||||
*输出：*
|
||||
|
||||
```javascript
|
||||
/*
|
||||
SelectElements id,name
|
||||
TableName user1
|
||||
*/
|
||||
|
||||
```
|
||||
|
||||
> 提示：使用 Listener 模式时，节点的方法名称可以在对应 SQL 目录下的 Listener 文件中查找
|
||||
@ -215,10 +232,14 @@ const parser = new FlinkSQL();
|
||||
const sql = 'SHOW TABLES;\nSELECT * FROM tb;';
|
||||
const sqlSlices = parser.splitSQLByStatement(sql);
|
||||
console.log(sqlSlices)
|
||||
```
|
||||
|
||||
*输出：*
|
||||
|
||||
```javascript
|
||||
/*
|
||||
[
|
||||
{
|
||||
{
|
||||
startIndex: 0,
|
||||
endIndex: 11,
|
||||
startLine: 1,
|
||||
@ -226,8 +247,8 @@ console.log(sqlSlices)
|
||||
startColumn: 1,
|
||||
endColumn: 12,
|
||||
text: 'SHOW TABLES;'
|
||||
},
|
||||
{
|
||||
},
|
||||
{
|
||||
startIndex: 13,
|
||||
endIndex: 29,
|
||||
startLine: 2,
|
||||
@ -235,17 +256,16 @@ console.log(sqlSlices)
|
||||
startColumn: 1,
|
||||
endColumn: 17,
|
||||
text: 'SELECT * FROM tb;'
|
||||
}
|
||||
}
|
||||
]
|
||||
*/
|
||||
|
||||
```
|
||||
|
||||
### 自动补全（Code Completion）
|
||||
在 sql 的指定位置上获取自动补全信息，以 `FlinkSQL` 为例：
|
||||
在 sql 文本的指定位置上获取自动补全信息，以 `FlinkSQL` 为例：
|
||||
|
||||
调用 `getSuggestionAtCaretPosition` 方法，传入 sql 内容和需要自动补全的位置的行列号。
|
||||
+ 获取关键字候选项列表
|
||||
调用 `getSuggestionAtCaretPosition` 方法，传入 sql 内容和指定位置的行列号。
|
||||
+ **获取关键字候选项列表**
|
||||
|
||||
```javascript
|
||||
import { FlinkSQL } from 'dt-sql-parser';
|
||||
@ -254,12 +274,15 @@ console.log(sqlSlices)
|
||||
const pos = { lineNumber: 1, column: 16 }; // 最后一个位置
|
||||
const keywords = parser.getSuggestionAtCaretPosition(sql, pos)?.keywords;
|
||||
console.log(keywords);
|
||||
```
|
||||
*输出：*
|
||||
|
||||
```javascript
|
||||
/*
|
||||
[ 'CATALOG', 'FUNCTION', 'TEMPORARY', 'VIEW', 'DATABASE', 'TABLE' ]
|
||||
*/
|
||||
```
|
||||
+ 获取语法相关自动补全信息
|
||||
+ **获取语法相关自动补全信息**
|
||||
```javascript
|
||||
const parser = new FlinkSQL();
|
||||
const sql = 'SELECT * FROM tb';
|
||||
@ -267,6 +290,12 @@ console.log(sqlSlices)
|
||||
const syntaxSuggestions = parser.getSuggestionAtCaretPosition(sql, pos)?.syntax;
|
||||
console.log(syntaxSuggestions);
|
||||
|
||||
|
||||
```
|
||||
|
||||
*输出：*
|
||||
|
||||
```javascript
|
||||
/*
|
||||
[
|
||||
{
|
||||
@ -298,7 +327,7 @@ console.log(sqlSlices)
|
||||
]
|
||||
*/
|
||||
```
|
||||
语法相关自动补全信息返回一个数组，数组中每一项代表该位置可以填写什么语法，比如上例中的输出结果代表该位置可以填写**表名**或者**视图名称**。其中 `syntaxContextType` 是可以补全的语法类型，`wordRanges` 则是已经填写的内容。
|
||||
语法相关自动补全信息返回一个数组，数组中每一项代表该位置可以填写什么语法，比如上例中的输出结果代表该位置可以填写**表名**或者**视图名称**。其中 `syntaxContextType` 是可以补全的语法类型，`wordRanges` 是已经填写的内容。
|
||||
|
||||
### 其他 API
|
||||
|
||||
|
||||
145
README.md
145
README.md
@ -2,8 +2,6 @@
|
||||
|
||||
[![NPM version][npm-image]][npm-url] [![NPM downloads][download-img]][download-url] [![Chat][online-chat-img]][online-chat-url]
|
||||
|
||||
English | [简体中文](./README-zh_CN.md)
|
||||
|
||||
[npm-image]: https://img.shields.io/npm/v/dt-sql-parser.svg?style=flat-square
|
||||
[npm-url]: https://www.npmjs.com/package/dt-sql-parser
|
||||
|
||||
@ -13,31 +11,33 @@ English | [简体中文](./README-zh_CN.md)
|
||||
[online-chat-img]: https://img.shields.io/discord/920616811261743104?logo=Molecule
|
||||
[online-chat-url]: https://discord.gg/uVvq6mfPfa
|
||||
|
||||
English | [简体中文](./README-zh_CN.md)
|
||||
|
||||
dt-sql-parser is a **SQL Parser** project built with [ANTLR4](https://github.com/antlr/antlr4), and it's mainly for the **BigData** field. The [ANTLR4](https://github.com/antlr/antlr4) generated the basic Parser, Visitor, and Listener, so it's easy to complete the **syntax validation**, **tokenizer**, **traverse** the AST, and so on features.
|
||||
|
||||
Additionally, it provides auxiliary functions such as **SQL splitting** and **code completion**.
|
||||
|
||||
**Supported SQL**:
|
||||
|
||||
- Generic SQL (MySQL)
|
||||
- MySQL
|
||||
- Flink SQL
|
||||
- Spark SQL
|
||||
- Hive SQL
|
||||
- PL/SQL
|
||||
- PostgreSQL
|
||||
- Trino SQL
|
||||
- Impala SQL
|
||||
|
||||
**Supported auxiliary methods**
|
||||
|
||||
| SQL Type | SQL Spliting | Code Completion |
|
||||
| ----------- | ------------ | --------------- |
|
||||
| Generic SQL | WIP | WIP |
|
||||
| Flink SQL | ✅ | ✅ |
|
||||
| Spark SQL | ✅ | ✅ |
|
||||
| Hive SQL | ✅ | ✅ |
|
||||
| PL/SQL | WIP | WIP |
|
||||
| PostgreSQL | ✅ | ✅ |
|
||||
| Trino SQL | ✅ | ✅ |
|
||||
| MySQL | ✅ | ✅ |
|
||||
| Flink SQL | ✅ | ✅ |
|
||||
| Spark SQL | ✅ | ✅ |
|
||||
| Hive SQL | ✅ | ✅ |
|
||||
| PostgreSQL | ✅ | ✅ |
|
||||
| Trino SQL | ✅ | ✅ |
|
||||
| Impala SQL | ✅ | ✅ |
|
||||
|
||||
>Tips: This project is the default for Javascript language, also you can try to compile it to other languages if you need.
|
||||
|
||||
@ -45,8 +45,7 @@ Additionally, it provides auxiliary functions such as **SQL splitting** and **co
|
||||
|
||||
## Integrating SQL Parser with Monaco Editor
|
||||
|
||||
We have provided a [monaco-sql-languages](https://github.com/DTStack/monaco-sql-languages) package, you can integrate with `monaco-editor`
|
||||
easily.
|
||||
We have provided [monaco-sql-languages](https://github.com/DTStack/monaco-sql-languages), it is easily to integrate with `monaco-editor`.
|
||||
|
||||
<br/>
|
||||
|
||||
@ -65,56 +64,57 @@ yarn add dt-sql-parser
|
||||
## Usage
|
||||
We recommend learning the Fundamentals usage before continuing. The dt-sql-parser library provides SQL Parser classes for different types of SQL.
|
||||
```javascript
|
||||
import { GenericSQL, FlinkSQL, SparkSQL, HiveSQL, PLSQL, PostgresSQL, TrinoSQL } from 'dt-sql-parser';
|
||||
import { MySQL, FlinkSQL, SparkSQL, HiveSQL, PostgresSQL, TrinoSQL, ImpalaSQL } from 'dt-sql-parser';
|
||||
```
|
||||
|
||||
Before using syntax validation, code completion, and other features, it is necessary to instantiate the Parser of the relevant SQL type.
|
||||
For instance, one can consider using `GenericSQL` as an example:
|
||||
For instance, one can consider using `MySQL` as an example:
|
||||
```javascript
|
||||
const parser = new GenericSQL();
|
||||
const parser = new MySQL();
|
||||
```
|
||||
|
||||
The following usage examples will utilize the `GenericSQL`, and the Parser for other SQL types will be used in a similar manner as `GenericSQL`.
|
||||
The following usage examples will utilize the `MySQL`, and the Parser for other SQL types will be used in a similar manner as `MySQL`.
|
||||
|
||||
### Syntax Validation
|
||||
```javascript
|
||||
import GenericSQL from 'dt-sql-parser/dist/parser/generic';
|
||||
import { MySQL } from 'dt-sql-parser';
|
||||
|
||||
const parser = new GenericSQL();
|
||||
const parser = new MySQL();
|
||||
|
||||
const correctSql = 'select id,name from user1;';
|
||||
const errors = parser.validate(correctSql);
|
||||
console.log(errors);
|
||||
```
|
||||
|
||||
Output:
|
||||
*output:*
|
||||
|
||||
```javascript
|
||||
/*
|
||||
/*
|
||||
[]
|
||||
*/
|
||||
```
|
||||
|
||||
Validate failed:
|
||||
**Validate failed:**
|
||||
|
||||
```javascript
|
||||
const incorrectSql = 'selec id,name from user1;'
|
||||
const errors = parser.validate(incorrectSql);
|
||||
console.log(errors);
|
||||
|
||||
```
|
||||
|
||||
Output:
|
||||
*output:*
|
||||
|
||||
```javascript
|
||||
/*
|
||||
[
|
||||
{
|
||||
endCol: 5,
|
||||
endLine: 1,
|
||||
startCol: 0,
|
||||
startLine: 1,
|
||||
message: "mismatched input 'SELEC' expecting {<EOF>, 'ALTER', 'ANALYZE', 'CALL', 'CHANGE', 'CHECK', 'CREATE', 'DELETE', 'DESC', 'DESCRIBE', 'DROP', 'EXPLAIN', 'GET', 'GRANT', 'INSERT', 'KILL', 'LOAD', 'LOCK', 'OPTIMIZE', 'PURGE', 'RELEASE', 'RENAME', 'REPLACE', 'RESIGNAL', 'REVOKE', 'SELECT', 'SET', 'SHOW', 'SIGNAL', 'UNLOCK', 'UPDATE', 'USE', 'BEGIN', 'BINLOG', 'CACHE', 'CHECKSUM', 'COMMIT', 'DEALLOCATE', 'DO', 'FLUSH', 'HANDLER', 'HELP', 'INSTALL', 'PREPARE', 'REPAIR', 'RESET', 'ROLLBACK', 'SAVEPOINT', 'START', 'STOP', 'TRUNCATE', 'UNINSTALL', 'XA', 'EXECUTE', 'SHUTDOWN', '--', '(', ';'}"
|
||||
}
|
||||
{
|
||||
endCol: 5,
|
||||
endLine: 1,
|
||||
startCol: 0,
|
||||
startLine: 1,
|
||||
message: "mismatched input 'SELEC' expecting {<EOF>, 'ALTER', 'ANALYZE', 'CALL', 'CHANGE', 'CHECK', 'CREATE', 'DELETE', 'DESC', 'DESCRIBE', 'DROP', 'EXPLAIN', 'GET', 'GRANT', 'INSERT', 'KILL', 'LOAD', 'LOCK', 'OPTIMIZE', 'PURGE', 'RELEASE', 'RENAME', 'REPLACE', 'RESIGNAL', 'REVOKE', 'SELECT', 'SET', 'SHOW', 'SIGNAL', 'UNLOCK', 'UPDATE', 'USE', 'BEGIN', 'BINLOG', 'CACHE', 'CHECKSUM', 'COMMIT', 'DEALLOCATE', 'DO', 'FLUSH', 'HANDLER', 'HELP', 'INSTALL', 'PREPARE', 'REPAIR', 'RESET', 'ROLLBACK', 'SAVEPOINT', 'START', 'STOP', 'TRUNCATE', 'UNINSTALL', 'XA', 'EXECUTE', 'SHUTDOWN', '--', '(', ';'}"
|
||||
}
|
||||
]
|
||||
*/
|
||||
```
|
||||
@ -127,26 +127,31 @@ returns an array object includes **error** message.
|
||||
Get all **tokens** by the Parser:
|
||||
|
||||
```javascript
|
||||
import GenericSQL from 'dt-sql-parser/dist/parser/generic';
|
||||
import { MySQL } from 'dt-sql-parser';
|
||||
|
||||
const parser = new GenericSQL()
|
||||
const parser = new MySQL()
|
||||
const sql = 'select id,name,sex from user1;'
|
||||
const tokens = parser.getAllTokens(sql)
|
||||
console.log(tokens)
|
||||
```
|
||||
|
||||
*output:*
|
||||
|
||||
```javascript
|
||||
/*
|
||||
[
|
||||
{
|
||||
channel: 0
|
||||
column: 0
|
||||
line: 1
|
||||
source: [SqlLexer, InputStream]
|
||||
start: 0
|
||||
stop: 5
|
||||
tokenIndex: -1
|
||||
type: 137
|
||||
_text: null
|
||||
},
|
||||
...
|
||||
{
|
||||
channel: 0
|
||||
column: 0
|
||||
line: 1
|
||||
source: [SqlLexer, InputStream]
|
||||
start: 0
|
||||
stop: 5
|
||||
tokenIndex: -1
|
||||
type: 137
|
||||
_text: null
|
||||
},
|
||||
...
|
||||
]
|
||||
*/
|
||||
```
|
||||
@ -156,14 +161,13 @@ console.log(tokens)
|
||||
Traverse the tree node by the Visitor:
|
||||
|
||||
```javascript
|
||||
import GenericSQL from 'dt-sql-parser/dist/parser/generic';
|
||||
import { SqlParserVisitor } from 'dt-sql-parser/dist/parser/generic/SqlParserVisitor';
|
||||
import { MySQL, MySqlParserVisitor } from 'dt-sql-parser';
|
||||
|
||||
const parser = new GenericSQL()
|
||||
const parser = new MySQL()
|
||||
const sql = `select id,name from user1;`
|
||||
// parseTree
|
||||
const tree = parser.parse(sql)
|
||||
class MyVisitor extends SqlParserVisitor {
|
||||
class MyVisitor extends MySqlParserVisitor {
|
||||
// overwrite visitTableName
|
||||
visitTableName(ctx) {
|
||||
let tableName = ctx.getText().toLowerCase()
|
||||
@ -177,12 +181,15 @@ class MyVisitor extends SqlParserVisitor {
|
||||
}
|
||||
const visitor = new MyVisitor()
|
||||
visitor.visit(tree)
|
||||
```
|
||||
|
||||
*output:*
|
||||
|
||||
```javascript
|
||||
/*
|
||||
SelectElements id,name
|
||||
TableName user1
|
||||
*/
|
||||
|
||||
```
|
||||
|
||||
> Tips: The node's method name can be found in the Visitor file under the corresponding SQL directory
|
||||
@ -192,14 +199,13 @@ TableName user1
|
||||
Access the specified node in the AST by the Listener
|
||||
|
||||
```javascript
|
||||
import GenericSQL from 'dt-sql-parser/dist/parser/generic';
|
||||
import { SqlParserListener } from 'dt-sql-parser/dist/parser/generic/SqlParserListener';
|
||||
import { MySQL, MySqlParserListener } from 'dt-sql-parser';
|
||||
|
||||
const parser = new GenericSQL();
|
||||
const parser = new MySQL();
|
||||
const sql = 'select id,name from user1;'
|
||||
// parseTree
|
||||
const tree = parser.parse(sql)
|
||||
class MyListener extends SqlParserListener {
|
||||
class MyListener extends MySqlParserListener {
|
||||
enterTableName(ctx) {
|
||||
let tableName = ctx.getText().toLowerCase()
|
||||
console.log('TableName', tableName)
|
||||
@ -212,11 +218,15 @@ class MyListener extends SqlParserListener {
|
||||
const listenTableName = new MyListener();
|
||||
parser.listen(listenTableName, tree);
|
||||
|
||||
```
|
||||
|
||||
*output:*
|
||||
|
||||
```javascript
|
||||
/*
|
||||
SelectElements id,name
|
||||
TableName user1
|
||||
*/
|
||||
|
||||
```
|
||||
|
||||
> Tips: The node's method name can be found in the Listener file under the corresponding SQL directory
|
||||
@ -229,10 +239,14 @@ const parser = new FlinkSQL();
|
||||
const sql = 'SHOW TABLES;\nSELECT * FROM tb;';
|
||||
const sqlSlices = parser.splitSQLByStatement(sql);
|
||||
console.log(sqlSlices)
|
||||
```
|
||||
|
||||
*output:*
|
||||
|
||||
```javascript
|
||||
/*
|
||||
[
|
||||
{
|
||||
{
|
||||
startIndex: 0,
|
||||
endIndex: 11,
|
||||
startLine: 1,
|
||||
@ -240,8 +254,8 @@ console.log(sqlSlices)
|
||||
startColumn: 1,
|
||||
endColumn: 12,
|
||||
text: 'SHOW TABLES;'
|
||||
},
|
||||
{
|
||||
},
|
||||
{
|
||||
startIndex: 13,
|
||||
endIndex: 29,
|
||||
startLine: 2,
|
||||
@ -249,10 +263,9 @@ console.log(sqlSlices)
|
||||
startColumn: 1,
|
||||
endColumn: 17,
|
||||
text: 'SELECT * FROM tb;'
|
||||
}
|
||||
}
|
||||
]
|
||||
*/
|
||||
|
||||
```
|
||||
|
||||
### Code Completion
|
||||
@ -260,7 +273,7 @@ Obtaining code completion information at a specified position in SQL.
|
||||
We can refer to the example of using `FlinkSQL`.
|
||||
|
||||
Invoke the `getSuggestionAtCaretPosition` method, pass the SQL content and the row and column numbers indicating the position where code completion is desired.
|
||||
+ keyword candidates list
|
||||
+ **keyword candidates list**
|
||||
|
||||
```javascript
|
||||
import { FlinkSQL } from 'dt-sql-parser';
|
||||
@ -269,19 +282,23 @@ Invoke the `getSuggestionAtCaretPosition` method, pass the SQL content and the r
|
||||
const pos = { lineNumber: 1, column: 16 }; // the end position
|
||||
const keywords = parser.getSuggestionAtCaretPosition(sql, pos)?.keywords;
|
||||
console.log(keywords);
|
||||
|
||||
```
|
||||
*output:*
|
||||
```javascript
|
||||
/*
|
||||
[ 'CATALOG', 'FUNCTION', 'TEMPORARY', 'VIEW', 'DATABASE', 'TABLE' ]
|
||||
*/
|
||||
*/
|
||||
```
|
||||
+ Obtaining information related to grammar completion
|
||||
+ **Obtaining information related to grammar completion**
|
||||
```javascript
|
||||
const parser = new FlinkSQL();
|
||||
const sql = 'SELECT * FROM tb';
|
||||
const pos = { lineNumber: 1, column: 16 }; // after 'tb'
|
||||
const syntaxSuggestions = parser.getSuggestionAtCaretPosition(sql, pos)?.syntax;
|
||||
console.log(syntaxSuggestions);
|
||||
|
||||
```
|
||||
*output:*
|
||||
```javascript
|
||||
/*
|
||||
[
|
||||
{
|
||||
|
||||
@ -1,19 +0,0 @@
|
||||
# RoadMap
|
||||
|
||||
## Supported SQL Language
|
||||
|
||||
- Generic SQL
|
||||
|
||||
<https://github.com/mysql/mysql-workbench/tree/8.0/library/parsers/grammars>
|
||||
- [x] PLSQL
|
||||
- [x] Hive SQL
|
||||
- [x] Unify parser generate to Antlr4
|
||||
- [ ] Impala SQL
|
||||
- [ ] Spark SQL
|
||||
- [ ] FLink SQL
|
||||
|
||||
## TODO
|
||||
|
||||
- Libra SQL
|
||||
- TiDB
|
||||
MySQL Compatible Syntax
|
||||
@ -1,31 +0,0 @@
|
||||
# Tutorials
|
||||
|
||||
## Antlr4 installation
|
||||
|
||||
## How to extend new grammar
|
||||
|
||||
## How to expose Javascript interface in this project
|
||||
|
||||
## Integrate with Monaco Editor
|
||||
|
||||
## Release
|
||||
|
||||
npm run script
|
||||
|
||||
```bash
|
||||
npm run release -- --release-as minor
|
||||
```
|
||||
|
||||
Or
|
||||
|
||||
```bash
|
||||
npm run release -- --release-as 1.1.0
|
||||
```
|
||||
|
||||
## Reference
|
||||
|
||||
- <https://tomassetti.me/writing-a-browser-based-editor-using-monaco-and-antlr/>
|
||||
- [SQL](https://en.wikipedia.org/wiki/SQL)
|
||||
- [FlinkSQL](https://github.com/apache/flink/blob/master/flink-table/flink-sql-parser/src/test/java/org/apache/flink/sql/parser/CreateTableLikeTest.java)
|
||||
- [antlr4 grammar](https://github.com/antlr/grammars-v4/tree/master/sql)
|
||||
- <https://github.com/apache/spark/blob/master/sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4>
|
||||
@ -1,6 +1,7 @@
|
||||
{
|
||||
"name": "dt-sql-parser",
|
||||
"version": "4.0.0-beta.4.7",
|
||||
"authors": "DTStack Corporation",
|
||||
"description": "SQL Parsers for BigData, built with antlr4",
|
||||
"keywords": [
|
||||
"sql",
|
||||
@ -29,7 +30,6 @@
|
||||
"lint-fix": "prettier --write '**/*.ts' --config ./.prettierrc",
|
||||
"cleanComment": "node ./scripts/cleanCommentCli.js"
|
||||
},
|
||||
"author": "dt-insight-front",
|
||||
"license": "MIT",
|
||||
"devDependencies": {
|
||||
"@commitlint/cli": "^17.7.2",
|
||||
|
||||
Loading…
Reference in New Issue
Block a user