docs: update docs (#224)
* docs: change authours field * docs: upgrade docs
This commit is contained in:
parent
329f161d2d
commit
7c75164449
@ -1,42 +1,48 @@
|
|||||||
# dt-sql-parser
|
# dt-sql-parser
|
||||||
|
|
||||||
[![NPM version][npm-image]][npm-url]
|
[![NPM version][npm-image]][npm-url] [![NPM downloads][download-img]][download-url] [![Chat][online-chat-img]][online-chat-url]
|
||||||
|
|
||||||
[npm-image]: https://img.shields.io/npm/v/dt-sql-parser.svg?style=flat-square
|
[npm-image]: https://img.shields.io/npm/v/dt-sql-parser.svg?style=flat-square
|
||||||
[npm-url]: https://www.npmjs.com/package/dt-sql-parser
|
[npm-url]: https://www.npmjs.com/package/dt-sql-parser
|
||||||
|
|
||||||
|
[download-img]: https://img.shields.io/npm/dm/dt-sql-parser.svg?style=flat
|
||||||
|
[download-url]: https://www.npmjs.com/package/dt-sql-parser
|
||||||
|
|
||||||
|
[online-chat-img]: https://img.shields.io/discord/920616811261743104?logo=Molecule
|
||||||
|
[online-chat-url]: https://discord.gg/uVvq6mfPfa
|
||||||
|
|
||||||
[English](./README.md) | 简体中文
|
[English](./README.md) | 简体中文
|
||||||
|
|
||||||
dt-sql-parser 是一个基于 [ANTLR4](https://github.com/antlr/antlr4) 开发的, 针对大数据领域的 **SQL Parser** 项目。通过[ANTLR4](https://github.com/antlr/antlr4) 默认生成的 Parser、Visitor 和 Listener 对象,我们可以轻松的做到对 SQL 语句的**语法检查**(Syntax Validation)、**词法分析**(Tokenizer)、 **遍历 AST** 节点等功能。此外,还提供了一些辅助方法, 例如 **SQL 切割(Split)**、**自动补全**等。
|
dt-sql-parser 是一个基于 [ANTLR4](https://github.com/antlr/antlr4) 开发的, 针对大数据领域的 **SQL Parser** 项目。通过[ANTLR4](https://github.com/antlr/antlr4) 生成的 Parser、Visitor 和 Listener,我们可以轻松的做到对 SQL 语句的**语法检查**(Syntax Validation)、**词法分析**(Tokenizer)、 **遍历 AST** 节点等功能。此外,还提供了一些辅助方法, 例如 **SQL 切割(Split)**、**自动补全**等。
|
||||||
|
|
||||||
**已支持的 SQL 类型:**
|
**已支持的 SQL 类型:**
|
||||||
|
|
||||||
- Generic SQL (MySQL)
|
- MySQL
|
||||||
- Flink SQL
|
- Flink SQL
|
||||||
- Spark SQL
|
- Spark SQL
|
||||||
- Hive SQL
|
- Hive SQL
|
||||||
- PL/SQL
|
|
||||||
- PostgreSQL
|
- PostgreSQL
|
||||||
- Trino SQL
|
- Trino SQL
|
||||||
|
- Impala SQL
|
||||||
|
|
||||||
**SQL 辅助方法支持**
|
**SQL 辅助方法支持**
|
||||||
|
|
||||||
| SQL 类型 | SQL 切割 | 自动补全 |
|
| SQL 类型 | SQL 切割 | 自动补全 |
|
||||||
| ----------- | -------- | -------- |
|
| ----------- | -------- | -------- |
|
||||||
| Generic SQL | WIP | WIP |
|
| MySQL | ✅ | ✅ |
|
||||||
| Flink SQL | ✅ | ✅ |
|
| Flink SQL | ✅ | ✅ |
|
||||||
| Spark SQL | ✅ | ✅ |
|
| Spark SQL | ✅ | ✅ |
|
||||||
| Hive SQL | ✅ | ✅ |
|
| Hive SQL | ✅ | ✅ |
|
||||||
| PL/SQL | WIP | WIP |
|
|
||||||
| PostgreSQL | ✅ | ✅ |
|
| PostgreSQL | ✅ | ✅ |
|
||||||
| Trino SQL | ✅ | ✅ |
|
| Trino SQL | ✅ | ✅ |
|
||||||
|
| Impala SQL | ✅ | ✅ |
|
||||||
|
|
||||||
> 提示:当前的 Parser 是 `Javascript` 语言版本,如果有必要,可以尝试编译 Grammar 文件到其他目标语言
|
> 提示:当前的 Parser 是 `Javascript` 语言版本,如果有必要,可以尝试编译 Grammar 文件到其他目标语言。
|
||||||
|
|
||||||
<br/>
|
<br/>
|
||||||
|
|
||||||
## 与 MonacoEditor 集成
|
## 与 MonacoEditor 集成
|
||||||
我们提供了一个[monaco-sql-languages](https://github.com/DTStack/monaco-sql-languages)包,你可以轻易的将`dt-sql-parser`与`monaco-editor`集成。
|
我们提供了[monaco-sql-languages](https://github.com/DTStack/monaco-sql-languages),通过它你可以轻易的将`dt-sql-parser`与`monaco-editor`集成。
|
||||||
|
|
||||||
<br/>
|
<br/>
|
||||||
|
|
||||||
@ -55,28 +61,28 @@ yarn add dt-sql-parser
|
|||||||
## 使用
|
## 使用
|
||||||
在开始使用前,需要先了解基本用法。`dt-sql-parser` 为不同类型的 SQL 分别提供相应的 SQL Parser 类:
|
在开始使用前,需要先了解基本用法。`dt-sql-parser` 为不同类型的 SQL 分别提供相应的 SQL Parser 类:
|
||||||
```javascript
|
```javascript
|
||||||
import { GenericSQL, FlinkSQL, SparkSQL, HiveSQL, PLSQL, PostgresSQL, TrinoSQL } from 'dt-sql-parser';
|
import { MySQL, FlinkSQL, SparkSQL, HiveSQL, PostgresSQL, TrinoSQL, ImpalaSQL } from 'dt-sql-parser';
|
||||||
```
|
```
|
||||||
|
|
||||||
在使用语法校验,自动补全等功能之前,需要先实例化对应 SQL 类型的 Parser,以 `GenericSQL` 为例:
|
在使用语法校验,自动补全等功能之前,需要先实例化对应 SQL 类型的 Parser,以 `MySQL` 为例:
|
||||||
```javascript
|
```javascript
|
||||||
const parser = new GenericSQL();
|
const parser = new MySQL();
|
||||||
```
|
```
|
||||||
|
|
||||||
下文中的使用示例将使用 `GenericSQL`,其他 SQL 类型的 Parser 使用方式与`GenericSQL` 相同。
|
下文中的使用示例将使用 `MySQL`,其他 SQL 类型的 Parser 使用方式与`MySQL` 相同。
|
||||||
|
|
||||||
### 语法校验(Syntax Validation)
|
### 语法校验(Syntax Validation)
|
||||||
```javascript
|
```javascript
|
||||||
import { GenericSQL } from 'dt-sql-parser';
|
import { MySQL } from 'dt-sql-parser';
|
||||||
|
|
||||||
const parser = new GenericSQL();
|
const parser = new MySQL();
|
||||||
|
|
||||||
const correctSql = 'select id,name from user1;';
|
const correctSql = 'select id,name from user1;';
|
||||||
const errors = parser.validate(correctSql);
|
const errors = parser.validate(correctSql);
|
||||||
console.log(errors);
|
console.log(errors);
|
||||||
```
|
```
|
||||||
|
|
||||||
输出:
|
*输出:*
|
||||||
|
|
||||||
```javascript
|
```javascript
|
||||||
/*
|
/*
|
||||||
@ -84,7 +90,7 @@ console.log(errors);
|
|||||||
*/
|
*/
|
||||||
```
|
```
|
||||||
|
|
||||||
校验失败示例:
|
**校验失败示例:**
|
||||||
|
|
||||||
```javascript
|
```javascript
|
||||||
const incorrectSql = 'selec id,name from user1;'
|
const incorrectSql = 'selec id,name from user1;'
|
||||||
@ -92,7 +98,7 @@ const errors = parser.validate(incorrectSql);
|
|||||||
console.log(errors);
|
console.log(errors);
|
||||||
```
|
```
|
||||||
|
|
||||||
输出:
|
*输出:*
|
||||||
|
|
||||||
```javascript
|
```javascript
|
||||||
/*
|
/*
|
||||||
@ -112,15 +118,20 @@ console.log(errors);
|
|||||||
|
|
||||||
### 词法分析(Tokenizer)
|
### 词法分析(Tokenizer)
|
||||||
|
|
||||||
必要场景下,可单独对 SQL 语句进行词法分析,获取所有的 Tokens 对象:
|
部分场景下,可以通过 `getAllTokens` 单独对 SQL 语句进行词法分析,获取所有的 Tokens 对象:
|
||||||
|
|
||||||
```javascript
|
```javascript
|
||||||
import { GenericSQL } from 'dt-sql-parser';
|
import { MySQL } from 'dt-sql-parser';
|
||||||
|
|
||||||
const parser = new GenericSQL()
|
const parser = new MySQL()
|
||||||
const sql = 'select id,name,sex from user1;'
|
const sql = 'select id,name,sex from user1;'
|
||||||
const tokens = parser.getAllTokens(sql)
|
const tokens = parser.getAllTokens(sql)
|
||||||
console.log(tokens)
|
console.log(tokens)
|
||||||
|
```
|
||||||
|
|
||||||
|
*输出:*
|
||||||
|
|
||||||
|
```javascript
|
||||||
/*
|
/*
|
||||||
[
|
[
|
||||||
{
|
{
|
||||||
@ -144,13 +155,13 @@ console.log(tokens)
|
|||||||
使用 Visitor 模式访问 AST 中的指定节点
|
使用 Visitor 模式访问 AST 中的指定节点
|
||||||
|
|
||||||
```javascript
|
```javascript
|
||||||
import { GenericSQL, SqlParserVisitor } from 'dt-sql-parser';
|
import { MySQL, MySqlParserVisitor } from 'dt-sql-parser';
|
||||||
|
|
||||||
const parser = new GenericSQL()
|
const parser = new MySQL()
|
||||||
const sql = `select id,name from user1;`
|
const sql = `select id,name from user1;`
|
||||||
// parseTree
|
// parseTree
|
||||||
const tree = parser.parse(sql)
|
const tree = parser.parse(sql)
|
||||||
class MyVisitor extends SqlParserVisitor {
|
class MyVisitor extends MySqlParserVisitor {
|
||||||
// 重写 visitTableName 方法
|
// 重写 visitTableName 方法
|
||||||
visitTableName(ctx) {
|
visitTableName(ctx) {
|
||||||
let tableName = ctx.getText().toLowerCase()
|
let tableName = ctx.getText().toLowerCase()
|
||||||
@ -164,12 +175,15 @@ class MyVisitor extends SqlParserVisitor {
|
|||||||
}
|
}
|
||||||
const visitor = new MyVisitor()
|
const visitor = new MyVisitor()
|
||||||
visitor.visit(tree)
|
visitor.visit(tree)
|
||||||
|
```
|
||||||
|
|
||||||
|
*输出:*
|
||||||
|
|
||||||
|
```javascript
|
||||||
/*
|
/*
|
||||||
SelectElements id,name
|
SelectElements id,name
|
||||||
TableName user1
|
TableName user1
|
||||||
*/
|
*/
|
||||||
|
|
||||||
```
|
```
|
||||||
|
|
||||||
> 提示:使用 Visitor 模式时,节点的方法名称可以在对应 SQL 目录下的 Visitor 文件中查找
|
> 提示:使用 Visitor 模式时,节点的方法名称可以在对应 SQL 目录下的 Visitor 文件中查找
|
||||||
@ -179,13 +193,13 @@ TableName user1
|
|||||||
Listener 模式,利用 [ANTLR4](https://github.com/antlr/antlr4) 提供的 `ParseTreeWalker` 对象遍历 AST,进入各个节点时调用对应的方法。
|
Listener 模式,利用 [ANTLR4](https://github.com/antlr/antlr4) 提供的 `ParseTreeWalker` 对象遍历 AST,进入各个节点时调用对应的方法。
|
||||||
|
|
||||||
```javascript
|
```javascript
|
||||||
import { GenericSQL, SqlParserListener } from 'dt-sql-parser';
|
import { MySQL, MySqlParserListener } from 'dt-sql-parser';
|
||||||
|
|
||||||
const parser = new GenericSQL();
|
const parser = new MySQL();
|
||||||
const sql = 'select id,name from user1;'
|
const sql = 'select id,name from user1;'
|
||||||
// parseTree
|
// parseTree
|
||||||
const tree = parser.parse(sql)
|
const tree = parser.parse(sql)
|
||||||
class MyListener extends SqlParserListener {
|
class MyListener extends MySqlParserListener {
|
||||||
enterTableName(ctx) {
|
enterTableName(ctx) {
|
||||||
let tableName = ctx.getText().toLowerCase()
|
let tableName = ctx.getText().toLowerCase()
|
||||||
console.log('TableName', tableName)
|
console.log('TableName', tableName)
|
||||||
@ -197,12 +211,15 @@ class MyListener extends SqlParserListener {
|
|||||||
}
|
}
|
||||||
const listenTableName = new MyListener();
|
const listenTableName = new MyListener();
|
||||||
parser.listen(listenTableName, tree);
|
parser.listen(listenTableName, tree);
|
||||||
|
```
|
||||||
|
|
||||||
|
*输出:*
|
||||||
|
|
||||||
|
```javascript
|
||||||
/*
|
/*
|
||||||
SelectElements id,name
|
SelectElements id,name
|
||||||
TableName user1
|
TableName user1
|
||||||
*/
|
*/
|
||||||
|
|
||||||
```
|
```
|
||||||
|
|
||||||
> 提示:使用 Listener 模式时,节点的方法名称可以在对应 SQL 目录下的 Listener 文件中查找
|
> 提示:使用 Listener 模式时,节点的方法名称可以在对应 SQL 目录下的 Listener 文件中查找
|
||||||
@ -215,7 +232,11 @@ const parser = new FlinkSQL();
|
|||||||
const sql = 'SHOW TABLES;\nSELECT * FROM tb;';
|
const sql = 'SHOW TABLES;\nSELECT * FROM tb;';
|
||||||
const sqlSlices = parser.splitSQLByStatement(sql);
|
const sqlSlices = parser.splitSQLByStatement(sql);
|
||||||
console.log(sqlSlices)
|
console.log(sqlSlices)
|
||||||
|
```
|
||||||
|
|
||||||
|
*输出:*
|
||||||
|
|
||||||
|
```javascript
|
||||||
/*
|
/*
|
||||||
[
|
[
|
||||||
{
|
{
|
||||||
@ -238,14 +259,13 @@ console.log(sqlSlices)
|
|||||||
}
|
}
|
||||||
]
|
]
|
||||||
*/
|
*/
|
||||||
|
|
||||||
```
|
```
|
||||||
|
|
||||||
### 自动补全(Code Completion)
|
### 自动补全(Code Completion)
|
||||||
在 sql 的指定位置上获取自动补全信息,以 `FlinkSQL` 为例:
|
在 sql 文本的指定位置上获取自动补全信息,以 `FlinkSQL` 为例:
|
||||||
|
|
||||||
调用 `getSuggestionAtCaretPosition` 方法,传入 sql 内容和需要自动补全的位置的行列号。
|
调用 `getSuggestionAtCaretPosition` 方法,传入 sql 内容和指定位置的行列号。
|
||||||
+ 获取关键字候选项列表
|
+ **获取关键字候选项列表**
|
||||||
|
|
||||||
```javascript
|
```javascript
|
||||||
import { FlinkSQL } from 'dt-sql-parser';
|
import { FlinkSQL } from 'dt-sql-parser';
|
||||||
@ -254,12 +274,15 @@ console.log(sqlSlices)
|
|||||||
const pos = { lineNumber: 1, column: 16 }; // 最后一个位置
|
const pos = { lineNumber: 1, column: 16 }; // 最后一个位置
|
||||||
const keywords = parser.getSuggestionAtCaretPosition(sql, pos)?.keywords;
|
const keywords = parser.getSuggestionAtCaretPosition(sql, pos)?.keywords;
|
||||||
console.log(keywords);
|
console.log(keywords);
|
||||||
|
```
|
||||||
|
*输出:*
|
||||||
|
|
||||||
|
```javascript
|
||||||
/*
|
/*
|
||||||
[ 'CATALOG', 'FUNCTION', 'TEMPORARY', 'VIEW', 'DATABASE', 'TABLE' ]
|
[ 'CATALOG', 'FUNCTION', 'TEMPORARY', 'VIEW', 'DATABASE', 'TABLE' ]
|
||||||
*/
|
*/
|
||||||
```
|
```
|
||||||
+ 获取语法相关自动补全信息
|
+ **获取语法相关自动补全信息**
|
||||||
```javascript
|
```javascript
|
||||||
const parser = new FlinkSQL();
|
const parser = new FlinkSQL();
|
||||||
const sql = 'SELECT * FROM tb';
|
const sql = 'SELECT * FROM tb';
|
||||||
@ -267,6 +290,12 @@ console.log(sqlSlices)
|
|||||||
const syntaxSuggestions = parser.getSuggestionAtCaretPosition(sql, pos)?.syntax;
|
const syntaxSuggestions = parser.getSuggestionAtCaretPosition(sql, pos)?.syntax;
|
||||||
console.log(syntaxSuggestions);
|
console.log(syntaxSuggestions);
|
||||||
|
|
||||||
|
|
||||||
|
```
|
||||||
|
|
||||||
|
*输出:*
|
||||||
|
|
||||||
|
```javascript
|
||||||
/*
|
/*
|
||||||
[
|
[
|
||||||
{
|
{
|
||||||
@ -298,7 +327,7 @@ console.log(sqlSlices)
|
|||||||
]
|
]
|
||||||
*/
|
*/
|
||||||
```
|
```
|
||||||
语法相关自动补全信息返回一个数组,数组中每一项代表该位置可以填写什么语法,比如上例中的输出结果代表该位置可以填写**表名**或者**视图名称**。其中 `syntaxContextType` 是可以补全的语法类型,`wordRanges` 则是已经填写的内容。
|
语法相关自动补全信息返回一个数组,数组中每一项代表该位置可以填写什么语法,比如上例中的输出结果代表该位置可以填写**表名**或者**视图名称**。其中 `syntaxContextType` 是可以补全的语法类型,`wordRanges` 是已经填写的内容。
|
||||||
|
|
||||||
### 其他 API
|
### 其他 API
|
||||||
|
|
||||||
|
85
README.md
85
README.md
@ -2,8 +2,6 @@
|
|||||||
|
|
||||||
[![NPM version][npm-image]][npm-url] [![NPM downloads][download-img]][download-url] [![Chat][online-chat-img]][online-chat-url]
|
[![NPM version][npm-image]][npm-url] [![NPM downloads][download-img]][download-url] [![Chat][online-chat-img]][online-chat-url]
|
||||||
|
|
||||||
English | [简体中文](./README-zh_CN.md)
|
|
||||||
|
|
||||||
[npm-image]: https://img.shields.io/npm/v/dt-sql-parser.svg?style=flat-square
|
[npm-image]: https://img.shields.io/npm/v/dt-sql-parser.svg?style=flat-square
|
||||||
[npm-url]: https://www.npmjs.com/package/dt-sql-parser
|
[npm-url]: https://www.npmjs.com/package/dt-sql-parser
|
||||||
|
|
||||||
@ -13,31 +11,33 @@ English | [简体中文](./README-zh_CN.md)
|
|||||||
[online-chat-img]: https://img.shields.io/discord/920616811261743104?logo=Molecule
|
[online-chat-img]: https://img.shields.io/discord/920616811261743104?logo=Molecule
|
||||||
[online-chat-url]: https://discord.gg/uVvq6mfPfa
|
[online-chat-url]: https://discord.gg/uVvq6mfPfa
|
||||||
|
|
||||||
|
English | [简体中文](./README-zh_CN.md)
|
||||||
|
|
||||||
dt-sql-parser is a **SQL Parser** project built with [ANTLR4](https://github.com/antlr/antlr4), and it's mainly for the **BigData** field. The [ANTLR4](https://github.com/antlr/antlr4) generated the basic Parser, Visitor, and Listener, so it's easy to complete the **syntax validation**, **tokenizer**, **traverse** the AST, and so on features.
|
dt-sql-parser is a **SQL Parser** project built with [ANTLR4](https://github.com/antlr/antlr4), and it's mainly for the **BigData** field. The [ANTLR4](https://github.com/antlr/antlr4) generated the basic Parser, Visitor, and Listener, so it's easy to complete the **syntax validation**, **tokenizer**, **traverse** the AST, and so on features.
|
||||||
|
|
||||||
Additionally, it provides auxiliary functions such as **SQL splitting** and **code completion**.
|
Additionally, it provides auxiliary functions such as **SQL splitting** and **code completion**.
|
||||||
|
|
||||||
**Supported SQL**:
|
**Supported SQL**:
|
||||||
|
|
||||||
- Generic SQL (MySQL)
|
- MySQL
|
||||||
- Flink SQL
|
- Flink SQL
|
||||||
- Spark SQL
|
- Spark SQL
|
||||||
- Hive SQL
|
- Hive SQL
|
||||||
- PL/SQL
|
|
||||||
- PostgreSQL
|
- PostgreSQL
|
||||||
- Trino SQL
|
- Trino SQL
|
||||||
|
- Impala SQL
|
||||||
|
|
||||||
**Supported auxiliary methods**
|
**Supported auxiliary methods**
|
||||||
|
|
||||||
| SQL Type | SQL Spliting | Code Completion |
|
| SQL Type | SQL Spliting | Code Completion |
|
||||||
| ----------- | ------------ | --------------- |
|
| ----------- | ------------ | --------------- |
|
||||||
| Generic SQL | WIP | WIP |
|
| MySQL | ✅ | ✅ |
|
||||||
| Flink SQL | ✅ | ✅ |
|
| Flink SQL | ✅ | ✅ |
|
||||||
| Spark SQL | ✅ | ✅ |
|
| Spark SQL | ✅ | ✅ |
|
||||||
| Hive SQL | ✅ | ✅ |
|
| Hive SQL | ✅ | ✅ |
|
||||||
| PL/SQL | WIP | WIP |
|
|
||||||
| PostgreSQL | ✅ | ✅ |
|
| PostgreSQL | ✅ | ✅ |
|
||||||
| Trino SQL | ✅ | ✅ |
|
| Trino SQL | ✅ | ✅ |
|
||||||
|
| Impala SQL | ✅ | ✅ |
|
||||||
|
|
||||||
>Tips: This project is the default for Javascript language, also you can try to compile it to other languages if you need.
|
>Tips: This project is the default for Javascript language, also you can try to compile it to other languages if you need.
|
||||||
|
|
||||||
@ -45,8 +45,7 @@ Additionally, it provides auxiliary functions such as **SQL splitting** and **co
|
|||||||
|
|
||||||
## Integrating SQL Parser with Monaco Editor
|
## Integrating SQL Parser with Monaco Editor
|
||||||
|
|
||||||
We have provided a [monaco-sql-languages](https://github.com/DTStack/monaco-sql-languages) package, you can integrate with `monaco-editor`
|
We have provided [monaco-sql-languages](https://github.com/DTStack/monaco-sql-languages), it is easily to integrate with `monaco-editor`.
|
||||||
easily.
|
|
||||||
|
|
||||||
<br/>
|
<br/>
|
||||||
|
|
||||||
@ -65,29 +64,29 @@ yarn add dt-sql-parser
|
|||||||
## Usage
|
## Usage
|
||||||
We recommend learning the Fundamentals usage before continuing. The dt-sql-parser library provides SQL Parser classes for different types of SQL.
|
We recommend learning the Fundamentals usage before continuing. The dt-sql-parser library provides SQL Parser classes for different types of SQL.
|
||||||
```javascript
|
```javascript
|
||||||
import { GenericSQL, FlinkSQL, SparkSQL, HiveSQL, PLSQL, PostgresSQL, TrinoSQL } from 'dt-sql-parser';
|
import { MySQL, FlinkSQL, SparkSQL, HiveSQL, PostgresSQL, TrinoSQL, ImpalaSQL } from 'dt-sql-parser';
|
||||||
```
|
```
|
||||||
|
|
||||||
Before using syntax validation, code completion, and other features, it is necessary to instantiate the Parser of the relevant SQL type.
|
Before using syntax validation, code completion, and other features, it is necessary to instantiate the Parser of the relevant SQL type.
|
||||||
For instance, one can consider using `GenericSQL` as an example:
|
For instance, one can consider using `MySQL` as an example:
|
||||||
```javascript
|
```javascript
|
||||||
const parser = new GenericSQL();
|
const parser = new MySQL();
|
||||||
```
|
```
|
||||||
|
|
||||||
The following usage examples will utilize the `GenericSQL`, and the Parser for other SQL types will be used in a similar manner as `GenericSQL`.
|
The following usage examples will utilize the `MySQL`, and the Parser for other SQL types will be used in a similar manner as `MySQL`.
|
||||||
|
|
||||||
### Syntax Validation
|
### Syntax Validation
|
||||||
```javascript
|
```javascript
|
||||||
import GenericSQL from 'dt-sql-parser/dist/parser/generic';
|
import { MySQL } from 'dt-sql-parser';
|
||||||
|
|
||||||
const parser = new GenericSQL();
|
const parser = new MySQL();
|
||||||
|
|
||||||
const correctSql = 'select id,name from user1;';
|
const correctSql = 'select id,name from user1;';
|
||||||
const errors = parser.validate(correctSql);
|
const errors = parser.validate(correctSql);
|
||||||
console.log(errors);
|
console.log(errors);
|
||||||
```
|
```
|
||||||
|
|
||||||
Output:
|
*output:*
|
||||||
|
|
||||||
```javascript
|
```javascript
|
||||||
/*
|
/*
|
||||||
@ -95,15 +94,16 @@ Output:
|
|||||||
*/
|
*/
|
||||||
```
|
```
|
||||||
|
|
||||||
Validate failed:
|
**Validate failed:**
|
||||||
|
|
||||||
```javascript
|
```javascript
|
||||||
const incorrectSql = 'selec id,name from user1;'
|
const incorrectSql = 'selec id,name from user1;'
|
||||||
const errors = parser.validate(incorrectSql);
|
const errors = parser.validate(incorrectSql);
|
||||||
console.log(errors);
|
console.log(errors);
|
||||||
|
|
||||||
```
|
```
|
||||||
|
|
||||||
Output:
|
*output:*
|
||||||
|
|
||||||
```javascript
|
```javascript
|
||||||
/*
|
/*
|
||||||
@ -127,12 +127,17 @@ returns an array object includes **error** message.
|
|||||||
Get all **tokens** by the Parser:
|
Get all **tokens** by the Parser:
|
||||||
|
|
||||||
```javascript
|
```javascript
|
||||||
import GenericSQL from 'dt-sql-parser/dist/parser/generic';
|
import { MySQL } from 'dt-sql-parser';
|
||||||
|
|
||||||
const parser = new GenericSQL()
|
const parser = new MySQL()
|
||||||
const sql = 'select id,name,sex from user1;'
|
const sql = 'select id,name,sex from user1;'
|
||||||
const tokens = parser.getAllTokens(sql)
|
const tokens = parser.getAllTokens(sql)
|
||||||
console.log(tokens)
|
console.log(tokens)
|
||||||
|
```
|
||||||
|
|
||||||
|
*output:*
|
||||||
|
|
||||||
|
```javascript
|
||||||
/*
|
/*
|
||||||
[
|
[
|
||||||
{
|
{
|
||||||
@ -156,14 +161,13 @@ console.log(tokens)
|
|||||||
Traverse the tree node by the Visitor:
|
Traverse the tree node by the Visitor:
|
||||||
|
|
||||||
```javascript
|
```javascript
|
||||||
import GenericSQL from 'dt-sql-parser/dist/parser/generic';
|
import { MySQL, MySqlParserVisitor } from 'dt-sql-parser';
|
||||||
import { SqlParserVisitor } from 'dt-sql-parser/dist/parser/generic/SqlParserVisitor';
|
|
||||||
|
|
||||||
const parser = new GenericSQL()
|
const parser = new MySQL()
|
||||||
const sql = `select id,name from user1;`
|
const sql = `select id,name from user1;`
|
||||||
// parseTree
|
// parseTree
|
||||||
const tree = parser.parse(sql)
|
const tree = parser.parse(sql)
|
||||||
class MyVisitor extends SqlParserVisitor {
|
class MyVisitor extends MySqlParserVisitor {
|
||||||
// overwrite visitTableName
|
// overwrite visitTableName
|
||||||
visitTableName(ctx) {
|
visitTableName(ctx) {
|
||||||
let tableName = ctx.getText().toLowerCase()
|
let tableName = ctx.getText().toLowerCase()
|
||||||
@ -177,12 +181,15 @@ class MyVisitor extends SqlParserVisitor {
|
|||||||
}
|
}
|
||||||
const visitor = new MyVisitor()
|
const visitor = new MyVisitor()
|
||||||
visitor.visit(tree)
|
visitor.visit(tree)
|
||||||
|
```
|
||||||
|
|
||||||
|
*output:*
|
||||||
|
|
||||||
|
```javascript
|
||||||
/*
|
/*
|
||||||
SelectElements id,name
|
SelectElements id,name
|
||||||
TableName user1
|
TableName user1
|
||||||
*/
|
*/
|
||||||
|
|
||||||
```
|
```
|
||||||
|
|
||||||
> Tips: The node's method name can be found in the Visitor file under the corresponding SQL directory
|
> Tips: The node's method name can be found in the Visitor file under the corresponding SQL directory
|
||||||
@ -192,14 +199,13 @@ TableName user1
|
|||||||
Access the specified node in the AST by the Listener
|
Access the specified node in the AST by the Listener
|
||||||
|
|
||||||
```javascript
|
```javascript
|
||||||
import GenericSQL from 'dt-sql-parser/dist/parser/generic';
|
import { MySQL, MySqlParserListener } from 'dt-sql-parser';
|
||||||
import { SqlParserListener } from 'dt-sql-parser/dist/parser/generic/SqlParserListener';
|
|
||||||
|
|
||||||
const parser = new GenericSQL();
|
const parser = new MySQL();
|
||||||
const sql = 'select id,name from user1;'
|
const sql = 'select id,name from user1;'
|
||||||
// parseTree
|
// parseTree
|
||||||
const tree = parser.parse(sql)
|
const tree = parser.parse(sql)
|
||||||
class MyListener extends SqlParserListener {
|
class MyListener extends MySqlParserListener {
|
||||||
enterTableName(ctx) {
|
enterTableName(ctx) {
|
||||||
let tableName = ctx.getText().toLowerCase()
|
let tableName = ctx.getText().toLowerCase()
|
||||||
console.log('TableName', tableName)
|
console.log('TableName', tableName)
|
||||||
@ -212,11 +218,15 @@ class MyListener extends SqlParserListener {
|
|||||||
const listenTableName = new MyListener();
|
const listenTableName = new MyListener();
|
||||||
parser.listen(listenTableName, tree);
|
parser.listen(listenTableName, tree);
|
||||||
|
|
||||||
|
```
|
||||||
|
|
||||||
|
*output:*
|
||||||
|
|
||||||
|
```javascript
|
||||||
/*
|
/*
|
||||||
SelectElements id,name
|
SelectElements id,name
|
||||||
TableName user1
|
TableName user1
|
||||||
*/
|
*/
|
||||||
|
|
||||||
```
|
```
|
||||||
|
|
||||||
> Tips: The node's method name can be found in the Listener file under the corresponding SQL directory
|
> Tips: The node's method name can be found in the Listener file under the corresponding SQL directory
|
||||||
@ -229,7 +239,11 @@ const parser = new FlinkSQL();
|
|||||||
const sql = 'SHOW TABLES;\nSELECT * FROM tb;';
|
const sql = 'SHOW TABLES;\nSELECT * FROM tb;';
|
||||||
const sqlSlices = parser.splitSQLByStatement(sql);
|
const sqlSlices = parser.splitSQLByStatement(sql);
|
||||||
console.log(sqlSlices)
|
console.log(sqlSlices)
|
||||||
|
```
|
||||||
|
|
||||||
|
*output:*
|
||||||
|
|
||||||
|
```javascript
|
||||||
/*
|
/*
|
||||||
[
|
[
|
||||||
{
|
{
|
||||||
@ -252,7 +266,6 @@ console.log(sqlSlices)
|
|||||||
}
|
}
|
||||||
]
|
]
|
||||||
*/
|
*/
|
||||||
|
|
||||||
```
|
```
|
||||||
|
|
||||||
### Code Completion
|
### Code Completion
|
||||||
@ -260,7 +273,7 @@ Obtaining code completion information at a specified position in SQL.
|
|||||||
We can refer to the example of using `FlinkSQL`.
|
We can refer to the example of using `FlinkSQL`.
|
||||||
|
|
||||||
Invoke the `getSuggestionAtCaretPosition` method, pass the SQL content and the row and column numbers indicating the position where code completion is desired.
|
Invoke the `getSuggestionAtCaretPosition` method, pass the SQL content and the row and column numbers indicating the position where code completion is desired.
|
||||||
+ keyword candidates list
|
+ **keyword candidates list**
|
||||||
|
|
||||||
```javascript
|
```javascript
|
||||||
import { FlinkSQL } from 'dt-sql-parser';
|
import { FlinkSQL } from 'dt-sql-parser';
|
||||||
@ -269,19 +282,23 @@ Invoke the `getSuggestionAtCaretPosition` method, pass the SQL content and the r
|
|||||||
const pos = { lineNumber: 1, column: 16 }; // the end position
|
const pos = { lineNumber: 1, column: 16 }; // the end position
|
||||||
const keywords = parser.getSuggestionAtCaretPosition(sql, pos)?.keywords;
|
const keywords = parser.getSuggestionAtCaretPosition(sql, pos)?.keywords;
|
||||||
console.log(keywords);
|
console.log(keywords);
|
||||||
|
```
|
||||||
|
*output:*
|
||||||
|
```javascript
|
||||||
/*
|
/*
|
||||||
[ 'CATALOG', 'FUNCTION', 'TEMPORARY', 'VIEW', 'DATABASE', 'TABLE' ]
|
[ 'CATALOG', 'FUNCTION', 'TEMPORARY', 'VIEW', 'DATABASE', 'TABLE' ]
|
||||||
*/
|
*/
|
||||||
```
|
```
|
||||||
+ Obtaining information related to grammar completion
|
+ **Obtaining information related to grammar completion**
|
||||||
```javascript
|
```javascript
|
||||||
const parser = new FlinkSQL();
|
const parser = new FlinkSQL();
|
||||||
const sql = 'SELECT * FROM tb';
|
const sql = 'SELECT * FROM tb';
|
||||||
const pos = { lineNumber: 1, column: 16 }; // after 'tb'
|
const pos = { lineNumber: 1, column: 16 }; // after 'tb'
|
||||||
const syntaxSuggestions = parser.getSuggestionAtCaretPosition(sql, pos)?.syntax;
|
const syntaxSuggestions = parser.getSuggestionAtCaretPosition(sql, pos)?.syntax;
|
||||||
console.log(syntaxSuggestions);
|
console.log(syntaxSuggestions);
|
||||||
|
```
|
||||||
|
*output:*
|
||||||
|
```javascript
|
||||||
/*
|
/*
|
||||||
[
|
[
|
||||||
{
|
{
|
||||||
|
@ -1,19 +0,0 @@
|
|||||||
# RoadMap
|
|
||||||
|
|
||||||
## Supported SQL Language
|
|
||||||
|
|
||||||
- Generic SQL
|
|
||||||
|
|
||||||
<https://github.com/mysql/mysql-workbench/tree/8.0/library/parsers/grammars>
|
|
||||||
- [x] PLSQL
|
|
||||||
- [x] Hive SQL
|
|
||||||
- [x] Unify parser generate to Antlr4
|
|
||||||
- [ ] Impala SQL
|
|
||||||
- [ ] Spark SQL
|
|
||||||
- [ ] FLink SQL
|
|
||||||
|
|
||||||
## TODO
|
|
||||||
|
|
||||||
- Libra SQL
|
|
||||||
- TiDB
|
|
||||||
MySQL Compatible Syntax
|
|
@ -1,31 +0,0 @@
|
|||||||
# Tutorials
|
|
||||||
|
|
||||||
## Antlr4 installation
|
|
||||||
|
|
||||||
## How to extend new grammar
|
|
||||||
|
|
||||||
## How to expose Javascript interface in this project
|
|
||||||
|
|
||||||
## Integrate with Monaco Editor
|
|
||||||
|
|
||||||
## Release
|
|
||||||
|
|
||||||
npm run script
|
|
||||||
|
|
||||||
```bash
|
|
||||||
npm run release -- --release-as minor
|
|
||||||
```
|
|
||||||
|
|
||||||
Or
|
|
||||||
|
|
||||||
```bash
|
|
||||||
npm run release -- --release-as 1.1.0
|
|
||||||
```
|
|
||||||
|
|
||||||
## Reference
|
|
||||||
|
|
||||||
- <https://tomassetti.me/writing-a-browser-based-editor-using-monaco-and-antlr/>
|
|
||||||
- [SQL](https://en.wikipedia.org/wiki/SQL)
|
|
||||||
- [FlinkSQL](https://github.com/apache/flink/blob/master/flink-table/flink-sql-parser/src/test/java/org/apache/flink/sql/parser/CreateTableLikeTest.java)
|
|
||||||
- [antlr4 grammar](https://github.com/antlr/grammars-v4/tree/master/sql)
|
|
||||||
- <https://github.com/apache/spark/blob/master/sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4>
|
|
@ -1,6 +1,7 @@
|
|||||||
{
|
{
|
||||||
"name": "dt-sql-parser",
|
"name": "dt-sql-parser",
|
||||||
"version": "4.0.0-beta.4.7",
|
"version": "4.0.0-beta.4.7",
|
||||||
|
"authors": "DTStack Corporation",
|
||||||
"description": "SQL Parsers for BigData, built with antlr4",
|
"description": "SQL Parsers for BigData, built with antlr4",
|
||||||
"keywords": [
|
"keywords": [
|
||||||
"sql",
|
"sql",
|
||||||
@ -29,7 +30,6 @@
|
|||||||
"lint-fix": "prettier --write '**/*.ts' --config ./.prettierrc",
|
"lint-fix": "prettier --write '**/*.ts' --config ./.prettierrc",
|
||||||
"cleanComment": "node ./scripts/cleanCommentCli.js"
|
"cleanComment": "node ./scripts/cleanCommentCli.js"
|
||||||
},
|
},
|
||||||
"author": "dt-insight-front",
|
|
||||||
"license": "MIT",
|
"license": "MIT",
|
||||||
"devDependencies": {
|
"devDependencies": {
|
||||||
"@commitlint/cli": "^17.7.2",
|
"@commitlint/cli": "^17.7.2",
|
||||||
|
Loading…
Reference in New Issue
Block a user