java.lang.Object
- com.alibaba.druid.sql.parser.Lexer

Direct Known Subclasses:

BigQueryLexer, BlinkLexer, CKLexer, DB2Lexer, H2Lexer, HiveLexer, MySqlLexer, OracleLexer, OscarLexer, PGLexer, PhoenixLexer, PrestoLexer, SnowflakeLexer, SQLServerLexer, StarRocksLexer, TDLexer
```
public class Lexer
extends Object
```
Author:

wenshao [szujobs@hotmail.com]

Nested Class Summary

Nested Classes
Modifier and Type Class Description

static interface Lexer.CommentHandler

static class Lexer.SavePoint

Field Summary

Fields
Modifier and Type	Field	Description
`protected char[]`	`buf`
`protected int`	`bufPos`
`protected char`	`ch`
`protected int`	`commentCount`
`protected Lexer.CommentHandler`	`commentHandler`
`protected List<String>`	`comments`
`protected DbType`	`dbType`
`protected DialectFeature`	`dialectFeature`
`protected boolean`	`endOfComment`
`protected int`	`features`
`protected long`	`hash`
`protected long`	`hashLCase`
`protected boolean`	`keepComments`
`protected boolean`	`keepSourceLocation`
`protected Keywords`	`keywords`
`protected int`	`line`
`protected int`	`lines`
`protected int`	`mark`
`protected boolean`	`numberExp`
`protected int`	`numberSale`
`protected boolean`	`optimizedForParameterized`
`protected int`	`pos`
`protected int`	`posColumn`
`protected int`	`posLine`
`protected boolean`	`skipComment`
`protected int`	`startPos`
`protected String`	`stringVal`
`protected static SymbolTable`	`symbols_l2`
`String`	`text`
`protected TimeZone`	`timeZone`
`protected Token`	`token`

Constructor Summary

Constructors
Constructor	Description
`Lexer(char[] input, int inputLength, boolean skipComment)`
`Lexer(String input)`
`Lexer(String input, boolean skipComment)`
`Lexer(String input, DbType dbType)`
`Lexer(String input, Lexer.CommentHandler commentHandler)`
`Lexer(String input, Lexer.CommentHandler commentHandler, DbType dbType)`

Method Summary

All Methods Static Methods Instance Methods Concrete Methods Deprecated Methods
Modifier and Type	Method	Description
`protected void`	`addComment(String comment)`
`String`	`addSymbol()`
`void`	`arraycopy(int srcPos, char[] dest, int destPos, int length)`
`int`	`bp()`
`char`	`charAt(int index)`
`void`	`computeRowAndColumn()`
`void`	`computeRowAndColumn(SQLObject x)`
`void`	`config(SQLParserFeature feature, boolean state)`
`char`	`current()`
`BigDecimal`	`decimalValue()`
`protected boolean`	`dialectFeatureEnabled(DialectFeature.LexerFeature feature)`
`int`	`getCommentCount()`
`Lexer.CommentHandler`	`getCommentHandler()`
`List<String>`	`getComments()`
`DbType`	`getDbType()`
`Keywords`	`getKeywords()`
`int`	`getLine()`
`int`	`getPosColumn()`
`int`	`getPosLine()`
`String`	`getSource()`
`TimeZone`	`getTimeZone()`
`boolean`	`hasComment()`
`long`	`hashLCase()`
`String`	`hexString()`
`boolean`	`identifierEquals(long hash_lower)`
`boolean`	`identifierEquals(String text)`
`String`	`info()`
`protected void`	`initBuff(int size)`
`protected void`	`initDialectFeature()`
`Number`	`integerValue()`
`boolean`	`isAllowComment()`
`boolean`	`isDigit(char ch)`
`protected static boolean`	`isDigit2(char ch)`
`boolean`	`isEnabled(SQLParserFeature feature)`
`boolean`	`isEndOfComment()`
`boolean`	`isEOF()`
`protected boolean`	`isFirstIdentifierChar0(char ch)`
`boolean`	`isKeepComments()`
`boolean`	`isKeepSourceLocation()`
`boolean`	`isNegativeIntegerValue()`
`protected boolean`	`isSafeComment(String comment)`
`protected void`	`lexError(String key, Object... args)`	Report an error at the given position using the provided arguments.
`protected Keywords`	`loadKeywords()`
`Lexer.SavePoint`	`mark()`	Deprecated.
`Lexer.SavePoint`	`markOut()`
`boolean`	`nextIf(Token token)`
`boolean`	`nextIfComma()`
`boolean`	`nextIfIdentifier(long hashCode54)`
`boolean`	`nextIfIdentifier(String identifier)`
`void`	`nextPath()`
`void`	`nextToken()`
`void`	`nextTokenAlias()`
`void`	`nextTokenBy()`
`void`	`nextTokenComma()`
`void`	`nextTokenCommaValue()`
`void`	`nextTokenEq()`
`void`	`nextTokenForSet()`
`void`	`nextTokenIdent()`
`void`	`nextTokenLParen()`
`void`	`nextTokenNotOrNull()`
`protected void`	`nextTokenQues()`
`void`	`nextTokenValue()`
`int`	`nextVarIndex()`
`SQLNumberExpr`	`numberExpr()`
`SQLNumberExpr`	`numberExpr(boolean negate)`
`SQLNumberExpr`	`numberExpr(SQLObject parent)`
`String`	`numberString()`
`static String`	`parameterize(String sql, DbType dbType)`
`int`	`pos()`	Return the current token's position: a 0-based offset from beginning of the raw input stream (before unicode translation)
`protected void`	`putChar(char ch)`	Append a character to sbuf.
`List<String>`	`readAndResetComments()`
`void`	`reset()`	Deprecated.
`void`	`reset(int pos)`
`void`	`reset(int mark, char markChar, Token token)`	Deprecated.
`void`	`reset(Lexer.SavePoint savePoint)`
`protected void`	`scanAlias()`
`protected void`	`scanChar()`
`void`	`scanComment()`
`void`	`scanHexaDecimal()`
`void`	`scanIdentifier()`
`protected void`	`scanIdentifier0()`
`protected void`	`scanLBracket()`
`List<String>`	`scanLineArgument()`
`protected void`	`scanMultiLineComment()`
`void`	`scanNumber()`
`void`	`scanSharp()`
`protected void`	`scanSingleLineComment()`
`SQLType`	`scanSQLType()`
`SQLType`	`scanSQLTypeV2()`
`protected void`	`scanString()`
`protected void`	`scanString2()`
`protected void`	`scanString2_d()`
`void`	`scanVariable()`
`protected void`	`scanVariable_at()`
`void`	`setAllowComment(boolean allowComment)`
`void`	`setCommentHandler(Lexer.CommentHandler commentHandler)`
`void`	`setKeepComments(boolean keepComments)`
`void`	`setTimeZone(TimeZone timeZone)`
`void`	`setToken(Token token)`
`void`	`skipToEOF()`
`boolean`	`skipToNextLine(int startPosition)`
`boolean`	`skipToNextLineOrParameter(int startPosition)`
`String`	`stringVal()`	The value of a literal token, recorded as a string.
`char[]`	`sub_chars(int offset, int count)`
`String`	`subString(int offset)`
`String`	`subString(int offset, int count)`
`Token`	`token()`	Return the current token, set by nextToken().
`protected void`	`unscan()`

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

symbols_l2

protected static SymbolTable symbols_l2

features
```
protected int features
```

timeZone
```
protected TimeZone timeZone
```

text
```
public final String text
```

pos
```
protected int pos
```

mark
```
protected int mark
```

numberSale
```
protected int numberSale
```

numberExp
```
protected boolean numberExp
```

ch
```
protected char ch
```

buf
```
protected char[] buf
```

bufPos
```
protected int bufPos
```

token
```
protected Token token
```

keywords
```
protected Keywords keywords
```

stringVal
```
protected String stringVal
```

hashLCase
```
protected long hashLCase
```

hash
```
protected long hash
```

commentCount
```
protected int commentCount
```

comments
```
protected List<String> comments
```

skipComment
```
protected boolean skipComment
```

commentHandler

protected Lexer.CommentHandler commentHandler

endOfComment
```
protected boolean endOfComment
```

keepComments
```
protected boolean keepComments
```

line
```
protected int line
```

lines
```
protected int lines
```

dbType
```
protected DbType dbType
```

optimizedForParameterized

protected boolean optimizedForParameterized

keepSourceLocation
```
protected boolean keepSourceLocation
```

startPos
```
protected int startPos
```

posLine
```
protected int posLine
```

posColumn
```
protected int posColumn
```

dialectFeature

protected DialectFeature dialectFeature

Constructor Detail

Lexer
```
public Lexer(String input)
```

Lexer

public Lexer(String input,
             DbType dbType)

Lexer

public Lexer(String input,
             Lexer.CommentHandler commentHandler)

Lexer

public Lexer(String input,
             Lexer.CommentHandler commentHandler,
             DbType dbType)

Lexer

public Lexer(String input,
             boolean skipComment)

Lexer

public Lexer(char[] input,
             int inputLength,
             boolean skipComment)

Method Detail

loadKeywords
```
protected Keywords loadKeywords()
```

isKeepSourceLocation
```
public boolean isKeepSourceLocation()
```

isKeepComments
```
public boolean isKeepComments()
```

setKeepComments

public void setKeepComments(boolean keepComments)

getCommentHandler

public Lexer.CommentHandler getCommentHandler()

setCommentHandler

public void setCommentHandler(Lexer.CommentHandler commentHandler)

charAt

public final char charAt(int index)

addSymbol
```
public final String addSymbol()
```

subString

public final String subString(int offset,
                              int count)

subString

public final String subString(int offset)

sub_chars

public final char[] sub_chars(int offset,
                              int count)

initBuff
```
protected void initBuff(int size)
```

arraycopy

public void arraycopy(int srcPos,
                      char[] dest,
                      int destPos,
                      int length)

isAllowComment
```
public boolean isAllowComment()
```

setAllowComment

public void setAllowComment(boolean allowComment)

nextVarIndex
```
public int nextVarIndex()
```

getKeywords
```
public Keywords getKeywords()
```

mark

@Deprecated
public Lexer.SavePoint mark()

Deprecated.

markOut
```
public Lexer.SavePoint markOut()
```

reset

public void reset(Lexer.SavePoint savePoint)

reset
```
@Deprecated
public void reset()
```
Deprecated.

reset
```
public void reset(int pos)
```

initDialectFeature
```
protected void initDialectFeature()
```

scanChar
```
protected final void scanChar()
```

unscan
```
protected void unscan()
```

isEOF
```
public boolean isEOF()
```

lexError

protected void lexError(String key,
                        Object... args)

Report an error at the given position using the provided arguments.

token
```
public final Token token()
```
Return the current token, set by nextToken().

nextIf

public boolean nextIf(Token token)

nextIfComma
```
public final boolean nextIfComma()
```

nextIfIdentifier

public final boolean nextIfIdentifier(String identifier)

nextIfIdentifier

public final boolean nextIfIdentifier(long hashCode54)

setToken
```
public void setToken(Token token)
```

getDbType
```
public final DbType getDbType()
```

info
```
public String info()
```

nextTokenComma
```
public final void nextTokenComma()
```

nextTokenCommaValue

public final void nextTokenCommaValue()

nextTokenEq
```
public final void nextTokenEq()
```

nextTokenLParen
```
public final void nextTokenLParen()
```

nextTokenValue
```
public final void nextTokenValue()
```

nextTokenBy
```
public final void nextTokenBy()
```

nextTokenNotOrNull

public final void nextTokenNotOrNull()

nextTokenIdent
```
public final void nextTokenIdent()
```

scanSQLType
```
public final SQLType scanSQLType()
```

scanSQLTypeV2
```
public final SQLType scanSQLTypeV2()
```

nextTokenAlias
```
public final void nextTokenAlias()
```

nextPath
```
public final void nextPath()
```

nextTokenForSet
```
public final void nextTokenForSet()
```

skipToNextLine

public final boolean skipToNextLine(int startPosition)

skipToNextLineOrParameter

public final boolean skipToNextLineOrParameter(int startPosition)

nextTokenQues
```
protected void nextTokenQues()
```

nextToken
```
public final void nextToken()
```

scanLBracket
```
protected void scanLBracket()
```

scanString
```
protected void scanString()
```

scanString2
```
protected final void scanString2()
```

scanString2_d
```
protected final void scanString2_d()
```

scanAlias
```
protected void scanAlias()
```

scanSharp
```
public void scanSharp()
```

scanVariable
```
public void scanVariable()
```

scanVariable_at
```
protected void scanVariable_at()
```

scanComment
```
public void scanComment()
```

scanLineArgument

public List<String> scanLineArgument()

scanMultiLineComment
```
protected void scanMultiLineComment()
```

scanSingleLineComment

protected void scanSingleLineComment()

scanIdentifier
```
public void scanIdentifier()
```

isFirstIdentifierChar0

protected boolean isFirstIdentifierChar0(char ch)

scanIdentifier0
```
protected void scanIdentifier0()
```

scanNumber
```
public void scanNumber()
```

scanHexaDecimal
```
public void scanHexaDecimal()
```

hexString
```
public String hexString()
```

isDigit

public final boolean isDigit(char ch)

isDigit2

protected static final boolean isDigit2(char ch)

putChar

protected final void putChar(char ch)

Append a character to sbuf.

pos
```
public final int pos()
```
Return the current token's position: a 0-based offset from beginning of the raw input stream (before unicode translation)

stringVal
```
public final String stringVal()
```
The value of a literal token, recorded as a string. For integers, leading 0x and 'l' suffixes are suppressed.

identifierEquals

public final boolean identifierEquals(String text)

identifierEquals

public final boolean identifierEquals(long hash_lower)

hashLCase
```
public final long hashLCase()
```

readAndResetComments

public final List<String> readAndResetComments()

isNegativeIntegerValue

public final boolean isNegativeIntegerValue()

integerValue
```
public final Number integerValue()
```

bp
```
public int bp()
```

current
```
public char current()
```

reset

@Deprecated
public void reset(int mark,
                  char markChar,
                  Token token)

Deprecated.

numberString
```
public final String numberString()
```

decimalValue
```
public BigDecimal decimalValue()
```

numberExpr
```
public SQLNumberExpr numberExpr()
```

numberExpr

public SQLNumberExpr numberExpr(SQLObject parent)

numberExpr

public SQLNumberExpr numberExpr(boolean negate)

hasComment
```
public boolean hasComment()
```

getCommentCount
```
public int getCommentCount()
```

skipToEOF
```
public void skipToEOF()
```

isEndOfComment
```
public boolean isEndOfComment()
```

isSafeComment

protected boolean isSafeComment(String comment)

addComment

protected void addComment(String comment)

getComments

public final List<String> getComments()

getLine
```
public int getLine()
```

computeRowAndColumn

public void computeRowAndColumn(SQLObject x)

computeRowAndColumn
```
public void computeRowAndColumn()
```

getPosLine
```
public int getPosLine()
```

getPosColumn
```
public int getPosColumn()
```

config

public void config(SQLParserFeature feature,
                   boolean state)

getTimeZone
```
public TimeZone getTimeZone()
```

setTimeZone

public void setTimeZone(TimeZone timeZone)

isEnabled

public final boolean isEnabled(SQLParserFeature feature)

parameterize

public static String parameterize(String sql,
                                  DbType dbType)

getSource
```
public String getSource()
```

dialectFeatureEnabled

protected boolean dialectFeatureEnabled(DialectFeature.LexerFeature feature)

Modifier and Type	Class	Description
`static interface`	`Lexer.CommentHandler`
`static class`	`Lexer.SavePoint`

Class Lexer

Nested Class Summary

Field Summary

Constructor Summary

Method Summary

Methods inherited from class java.lang.Object

Field Detail

symbols_l2

features

timeZone

text

pos

mark

numberSale

numberExp

ch

buf

bufPos

token

keywords

stringVal

hashLCase

hash

commentCount

comments

skipComment

commentHandler

endOfComment

keepComments

line

lines

dbType

optimizedForParameterized

keepSourceLocation

startPos

posLine

posColumn

dialectFeature

Constructor Detail

Lexer

Lexer

Lexer

Lexer

Lexer

Lexer

Method Detail

loadKeywords

isKeepSourceLocation

isKeepComments

setKeepComments

getCommentHandler

setCommentHandler

charAt

addSymbol

subString

subString

sub_chars

initBuff

arraycopy

isAllowComment

setAllowComment

nextVarIndex

getKeywords

mark

markOut

reset

reset

reset

initDialectFeature

scanChar

unscan

isEOF

lexError

token

nextIf

nextIfComma

nextIfIdentifier

nextIfIdentifier

setToken

getDbType