Class TToken
Unit
PasDoc_Tokenizer
Declaration
type TToken = class(TObject)
Description
Stores the exact type and additional information on one token.
Hierarchy
Overview
Fields
Methods
Properties
Description
Fields
 |
Data: string; |
|
the exact character representation of this token as it was found in the input file
|
|
Info: record |
|
additional information on this token as a variant record depending on the token's MyType
|
|
CommentContent: string; |
|
Contents of a comment token. This is defined only when MyType is in TokenCommentTypes or is TOK_DIRECTIVE. This is the text within the comment without comment delimiters. For TOK_DIRECTIVE you can safely assume that CommentContent[1] = '$'.
|
|
StringContent: string; |
|
Contents of the string token, that is: the value of the string literal. D only when MyType is TOK_STRING.
|
Methods
|
constructor Create(const TT: TTokenType); |
|
Create a token of and assign the argument token type to MyType
|
|
function GetTypeName: string; |
|
|
|
function IsSymbol(const ASymbolType: TSymbolType): Boolean; |
|
Does MyType is TOK_SYMBOL and Info.SymbolType is ASymbolType ?
|
|
function IsKeyWord(const AKeyWord: TKeyWord): Boolean; |
|
Does MyType is TOK_KEYWORD and Info.KeyWord is AKeyWord ?
|
|
function IsStandardDirective( const AStandardDirective: TStandardDirective): Boolean; |
|
Does MyType is TOK_IDENTIFIER and Info.StandardDirective is AStandardDirective ?
|
|
function Description: string; |
|
Few words long description of this token. Describes MyType and Data (for those tokens that tend to have short Data). Starts with lower letter.
|
Properties
|
property StreamName: string read FStreamName; |
|
StreamName is the name of the TStream from which this TToken was read. It is currently used to set TRawDescriptionInfo.StreamName.
|
|
property BeginPosition: Int64 read FBeginPosition; |
|
BeginPosition is the position in the stream of the start of the token. It is currently used to set TRawDescriptionInfo.BeginPosition.
|
|
property EndPosition: Int64 read FEndPosition; |
|
EndPosition is the position in the stream of the character immediately after the end of the token. It is currently used to set TRawDescriptionInfo.EndPosition.
|
Generated by PasDoc 0.15.0.
| 
