PasDoc's autodoc

Overview

Units

Class Hierarchy

Classes, Interfaces, Objects and Records

Types

Variables

Constants

Functions and Procedures

Identifiers

Unit dependency graph

Classes hierarchy graph

Class TTokenizer

Description
Hierarchy
Fields
Methods
Properties

Unit

PasDoc_Tokenizer

Declaration

type TTokenizer = class(TObject)

Description

Converts an input TStream to a sequence of TToken objects.

Hierarchy

  • TObject
  • TTokenizer

Overview

Fields

Protected FOnMessage: TPasDocMessageEvent;
Protected FVerbosity: Cardinal;
Protected BufferedChar: Char;
Protected EOS: Boolean;
Protected IsCharBuffered: Boolean;
Protected Row: Integer;
Protected Stream: TStream;
Protected FStreamName: string;
Protected FStreamPath: string;

Methods

Protected procedure DoError(const AMessage: string; const AArguments: array of const; const AExitCode: Word);
Protected procedure DoMessage(const AVerbosity: Cardinal; const MessageType: TPasDocMessageType; const AMessage: string; const AArguments: array of const);
Protected procedure CheckForDirective(const t: TToken);
Protected procedure ConsumeChar;
Protected function CreateSymbolToken(const st: TSymbolType; const s: string): TToken; overload;
Protected function CreateSymbolToken(const st: TSymbolType): TToken; overload;
Protected function GetChar(out c: AnsiChar): Integer;
Protected function PeekChar(out c: Char): Boolean;
Protected function ReadCommentType1: TToken;
Protected function ReadCommentType2: TToken;
Protected function ReadCommentType3: TToken;
Protected function ReadAttAssemblerRegister: TToken;
Protected function ReadLiteralString(var t: TToken): Boolean;
Protected function ReadToken(c: Char; const s: TCharSet; const TT: TTokenType; var t: TToken): Boolean;
Public constructor Create( const AStream: TStream; const OnMessageEvent: TPasDocMessageEvent; const VerbosityLevel: Cardinal; const AStreamName, AStreamPath: string);
Public destructor Destroy; override;
Public function HasData: Boolean;
Public function GetStreamInfo: string;
Public function GetToken: TToken;
Public function SkipUntilCompilerDirective: TToken;

Properties

Public property OnMessage: TPasDocMessageEvent read FOnMessage write FOnMessage;
Public property Verbosity: Cardinal read FVerbosity write FVerbosity;
Public property StreamName: string read FStreamName;
Public property StreamPath: string read FStreamPath;

Description

Fields

Protected FOnMessage: TPasDocMessageEvent;
 
Protected FVerbosity: Cardinal;
 
Protected BufferedChar: Char;

if IsCharBuffered is true, this field contains the buffered character

Protected EOS: Boolean;

true if end of stream Stream has been reached, false otherwise

Protected IsCharBuffered: Boolean;

if this is true, BufferedChar contains a buffered character; the next call to GetChar or PeekChar will return this character, not the next in the associated stream Stream

Protected Row: Integer;

current row in stream Stream; useful when giving error messages

Protected Stream: TStream;

the input stream this tokenizer is working on

Protected FStreamName: string;
 
Protected FStreamPath: string;
 

Methods

Protected procedure DoError(const AMessage: string; const AArguments: array of const; const AExitCode: Word);
 
Protected procedure DoMessage(const AVerbosity: Cardinal; const MessageType: TPasDocMessageType; const AMessage: string; const AArguments: array of const);
 
Protected procedure CheckForDirective(const t: TToken);
 
Protected procedure ConsumeChar;
 
Protected function CreateSymbolToken(const st: TSymbolType; const s: string): TToken; overload;
 
Protected function CreateSymbolToken(const st: TSymbolType): TToken; overload;

Uses default symbol representation, from SymbolNames[st]

Protected function GetChar(out c: AnsiChar): Integer;

Returns 1 on success or 0 on failure

Protected function PeekChar(out c: Char): Boolean;
 
Protected function ReadCommentType1: TToken;
 
Protected function ReadCommentType2: TToken;
 
Protected function ReadCommentType3: TToken;
 
Protected function ReadAttAssemblerRegister: TToken;
 
Protected function ReadLiteralString(var t: TToken): Boolean;
 
Protected function ReadToken(c: Char; const s: TCharSet; const TT: TTokenType; var t: TToken): Boolean;
 
Public constructor Create( const AStream: TStream; const OnMessageEvent: TPasDocMessageEvent; const VerbosityLevel: Cardinal; const AStreamName, AStreamPath: string);

Creates a TTokenizer and associates it with given input TStream. Note that AStream will be freed when this object will be freed.

Public destructor Destroy; override;

Releases all dynamically allocated memory.

Public function HasData: Boolean;
 
Public function GetStreamInfo: string;
 
Public function GetToken: TToken;
 
Public function SkipUntilCompilerDirective: TToken;

Skips all chars until it encounters either $ELSE or $ENDIF compiler defines.

Properties

Public property OnMessage: TPasDocMessageEvent read FOnMessage write FOnMessage;
 
Public property Verbosity: Cardinal read FVerbosity write FVerbosity;
 
Public property StreamName: string read FStreamName;
 
Public property StreamPath: string read FStreamPath;

This is the path where the underlying file of this stream is located.

It may be an absolute path or a relative path. Relative paths are always resolved vs pasdoc current directory. This way user can give relative paths in command-line when writing Pascal source filenames to parse.

In particular, this may be '' to indicate current dir.

It's always specified like it was processed by IncludeTrailingPathDelimiter, so it has trailing PathDelim included (unless it was '', in which case it remains empty).

Generated by PasDoc 0.15.0.