Data Structures | Public Types | Public Member Functions | Static Public Attributes | Private Member Functions | Private Attributes | Friends
token Class Reference

A token holds items read from Istream. More...

Collaboration diagram for token:
Collaboration graph
[legend]

Data Structures

class  compound
 Abstract base class for complex tokens. More...
 
class  Compound
 A templated class for holding compound tokens. More...
 

Public Types

enum  tokenType {
  UNDEFINED, PUNCTUATION, WORD, VARIABLE,
  STRING, VERBATIMSTRING, LABEL, FLOAT_SCALAR,
  DOUBLE_SCALAR, COMPOUND, ERROR
}
 Enumeration defining the types of token. More...
 
enum  punctuationToken {
  NULL_TOKEN = '\0', SPACE = ' ', TAB = '\t', NL = '\n',
  END_STATEMENT = ';', BEGIN_LIST = '(', END_LIST = ')', BEGIN_SQR = '[',
  END_SQR = ']', BEGIN_BLOCK = '{', END_BLOCK = '}', COLON = ':',
  COMMA = ',', HASH = '#', BEGIN_STRING = '"', END_STRING = BEGIN_STRING,
  ASSIGN = '=', ADD = '+', SUBTRACT = '-', MULTIPLY = '*',
  DIVIDE = '/'
}
 Standard punctuation tokens. More...
 

Public Member Functions

 token ()
 Construct null. More...
 
 token (const token &)
 Construct as copy. More...
 
 token (punctuationToken, label lineNumber=0)
 Construct punctuation character token. More...
 
 token (const word &, label lineNumber=0)
 Construct word token. More...
 
 token (const string &, label lineNumber=0)
 Construct string token. More...
 
 token (const label, label lineNumber=0)
 Construct label token. More...
 
 token (const floatScalar, label lineNumber=0)
 Construct floatScalar token. More...
 
 token (const doubleScalar, label lineNumber=0)
 Construct doubleScalar token. More...
 
 token (Istream &)
 Construct from Istream. More...
 
 ~token ()
 Destructor. More...
 
tokenType type () const
 
tokenTypetype ()
 
bool good () const
 
bool undefined () const
 
bool error () const
 
bool isPunctuation () const
 
punctuationToken pToken () const
 
bool isWord () const
 
const wordwordToken () const
 
bool isVariable () const
 
bool isString () const
 
const stringstringToken () const
 
bool isLabel () const
 
label labelToken () const
 
bool isFloatScalar () const
 
floatScalar floatScalarToken () const
 
bool isDoubleScalar () const
 
doubleScalar doubleScalarToken () const
 
bool isScalar () const
 
scalar scalarToken () const
 
bool isNumber () const
 
scalar number () const
 
bool isCompound () const
 
const compoundcompoundToken () const
 
compoundtransferCompoundToken (const Istream &is)
 
label lineNumber () const
 
labellineNumber ()
 
void setBad ()
 Set bad. More...
 
InfoProxy< tokeninfo () const
 Return info proxy. More...
 
void operator= (const token &)
 
void operator= (const punctuationToken)
 
void operator= (word *)
 
void operator= (const word &)
 
void operator= (string *)
 
void operator= (const string &)
 
void operator= (const label)
 
void operator= (const floatScalar)
 
void operator= (const doubleScalar)
 
void operator= (compound *)
 
bool operator== (const token &) const
 
bool operator== (const punctuationToken) const
 
bool operator== (const word &) const
 
bool operator== (const string &) const
 
bool operator== (const label) const
 
bool operator== (const floatScalar) const
 
bool operator== (const doubleScalar) const
 
bool operator!= (const token &) const
 
bool operator!= (const punctuationToken) const
 
bool operator!= (const word &) const
 
bool operator!= (const string &) const
 
bool operator!= (const label) const
 
bool operator!= (const floatScalar) const
 
bool operator!= (const doubleScalar) const
 

Static Public Attributes

static token undefinedToken
 Static undefined token. More...
 
static const char *const typeName = "token"
 

Private Member Functions

void clear ()
 Clear any allocated storage (word or string) More...
 
void parseError (const char *expected) const
 

Private Attributes

tokenType type_
 The token type. More...
 
union {
   punctuationToken   punctuationToken_
 
   word *   wordTokenPtr_
 
   string *   stringTokenPtr_
 
   label   labelToken_
 
   floatScalar   floatScalarToken_
 
   doubleScalar   doubleScalarToken_
 
   compound *   compoundTokenPtr_
 
}; 
 Anonymous Union of token types. More...
 
label lineNumber_
 Line number in the file this token was read from. More...
 

Friends

Istreamoperator>> (Istream &, token &)
 
Ostreamoperator<< (Ostream &, const token &)
 
Ostreamoperator<< (Ostream &, const punctuationToken &)
 
ostream & operator<< (ostream &, const punctuationToken &)
 
ostream & operator<< (ostream &, const InfoProxy< token > &)
 

Detailed Description

A token holds items read from Istream.

Source files

Definition at line 67 of file token.H.

Member Enumeration Documentation

◆ tokenType

enum tokenType

Enumeration defining the types of token.

Enumerator
UNDEFINED 
PUNCTUATION 
WORD 
VARIABLE 
STRING 
VERBATIMSTRING 
LABEL 
FLOAT_SCALAR 
DOUBLE_SCALAR 
COMPOUND 
ERROR 

Definition at line 73 of file token.H.

◆ punctuationToken

Standard punctuation tokens.

Enumerator
NULL_TOKEN 
SPACE 
TAB 
NL 
END_STATEMENT 
BEGIN_LIST 
END_LIST 
BEGIN_SQR 
END_SQR 
BEGIN_BLOCK 
END_BLOCK 
COLON 
COMMA 
HASH 
BEGIN_STRING 
END_STRING 
ASSIGN 
ADD 
SUBTRACT 
MULTIPLY 
DIVIDE 

Definition at line 92 of file token.H.

Constructor & Destructor Documentation

◆ token() [1/9]

token ( )
inline

Construct null.

Definition at line 65 of file tokenI.H.

◆ token() [2/9]

token ( const token t)
inline

◆ token() [3/9]

token ( punctuationToken  p,
label  lineNumber = 0 
)
inline

Construct punctuation character token.

Definition at line 119 of file tokenI.H.

◆ token() [4/9]

token ( const word w,
label  lineNumber = 0 
)
inline

Construct word token.

Definition at line 127 of file tokenI.H.

◆ token() [5/9]

token ( const string s,
label  lineNumber = 0 
)
inline

Construct string token.

Definition at line 135 of file tokenI.H.

◆ token() [6/9]

token ( const label  l,
label  lineNumber = 0 
)
inline

Construct label token.

Definition at line 143 of file tokenI.H.

◆ token() [7/9]

token ( const floatScalar  s,
label  lineNumber = 0 
)
inline

Construct floatScalar token.

Definition at line 151 of file tokenI.H.

◆ token() [8/9]

token ( const doubleScalar  s,
label  lineNumber = 0 
)
inline

Construct doubleScalar token.

Definition at line 159 of file tokenI.H.

◆ token() [9/9]

token ( Istream is)

Construct from Istream.

Definition at line 34 of file tokenIO.C.

References Istream::read().

Here is the call graph for this function:

◆ ~token()

~token ( )
inline

Destructor.

Definition at line 170 of file tokenI.H.

References token::clear().

Here is the call graph for this function:

Member Function Documentation

◆ clear()

void clear ( )
inlineprivate

Clear any allocated storage (word or string)

Definition at line 36 of file tokenI.H.

References token::COMPOUND, token::compoundTokenPtr_, refCount::okToDelete(), token::STRING, token::stringTokenPtr_, token::type_, token::UNDEFINED, token::VARIABLE, token::VERBATIMSTRING, token::WORD, and token::wordTokenPtr_.

Referenced by token::operator=(), Foam::operator>>(), token::setBad(), and token::~token().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ parseError()

void parseError ( const char *  expected) const
private

Definition at line 43 of file token.C.

References Foam::endl(), Foam::FatalIOError, and token::info().

Referenced by token::compoundToken(), token::doubleScalarToken(), token::floatScalarToken(), token::labelToken(), token::number(), token::pToken(), token::scalarToken(), token::stringToken(), and token::wordToken().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ type() [1/2]

token::tokenType & type ( ) const
inline

Definition at line 178 of file tokenI.H.

References token::type_.

Referenced by Foam::operator<<(), ISstream::read(), UIPstream::read(), prefixOSstream::write(), OSstream::write(), UOPstream::write(), and primitiveEntry::write().

Here is the caller graph for this function:

◆ type() [2/2]

tokenType& type ( )
inline

◆ good()

bool good ( ) const
inline

Definition at line 188 of file tokenI.H.

References token::ERROR, token::type_, and token::UNDEFINED.

Referenced by entry::getKeyword(), Foam::operator>>(), and primitiveEntry::read().

Here is the caller graph for this function:

◆ undefined()

bool undefined ( ) const
inline

Definition at line 193 of file tokenI.H.

References token::type_, and token::UNDEFINED.

◆ error()

bool error ( ) const
inline

Definition at line 198 of file tokenI.H.

References token::ERROR, and token::type_.

Referenced by dimensionSet::parse().

Here is the caller graph for this function:

◆ isPunctuation()

bool isPunctuation ( ) const
inline

◆ pToken()

token::punctuationToken pToken ( ) const
inline

◆ isWord()

bool isWord ( ) const
inline

◆ wordToken()

const word & wordToken ( ) const
inline

◆ isVariable()

bool isVariable ( ) const
inline

Definition at line 239 of file tokenI.H.

References token::type_, and token::VARIABLE.

Referenced by primitiveEntry::append().

Here is the caller graph for this function:

◆ isString()

bool isString ( ) const
inline

Definition at line 244 of file tokenI.H.

References token::STRING, token::type_, token::VARIABLE, and token::VERBATIMSTRING.

Referenced by entry::getKeyword(), and Foam::operator>>().

Here is the caller graph for this function:

◆ stringToken()

const string & stringToken ( ) const
inline

◆ isLabel()

bool isLabel ( ) const
inline

◆ labelToken()

label labelToken ( ) const
inline

◆ isFloatScalar()

bool isFloatScalar ( ) const
inline

Definition at line 280 of file tokenI.H.

References token::FLOAT_SCALAR, and token::type_.

◆ floatScalarToken()

floatScalar floatScalarToken ( ) const
inline

Definition at line 285 of file tokenI.H.

References token::FLOAT_SCALAR, token::floatScalarToken_, token::parseError(), and token::type_.

Referenced by Foam::operator<<().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ isDoubleScalar()

bool isDoubleScalar ( ) const
inline

Definition at line 299 of file tokenI.H.

References token::DOUBLE_SCALAR, and token::type_.

◆ doubleScalarToken()

doubleScalar doubleScalarToken ( ) const
inline

Definition at line 304 of file tokenI.H.

References token::DOUBLE_SCALAR, token::doubleScalarToken_, token::parseError(), and token::type_.

Referenced by Foam::operator<<().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ isScalar()

bool isScalar ( ) const
inline

Definition at line 318 of file tokenI.H.

References token::DOUBLE_SCALAR, token::FLOAT_SCALAR, and token::type_.

Referenced by token::isNumber(), and token::number().

Here is the caller graph for this function:

◆ scalarToken()

scalar scalarToken ( ) const
inline

Definition at line 323 of file tokenI.H.

References token::DOUBLE_SCALAR, token::doubleScalarToken_, token::FLOAT_SCALAR, token::floatScalarToken_, token::parseError(), and token::type_.

Referenced by token::number().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ isNumber()

bool isNumber ( ) const
inline

Definition at line 340 of file tokenI.H.

References token::isScalar(), token::LABEL, and token::type_.

Referenced by Time::findTimes(), Foam::operator>>(), dimensionSet::parse(), dimensionSet::read(), and Reaction::specieCoeffs::specieCoeffs().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ number()

scalar number ( ) const
inline

◆ isCompound()

bool isCompound ( ) const
inline

Definition at line 362 of file tokenI.H.

References token::COMPOUND, and token::type_.

Referenced by genericFvPatchField< Type >::genericFvPatchField(), and genericPointPatchField< Type >::genericPointPatchField().

Here is the caller graph for this function:

◆ compoundToken()

const token::compound & compoundToken ( ) const
inline

Definition at line 367 of file tokenI.H.

References token::COMPOUND, token::compoundTokenPtr_, token::parseError(), and token::type_.

Referenced by genericFvPatchField< Type >::genericFvPatchField(), genericPointPatchField< Type >::genericPointPatchField(), and Foam::operator<<().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ transferCompoundToken()

Foam::token::compound & transferCompoundToken ( const Istream is)

Definition at line 93 of file token.C.

References Foam::abort(), Foam::FatalIOError, and FatalIOErrorInFunction.

Referenced by genericFvPatchField< Type >::genericFvPatchField(), and genericPointPatchField< Type >::genericPointPatchField().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ lineNumber() [1/2]

label & lineNumber ( ) const
inline

Definition at line 381 of file tokenI.H.

References token::lineNumber_.

Referenced by Foam::operator<<(), ISstream::read(), UIPstream::read(), and ITstream::read().

Here is the caller graph for this function:

◆ lineNumber() [2/2]

label& lineNumber ( )
inline

◆ setBad()

void setBad ( )
inline

Set bad.

Definition at line 392 of file tokenI.H.

References token::clear(), token::ERROR, and token::type_.

Referenced by ISstream::read(), UIPstream::read(), and ISstream::readWordToken().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ info()

InfoProxy<token> info ( ) const
inline

◆ operator=() [1/10]

void operator= ( const token t)
inline

◆ operator=() [2/10]

void operator= ( const punctuationToken  p)
inline

Definition at line 449 of file tokenI.H.

References token::clear(), p, token::PUNCTUATION, token::punctuationToken_, and token::type_.

Here is the call graph for this function:

◆ operator=() [3/10]

void operator= ( word wPtr)
inline

Definition at line 456 of file tokenI.H.

References token::clear(), token::type_, token::WORD, and token::wordTokenPtr_.

Here is the call graph for this function:

◆ operator=() [4/10]

void operator= ( const word w)
inline

Definition at line 463 of file tokenI.H.

References token::operator=(), and w().

Here is the call graph for this function:

◆ operator=() [5/10]

void operator= ( string sPtr)
inline

Definition at line 468 of file tokenI.H.

References token::clear(), token::STRING, token::stringTokenPtr_, and token::type_.

Here is the call graph for this function:

◆ operator=() [6/10]

void operator= ( const string s)
inline

Definition at line 475 of file tokenI.H.

References token::operator=(), and s().

Here is the call graph for this function:

◆ operator=() [7/10]

void operator= ( const label  l)
inline

Definition at line 480 of file tokenI.H.

References token::clear(), token::LABEL, token::labelToken_, and token::type_.

Here is the call graph for this function:

◆ operator=() [8/10]

void operator= ( const floatScalar  s)
inline

Definition at line 487 of file tokenI.H.

References token::clear(), token::FLOAT_SCALAR, token::floatScalarToken_, s(), and token::type_.

Here is the call graph for this function:

◆ operator=() [9/10]

void operator= ( const doubleScalar  s)
inline

Definition at line 494 of file tokenI.H.

References token::clear(), token::DOUBLE_SCALAR, token::doubleScalarToken_, s(), and token::type_.

Here is the call graph for this function:

◆ operator=() [10/10]

void operator= ( token::compound cPtr)
inline

Definition at line 501 of file tokenI.H.

References token::clear(), token::COMPOUND, token::compoundTokenPtr_, and token::type_.

Here is the call graph for this function:

◆ operator==() [1/7]

bool operator== ( const token t) const
inline

◆ operator==() [2/7]

bool operator== ( const punctuationToken  p) const
inline

Definition at line 551 of file tokenI.H.

References p, token::PUNCTUATION, token::punctuationToken_, and token::type_.

◆ operator==() [3/7]

bool operator== ( const word w) const
inline

Definition at line 556 of file tokenI.H.

References token::type_, w(), token::WORD, and token::wordToken().

Here is the call graph for this function:

◆ operator==() [4/7]

bool operator== ( const string s) const
inline

Definition at line 561 of file tokenI.H.

References s(), token::STRING, token::stringToken(), token::type_, token::VARIABLE, and token::VERBATIMSTRING.

Here is the call graph for this function:

◆ operator==() [5/7]

bool operator== ( const label  l) const
inline

Definition at line 570 of file tokenI.H.

References token::LABEL, token::labelToken_, and token::type_.

◆ operator==() [6/7]

bool operator== ( const floatScalar  s) const
inline

Definition at line 575 of file tokenI.H.

References Foam::equal(), token::FLOAT_SCALAR, token::floatScalarToken_, s(), and token::type_.

Here is the call graph for this function:

◆ operator==() [7/7]

bool operator== ( const doubleScalar  s) const
inline

Definition at line 580 of file tokenI.H.

References token::DOUBLE_SCALAR, token::doubleScalarToken_, Foam::equal(), s(), and token::type_.

Here is the call graph for this function:

◆ operator!=() [1/7]

bool operator!= ( const token t) const
inline

Definition at line 585 of file tokenI.H.

References token::operator==().

Here is the call graph for this function:

◆ operator!=() [2/7]

bool operator!= ( const punctuationToken  p) const
inline

Definition at line 590 of file tokenI.H.

References token::operator==(), and p.

Here is the call graph for this function:

◆ operator!=() [3/7]

bool operator!= ( const word w) const
inline

Definition at line 595 of file tokenI.H.

References token::operator==(), and w().

Here is the call graph for this function:

◆ operator!=() [4/7]

bool operator!= ( const string s) const
inline

Definition at line 600 of file tokenI.H.

References token::operator==(), and s().

Here is the call graph for this function:

◆ operator!=() [5/7]

bool operator!= ( const label  l) const
inline

Definition at line 615 of file tokenI.H.

References token::operator==().

Here is the call graph for this function:

◆ operator!=() [6/7]

bool operator!= ( const floatScalar  s) const
inline

Definition at line 605 of file tokenI.H.

References token::operator==(), and s().

Here is the call graph for this function:

◆ operator!=() [7/7]

bool operator!= ( const doubleScalar  s) const
inline

Definition at line 610 of file tokenI.H.

References token::operator==(), and s().

Here is the call graph for this function:

Friends And Related Function Documentation

◆ operator>>

Istream& operator>> ( Istream ,
token  
)
friend

◆ operator<< [1/4]

Ostream& operator<< ( Ostream ,
const token  
)
friend

◆ operator<< [2/4]

Ostream& operator<< ( Ostream ,
const punctuationToken  
)
friend

◆ operator<< [3/4]

ostream& operator<< ( ostream &  ,
const punctuationToken  
)
friend

◆ operator<< [4/4]

ostream& operator<< ( ostream &  ,
const InfoProxy< token > &   
)
friend

Field Documentation

◆ undefinedToken

token undefinedToken
static

Static undefined token.

Definition at line 241 of file token.H.

Referenced by Istream::peekBack(), and ITstream::read().

◆ type_

tokenType type_
private

◆ punctuationToken_

punctuationToken punctuationToken_

◆ wordTokenPtr_

word* wordTokenPtr_

◆ stringTokenPtr_

string* stringTokenPtr_

◆ labelToken_

label labelToken_

◆ floatScalarToken_

floatScalar floatScalarToken_

◆ doubleScalarToken_

doubleScalar doubleScalarToken_

◆ compoundTokenPtr_

compound* compoundTokenPtr_
mutable

◆ @1

union { ... }

Anonymous Union of token types.

◆ lineNumber_

label lineNumber_
private

Line number in the file this token was read from.

Definition at line 264 of file token.H.

Referenced by token::lineNumber(), and token::operator=().

◆ typeName

const char *const typeName = "token"
static

Definition at line 280 of file token.H.


The documentation for this class was generated from the following files: