| Home | Libraries | People | FAQ | More

##### Lexer

###### Description

The Lexer is the most fundamental concept. A Lexer has a member function, `collect`, that accepts a token definition container `Def`, and a the name of the lexer state the token definitions of the lexer component need to be added to (a string). It doesn't return anything (return type is `void`). Each Lexer can represent a specific pattern or algorithm, or it can be a more complex lexer component formed as a composition of other Lexer's. Additionally, a Lexer exposes a member `add_actions`, that accepts the token definition container `Def`, while returning nothing (again, the returned type is `void`).

Notation

`l`

  A `Lexer`.

`L`

  A `Lexer` type.

`Def`

  A token definition container type.

`State`

  A type used to represent lexer state names.

###### Valid Expressions

In the expressions below, the behavior of the lexer component, `l`, is left unspecified in the base `Lexer` concept. These are specified in subsequent, more refined concepts and by the actual models thereof.

For any Lexer the following expressions must be valid:

Expression | Semantics | Return type
--- | --- | ---
`l.collect(def, state, targetstate)` | Add all token definitions provided by this Lexer instance to the lexer state `state` of the token definition container `def`. After matching this token, the lexer should be switched into the state `targetstate` (optional) | `void`
`l.add_actions(def)` | Add all semantic actions provided by this Lexer instance to the token definition container `def`. | `void`

###### Type Expressions

Expression | Description
--- | ---
`traits::is_lexer<L>::type` | Metafunction that evaluates to `mpl::true_` if a certain type, `L` is a Lexer, `mpl::false_` otherwise (See MPL Boolean Constant).

###### Postcondition

Upon return from `l.collect` the following post conditions should hold:

- On return, `def` holds all token definitions defined in the Lexer, `l`. This includes all Lexer's contained inside `l`.

Upon return from `l.add_actions` the following post conditions should hold:

- On return, `def` holds all semantic actions correctly associated with the corresponding token definitions as defined in the Lexer, `l`. This includes all semantic actions defined by the Lexer's contained inside `l`.

###### Models

All lexer components in Spirit.Lex are models of the Lexer concept.