token

token — A unit of information.

Synopsis

token ::=

Description

A token identifies a unit of information. Usually, tokens are the result of some processing pass that has performed lexical analysis and divided a data set into the smallest units of information used for subsequent processing.

Exactly what constitutes a token varies by context.

Processing expectations

Formatted inline.

Attributes

Common linking attributes.

any attribute

Any attribute in any other explicit namespace

Children

The following elements occur in token: text, abbrev, acronym, alt, anchor, annotation, biblioref, coref, date, emphasis (db._emphasis), firstterm (db._firstterm), footnote, footnoteref, foreignphrase (db._foreignphrase), glossterm (db._glossterm), indexterm (db.indexterm.endofrange), indexterm (db.indexterm.singular), indexterm (db.indexterm.startofrange), inlinemediaobject, link, olink, phrase (db._phrase), quote (db._quote), remark, replaceable, subscript, superscript, trademark, wordasword, xref.

See Also

Related elements: classname, property, symbol, type.

Examples

1 |<article xmlns='http://docbook.org/ns/docbook'>
  |<title>Example token</title>
  | 
  |<para>In parsing, line ends are turned into the <token>CRLF</token>, all other
5 |whitespace becomes <token>WHITESP</token>.
  |</para>
  | 
  |</article>

In parsing, line ends are turned into the CRLF, all other whitespace becomes WHITESP.