pt::pe::op - Parsing Expression Utilities

Api

::pt::pe::opdropdropsetpe
This command removes all occurences of any of the nonterminals symbols in the set dropset from the
parsing expression pe, and simplifies it. This may result in the expression becoming "epsilon",
i.e. matching nothing.

::pt::pe::oprenamentntnewpe
This command renames all occurences of the nonterminal nt in the parsing expression pe into ntnew.

::pt::pe::opcalledpe
This command extracts the set of all nonterminal symbols used, i.e. 'called', in the parsing
expression pe.

::pt::pe::opflattenpe
This command transforms the parsing expression by eliminating sequences nested in sequences, and
choices in choices, lifting the children of the nested expression into the parent. It further
eliminates all sequences and choices with only one child, as these are redundant.

The resulting parsing expression is returned as the result of the command.

::pt::pe::opfusecharspe
This command transforms the parsing expression by fusing adjacent terminals in sequences and
adjacent terminals and ranges in choices, it (re)constructs highlevel strings and characterclasses.

The resulting pseudo-parsing expression is returned as the result of the command and may contain
the pseudo-operators str for character sequences, aka strings, and cl for character choices, aka
character classes.

The result is called a pseudo-parsingexpression because it is not a true parsing expression
anymore, and will fail a check with ::pt::pegverify if the new pseudo-operators are present in
the result, but is otherwise of sound structure for a parsing expression. Notably, the commands
::pt::pegbottomup and ::pt::pegtopdown will process them without trouble.

Bugs, Ideas, Feedback

       This document, and the package it describes, will undoubtedly contain bugs and  other  problems.   Please
       report  such  in  the  category pt of the TcllibTrackers [http://core.tcl.tk/tcllib/reportlist].  Please
       also report any ideas for enhancements you may have for either package and/or documentation.

       When proposing code changes, please provide unifieddiffs, i.e the output of diff-u.

       Note further that attachments are strongly preferred over inlined patches. Attachments  can  be  made  by
       going  to the Edit form of the ticket immediately after its creation, and then using the left-most button
       in the secondary navigation bar.

Copyright

       Copyright (c) 2009 Andreas Kupries <andreas_kupries@users.sourceforge.net>

tcllib                                                1.0.2                                     pt::pe::op(3tcl)

Description

       Are  you lost ?  Do you have trouble understanding this document ?  In that case please read the overview
       provided by the IntroductiontoParserTools. This document is the entrypoint to  the  whole  system  the
       current package is a part of.

       This  package  provides  additional  commands  to  work with the serializations of parsing expressions as
       managed by the PEG and related packages, and specified in section PEserializationformat.

       This is an internal package, for use by the higher level packages handling PEGs,  their  conversion  into
       and out of various other formats, or other uses.

Keywords

       EBNF,  LL(k),  PEG,  TDPL,  context-free  languages,  expression,  grammar,  matching,  parser,   parsing
       expression,  parsing  expression grammar, push down automaton, recursive descent, state, top-down parsing
       languages, transducer

Name

       pt::pe::op - Parsing Expression Utilities

Pe Serialization Format

Here we specify the format used by the Parser Tools to serialize Parsing Expressions as immutable values
for transport, comparison, etc.

We distinguish between regular and canonical serializations. While a parsing expression may have more
than one regular serialization only exactly one of them will be canonical.

Regular serialization

AtomicParsingExpressions

[1] The string epsilon is an atomic parsing expression. It matches the empty string.

[2] The string dot is an atomic parsing expression. It matches any character.

[3] The string alnum is an atomic parsing expression. It matches any Unicode alphabet or
digit character. This is a custom extension of PEs based on Tcl's builtin command
stringis.

[4] The string alpha is an atomic parsing expression. It matches any Unicode alphabet
character. This is a custom extension of PEs based on Tcl's builtin command stringis.

[5] The string ascii is an atomic parsing expression. It matches any Unicode character
below U0080. This is a custom extension of PEs based on Tcl's builtin command stringis.

[6] The string control is an atomic parsing expression. It matches any Unicode control
character. This is a custom extension of PEs based on Tcl's builtin command stringis.

[7] The string digit is an atomic parsing expression. It matches any Unicode digit
character. Note that this includes characters outside of the [0..9] range. This is a
custom extension of PEs based on Tcl's builtin command stringis.

[8] The string graph is an atomic parsing expression. It matches any Unicode printing
character, except for space. This is a custom extension of PEs based on Tcl's
builtin command stringis.

[9] The string lower is an atomic parsing expression. It matches any Unicode lower-case
alphabet character. This is a custom extension of PEs based on Tcl's builtin command
stringis.

[10] The string print is an atomic parsing expression. It matches any Unicode printing
character, including space. This is a custom extension of PEs based on Tcl's builtin
command stringis.

[11] The string punct is an atomic parsing expression. It matches any Unicode punctuation
character. This is a custom extension of PEs based on Tcl's builtin command stringis.

[12] The string space is an atomic parsing expression. It matches any Unicode space
character. This is a custom extension of PEs based on Tcl's builtin command stringis.

[13] The string upper is an atomic parsing expression. It matches any Unicode upper-case
alphabet character. This is a custom extension of PEs based on Tcl's builtin command
stringis.

[14] The string wordchar is an atomic parsing expression. It matches any Unicode word
character. This is any alphanumeric character (see alnum), and any connector
punctuation characters (e.g. underscore). This is a custom extension of PEs based
on Tcl's builtin command stringis.

[15] The string xdigit is an atomic parsing expression. It matches any hexadecimal digit
character. This is a custom extension of PEs based on Tcl's builtin command stringis.

[16] The string ddigit is an atomic parsing expression. It matches any decimal digit
character. This is a custom extension of PEs based on Tcl's builtin command regexp.

[17] The expression [list t x] is an atomic parsing expression. It matches the terminal
string x.

[18] The expression [list n A] is an atomic parsing expression. It matches the
nonterminal A.

CombinedParsingExpressions

[1] For parsing expressions e1, e2, ... the result of [list / e1e2 ... ] is a parsing
expression as well. This is the orderedchoice, aka prioritizedchoice.

[2] For parsing expressions e1, e2, ... the result of [list x e1e2 ... ] is a parsing
expression as well. This is the sequence.

[3] For a parsing expression e the result of [list * e] is a parsing expression as well.
This is the kleeneclosure, describing zero or more repetitions.

[4] For a parsing expression e the result of [list + e] is a parsing expression as well.
This is the positivekleeneclosure, describing one or more repetitions.

[5] For a parsing expression e the result of [list & e] is a parsing expression as well.
This is the andlookaheadpredicate.

[6] For a parsing expression e the result of [list ! e] is a parsing expression as well.
This is the notlookaheadpredicate.

[7] For a parsing expression e the result of [list ? e] is a parsing expression as well.
This is the optionalinput.

Canonical serialization
The canonical serialization of a parsing expression has the format as specified in the previous
item, and then additionally satisfies the constraints below, which make it unique among all the
possible serializations of this parsing expression.

[1] The string representation of the value is the canonical representation of a pure Tcl list.
I.e. it does not contain superfluous whitespace.

[2] Terminals are not encoded as ranges (where start and end of the range are identical).

EXAMPLE
Assuming the parsing expression shown on the right-hand side of the rule

Expression <- Term (AddOp Term)*

then its canonical serialization (except for whitespace) is

{x {n Term} {* {x {n AddOp} {n Term}}}}

Synopsis

       package require Tcl8.59

       package require pt::pe::op?1.0.2?

       package require pt::pe?1?

       package require struct::set::pt::pe::opdropdropsetpe::pt::pe::oprenamentntnewpe::pt::pe::opcalledpe::pt::pe::opflattenpe::pt::pe::opfusecharspe

________________________________________________________________________________________________________________