grammar::me::cpu::core - ME virtual machine state manipulation

Api

The package directly provides only a single command, and all the functionality is made available through
its methods.

::grammar::me::cpu::coredisasmasm
This method returns a list containing a disassembly of the match instructions in asm. The format
of asm is specified in the section MATCHPROGRAMREPRESENTATION.

Each element of the result contains instruction label, instruction name, and the instruction
arguments, in this order. The label can be the empty string. Jump destinations are shown as
labels, strings and tokens unencoded. Token names are prefixed with their numeric id, if, and only
if a tokmap is defined. The two components are separated by a colon.

::grammar::me::cpu::coreasmasm
This method returns code in the format as specified in section MATCHPROGRAMREPRESENTATION
generated from ME assembly code asm, which is in the format as returned by the method disasm.

::grammar::me::cpu::corenewasm
This method creates state value for a ME virtual machine in its initial state and returns it as
its result.

The argument matchcode contains a Tcl representation of the match instructions the machine has to
execute while parsing the input stream. Its format is specified in the section MATCHPROGRAMREPRESENTATION.

The tokmap argument taken by the implementation provided by the package grammar::me::tcl is here
hidden inside of the match instructions and therefore not needed.

::grammar::me::cpu::corelcstatelocation
This method takes the state value of a ME virtual machine and uses it to convert a location in the
input stream (as offset) into a line number and column index. The result of the method is a
2-element list containing the two pieces in the order mentioned in the previous sentence.

Note that the method cannot convert locations which the machine has not yet read from the input
stream. In other words, if the machine has read 7 characters so far it is possible to convert the
offsets 0 to 6, but nothing beyond that. This also shows that it is not possible to convert
offsets which refer to locations before the beginning of the stream.

This utility allows higher levels to convert the location offsets found in the error status and
the AST into more human readable data.

::grammar::me::cpu::coretokstate ?from ?to??
This method takes the state value of a ME virtual machine and returns a Tcl list containing the
part of the input stream between the locations from and to (both inclusive). If to is not
specified it will default to the value of from. If from is not specified either the whole input
stream is returned.

This method places the same restrictions on its location arguments as the method lc.

::grammar::me::cpu::corepcstate
This method takes the state value of a ME virtual machine and returns the current value of the
stored program counter.

::grammar::me::cpu::coreiseofstate
This method takes the state value of a ME virtual machine and returns the current value of the
stored eof flag.

::grammar::me::cpu::coreatstate
This method takes the state value of a ME virtual machine and returns the current location in the
input stream.

::grammar::me::cpu::coreccstate
This method takes the state value of a ME virtual machine and returns the current token.

::grammar::me::cpu::coresvstate
This method takes the state value of a ME virtual machine and returns the current semantic value
stored in it. This is an abstract syntax tree as specified in the document grammar::me_ast,
section ASTVALUES.

::grammar::me::cpu::coreokstate
This method takes the state value of a ME virtual machine and returns the match status stored in
it.

::grammar::me::cpu::coreerrorstate
This method takes the state value of a ME virtual machine and returns the current error status
stored in it.

::grammar::me::cpu::corelstkstate
This method takes the state value of a ME virtual machine and returns the location stack.

::grammar::me::cpu::coreastkstate
This method takes the state value of a ME virtual machine and returns the AST stack.

::grammar::me::cpu::coremstkstate
This method takes the state value of a ME virtual machine and returns the AST marker stack.

::grammar::me::cpu::coreestkstate
This method takes the state value of a ME virtual machine and returns the error stack.

::grammar::me::cpu::corerstkstate
This method takes the state value of a ME virtual machine and returns the subroutine return stack.

::grammar::me::cpu::corencstate
This method takes the state value of a ME virtual machine and returns the nonterminal match cache
as a dictionary.

::grammar::me::cpu::coreaststate
This method takes the state value of a ME virtual machine and returns the abstract syntax tree
currently at the top of the AST stack stored in it. This is an abstract syntax tree as specified
in the document grammar::me_ast, section ASTVALUES.

::grammar::me::cpu::corehaltedstate
This method takes the state value of a ME virtual machine and returns the current halt status
stored in it, i.e. if the machine has stopped or not.

::grammar::me::cpu::corecodestate
This method takes the state value of a ME virtual machine and returns the code stored in it, i.e.
the instructions executed by the machine.

::grammar::me::cpu::coreeofstatevar
This method takes the state value of a ME virtual machine as stored in the variable named by
statevar and modifies it so that the eof flag inside is set. This signals to the machine that
whatever token are in the input queue are the last to be processed. There will be no more.

::grammar::me::cpu::coreputstatevartoklexlinecol
This method takes the state value of a ME virtual machine as stored in the variable named by
statevar and modifies it so that the token tok is added to the end of the input queue, with
associated lexeme data lex and line/column information.

The operation will fail with an error if the eof flag of the machine has been set through the
method eof.

::grammar::me::cpu::corerunstatevar ?n?
This method takes the state value of a ME virtual machine as stored in the variable named by
statevar, executes a number of instructions and stores the state resulting from their
modifications back into the variable.

The execution loop will run until either

• n instructions have been executed, or

• a halt instruction was executed, or

• the input queue is empty and the code is asking for more tokens to process.

If no limit n was set only the last two conditions are checked for.

MATCHPROGRAMREPRESENTATION
A match program is represented by nested Tcl list. The first element, asm, is a list of integer numbers,
the instructions to execute, and their arguments. The second element, pool, is a list of strings,
referenced by the instructions, for error messages, token names, etc. The third element, tokmap, provides
ordering information for the tokens, mapping their names to their numerical rank. This element can be
empty, forcing lexicographic comparison when matching ranges.

All ME instructions are encoded as integer numbers, with the mapping given below. A number of the
instructions, those which handle error messages, have been given an additional argument to supply that
message explicitly instead of having it constructed from token names, etc. This allows the machine state
to store only the message ids instead of the full strings.

Jump destination arguments are absolute indices into the asm element, refering to the instruction to jump
to. Any string arguments are absolute indices into the pool element. Tokens, characters, messages, and
token (actually character) classes to match are coded as references into the pool as well.

[1] "ict_advancemessage"

[2] "ict_match_tokentokmessage"

[3] "ict_match_tokrangetokbegintokendmessage"

[4] "ict_match_tokclasscodemessage"

[5] "inc_restorebranchlabelnt"

[6] "inc_savent"

[7] "icf_ntcallbranchlabel"

[8] "icf_ntreturn"

[9] "iok_ok"

[10] "iok_fail"

[11] "iok_negate"

[12] "icf_jalwaysbranchlabel"

[13] "icf_jokbranchlabel"

[14] "icf_jfailbranchlabel"

[15] "icf_halt"

[16] "icl_push"

[17] "icl_rewind"

[18] "icl_pop"

[19] "ier_push"

[20] "ier_clear"

[21] "ier_nonterminalmessage"

[22] "ier_merge"

[23] "isv_clear"

[24] "isv_terminal"

[25] "isv_nonterminal_leafnt"

[26] "isv_nonterminal_rangent"

[27] "isv_nonterminal_reducent"

[28] "ias_push"

[29] "ias_mark"

[30] "ias_mrewind"

[31] "ias_mpop"

Bugs, Ideas, Feedback

       This document, and the package it describes, will undoubtedly contain bugs and  other  problems.   Please
       report  such  in  the  category grammar_me of the TcllibTrackers [http://core.tcl.tk/tcllib/reportlist].
       Please also report any ideas for enhancements you may have for either package and/or documentation.

       When proposing code changes, please provide unifieddiffs, i.e the output of diff-u.

       Note further that attachments are strongly preferred over inlined patches. Attachments  can  be  made  by
       going  to the Edit form of the ticket immediately after its creation, and then using the left-most button
       in the secondary navigation bar.

Copyright

       Copyright (c) 2005-2006 Andreas Kupries <andreas_kupries@users.sourceforge.net>

tcllib                                                 0.4                          grammar::me::cpu::core(3tcl)

Cpu State

       A state value is a list containing the following elements, in the order listed below:

       [1]    code: Match instructions, see MATCHPROGRAMREPRESENTATION.

       [2]    pc:   Program counter, int.

       [3]    halt: Halt flag, boolean.

       [4]    eof:  Eof flag, boolean

       [5]    tc:   Terminal cache, and input queue. Structure see below.

       [6]    cl:   Current location, int.

       [7]    ct:   Current token, string.

       [8]    ok:   Match status, boolean.

       [9]    sv:   Semantic value, list.

       [10]   er:   Error status, list.

       [11]   ls:   Location stack, list.

       [12]   as:   AST stack, list.

       [13]   ms:   AST marker stack, list.

       [14]   es:   Error stack, list.

       [15]   rs:   Return stack, list.

       [16]   nc:   Nonterminal cache, dictionary.

       tc, the input queue of tokens waiting for processing and the terminal cache containing the tokens already
       processing  are  one  unified  data  structure  simply holding all tokens and their information, with the
       current location separating that which has been processed from that which is waiting.   Each  element  of
       the queue/cache is a list containing the token, its lexeme information, line number, and column index, in
       this order.

       All  stacks  have  their  top element aat the end, i.e. pushing an item is equivalent to appending to the
       list representing the stack, and popping it removes the last element.

       er, the error status is either empty or a list of two elements, a location in the input, and  a  list  of
       messages, encoded as references into the pool element of the code.

       nc,  the  nonterminal  cache  is  keyed  by nonterminal name and location, each value a four-element list
       containing current location, match status, semantic value, and error status, in this order.

Description

       This  package  provides  an  implementation  of  the ME virtual machine.  Please go and read the document
       grammar::me_intro first if you do not know what a ME virtual machine is.

       This implementation represents each ME virtual machine as a Tcl value and provides commands to manipulate
       and query such values to show the effects of executing instructions,  adding  tokens,  retrieving  state,
       etc.

       The  values  fully  follow  the  paradigm  of  Tcl that every value is a string and while also allowing C
       implementations for a proper Tcl_ObjType to keep all  the  important  data  in  native  data  structures.
       Because  of  the  latter  it  is recommended to access the state values only through the commands of this
       package to ensure that internal representation is not shimmered away.

       The actual structure used by all state values is described in section CPUSTATE.

Keywords

       grammar, parsing, virtual machine

Name

       grammar::me::cpu::core - ME virtual machine state manipulation

Synopsis

       package require Tcl8.59

       package require grammar::me::cpu::core?0.4?::grammar::me::cpu::coredisasmasm::grammar::me::cpu::coreasmasm::grammar::me::cpu::corenewasm::grammar::me::cpu::corelcstatelocation::grammar::me::cpu::coretokstate ?from ?to??

       ::grammar::me::cpu::corepcstate::grammar::me::cpu::coreiseofstate::grammar::me::cpu::coreatstate::grammar::me::cpu::coreccstate::grammar::me::cpu::coresvstate::grammar::me::cpu::coreokstate::grammar::me::cpu::coreerrorstate::grammar::me::cpu::corelstkstate::grammar::me::cpu::coreastkstate::grammar::me::cpu::coremstkstate::grammar::me::cpu::coreestkstate::grammar::me::cpu::corerstkstate::grammar::me::cpu::corencstate::grammar::me::cpu::coreaststate::grammar::me::cpu::corehaltedstate::grammar::me::cpu::corecodestate::grammar::me::cpu::coreeofstatevar::grammar::me::cpu::coreputstatevartoklexlinecol::grammar::me::cpu::corerunstatevar ?n?

________________________________________________________________________________________________________________