\input texinfo   @c -*-texinfo-*-
@comment The source is gforth.texi.in, from which gforth.texi is generated

@c s/@c <ebook>/@/g this if you want a smartphone/ebook PDF
@c <ebook>iftex
@c <ebook>internalpagesizes{150mm}{97mm}{-0.666in}{-0.95in}{0in}{0in}{160mm}{100mm}
@c <ebook>contentsrightmargin=0pt
@c <ebook>tableindent=0.1in
@c <ebook>itemindent=0.2in
@c <ebook>tolerance=700
@c <ebook>defbodyindent=0.1in
@c <ebook>lispnarrowing=0pt
@c <ebook>end iftex

@comment TODO: nac29jan99 - a list of things to add in the next edit:
@comment 1. x-ref all ambiguous or implementation-defined features?
@comment 2. Describe the use of Auser Avariable AConstant A, etc.
@comment 3. words in miscellaneous section need a home.
@comment 4. search for TODO for other minor and major works required.
@comment 5. [rats] change all @var to @i in Forth source so that info
@comment    file looks decent.
@c          Not an improvement IMO - anton
@c          and anyway, this should be taken up
@c          with Karl Berry (the texinfo guy) - anton
@c
@c Karl Berry writes:
@c  If they don't like the all-caps for @var Info output, all I can say is
@c  that it's always been that way, and the usage of all-caps for
@c  metavariables has a long tradition.  I think it's best to just let it be
@c  what it is, for the sake of consistency among manuals.
@c
@comment .. would be useful to have a word that identified all deferred words
@comment should semantics stuff in intro be moved to another section

@c POSTPONE, COMPILE, [COMPILE], LITERAL should have their own section

@comment %**start of header (This is for running Texinfo on a region.)
@setfilename gforth.info
@include version.texi
@settitle Gforth Manual
@c @syncodeindex pg cp

@c Don't use UTF-8; it will produce a large number of non-ASCII chars.
@c Instead, recode afterwards if necessary.
@documentencoding UTF-8


@macro progstyle {}
Programming style note:

@end macro

@macro assignment {}
@table @i
@item Assignment:
@end macro
@macro endassignment {}
@end table
@end macro

@comment macros for beautifying glossary entries
@macro GLOSS-START {}
@iftex
@ninerm
@end iftex
@end macro

@macro GLOSS-END {}
@iftex
@rm
@end iftex
@end macro

@comment %**end of header (This is for running Texinfo on a region.)
@copying
This manual is for Gforth (version @value{VERSION}, @value{UPDATED}),
a fast and portable implementation of the Standard Forth language.  It
serves as reference manual, but it also contains an introduction to
Forth and a Forth tutorial.

Authors: Bernd Paysan, Anton Ertl, Gerald Wodni
Copyright @copyright{} 1995, 1996, 1997, 1998, 2000, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014,2015,2016,2017,2018,2019,2020,2021,2022,2023 Free Software Foundation, Inc.

@quotation
Permission is granted to copy, distribute and/or modify this document
under the terms of the GNU Free Documentation License, Version 1.1 or
any later version published by the Free Software Foundation; with no
Invariant Sections, with the Front-Cover texts being ``A GNU Manual,''
and with the Back-Cover Texts as in (a) below.  A copy of the
license is included in the section entitled ``GNU Free Documentation
License.''

(a) The FSF's Back-Cover Text is: ``You have freedom to copy and modify
this GNU Manual, like GNU software.  Copies published by the Free
Software Foundation raise funds for GNU development.''
@end quotation
@end copying

@dircategory Software development
@direntry
* Gforth: (gforth).             A fast interpreter for the Forth language.
@end direntry
@c The Texinfo manual also recommends doing this, but for Gforth it may
@c  not make much sense
@c @dircategory Individual utilities
@c @direntry
@c * Gforth: (gforth)Invoking Gforth.      gforth, gforth-fast, gforthmi
@c @end direntry

@titlepage
@title Gforth
@subtitle for version @value{VERSION}, @value{UPDATED}
@author Neal Crook
@author Anton Ertl
@author David Kuehling
@author Bernd Paysan
@author Jens Wilke
@author Gerald Wodni
@page
@vskip 0pt plus 1filll
@insertcopying
@end titlepage

@contents

@ifnottex
@node Top, Goals, (dir), (dir)
@top Gforth

@insertcopying

@menu
* Goals::                       About the Gforth Project
* Gforth Environment::          Starting (and exiting) Gforth
* Tutorial::                    Hands-on Forth Tutorial
* Introduction::                An introduction to Standard Forth
* Literals in source code::
* Words::                       Forth words available in Gforth
* Error messages::              How to interpret them
* Tools::                       Programming tools
* Standard conformance::        Implementation-defined options etc.
* Standard vs Extensions::      Should I use extensions?
* Model::                       The abstract machine of Gforth
* Integrating Gforth::          Forth as scripting language for applications
* Emacs and Gforth::            The Gforth Mode
* Image Files::                 @code{.fi} files contain compiled code
* Engine::                      The inner interpreter and the primitives
* Cross Compiler::              The Cross Compiler
* MINOS2::                      The GUI library
* Bugs::                        How to report them
* Origin::                      Authors and ancestors of Gforth
* Forth-related information::   Books and places to look on the WWW
* Licenses::
* Word Index::                  An item for each Forth word
* Concept Index::               A menu covering many topics

@detailmenu
 --- The Detailed Node Listing ---

Goals of Gforth

* Stability Goals::

Gforth Environment

* Invoking Gforth::             Getting in
* Leaving Gforth::              Getting out
* Help on Gforth::              Getting help
* Command-line editing::        Gforth's command-line editor
* Environment variables::       that affect how Gforth starts up
* Gforth Files::                What gets installed and where
* Gforth in pipes::             Piping into or out of Gforth
* Startup speed::               When 14ms is not fast enough ...

Forth Tutorial

* Starting Gforth Tutorial::
* Syntax Tutorial::
* Crash Course Tutorial::
* Stack Tutorial::
* Arithmetics Tutorial::
* Stack Manipulation Tutorial::
* Using files for Forth code Tutorial::
* Comments Tutorial::
* Colon Definitions Tutorial::
* Decompilation Tutorial::
* Stack-Effect Comments Tutorial::
* Types Tutorial::
* Factoring Tutorial::
* Designing the stack effect Tutorial::
* Local Variables Tutorial::
* Conditional execution Tutorial::
* Flags and Comparisons Tutorial::
* General Loops Tutorial::
* Counted loops Tutorial::
* Recursion Tutorial::
* Leaving definitions or loops Tutorial::
* Return Stack Tutorial::
* Memory Tutorial::
* Characters and Strings Tutorial::
* Alignment Tutorial::
* Floating Point Tutorial::
* Files Tutorial::
* Interpretation and Compilation Semantics and Immediacy Tutorial::
* Execution Tokens Tutorial::
* Exceptions Tutorial::
* Defining Words Tutorial::
* Arrays and Records Tutorial::
* POSTPONE Tutorial::
* Literal Tutorial::
* Advanced macros Tutorial::
* Compilation Tokens Tutorial::
* Wordlists and Search Order Tutorial::

An Introduction to Standard Forth

* Introducing the Text Interpreter::
* Stacks and Postfix notation::
* Your first definition::
* How does that work?::
* Forth is written in Forth::
* Review - elements of a Forth system::
* Where to go next::
* Exercises::

Forth Words

* Notation::
* Case insensitivity::
* Comments::
* Boolean Flags::
* Arithmetic::
* Stack Manipulation::
* Memory::
* Strings and Characters::
* Control Structures::
* Defining Words::
* Interpretation and Compilation Semantics::
* Tokens for Words::
* Compiling words::
* The Text Interpreter::
* The Input Stream::
* Word Lists::
* Environmental Queries::
* Files::
* Blocks::
* Other I/O::
* OS command line arguments::
* Locals::
* Structures::
* Object-oriented Forth::
* Regular Expressions::
* Programming Tools::
* Multitasker::
* C Interface::
* Assembler and Code Words::
* Carnal words::
* Passing Commands to the OS::
* Keeping track of Time::
* Miscellaneous Words::

Arithmetic

* Single precision::
* Double precision::            Double-cell integer arithmetic
* Mixed precision::             Operations with single and double-cell integers
* Integer division::
* Two-stage integer division::
* Bitwise operations::
* Numeric comparison::
* Floating Point::

Stack Manipulation

* Data stack::
* Floating point stack::
* Return stack::
* Locals stack::
* Stack pointer manipulation::

Memory

* Memory model::
* Dictionary allocation::
* Heap Allocation::
* Memory Access::
* Special Memory Accesses::
* Address arithmetic::
* Memory Blocks::

Strings and Characters

* Characters::
* String representations::
* String and character literals::
* String words::
* $tring words::
* Counted string words::

Control Structures

* Selection::                   IF ... ELSE ... ENDIF
* Simple Loops::                BEGIN ...
* Counted Loops::               DO
* BEGIN loops with multiple exits::
* General control structures with CASE::
* Arbitrary control structures::
* Calls and returns::
* Exception Handling::

Defining Words

* CREATE::
* Variables::                   Variables and user variables
* Constants::
* Values::                      Initialised variables
* Varues::
* Colon Definitions::
* Anonymous Definitions::       Definitions without names
* Quotations::
* Supplying names::             Passing definition names as strings
* User-defined Defining Words::
* Deferred Words::              Allow forward references
* Forward::                     Auto-resolved forward references
* Aliases::

User-defined Defining Words

* User-defined defining words with colon definitions::
* User-defined defining words using CREATE::
* CREATE..DOES> applications::
* CREATE..DOES> details::
* Advanced does> usage example::
* User-defined TO and DEFER@@::
* User-defined compile-comma::
* Creating from a prototype::
* Making a word current::
* Const-does>::

Interpretation and Compilation Semantics

* Combined words::

Tokens for Words

* Execution token::             represents execution/interpretation semantics
* Name token::                  represents named words
* Compilation token::           represents compilation semantics

Compiling words

* Literals::                    Compiling data values
* Macros::                      Compiling words

The Text Interpreter

* Input Sources::
* Number Conversion::
* Interpret/Compile states::
* Interpreter Directives::
* Recognizers::
* Text Interpreter Hooks::

Recognizers

* Default Recognizers::
* Dealing with existing Recognizers::
* Defining Recognizers::

Word Lists

* Vocabularies::
* Why use word lists?::
* Word list example::

Files

* Forth source files::
* General files::
* Redirection::
* Directories::
* Search Paths::

Search Paths

* Source Search Paths::
* General Search Paths::

Other I/O

* Simple numeric output::       Predefined formats
* Formatted numeric output::    Formatted (pictured) output
* Floating-point output::
* Miscellaneous output::
* Displaying characters and strings::  Other stuff
* Terminal output::             Cursor positioning etc.
* Single-key input::
* Line input and conversion::
* Pipes::                       How to create your own pipes
* Xchars and Unicode::          Non-ASCII characters
* i18n and l10n::               Internationalization and Localization
* Substitute::                  Text macro substitution
* CSV Reader::                  Data import facility

Locals

* Gforth locals::
* Standard Forth locals::

Gforth locals

* Locals definition words::
* Where are locals visible by name?::
* How long do locals live?::
* Locals programming style::
* Locals implementation::
* Closures::

Structures

* Why explicit structure support?::
* Structure Usage::
* Structure Naming Convention::
* Structure Implementation::
* Structure Glossary::
* Forth200x Structures::

Object-oriented Forth

* Why object-oriented programming?::
* Object-Oriented Terminology::
* Objects::
* OOF::
* Mini-OOF::
* Mini-OOF2::
* Comparison with other object models::

The @file{objects.fs} model

* Properties of the Objects model::
* Basic Objects Usage::
* The Objects base class::
* Creating objects::
* Object-Oriented Programming Style::
* Class Binding::
* Method conveniences::
* Classes and Scoping::
* Dividing classes::
* Object Interfaces::
* Objects Implementation::
* Objects Glossary::

The @file{oof.fs} model

* Properties of the OOF model::
* Basic OOF Usage::
* The OOF base class::
* Class Declaration::
* Class Implementation::

The @file{mini-oof.fs} model

* Basic Mini-OOF Usage::
* Mini-OOF Example::
* Mini-OOF Implementation::

Programming Tools

* Locating source code definitions::  and editing
* Locating uses of a word::     @code{where}.
* Locating exception source::   Examining backtraces.
* Examining compiled code::     @code{see} and friends.
* Examining data::              Stacks and Memory.
* Forgetting words::            Usually before reloading.
* Debugging::                   By inserting @code{~~} tracers.
* Assertions::                  Making your programs self-checking.
* Singlestep Debugger::         Executing your program word by word.
* Code Coverage::               Measure execution frequency.

Multitasker

* Pthreads::                    Native Unix multitasker
* Cilk::                        Implements Cilk-like spawn+sync

Pthreads

* Basic multi-tasking::
* Task-local data::
* Semaphores::
* Hardware operations for multi-tasking::
* Message queues::

C Interface

* Calling C Functions::
* Declaring C Functions::
* Calling C function pointers::
* Defining library interfaces::
* Declaring OS-level libraries::
* Callbacks::
* C interface internals::
* Low-Level C Interface Words::
* Automated interface generation using SWIG::
* Migrating the C interface from earlier Gforth::

Assembler and Code Words

* Assembler Definitions::       Definitions in assembly language
* Common Assembler::            Assembler Syntax
* Common Disassembler::
* 386 Assembler::               Deviations and special cases
* AMD64 Assembler::
* Alpha Assembler::             Deviations and special cases
* MIPS assembler::              Deviations and special cases
* PowerPC assembler::           Deviations and special cases
* ARM Assembler::               Deviations and special cases
* Other assemblers::            How to write them

Carnal words

* Header fields::
* Header methods::
* Threading Words::

Tools

* Standard Report::             Report the words used, sorted by wordset.
* Stack depth changes::         Where does this stack item come from?

Standard conformance

* The Core Words::
* The optional Block word set::
* The optional Double Number word set::
* The optional Exception word set::
* The optional Facility word set::
* The optional File-Access word set::
* The optional Floating-Point word set::
* The optional Locals word set::
* The optional Memory-Allocation word set::
* The optional Programming-Tools word set::
* The optional Search-Order word set::

The Core Words

* core-idef::                   Implementation Defined Options                   
* core-ambcond::                Ambiguous Conditions                
* core-other::                  Other System Documentation                  

The optional Block word set

* block-idef::                  Implementation Defined Options
* block-ambcond::               Ambiguous Conditions               
* block-other::                 Other System Documentation                 

The optional Double Number word set

* double-ambcond::              Ambiguous Conditions              

The optional Exception word set

* exception-idef::              Implementation Defined Options              

The optional Facility word set

* facility-idef::               Implementation Defined Options               
* facility-ambcond::            Ambiguous Conditions            

The optional File-Access word set

* file-idef::                   Implementation Defined Options
* file-ambcond::                Ambiguous Conditions                

The optional Floating-Point word set

* floating-idef::               Implementation Defined Options
* floating-ambcond::            Ambiguous Conditions            

The optional Locals word set

* locals-idef::                 Implementation Defined Options                 
* locals-ambcond::              Ambiguous Conditions              

The optional Memory-Allocation word set

* memory-idef::                 Implementation Defined Options                 

The optional Programming-Tools word set

* programming-idef::            Implementation Defined Options            
* programming-ambcond::         Ambiguous Conditions         

The optional Search-Order word set

* search-idef::                 Implementation Defined Options                 
* search-ambcond::              Ambiguous Conditions              

Emacs and Gforth

* Installing gforth.el::        Making Emacs aware of Forth.
* Emacs Tags::                  Viewing the source of a word in Emacs.
* Hilighting::                  Making Forth code look prettier.
* Auto-Indentation::            Customizing auto-indentation.
* Blocks Files::                Reading and writing blocks files.

Image Files

* Image Licensing Issues::      Distribution terms for images.
* Image File Background::       Why have image files?
* Non-Relocatable Image Files::  don't always work.
* Data-Relocatable Image Files::  are better.
* Fully Relocatable Image Files::  better yet.
* Stack and Dictionary Sizes::  Setting the default sizes for an image.
* Running Image Files::         @code{gforth -i @i{file}} or @i{file}.
* Modifying the Startup Sequence::  and turnkey applications.

Fully Relocatable Image Files

* gforthmi::                    The normal way
* cross.fs::                    The hard way

Engine

* Portability::
* Threading::
* Primitives::
* Performance::

Threading

* Scheduling::
* Direct or Indirect Threaded?::
* Dynamic Superinstructions::
* DOES>::

Primitives

* Automatic Generation::
* TOS Optimization::
* Produced code::

Cross Compiler

* Using the Cross Compiler::
* How the Cross Compiler Works::

MINOS2, a GUI library

* MINOS2 object framework::
* MINOS2 tutorial::

MINOS2 object framework

* actor methods::
* widget methods::

Licenses

* GNU Free Documentation License::  License for copying this manual.
* Copying::                     GPL (for copying this software).

@end detailmenu
@end menu

@end ifnottex

@c ----------------------------------------------------------
@iftex
@unnumbered Preface
@cindex Preface
This manual documents Gforth. Some introductory material is provided for
readers who are unfamiliar with Forth or who are migrating to Gforth
from other Forth compilers. However, this manual is primarily a
reference manual.
@end iftex

@comment TODO much more blurb here.

@c ******************************************************************
@node Goals, Gforth Environment, Top, Top
@comment node-name,     next,           previous, up
@chapter Goals of Gforth
@cindex goals of the Gforth project
The goal of the Gforth Project is to develop a standard model for
Standard Forth. This can be split into several subgoals:

@itemize @bullet
@item
Gforth should conform to the Forth Standard.
@item
It should be a model, i.e. it should define all the
implementation-dependent things.
@item
It should become standard, i.e. widely accepted and used. This goal
is the most difficult one.
@end itemize

To achieve these goals Gforth should be
@itemize @bullet
@item
Similar to previous models (fig-Forth, F83)
@item
Powerful. It should provide for all the things that are considered
necessary today and even some that are not yet considered necessary.
@item
Efficient. It should not get the reputation of being exceptionally
slow.
@item
Free.
@item
Available on many machines/easy to port.
@end itemize

Have we achieved these goals? Gforth conforms to the Forth-94 (ANS
Forth) and Forth-2012 standards.  We have changed some of the internal
data structures (in particular, the headers) over time, so Gforth
cannot be considered a stable model.  It certainly has not yet become
a de facto standard, but it appears to be quite popular. It has some
similarities to and some differences from previous models. It has some
powerful features, but not yet everything that we envisioned. We
certainly have achieved our execution speed goals
(@pxref{Performance})@footnote{However, in 1998 the bar was raised
when the major commercial Forth vendors switched to native code
compilers.}.  It is free and available on many machines.


@menu
* Stability Goals::
@end menu

@c ******************************************************************
@node Stability Goals,  , Goals, Goals
@section Stability Goals
@cindex Gforth stability
@cindex stability of Gforth

Programs that work on earlier versions of Gforth should also work on
newer versions.  However, there are some caveats:

Internal data structures (including the representation of code) of
Gforth may change between versions, unless they are documented.

Moreover, we only feel obliged to keep standard words (i.e., with
standard wordset names) and words documented as permanent Gforth
extensions (with wordset name @code{gforth} or
@code{gforth-@i{<version>}}, @pxref{Notation}).  Other words may be
removed in newer releases.

In particular, you may find a word by using @code{locate} or otherwise
inspecting Gforth's source code.  You can see the wordset in a comment
right after the stack-effect comment.  If there is no wordset for a
word, it is an internal factor and may be removed in a future version.
If the wordset is @code{gforth-experimental}, @code{gforth-internal},
or @code{gforth-obsolete}, the word may also be removed in a future
version.

If you want to use a particular word that is not marked as permanent,
please let us know, and we will consider to add the word as permanent
word (or we may suggest an alternative to using this word).

@c ******************************************************************
@node Gforth Environment, Tutorial, Goals, Top
@chapter Gforth Environment
@cindex Gforth environment

Note: ultimately, the Gforth man page will be auto-generated from the
material in this chapter.

@menu
* Invoking Gforth::             Getting in
* Leaving Gforth::              Getting out
* Help on Gforth::              Getting help
* Command-line editing::        Gforth's command-line editor
* Environment variables::       that affect how Gforth starts up
* Gforth Files::                What gets installed and where
* Gforth in pipes::             Piping into or out of Gforth
* Startup speed::               When 14ms is not fast enough ...
@end menu

For related information about the creation of images see @ref{Image Files}.

@comment ----------------------------------------------
@node Invoking Gforth, Leaving Gforth, Gforth Environment, Gforth Environment
@section Invoking Gforth
@cindex invoking Gforth
@cindex running Gforth
@cindex command-line options
@cindex options on the command line
@cindex flags on the command line

Gforth is made up of two parts; an executable ``engine'' (named
@command{gforth} or @command{gforth-fast}) and an image file. To start it, you
will usually just say @code{gforth} -- this automatically loads the
default image file @file{gforth.fi}. In many other cases the default
Gforth image will be invoked like this:
@example
gforth [file | -e forth-code] ...
@end example
@noindent
This interprets the contents of the files and the Forth code in the order they
are given.

In addition to the @command{gforth} engine, there is also an engine
called @command{gforth-fast}, which is faster, but gives less
informative error messages (@pxref{Error messages}) and may catch some
errors (in particular, stack underflows and integer division errors)
later or not at all.  You should use it for debugged,
performance-critical programs.

Moreover, there is an engine called @command{gforth-itc}, which is
useful in some backwards-compatibility situations (@pxref{Direct or
Indirect Threaded?}).

In general, the command line looks like this:

@example
gforth[-fast] [engine options] [image options]
@end example

The engine options must come before the rest of the command
line. They are:

@c !! vollstaendig?
@table @code
@cindex -i, command-line option
@cindex --image-file, command-line option
@item --image-file @i{file}
@itemx -i @i{file}
Loads the Forth image @i{file} instead of the default
@file{gforth.fi} (@pxref{Image Files}).

@cindex --appl-image, command-line option
@item --appl-image @i{file}
Loads the image @i{file} and leaves all further command-line arguments
to the image (instead of processing them as engine options).  This is
useful for building executable application images on Unix, built with
@code{gforthmi --application ...}.

@cindex --path, command-line option
@cindex -p, command-line option
@item --path @i{path}
@itemx -p @i{path}
Uses @i{path} for searching the image file and Forth source code files
instead of the default in the environment variable @code{GFORTHPATH} or
the path specified at installation time and the working directory
@code{.} (e.g., @file{/usr/local/share/gforth/0.2.0:.}). A path is given
as a list of directories, separated by @samp{:} (previous versions had
@samp{;} for other OSes, but since Cygwin now only accepts
@code{/cygdrive/<letter>}, and we dropped support for OS/2 and MS-DOS,
it is @samp{:} everywhere).

@cindex --dictionary-size, command-line option
@cindex -m, command-line option
@cindex @i{size} parameters for command-line options
@cindex size of the dictionary and the stacks
@item --dictionary-size @i{size}
@itemx -m @i{size}
Allocate @i{size} space for the Forth dictionary space instead of
using the default specified in the image (typically 256K). The
@i{size} specification for this and subsequent options consists of
an integer and a unit (e.g.,
@code{4M}). The unit can be one of @code{b} (bytes), @code{e} (element
size, in this case Cells), @code{k} (kilobytes), @code{M} (Megabytes),
@code{G} (Gigabytes), and @code{T} (Terabytes). If no unit is specified,
@code{e} is used.

@cindex --data-stack-size, command-line option
@cindex -d, command-line option
@item --data-stack-size @i{size}
@itemx -d @i{size}
Allocate @i{size} space for the data stack instead of using the
default specified in the image (typically 16K).

@cindex --return-stack-size, command-line option
@cindex -r, command-line option
@item --return-stack-size @i{size}
@itemx -r @i{size}
Allocate @i{size} space for the return stack instead of using the
default specified in the image (typically 15K).

@cindex --fp-stack-size, command-line option
@cindex -f, command-line option
@item --fp-stack-size @i{size}
@itemx -f @i{size}
Allocate @i{size} space for the floating point stack instead of
using the default specified in the image (typically 15.5K). In this case
the unit specifier @code{e} refers to floating point numbers.

@cindex --locals-stack-size, command-line option
@cindex -l, command-line option
@item --locals-stack-size @i{size}
@itemx -l @i{size}
Allocate @i{size} space for the locals stack instead of using the
default specified in the image (typically 14.5K).

@cindex --vm-commit, command-line option
@cindex overcommit memory for dictionary and stacks
@cindex memory overcommit for dictionary and stacks
@item --vm-commit
Normally, Gforth tries to start up even if there is not enough virtual
memory for the dictionary and the stacks (using @code{MAP_NORESERVE}
on OSs that support it); so you can ask for a really big dictionary
and/or stacks, and as long as you don't use more virtual memory than
is available, everything will be fine (but if you use more, processes
get killed).  With this option you just use the default allocation
policy of the OS; for OSs that don't overcommit (e.g., Solaris), this
means that you cannot and should not ask for as big dictionary and
stacks, but once Gforth successfully starts up, out-of-memory won't
kill it.

@cindex -h, command-line option
@cindex --help, command-line option
@item --help
@itemx -h
Print a message about the command-line options

@cindex -v, command-line option
@cindex --version, command-line option
@item --version
@itemx -v
Print version and exit

@cindex --debug, command-line option
@item --debug
Print some information useful for debugging on startup.

@cindex --offset-image, command-line option
@item --offset-image
Start the dictionary at a slightly different position than would be used
otherwise (useful for creating data-relocatable images,
@pxref{Data-Relocatable Image Files}).

@cindex --no-offset-im, command-line option
@item --no-offset-im
Start the dictionary at the normal position.

@cindex --clear-dictionary, command-line option
@item --clear-dictionary
Initialize all bytes in the dictionary to 0 before loading the image
(@pxref{Data-Relocatable Image Files}).

@cindex --die-on-signal, command-line-option
@item --die-on-signal
Normally Gforth handles most signals (e.g., the user interrupt SIGINT,
or the segmentation violation SIGSEGV) by translating it into a Forth
@code{THROW}. With this option, Gforth exits if it receives such a
signal. This option is useful when the engine and/or the image might be
severely broken (such that it causes another signal before recovering
from the first); this option avoids endless loops in such cases.

@cindex --no-dynamic, command-line option
@cindex --dynamic, command-line option
@item --no-dynamic
@item --dynamic
Disable or enable dynamic superinstructions with replication
(@pxref{Dynamic Superinstructions}).

@cindex --no-super, command-line option
@item --no-super
Disable dynamic superinstructions, use just dynamic replication; this is
useful if you want to patch threaded code (@pxref{Dynamic
Superinstructions}).

@cindex --ss-number, command-line option
@item --ss-number=@var{N}
Use only the first @var{N} static superinstructions compiled into the
engine (default: use them all; note that only @code{gforth-fast} has
any).  This option is useful for measuring the performance impact of
static superinstructions.

@cindex --ss-min-..., command-line options
@item --ss-min-codesize
@item --ss-min-ls
@item --ss-min-lsu
@item --ss-min-nexts
Use specified metric for determining the cost of a primitive or static
superinstruction for static superinstruction selection.  @code{Codesize}
is the native code size of the primive or static superinstruction,
@code{ls} is the number of loads and stores, @code{lsu} is the number of
loads, stores, and updates, and @code{nexts} is the number of dispatches
(not taking dynamic superinstructions into account), i.e. every
primitive or static superinstruction has cost 1. Default:
@code{codesize} if you use dynamic code generation, otherwise
@code{nexts}.

@cindex --ss-greedy, command-line option
@item --ss-greedy
This option is useful for measuring the performance impact of static
superinstructions.  By default, an optimal shortest-path algorithm is
used for selecting static superinstructions.  With @option{--ss-greedy}
this algorithm is modified to assume that anything after the static
superinstruction currently under consideration is not combined into
static superinstructions.  With @option{--ss-min-nexts} this produces
the same result as a greedy algorithm that always selects the longest
superinstruction available at the moment.  E.g., if there are
superinstructions AB and BCD, then for the sequence A B C D the optimal
algorithm will select A BCD and the greedy algorithm will select AB C D.

@cindex --print-metrics, command-line option
@item --print-metrics
Prints some metrics used during static superinstruction selection:
@code{code size} is the actual size of the dynamically generated code.
@code{Metric codesize} is the sum of the codesize metrics as seen by
static superinstruction selection; there is a difference from @code{code
size}, because not all primitives and static superinstructions are
compiled into dynamically generated code, and because of markers.  The
other metrics correspond to the @option{ss-min-...} options.  This
option is useful for evaluating the effects of the @option{--ss-...}
options.

@end table

@cindex loading files at startup
@cindex executing code on startup
@cindex batch processing with Gforth
As explained above, the image-specific command-line arguments for the
default image @file{gforth.fi} consist of a sequence of filenames and
@code{-e @var{forth-code}} options that are interpreted in the sequence
in which they are given. The @code{-e @var{forth-code}} or
@code{--evaluate @var{forth-code}} option evaluates the Forth code. This
option takes only one argument; if you want to evaluate more Forth
words, you have to quote them or use @code{-e} several times. To exit
after processing the command line (instead of entering interactive mode)
append @code{-e bye} to the command line.  You can also process the
command-line arguments with a Forth program (@pxref{OS command line
arguments}).

@cindex versions, invoking other versions of Gforth
If you have several versions of Gforth installed, @code{gforth} will
invoke the version that was installed last. @code{gforth-@i{<version>}}
invokes a specific version. If your environment contains the variable
@code{GFORTHPATH}, you may want to override it by using the
@code{--path} option.

On startup, before processing any of the image option, the user
initialization file either specified in the environment variable
@code{GFORTH_ENV} or, if not set, @file{~/.config/gforthrc0} is included, if
it exists.  If @code{GFORTH_ENV} is ``@code{off},'' nothing is
included.  After processing all the image options and just before
printing the boot message, the user initialization file
@file{~/.config/gforthrc} from your home directory is included, unless the
option @code{--no-rc} is given.

Warning levels can be set with

@table @code
@cindex -W, command-line option
@item -W
Turn off warnings

@cindex -Won, command-line option
@item -Won
Turn on warnings (level 1)

@cindex -Wall, command-line option
@item -Wall
Turn on beginner warnings (level 2)

@cindex -Wpedantic, command-line option
@item -Wpedantic
Turn on pedantic warnings (level 3)

@cindex -Werror, command-line option
@item -Werror
Turn warnings into errors (level 4)
@end table

@comment ----------------------------------------------
@node Leaving Gforth, Help on Gforth, Invoking Gforth, Gforth Environment
@section Leaving Gforth
@cindex Gforth - leaving
@cindex leaving Gforth

You can leave Gforth by typing @code{bye} or @kbd{Ctrl-d} (at the start
of a line) or (if you invoked Gforth with the @code{--die-on-signal}
option) @kbd{Ctrl-c}. When you leave Gforth, all of your definitions and
data are discarded.  For ways of saving the state of the system before
leaving Gforth see @ref{Image Files}.

doc-bye

@comment ----------------------------------------------
@node Help on Gforth, Command-line editing, Leaving Gforth, Gforth Environment
@section Help on Gforth
@cindex help

Gforth has a simple, text-based online help system.

doc-help
doc-authors
doc-license

@comment ----------------------------------------------
@node Command-line editing, Environment variables, Help on Gforth, Gforth Environment
@section Command-line editing
@cindex command-line editing

Gforth maintains a history file that records every line that you type to
the text interpreter. This file is preserved between sessions, and is
used to provide a command-line recall facility; if you type @kbd{Ctrl-P}
repeatedly you can recall successively older commands from this (or
previous) session(s). The full list of command-line editing facilities is:

@itemize @bullet
@item
@kbd{Ctrl-p} (``previous'') (or up-arrow) to recall successively older
lines from the history buffer.
@item
@kbd{Ctrl-n} (``next'') (or down-arrow) to recall successively newer
lines from the history buffer.  If you moved to an older line earlier
and gave it to Gforth for text-interpretation, asking for the next
line as the first editing command gives you the next line after the
one you selected last time.
@item
@kbd{Ctrl-f} (or right-arrow) to move the cursor right, non-destructively.
@item
@kbd{Ctrl-b} (or left-arrow) to move the cursor left, non-destructively.
@item
@kbd{Ctrl-h} (backspace) to delete the character to the left of the cursor,
closing up the line.
@item
@kbd{Ctrl-k} to delete (``kill'') from the cursor to the end of the line.
@item
@kbd{Ctrl-a} to move the cursor to the start of the line.
@item
@kbd{Ctrl-e} to move the cursor to the end of the line.
@item
@key{RET} (@kbd{Ctrl-m}) or @key{LFD} (@kbd{Ctrl-j}) to submit the current
line.
@item
@key{TAB} to step through all possible full-word completions of the word
currently being typed.
@item
@kbd{Ctrl-d} on an empty line line to terminate Gforth (gracefully,
using @code{bye}). 
@item
@kbd{Ctrl-x} (or @code{Ctrl-d} on a non-empty line) to delete the
character under the cursor.
@end itemize

When editing, displayable characters are inserted to the left of the
cursor position; the line is always in ``insert'' (as opposed to
``overstrike'') mode.

@cindex history file
@cindex @file{.gforth-history}
On Unix systems, the history file is
@file{$HOME/.local/share/gforth/history} by default@footnote{i.e. it
is stored in the user's home directory.}. You can find out the name
and location of your history file using:

@example 
history-file type \ Unix-class systems

history-file type \ Other systems
history-dir  type
@end example

If you enter long definitions by hand, you can use a text editor to
paste them out of the history file into a Forth source file for reuse at
a later time.

Gforth never trims the size of the history file, so you should do this
periodically, if necessary.

@comment this is all defined in history.fs
@comment NAC TODO the ctrl-D behaviour can either do a bye or a beep.. how is that option
@comment chosen?


@comment ----------------------------------------------
@node Environment variables, Gforth Files, Command-line editing, Gforth Environment
@section Environment variables
@cindex environment variables

Gforth uses these environment variables:

@itemize @bullet
@item
@cindex @code{GFORTHHIST} -- environment variable
@code{GFORTHHIST} -- (Unix systems only) specifies the path for the
history file @file{.gforth-history}. Default:
@code{$HOME/.local/share/gforth/history}.

@item
@cindex @code{GFORTHPATH} -- environment variable
@code{GFORTHPATH} -- specifies the path used when searching for the
gforth image file and for Forth source-code files (usually @samp{.}, the
current working directory).  Path separator is @samp{:}, a typical path
would be @file{/usr/local/share/gforth/1.0:.}.

@item
@cindex @code{LANG} -- environment variable
@code{LANG} -- see @code{LC_CTYPE}

@item
@cindex @code{LC_ALL} -- environment variable
@code{LC_ALL} -- see @code{LC_CTYPE}

@item
@cindex @code{LC_CTYPE} -- environment variable
@code{LC_CTYPE} -- If this variable contains ``UTF-8'' on Gforth
startup, Gforth uses the UTF-8 encoding for strings internally and
expects its input and produces its output in UTF-8 encoding, otherwise
the encoding is 8bit (see @pxref{Xchars and Unicode}).  If this
environment variable is unset, Gforth looks in @code{LC_ALL}, and if
that is unset, in @code{LANG}.

@item
@cindex @code{GFORTHSYSTEMPREFIX} -- environment variable
@code{GFORTHSYSTEMPREFIX} -- specifies what to prepend to the argument
of @code{system} before passing it to C's @code{system()}.  Default:
@code{"./$COMSPEC /c "} on Windows, @code{""} on other OSs.  The
prefix and the command are directly concatenated, so if a space
between them is necessary, append it to the prefix.

@item
@cindex @code{GFORTH} -- environment variable
@code{GFORTH} -- used by @file{gforthmi}, @xref{gforthmi}.

@item
@cindex @code{GFORTHD} -- environment variable
@code{GFORTHD} -- used by @file{gforthmi}, @xref{gforthmi}.

@item
@cindex @code{TMP}, @code{TEMP} - environment variable
@code{TMP}, @code{TEMP} - (non-Unix systems only) used as a potential
location for the history file.
@end itemize

@comment also POSIXELY_CORRECT LINES COLUMNS HOME but no interest in
@comment mentioning these.

All the Gforth environment variables default to sensible values if they
are not set.


@comment ----------------------------------------------
@node Gforth Files, Gforth in pipes, Environment variables, Gforth Environment
@section Gforth files
@cindex Gforth files

When you install Gforth on a Unix system, it installs files in these
locations by default:

@itemize @bullet
@item
@file{/usr/local/bin/gforth}
@item
@file{/usr/local/bin/gforthmi}
@item
@file{/usr/local/man/man1/gforth.1} - man page.
@item
@file{/usr/local/info} - the Info version of this manual.
@item
@file{/usr/local/lib/gforth/<version>/...} - Gforth @file{.fi} files.
@item
@file{/usr/local/share/gforth/<version>/TAGS} - Emacs TAGS file.
@item
@file{/usr/local/share/gforth/<version>/...} - Gforth source files.
@item
@file{.../emacs/site-lisp/gforth.el} - Emacs gforth mode.
@end itemize

You can select different places for installation by using
@code{configure} options (listed with @code{configure --help}).

@comment ----------------------------------------------
@node Gforth in pipes, Startup speed, Gforth Files, Gforth Environment
@section Gforth in pipes
@cindex pipes, Gforth as part of

Gforth can be used in pipes created elsewhere (described in the
following).  It can also create pipes on its own (@pxref{Pipes}).

@cindex input from pipes
If you pipe into Gforth, your program should read with @code{read-file}
or @code{read-line} from @code{stdin} (@pxref{General files}).
@code{Key} does not recognize the end of input.  Words like
@code{accept} echo the input and are therefore usually not useful for
reading from a pipe.  You have to invoke the Forth program with an OS
command-line option, as you have no chance to use the Forth command line
(the text interpreter would try to interpret the pipe input).

@cindex output in pipes
You can output to a pipe with @code{type}, @code{emit}, @code{cr} etc.

@cindex silent exiting from Gforth
When you write to a pipe that has been closed at the other end, Gforth
receives a SIGPIPE signal (``pipe broken'').  Gforth translates this
into the exception @code{broken-pipe-error}.  If your application does
not catch that exception, the system catches it and exits, usually
silently (unless you were working on the Forth command line; then it
prints an error message and exits).  This is usually the desired
behaviour.

If you do not like this behaviour, you have to catch the exception
yourself, and react to it.

Here's an example of an invocation of Gforth that is usable in a pipe:

@example
gforth -e ": foo begin pad dup 10 stdin read-file throw dup while \
 type repeat ; foo bye"
@end example

This example just copies the input verbatim to the output.  A very
simple pipe containing this example looks like this:

@example
cat startup.fs |
gforth -e ": foo begin pad dup 80 stdin read-file throw dup while \
 type repeat ; foo bye"|
head
@end example

@cindex stderr and pipes
Pipes involving Gforth's @code{stderr} output do not work.

@comment ----------------------------------------------
@node Startup speed,  , Gforth in pipes, Gforth Environment
@section Startup speed
@cindex Startup speed
@cindex speed, startup

If Gforth is used for CGI scripts or in shell scripts, its startup
speed may become a problem.  On a 3GHz Core 2 Duo E8400 under 64-bit
Linux 2.6.27.8 with libc-2.7, @code{gforth-fast -e bye} takes 13.1ms
user and 1.2ms system time (@code{gforth -e bye} is faster on startup
with about 3.4ms user time and 1.2ms system time, because it subsumes
some of the options discussed below).

If startup speed is a problem, you may consider the following ways to
improve it; or you may consider ways to reduce the number of startups
(for example, by using Fast-CGI).  Note that the first steps below
improve the startup time at the cost of run-time (including
compile-time), so whether they are profitable depends on the balance
of these times in your application.

An easy step that influences Gforth startup speed is the use of a
number of options that increase run-time, but decrease image-loading
time.

The first of these that you should try is @code{--ss-number=0
--ss-states=1} because this option buys relatively little run-time
speedup and costs quite a bit of time at startup.  @code{gforth-fast
--ss-number=0 --ss-states=1 -e bye} takes about 2.8ms user and 1.5ms
system time.

The next option is @code{--no-dynamic} which has a substantial impact
on run-time (about a factor of 2 on several platforms), but still
makes startup speed a little faster: @code{gforth-fast --ss-number=0
--ss-states=1 --no-dynamic -e bye} consumes about 2.6ms user and 1.2ms
system time.

@c !! check
The next step to improve startup speed is to use a data-relocatable
image (@pxref{Data-Relocatable Image Files}).  This avoids the
relocation cost for the code in the image (but not for the data).
Note that the image is then specific to the particular binary you are
using (i.e., whether it is @code{gforth}, @code{gforth-fast}, and even
the particular build).  You create the data-relocatable image that
works with @code{./gforth-fast} with @code{GFORTHD="./gforth-fast
--no-dynamic" gforthmi gforthdr.fi} (the @code{--no-dynamic} is
required here or the image will not work).  And you run it with
@code{gforth-fast -i gforthdr.fi ... -e bye} (the flags discussed
above don't matter here, because they only come into play on
relocatable code).  @code{gforth-fast -i gforthdr.fi -e bye} takes
about 1.1ms user and 1.2ms system time.

One step further is to avoid all relocation cost and part of the
copy-on-write cost through using a non-relocatable image
(@pxref{Non-Relocatable Image Files}).  However, this has the
disadvantage that it does not work on operating systems with address
space randomization (the default in, e.g., Linux nowadays), or if the
dictionary moves for any other reason (e.g., because of a change of
the OS kernel or an updated library), so we cannot really recommend
it.  You create a non-relocatable image with @code{gforth-fast
--no-dynamic -e "savesystem gforthnr.fi bye"} (the @code{--no-dynamic}
is required here, too).  And you run it with @code{gforth-fast -i
gforthnr.fi ... -e bye} (again the flags discussed above don't
matter).  @code{gforth-fast -i gforthdr.fi -e bye} takes
about 0.9ms user and 0.9ms system time.

If the script you want to execute contains a significant amount of
code, it may be profitable to compile it into the image to avoid the
cost of compiling it at startup time.

@c ******************************************************************
@node Tutorial, Introduction, Gforth Environment, Top
@chapter Forth Tutorial
@cindex Tutorial
@cindex Forth Tutorial

@c Topics from nac's Introduction that could be mentioned:
@c press <ret> after each line
@c Prompt
@c numbers vs. words in dictionary on text interpretation
@c what happens on redefinition
@c parsing words (in particular, defining words)

The difference of this chapter from the Introduction
(@pxref{Introduction}) is that this tutorial is more fast-paced, should
be used while sitting in front of a computer, and covers much more
material, but does not explain how the Forth system works.

This tutorial can be used with any Standard-compliant Forth; any
Gforth-specific features are marked as such and you can skip them if
you work with another Forth.  This tutorial does not explain all
features of Forth, just enough to get you started and give you some
ideas about the facilities available in Forth.  Read the rest of the
manual when you are through this.

The intended way to use this tutorial is that you work through it while
sitting in front of the console, take a look at the examples and predict
what they will do, then try them out; if the outcome is not as expected,
find out why (e.g., by trying out variations of the example), so you
understand what's going on.  There are also some assignments that you
should solve.

This tutorial assumes that you have programmed before and know what,
e.g., a loop is.

@c !! explain compat library

@menu
* Starting Gforth Tutorial::
* Syntax Tutorial::
* Crash Course Tutorial::
* Stack Tutorial::
* Arithmetics Tutorial::
* Stack Manipulation Tutorial::
* Using files for Forth code Tutorial::
* Comments Tutorial::
* Colon Definitions Tutorial::
* Decompilation Tutorial::
* Stack-Effect Comments Tutorial::
* Types Tutorial::
* Factoring Tutorial::
* Designing the stack effect Tutorial::
* Local Variables Tutorial::
* Conditional execution Tutorial::
* Flags and Comparisons Tutorial::
* General Loops Tutorial::
* Counted loops Tutorial::
* Recursion Tutorial::
* Leaving definitions or loops Tutorial::
* Return Stack Tutorial::
* Memory Tutorial::
* Characters and Strings Tutorial::
* Alignment Tutorial::
* Floating Point Tutorial::
* Files Tutorial::
* Interpretation and Compilation Semantics and Immediacy Tutorial::
* Execution Tokens Tutorial::
* Exceptions Tutorial::
* Defining Words Tutorial::
* Arrays and Records Tutorial::
* POSTPONE Tutorial::
* Literal Tutorial::
* Advanced macros Tutorial::
* Compilation Tokens Tutorial::
* Wordlists and Search Order Tutorial::
@end menu

@node Starting Gforth Tutorial, Syntax Tutorial, Tutorial, Tutorial
@section Starting Gforth
@cindex starting Gforth tutorial
You can start Gforth by typing its name:

@example
gforth
@end example

That puts you into interactive mode; you can leave Gforth by typing
@code{bye}.  While in Gforth, you can edit the command line and access
the command line history with cursor keys, similar to bash.


@node Syntax Tutorial, Crash Course Tutorial, Starting Gforth Tutorial, Tutorial
@section Syntax
@cindex syntax tutorial

A @dfn{word} is a sequence of arbitrary characters (except white
space).  Words are separated by white space.  E.g., each of the
following lines contains exactly one word:

@example
word
!@@#$%^&*()
1234567890
5!a
@end example

A frequent beginner's error is to leave out necessary white space,
resulting in an error like @samp{Undefined word}; so if you see such an
error, check if you have put spaces wherever necessary.

@example
." hello, world" \ correct
."hello, world"  \ gives an "Undefined word" error
@end example

Gforth and most other Forth systems ignore differences in case (they are
case-insensitive), i.e., @samp{word} is the same as @samp{Word}.  If
your system is case-sensitive, you may have to type all the examples
given here in upper case.


@node Crash Course Tutorial, Stack Tutorial, Syntax Tutorial, Tutorial
@section Crash Course

Forth does not prevent you from shooting yourself in the foot.  Let's
try a few ways to crash Gforth:

@example
0 0 !
here execute
' catch >body 20 erase abort
' (quit1) >body 20 erase
@end example

The last two examples are guaranteed to destroy important parts of
Gforth (and most other systems), so you better leave Gforth afterwards
(if it has not finished by itself).  On some systems you may have to
kill gforth from outside (e.g., in Unix with @code{kill}).

You will find out later what these lines do and then you will get an
idea why they produce crashes.

Now that you know how to produce crashes (and that there's not much to
them), let's learn how to produce meaningful programs.


@node Stack Tutorial, Arithmetics Tutorial, Crash Course Tutorial, Tutorial
@section Stack
@cindex stack tutorial

The most obvious feature of Forth is the stack.  When you type in a
number, it is pushed on the stack.  You can display the contents of the
stack with @code{.s}.

@example
1 2 .s
3 .s
@end example

@code{.s} displays the top-of-stack to the right, i.e., the numbers
appear in @code{.s} output as they appeared in the input.

You can print the top element of the stack with @code{.}.

@example
1 2 3 . . .
@end example

In general, words consume their stack arguments (@code{.s} is an
exception).

@quotation Assignment
What does the stack contain after @code{5 6 7 .}?
@end quotation


@node Arithmetics Tutorial, Stack Manipulation Tutorial, Stack Tutorial, Tutorial
@section Arithmetics
@cindex arithmetics tutorial

The words @code{+}, @code{-}, @code{*}, @code{/}, and @code{mod} always
operate on the top two stack items:

@example
2 2 .s
+ .s
.
2 1 - .
7 3 mod .
@end example

The operands of @code{-}, @code{/}, and @code{mod} are in the same order
as in the corresponding infix expression (this is generally the case in
Forth).

Parentheses are superfluous (and not available), because the order of
the words unambiguously determines the order of evaluation and the
operands:

@example
3 4 + 5 * .
3 4 5 * + .
@end example

@quotation Assignment
What are the infix expressions corresponding to the Forth code above?
Write @code{6-7*8+9} in Forth notation@footnote{This notation is also
known as Postfix or RPN (Reverse Polish Notation).}.
@end quotation

To change the sign, use @code{negate}:

@example
2 negate .
@end example

@quotation Assignment
Convert -(-3)*4-5 to Forth.
@end quotation

@code{/mod} performs both @code{/} and @code{mod}.

@example
7 3 /mod . .
@end example

Reference: @ref{Arithmetic}.


@node Stack Manipulation Tutorial, Using files for Forth code Tutorial, Arithmetics Tutorial, Tutorial
@section Stack Manipulation
@cindex stack manipulation tutorial

Stack manipulation words rearrange the data on the stack.

@example
1 .s drop .s
1 .s dup .s drop drop .s
1 2 .s over .s drop drop drop
1 2 .s swap .s drop drop
1 2 3 .s rot .s drop drop drop
@end example

These are the most important stack manipulation words.  There are also
variants that manipulate twice as many stack items:

@example
1 2 3 4 .s 2swap .s 2drop 2drop
@end example

Two more stack manipulation words are:

@example
1 2 .s nip .s drop
1 2 .s tuck .s 2drop drop
@end example

@quotation Assignment
Replace @code{nip} and @code{tuck} with combinations of other stack
manipulation words.

@example
Given:          How do you get:
1 2 3           3 2 1           
1 2 3           1 2 3 2                 
1 2 3           1 2 3 3                 
1 2 3           1 3 3           
1 2 3           2 1 3           
1 2 3 4         4 3 2 1         
1 2 3           1 2 3 1 2 3             
1 2 3 4         1 2 3 4 1 2             
1 2 3
1 2 3           1 2 3 4                 
1 2 3           1 3             
@end example
@end quotation

@example
5 dup * .
@end example

@quotation Assignment
Write 17^3 and 17^4 in Forth, without writing @code{17} more than once.
Write a piece of Forth code that expects two numbers on the stack
(@var{a} and @var{b}, with @var{b} on top) and computes
@code{(a-b)(a+1)}.
@end quotation

Reference: @ref{Stack Manipulation}.


@node Using files for Forth code Tutorial, Comments Tutorial, Stack Manipulation Tutorial, Tutorial
@section Using files for Forth code
@cindex loading Forth code, tutorial
@cindex files containing Forth code, tutorial

While working at the Forth command line is convenient for one-line
examples and short one-off code, you probably want to store your source
code in files for convenient editing and persistence.  You can use your
favourite editor (Gforth includes Emacs support, @pxref{Emacs and
Gforth}) to create @var{file.fs} and use

@example
s" @var{file.fs}" included
@end example

to load it into your Forth system.  The file name extension I use for
Forth files is @samp{.fs}.

You can easily start Gforth with some files loaded like this:

@example
gforth @var{file1.fs} @var{file2.fs}
@end example

If an error occurs during loading these files, Gforth terminates,
whereas an error during @code{INCLUDED} within Gforth usually gives you
a Gforth command line.  Starting the Forth system every time gives you a
clean start every time, without interference from the results of earlier
tries.

I often put all the tests in a file, then load the code and run the
tests with

@example
gforth @var{code.fs} @var{tests.fs} -e bye
@end example

(often by performing this command with @kbd{C-x C-e} in Emacs).  The
@code{-e bye} ensures that Gforth terminates afterwards so that I can
restart this command without ado.

The advantage of this approach is that the tests can be repeated easily
every time the program ist changed, making it easy to catch bugs
introduced by the change.

Reference: @ref{Forth source files}.


@node Comments Tutorial, Colon Definitions Tutorial, Using files for Forth code Tutorial, Tutorial
@section Comments
@cindex comments tutorial

@example
\ That's a comment; it ends at the end of the line
( Another comment; it ends here: )  .s
@end example

@code{\} and @code{(} are ordinary Forth words and therefore have to be
separated with white space from the following text.

@example
\This gives an "Undefined word" error
@end example

The first @code{)} ends a comment started with @code{(}, so you cannot
nest @code{(}-comments; and you cannot comment out text containing a
@code{)} with @code{( ... )}@footnote{therefore it's a good idea to
avoid @code{)} in word names.}.

I use @code{\}-comments for descriptive text and for commenting out code
of one or more line; I use @code{(}-comments for describing the stack
effect, the stack contents, or for commenting out sub-line pieces of
code.

The Emacs mode @file{gforth.el} (@pxref{Emacs and Gforth}) supports
these uses by commenting out a region with @kbd{C-x \}, uncommenting a
region with @kbd{C-u C-x \}, and filling a @code{\}-commented region
with @kbd{M-q}.

Reference: @ref{Comments}.


@node Colon Definitions Tutorial, Decompilation Tutorial, Comments Tutorial, Tutorial
@section Colon Definitions
@cindex colon definitions, tutorial
@cindex definitions, tutorial
@cindex procedures, tutorial
@cindex functions, tutorial

are similar to procedures and functions in other programming languages.

@example
: squared ( n -- n^2 )
   dup * ;
5 squared .
7 squared .
@end example

@code{:} starts the colon definition; its name is @code{squared}.  The
following comment describes its stack effect.  The words @code{dup *}
are not executed, but compiled into the definition.  @code{;} ends the
colon definition.

The newly-defined word can be used like any other word, including using
it in other definitions:

@example
: cubed ( n -- n^3 )
   dup squared * ;
-5 cubed .
: fourth-power ( n -- n^4 )
   squared squared ;
3 fourth-power .
@end example

@quotation Assignment
Write colon definitions for @code{nip}, @code{tuck}, @code{negate}, and
@code{/mod} in terms of other Forth words, and check if they work (hint:
test your tests on the originals first).  Don't let the
@samp{redefined}-Messages spook you, they are just warnings.
@end quotation

Reference: @ref{Colon Definitions}.


@node Decompilation Tutorial, Stack-Effect Comments Tutorial, Colon Definitions Tutorial, Tutorial
@section Decompilation
@cindex decompilation tutorial
@cindex see tutorial

You can decompile colon definitions with @code{see}:

@example
see squared
see cubed
@end example

In Gforth @code{see} shows you a reconstruction of the source code from
the executable code.  Informations that were present in the source, but
not in the executable code, are lost (e.g., comments).

You can also decompile the predefined words:

@example
see .
see +
@end example


@node Stack-Effect Comments Tutorial, Types Tutorial, Decompilation Tutorial, Tutorial
@section Stack-Effect Comments
@cindex stack-effect comments, tutorial
@cindex --, tutorial
By convention the comment after the name of a definition describes the
stack effect: The part in front of the @samp{--} describes the state of
the stack before the execution of the definition, i.e., the parameters
that are passed into the colon definition; the part behind the @samp{--}
is the state of the stack after the execution of the definition, i.e.,
the results of the definition.  The stack comment only shows the top
stack items that the definition accesses and/or changes.

You should put a correct stack effect on every definition, even if it is
just @code{( -- )}.  You should also add some descriptive comment to
more complicated words (I usually do this in the lines following
@code{:}).  If you don't do this, your code becomes unreadable (because
you have to work through every definition before you can understand
any).

@quotation Assignment
The stack effect of @code{swap} can be written like this: @code{x1 x2 --
x2 x1}.  Describe the stack effect of @code{-}, @code{drop}, @code{dup},
@code{over}, @code{rot}, @code{nip}, and @code{tuck}.  Hint: When you
are done, you can compare your stack effects to those in this manual
(@pxref{Word Index}).
@end quotation

Sometimes programmers put comments at various places in colon
definitions that describe the contents of the stack at that place (stack
comments); i.e., they are like the first part of a stack-effect
comment. E.g.,

@example
: cubed ( n -- n^3 )
   dup squared  ( n n^2 ) * ;
@end example

In this case the stack comment is pretty superfluous, because the word
is simple enough.  If you think it would be a good idea to add such a
comment to increase readability, you should also consider factoring the
word into several simpler words (@pxref{Factoring Tutorial,,
Factoring}), which typically eliminates the need for the stack comment;
however, if you decide not to refactor it, then having such a comment is
better than not having it.

The names of the stack items in stack-effect and stack comments in the
standard, in this manual, and in many programs specify the type through
a type prefix, similar to Fortran and Hungarian notation.  The most
frequent prefixes are:

@table @code
@item n
signed integer
@item u
unsigned integer
@item c
character
@item f
Boolean flags, i.e. @code{false} or @code{true}.
@item a-addr,a-
Cell-aligned address
@item c-addr,c-
Char-aligned address (note that a Char may have two bytes in Windows NT)
@item xt
Execution token, same size as Cell
@item w,x
Cell, can contain an integer or an address.  It usually takes 32, 64 or
16 bits (depending on your platform and Forth system). A cell is more
commonly known as machine word, but the term @emph{word} already means
something different in Forth.
@item d
signed double-cell integer
@item ud
unsigned double-cell integer
@item r
Float (on the FP stack)
@end table

You can find a more complete list in @ref{Notation}.

@quotation Assignment
Write stack-effect comments for all definitions you have written up to
now.
@end quotation


@node Types Tutorial, Factoring Tutorial, Stack-Effect Comments Tutorial, Tutorial
@section Types
@cindex types tutorial

In Forth the names of the operations are not overloaded; so similar
operations on different types need different names; e.g., @code{+} adds
integers, and you have to use @code{f+} to add floating-point numbers.
The following prefixes are often used for related operations on
different types:

@table @code
@item (none)
signed integer
@item u
unsigned integer
@item c
character
@item d
signed double-cell integer
@item ud, du
unsigned double-cell integer
@item 2
two cells (not-necessarily double-cell numbers)
@item m, um
mixed single-cell and double-cell operations
@item f
floating-point (note that in stack comments @samp{f} represents flags,
and @samp{r} represents FP numbers; also, you need to include the
exponent part in literal FP numbers, @pxref{Floating Point Tutorial}).
@end table

If there are no differences between the signed and the unsigned variant
(e.g., for @code{+}), there is only the prefix-less variant.

Forth does not perform type checking, neither at compile time, nor at
run time.  If you use the wrong operation, the data are interpreted
incorrectly:

@example
-1 u.
@end example

If you have only experience with type-checked languages until now, and
have heard how important type-checking is, don't panic!  In my
experience (and that of other Forthers), type errors in Forth code are
usually easy to find (once you get used to it), the increased vigilance
of the programmer tends to catch some harder errors in addition to most
type errors, and you never have to work around the type system, so in
most situations the lack of type-checking seems to be a win (projects to
add type checking to Forth have not caught on).


@node Factoring Tutorial, Designing the stack effect Tutorial, Types Tutorial, Tutorial
@section Factoring
@cindex factoring tutorial

If you try to write longer definitions, you will soon find it hard to
keep track of the stack contents.  Therefore, good Forth programmers
tend to write only short definitions (e.g., three lines).  The art of
finding meaningful short definitions is known as factoring (as in
factoring polynomials).

Well-factored programs offer additional advantages: smaller, more
general words, are easier to test and debug and can be reused more and
better than larger, specialized words.

So, if you run into difficulties with stack management, when writing
code, try to define meaningful factors for the word, and define the word
in terms of those.  Even if a factor contains only two words, it is
often helpful.

Good factoring is not easy, and it takes some practice to get the knack
for it; but even experienced Forth programmers often don't find the
right solution right away, but only when rewriting the program.  So, if
you don't come up with a good solution immediately, keep trying, don't
despair.

@c example !!


@node Designing the stack effect Tutorial, Local Variables Tutorial, Factoring Tutorial, Tutorial
@section Designing the stack effect
@cindex Stack effect design, tutorial
@cindex design of stack effects, tutorial

In other languages you can use an arbitrary order of parameters for a
function; and since there is only one result, you don't have to deal with
the order of results, either.

In Forth (and other stack-based languages, e.g., PostScript) the
parameter and result order of a definition is important and should be
designed well.  The general guideline is to design the stack effect such
that the word is simple to use in most cases, even if that complicates
the implementation of the word.  Some concrete rules are:

@itemize @bullet

@item
Words consume all of their parameters (e.g., @code{.}).

@item
If there is a convention on the order of parameters (e.g., from
mathematics or another programming language), stick with it (e.g.,
@code{-}).

@item
If one parameter usually requires only a short computation (e.g., it is
a constant), pass it on the top of the stack.  Conversely, parameters
that usually require a long sequence of code to compute should be passed
as the bottom (i.e., first) parameter.  This makes the code easier to
read, because the reader does not need to keep track of the bottom item
through a long sequence of code (or, alternatively, through stack
manipulations). E.g., @code{!} (store, @pxref{Memory}) expects the
address on top of the stack because it is usually simpler to compute
than the stored value (often the address is just a variable).

@item
Similarly, results that are usually consumed quickly should be returned
on the top of stack, whereas a result that is often used in long
computations should be passed as bottom result.  E.g., the file words
like @code{open-file} return the error code on the top of stack, because
it is usually consumed quickly by @code{throw}; moreover, the error code
has to be checked before doing anything with the other results.

@end itemize

These rules are just general guidelines, don't lose sight of the overall
goal to make the words easy to use.  E.g., if the convention rule
conflicts with the computation-length rule, you might decide in favour
of the convention if the word will be used rarely, and in favour of the
computation-length rule if the word will be used frequently (because
with frequent use the cost of breaking the computation-length rule would
be quite high, and frequent use makes it easier to remember an
unconventional order).

@c example !! structure package


@node Local Variables Tutorial, Conditional execution Tutorial, Designing the stack effect Tutorial, Tutorial
@section Local Variables
@cindex local variables, tutorial

You can define local variables (@emph{locals}) in a colon definition:

@example
: swap @{ a b -- b a @}
  b a ;
1 2 swap .s 2drop
@end example

(If your Forth system does not support this syntax, include
@file{compat/anslocal.fs} first).

In this example @code{@{ a b -- b a @}} is the locals definition; it
takes two cells from the stack, puts the top of stack in @code{b} and
the next stack element in @code{a}.  @code{--} starts a comment ending
with @code{@}}.  After the locals definition, using the name of the
local will push its value on the stack.  You can omit the comment
part (@code{-- b a}):

@example
: swap ( x1 x2 -- x2 x1 )
  @{ a b @} b a ;
@end example

In Gforth you can have several locals definitions, anywhere in a colon
definition; in contrast, in a standard program you can have only one
locals definition per colon definition, and that locals definition must
be outside any control structure.

With locals you can write slightly longer definitions without running
into stack trouble.  However, I recommend trying to write colon
definitions without locals for exercise purposes to help you gain the
essential factoring skills.

@quotation Assignment
Rewrite your definitions until now with locals
@end quotation

Reference: @ref{Locals}.


@node Conditional execution Tutorial, Flags and Comparisons Tutorial, Local Variables Tutorial, Tutorial
@section Conditional execution
@cindex conditionals, tutorial
@cindex if, tutorial

In Forth you can use control structures only inside colon definitions.
An @code{if}-structure looks like this:

@example
: abs ( n1 -- +n2 )
    dup 0 < if
        negate
    endif ;
5 abs .
-5 abs .
@end example

@code{if} takes a flag from the stack.  If the flag is non-zero (true),
the following code is performed, otherwise execution continues after the
@code{endif} (or @code{else}).  @code{<} compares the top two stack
elements and produces a flag:

@example
1 2 < .
2 1 < .
1 1 < .
@end example

Actually the standard name for @code{endif} is @code{then}.  This
tutorial presents the examples using @code{endif}, because this is often
less confusing for people familiar with other programming languages
where @code{then} has a different meaning.  If your system does not have
@code{endif}, define it with

@example
: endif postpone then ; immediate
@end example

You can optionally use an @code{else}-part:

@example
: min ( n1 n2 -- n )
  2dup < if
    drop
  else
    nip
  endif ;
2 3 min .
3 2 min .
@end example

@quotation Assignment
Write @code{min} without @code{else}-part (hint: what's the definition
of @code{nip}?).
@end quotation

Reference: @ref{Selection}.


@node Flags and Comparisons Tutorial, General Loops Tutorial, Conditional execution Tutorial, Tutorial
@section Flags and Comparisons
@cindex flags tutorial
@cindex comparison tutorial

In a false-flag all bits are clear (0 when interpreted as integer).  In
a canonical true-flag all bits are set (-1 as a twos-complement signed
integer); in many contexts (e.g., @code{if}) any non-zero value is
treated as true flag.

@example
false .
true .
true hex u. decimal
@end example

Comparison words produce canonical flags:

@example
1 1 = .
1 0= .
0 1 < .
0 0 < .
-1 1 u< . \ type error, u< interprets -1 as large unsigned number
-1 1 < .
@end example

Gforth supports all combinations of the prefixes @code{0 u d d0 du f f0}
(or none) and the comparisons @code{= <> < > <= >=}.  Only a part of
these combinations are standard (for details see the standard,
@ref{Numeric comparison}, @ref{Floating Point} or @ref{Word Index}).

You can use @code{and or xor invert} as operations on canonical flags.
Actually they are bitwise operations:

@example
1 2 and .
1 2 or .
1 3 xor .
1 invert .
@end example

You can convert a zero/non-zero flag into a canonical flag with
@code{0<>} (and complement it on the way with @code{0=}; indeed, it is
more common to use @code{0=} instead of @code{invert} for canonical
flags).

@example
1 0= .
1 0<> .
@end example

While you can use @code{if} without @code{0<>} to test for
zero/non-zero, you sometimes need to use @code{0<>} when combining
zero/non-zero values with @code{and or xor} because of their bitwise
nature.  The simplest, least error-prone, and probably clearest way is
to use @code{0<>} in all these cases, but in some cases you can use
fewer @code{0<>}s.  Here are some stack effects, where @i{fc}
represents a canonical flag, and @i{fz} represents zero/non-zero
(every @i{fc} also works as @i{fz}):

@example
or  ( fz1 fz2 -- fz3 )
and ( fz1 fc  -- fz2 )
and ( fc  fz1 -- fz2 )
@end example

So, if you see code like this:

@example
( n1 n2 ) 0<> and if
@end example

This tests whether n1 and n2 are non-zero and if yes, performs the
code after @code{if}; it treats n1 as zero/non-zero and uses @code{0<>} to
convert n2 into a canonical flag; the @code{and} then produces an fz,
which is consumed by the @code{if}.

You can use the all-bits-set feature of canonical flags and the bitwise
operation of the Boolean operations to avoid @code{if}s:

@example
: foo ( n1 -- n2 )
  0= if
    14
  else
    0
  endif ;
0 foo .
1 foo .

: foo ( n1 -- n2 )
  0= 14 and ;
0 foo .
1 foo .
@end example

@quotation Assignment
Write @code{min} without @code{if}.
@end quotation

For reference, see @ref{Boolean Flags}, @ref{Numeric comparison}, and
@ref{Bitwise operations}.


@node General Loops Tutorial, Counted loops Tutorial, Flags and Comparisons Tutorial, Tutorial
@section General Loops
@cindex loops, indefinite, tutorial

The endless loop is the most simple one:

@example
: endless ( -- )
  0 begin
    dup . 1+
  again ;
endless
@end example

Terminate this loop by pressing @kbd{Ctrl-C} (in Gforth).  @code{begin}
does nothing at run-time, @code{again} jumps back to @code{begin}.

A loop with one exit at any place looks like this:

@example
: log2 ( +n1 -- n2 )
\ logarithmus dualis of n1>0, rounded down to the next integer
  assert( dup 0> )
  2/ 0 begin
    over 0> while
      1+ swap 2/ swap
  repeat
  nip ;
7 log2 .
8 log2 .
@end example

At run-time @code{while} consumes a flag; if it is 0, execution
continues behind the @code{repeat}; if the flag is non-zero, execution
continues behind the @code{while}.  @code{Repeat} jumps back to
@code{begin}, just like @code{again}.

In Forth there are a number of combinations/abbreviations, like
@code{1+}.  However, @code{2/} is not one of them; it shifts its
argument right by one bit (arithmetic shift right), and viewed as
division that always rounds towards negative infinity (floored
division), like Gforth's @code{/} (since Gforth 0.7), but unlike
@code{/} in many other Forth systems.

@example
-5 2 / . \ -2 or -3
-5 2/ .  \ -3
@end example

@code{assert(} is no standard word, but you can get it on systems other
than Gforth by including @file{compat/assert.fs}.  You can see what it
does by trying

@example
0 log2 .
@end example

Here's a loop with an exit at the end:

@example
: log2 ( +n1 -- n2 )
\ logarithmus dualis of n1>0, rounded down to the next integer
  assert( dup 0 > )
  -1 begin
    1+ swap 2/ swap
    over 0 <=
  until
  nip ;
@end example

@code{Until} consumes a flag; if it is zero, execution continues at
the @code{begin}, otherwise after the @code{until}.

@quotation Assignment
Write a definition for computing the greatest common divisor.
@end quotation

Reference: @ref{Simple Loops}.


@node Counted loops Tutorial, Recursion Tutorial, General Loops Tutorial, Tutorial
@section Counted loops
@cindex loops, counted, tutorial

@example
: ^ ( n1 u -- n )
\ n = the uth power of n1
  1 swap 0 u+do
    over *
  loop
  nip ;
3 2 ^ .
4 3 ^ .
@end example

@code{U+do} (from @file{compat/loops.fs}, if your Forth system doesn't
have it) takes two numbers of the stack @code{( u3 u4 -- )}, and then
performs the code between @code{u+do} and @code{loop} for @code{u3-u4}
times (or not at all, if @code{u3-u4<0}).

You can see the stack effect design rules at work in the stack effect of
the loop start words: Since the start value of the loop is more
frequently constant than the end value, the start value is passed on
the top-of-stack.

You can access the counter of a counted loop with @code{i}:

@example
: fac ( u -- u! )
  1 swap 1+ 1 u+do
    i *
  loop ;
5 fac .
7 fac .
@end example

There is also @code{+do}, which expects signed numbers (important for
deciding whether to enter the loop).

@quotation Assignment
Write a definition for computing the nth Fibonacci number.
@end quotation

You can also use increments other than 1:

@example
: up2 ( n1 n2 -- )
  +do
    i .
  2 +loop ;
10 0 up2

: down2 ( n1 n2 -- )
  -do
    i .
  2 -loop ;
0 10 down2
@end example

Reference: @ref{Counted Loops}.


@node Recursion Tutorial, Leaving definitions or loops Tutorial, Counted loops Tutorial, Tutorial
@section Recursion
@cindex recursion tutorial

Usually the name of a definition is not visible in the definition; but
earlier definitions are usually visible:

@example
1 0 / . \ "Floating-point unidentified fault" in Gforth on some platforms
: / ( n1 n2 -- n )
  dup 0= if
    -10 throw \ report division by zero
  endif
  /           \ old version
;
1 0 /
@end example

For recursive definitions you can use @code{recursive} (non-standard) or
@code{recurse}:

@example
: fac1 ( n -- n! ) recursive
 dup 0> if
   dup 1- fac1 *
 else
   drop 1
 endif ;
7 fac1 .

: fac2 ( n -- n! )
 dup 0> if
   dup 1- recurse *
 else
   drop 1
 endif ;
8 fac2 .
@end example

@quotation Assignment
Write a recursive definition for computing the nth Fibonacci number.
@end quotation

Reference (including indirect recursion): @xref{Calls and returns}.


@node Leaving definitions or loops Tutorial, Return Stack Tutorial, Recursion Tutorial, Tutorial
@section Leaving definitions or loops
@cindex leaving definitions, tutorial
@cindex leaving loops, tutorial

@code{EXIT} exits the current definition right away.  For every counted
loop that is left in this way, an @code{UNLOOP} has to be performed
before the @code{EXIT}:

@c !! real examples
@example
: ...
 ... u+do
   ... if
     ... unloop exit
   endif
   ...
 loop
 ... ;
@end example

@code{LEAVE} leaves the innermost counted loop right away:

@example
: ...
 ... u+do
   ... if
     ... leave
   endif
   ...
 loop
 ... ;
@end example

@c !! example

Reference: @ref{Calls and returns}, @ref{Counted Loops}.


@node Return Stack Tutorial, Memory Tutorial, Leaving definitions or loops Tutorial, Tutorial
@section Return Stack
@cindex return stack tutorial

In addition to the data stack Forth also has a second stack, the return
stack; most Forth systems store the return addresses of procedure calls
there (thus its name).  Programmers can also use this stack:

@example
: foo ( n1 n2 -- )
 .s
 >r .s
 r@@ .
 >r .s
 r@@ .
 r> .
 r@@ .
 r> . ;
1 2 foo
@end example

@code{>r} takes an element from the data stack and pushes it onto the
return stack; conversely, @code{r>} moves an element from the return to
the data stack; @code{r@@} pushes a copy of the top of the return stack
on the data stack.

Forth programmers usually use the return stack for storing data
temporarily, if using the data stack alone would be too complex, and
factoring and locals are not an option:

@example
: 2swap ( x1 x2 x3 x4 -- x3 x4 x1 x2 )
 rot >r rot r> ;
@end example

The return address of the definition and the loop control parameters of
counted loops usually reside on the return stack, so you have to take
all items, that you have pushed on the return stack in a colon
definition or counted loop, from the return stack before the definition
or loop ends.  You cannot access items that you pushed on the return
stack outside some definition or loop within the definition of loop.

If you miscount the return stack items, this usually ends in a crash:

@example
: crash ( n -- )
  >r ;
5 crash
@end example

You cannot mix using locals and using the return stack (according to the
standard; Gforth has no problem).  However, they solve the same
problems, so this shouldn't be an issue.

@quotation Assignment
Can you rewrite any of the definitions you wrote until now in a better
way using the return stack?
@end quotation

Reference: @ref{Return stack}.


@node Memory Tutorial, Characters and Strings Tutorial, Return Stack Tutorial, Tutorial
@section Memory
@cindex memory access/allocation tutorial

You can create a global variable @code{v} with

@example
variable v ( -- addr )
@end example

@code{v} pushes the address of a cell in memory on the stack.  This
cell was reserved by @code{variable}.  You can use @code{!} (store) to
store values from the stack into this cell and @code{@@} (fetch) to
load the value from memory onto the stack:

@example
v .
5 v ! .s
v @@ .
@end example

You can see a raw dump of memory with @code{dump}:

@example
v 1 cells .s dump
@end example

@code{Cells ( n1 -- n2 )} gives you the number of bytes (or, more
generally, address units (aus)) that @code{n1 cells} occupy.  You can
also reserve more memory:

@example
create v2 20 cells allot
v2 20 cells dump
@end example

creates a variable-like word @code{v2} and reserves 20 uninitialized
cells; the address pushed by @code{v2} points to the start of these 20
cells (@pxref{CREATE}).  You can use address arithmetic to access
these cells:

@example
3 v2 5 cells + !
v2 20 cells dump
@end example

You can reserve and initialize memory with @code{,}:

@example
create v3
  5 , 4 , 3 , 2 , 1 ,
v3 @@ .
v3 cell+ @@ .
v3 2 cells + @@ .
v3 5 cells dump
@end example

@quotation Assignment
Write a definition @code{vsum ( addr u -- n )} that computes the sum of
@code{u} cells, with the first of these cells at @code{addr}, the next
one at @code{addr cell+} etc.
@end quotation

The difference between @code{variable} and @code{create} is that
@code{variable} allots a cell, and that you cannot allot additional
memory to a variable in standard Forth.

You can also reserve memory without creating a new word:

@example
here 10 cells allot .
here .
@end example

The first @code{here} pushes the start address of the memory area, the
second @code{here} the address after the dictionary area.  You should
store the start address somewhere, or you will have a hard time
finding the memory area again.

@code{Allot} manages dictionary memory.  The dictionary memory contains
the system's data structures for words etc. on Gforth and most other
Forth systems.  It is managed like a stack: You can free the memory that
you have just @code{allot}ed with

@example
-10 cells allot
here .
@end example

Note that you cannot do this if you have created a new word in the
meantime (because then your @code{allot}ed memory is no longer on the
top of the dictionary ``stack'').

Alternatively, you can use @code{allocate} and @code{free} which allow
freeing memory in any order:

@example
10 cells allocate throw .s
20 cells allocate throw .s
swap
free throw
free throw
@end example

The @code{throw}s deal with errors (e.g., out of memory).

And there is also a
@uref{https://www.complang.tuwien.ac.at/forth/garbage-collection.zip,
garbage collector}, which eliminates the need to @code{free} memory
explicitly.

Reference: @ref{Memory}.


@node Characters and Strings Tutorial, Alignment Tutorial, Memory Tutorial, Tutorial
@section Characters and Strings
@cindex strings tutorial
@cindex characters tutorial

On the stack characters take up a cell, like numbers.  In memory they
have their own size (one 8-bit byte on most systems), and therefore
require their own words for memory access:

@example
create v4 
  104 c, 97 c, 108 c, 108 c, 111 c,
v4 4 chars + c@@ .
v4 5 chars dump
@end example

The preferred representation of strings on the stack is @code{addr
u-count}, where @code{addr} is the address of the first character and
@code{u-count} is the number of characters in the string.

@example
v4 5 type
@end example

You get a string constant with

@example
s" hello, world" .s
type
@end example

Make sure you have a space between @code{s"} and the string; @code{s"}
is a normal Forth word and must be delimited with white space (try what
happens when you remove the space).

However, this interpretive use of @code{s"} is quite restricted: the
string exists only until the next call of @code{s"} (some Forth systems
keep more than one of these strings, but usually they still have a
limited lifetime).

@example
s" hello," s" world" .s
type
type
@end example

You can also use @code{s"} in a definition, and the resulting
strings then live forever (well, for as long as the definition):

@example
: foo s" hello," s" world" ;
foo .s
type
type
@end example

@quotation Assignment
@code{Emit ( c -- )} types @code{c} as character (not a number).
Implement @code{type ( addr u -- )}.
@end quotation

Reference: @ref{Memory Blocks}.


@node Alignment Tutorial, Floating Point Tutorial, Characters and Strings Tutorial, Tutorial
@section Alignment
@cindex alignment tutorial
@cindex memory alignment tutorial

On many processors cells have to be aligned in memory, if you want to
access them with @code{@@} and @code{!} (and even if the processor does
not require alignment, access to aligned cells is faster).

@code{Create} aligns @code{here} (i.e., the place where the next
allocation will occur, and that the @code{create}d word points to).
Likewise, the memory produced by @code{allocate} starts at an aligned
address.  Adding a number of @code{cells} to an aligned address produces
another aligned address.

However, address arithmetic involving @code{char+} and @code{chars} can
create an address that is not cell-aligned.  @code{Aligned ( addr --
a-addr )} produces the next aligned address:

@example
v3 char+ aligned .s @@ .
v3 char+ .s @@ .
@end example

Similarly, @code{align} advances @code{here} to the next aligned
address:

@example
create v5 97 c,
here .
align here .
1000 ,
@end example

Note that you should use aligned addresses even if your processor does
not require them, if you want your program to be portable.

Reference: @ref{Address arithmetic}.

@node Floating Point Tutorial, Files Tutorial, Alignment Tutorial, Tutorial
@section Floating Point
@cindex floating point tutorial
@cindex FP tutorial

Floating-point (FP) numbers and arithmetic in Forth works mostly as one
might expect, but there are a few things worth noting:

The first point is not specific to Forth, but so important and yet not
universally known that I mention it here: FP numbers are not reals.
Many properties (e.g., arithmetic laws) that reals have and that one
expects of all kinds of numbers do not hold for FP numbers.  If you
want to use FP computations, you should learn about their problems and
how to avoid them; a good starting point is @cite{David Goldberg,
@uref{https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html,What Every
Computer Scientist Should Know About Floating-Point Arithmetic}, ACM
Computing Surveys 23(1):5@minus{}48, March 1991}.

In Forth source code literal FP numbers need an exponent, e.g.,
@code{1e0}; this can also be written shorter as @code{1e}, longer as
@code{+1.0e+0}, and many variations in between.  The reason for this is
that, for historical reasons, Forth interprets a decimal point alone
(e.g., @code{1.}) as indicating a double-cell integer.  Examples:

@example
2e 2e f+ f.
@end example

Another requirement for literal FP numbers is that the current base is
decimal; with a hex base @code{1e} is interpreted as an integer.

Forth has a separate stack for FP numbers in conformance with
Forth-2012.  One advantage of this model is that cells are not in the
way when accessing FP values, and vice versa.  Forth has a set of
words for manipulating the FP stack: @code{fdup fswap fdrop fover
frot} and (non-standard) @code{fnip ftuck fpick}.

FP arithmetic words are prefixed with @code{F}.  There is the usual
set @code{f+ f- f* f/ f** fnegate} as well as a number of words for
other functions, e.g., @code{fsqrt fsin fln fmin}.  One word that you
might expect is @code{f=}; but @code{f=} is non-standard, because FP
computation results are usually inaccurate, so exact comparison is
usually a mistake, and one should use approximate comparison.
Unfortunately, @code{f~}, the standard word for that purpose, is not
well designed, so Gforth provides @code{f~abs} and @code{f~rel} as
well.

And of course there are words for accessing FP numbers in memory
(@code{f@@ f!}), and for address arithmetic (@code{floats float+
faligned}).  There are also variants of these words with an @code{sf}
and @code{df} prefix for accessing IEEE format single-precision and
double-precision numbers in memory; their main purpose is for
accessing external FP data (e.g., that has been read from or will be
written to a file).

Here is an example of a dot-product word and its use:

@example
: v* ( f_addr1 nstride1 f_addr2 nstride2 ucount -- r )
  >r swap 2swap swap 0e r> 0 ?DO
    dup f@@ over + 2swap dup f@@ f* f+ over + 2swap
  LOOP
  2drop 2drop ;

create v 1.23e f, 4.56e f, 7.89e f,

v 1 floats  v 1 floats  3  v* f.
@end example

@quotation Assignment
Write a program to solve a quadratic equation.  Then read @cite{Henry
G. Baker,
@uref{https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.111.4448&rep=rep1&type=pdf,You
Could Learn a Lot from a Quadratic}, ACM SIGPLAN Notices,
33(1):30@minus{}39, January 1998}, and see if you can improve your
program.  Finally, find a test case where the original and the
improved version produce different results.
@end quotation

Reference: @ref{Floating Point}; @ref{Floating point stack};
@ref{Number Conversion}; @ref{Memory Access}; @ref{Address
arithmetic}.

@node Files Tutorial, Interpretation and Compilation Semantics and Immediacy Tutorial, Floating Point Tutorial, Tutorial
@section Files
@cindex files tutorial

This section gives a short introduction into how to use files inside
Forth. It's broken up into five easy steps:

@enumerate 1
@item Open an ASCII text file for input
@item Open a file for output
@item Read input file until string matches (or some other condition is met)
@item Write some lines from input (modified or not) to output
@item Close the files.
@end enumerate

Reference: @ref{General files}.

@subsection Open file for input

@example
s" foo.in"  r/o open-file throw Value fd-in
@end example

@subsection Create file for output

@example
s" foo.out" w/o create-file throw Value fd-out
@end example

The available file modes are r/o for read-only access, r/w for
read-write access, and w/o for write-only access. You could open both
files with r/w, too, if you like. All file words return error codes; for
most applications, it's best to pass there error codes with @code{throw}
to the outer error handler.

If you want words for opening and assigning, define them as follows:

@example
0 Value fd-in
0 Value fd-out
: open-input ( addr u -- )  r/o open-file throw to fd-in ;
: open-output ( addr u -- )  w/o create-file throw to fd-out ;
@end example

Usage example:

@example
s" foo.in" open-input
s" foo.out" open-output
@end example

@subsection Scan file for a particular line

@example
256 Constant max-line
Create line-buffer  max-line 2 + allot

: scan-file ( addr u -- )
  begin
      line-buffer max-line fd-in read-line throw
  while
         >r 2dup line-buffer r> compare 0=
     until
  else
     drop
  then
  2drop ;
@end example

@code{read-line ( addr u1 fd -- u2 flag ior )} reads up to u1 bytes into
the buffer at addr, and returns the number of bytes read, a flag that is
false when the end of file is reached, and an error code.

@code{compare ( addr1 u1 addr2 u2 -- n )} compares two strings and
returns zero if both strings are equal. It returns a positive number if
the first string is lexically greater, a negative if the second string
is lexically greater.

We haven't seen this loop here; it has two exits. Since the @code{while}
exits with the number of bytes read on the stack, we have to clean up
that separately; that's after the @code{else}.

Usage example:

@example
s" The text I search is here" scan-file
@end example

@subsection Copy input to output

@example
: copy-file ( -- )
  begin
      line-buffer max-line fd-in read-line throw
  while
      line-buffer swap fd-out write-line throw
  repeat 
  drop ;
@end example
@c !! does not handle long lines, no newline at end of file

@subsection Close files

@example
fd-in close-file throw
fd-out close-file throw
@end example

Likewise, you can put that into definitions, too:

@example
: close-input ( -- )  fd-in close-file throw ;
: close-output ( -- )  fd-out close-file throw ;
@end example

@quotation Assignment
How could you modify @code{copy-file} so that it copies until a second line is
matched? Can you write a program that extracts a section of a text file,
given the line that starts and the line that terminates that section?
@end quotation

@node Interpretation and Compilation Semantics and Immediacy Tutorial, Execution Tokens Tutorial, Files Tutorial, Tutorial
@section Interpretation and Compilation Semantics and Immediacy
@cindex semantics tutorial
@cindex interpretation semantics tutorial
@cindex compilation semantics tutorial
@cindex immediate, tutorial

When a word is compiled, it behaves differently from being interpreted.
E.g., consider @code{+}:

@example
1 2 + .
: foo + ;
@end example

These two behaviours are known as compilation and interpretation
semantics.  For normal words (e.g., @code{+}), the compilation semantics
is to append the interpretation semantics to the currently defined word
(@code{foo} in the example above).  I.e., when @code{foo} is executed
later, the interpretation semantics of @code{+} (i.e., adding two
numbers) will be performed.

However, there are words with non-default compilation semantics, e.g.,
the control-flow words like @code{if}.  You can use @code{immediate} to
change the compilation semantics of the last defined word to be equal to
the interpretation semantics:

@example
: [FOO] ( -- )
 5 . ; immediate

[FOO]
: bar ( -- )
  [FOO] ;
bar
see bar
@end example

Two conventions to mark words with non-default compilation semantics are
names with brackets (more frequently used) and to write them all in
upper case (less frequently used).

For some words, such as @code{if}, using their interpretation
semantics is usually a mistake, so we mark them as @code{compile-only}, and you get a warning when you interpret them.

@example
: flip ( -- )
 6 . ; compile-only \ but not immediate
flip

: flop ( -- )
 flip ;
flop
@end example

In this example, first the interpretation semantics of @code{flip} is
used (and you get a warning); the second use of @code{flip} uses the
compilation semantics (and you get no warning).  You can also see in
this example that compile-only is a property that is evaluated at text
interpretation time, not at run-time.

The text interpreter has two states: in interpret state, it performs the
interpretation semantics of words it encounters; in compile state, it
performs the compilation semantics of these words.

Among other things, @code{:} switches into compile state, and @code{;}
switches back to interpret state.  They contain the factors @code{]}
(switch to compile state) and @code{[} (switch to interpret state), that
do nothing but switch the state.

@example
: xxx ( -- )
  [ 5 . ]
;

xxx
see xxx
@end example

These brackets are also the source of the naming convention mentioned
above.

Reference: @ref{Interpretation and Compilation Semantics}.


@node Execution Tokens Tutorial, Exceptions Tutorial, Interpretation and Compilation Semantics and Immediacy Tutorial, Tutorial
@section Execution Tokens
@cindex execution tokens tutorial
@cindex XT tutorial

@code{' word} gives you the execution token (XT) of a word.  The XT is a
cell representing the interpretation semantics of a word.  You can
execute this semantics with @code{execute}:

@example
' + .s
1 2 rot execute .
@end example

The XT is similar to a function pointer in C.  However, parameter
passing through the stack makes it a little more flexible:

@example
: map-array ( ... addr u xt -- ... )
\ executes xt ( ... x -- ... ) for every element of the array starting
\ at addr and containing u elements
  @{ xt @}
  cells over + swap ?do
    i @@ xt execute
  1 cells +loop ;

create a 3 , 4 , 2 , -1 , 4 ,
a 5 ' . map-array .s
0 a 5 ' + map-array .
s" max-n" environment? drop .s
a 5 ' min map-array .
@end example

You can use map-array with the XTs of words that consume one element
more than they produce.  In theory you can also use it with other XTs,
but the stack effect then depends on the size of the array, which is
hard to understand.

Since XTs are cell-sized, you can store them in memory and manipulate
them on the stack like other cells.  You can also compile the XT into a
word with @code{compile,}:

@example
: foo1 ( n1 n2 -- n )
   [ ' + compile, ] ;
see foo1
@end example

This is non-standard, because @code{compile,} has no compilation
semantics in the standard, but it works in good Forth systems.  For the
broken ones, use

@example
: [compile,] compile, ; immediate

: foo1 ( n1 n2 -- n )
   [ ' + ] [compile,] ;
see foo1
@end example

@code{'} is a word with default compilation semantics; it parses the
next word when its interpretation semantics are executed, not during
compilation:

@example
: foo ( -- xt )
  ' ;
see foo
: bar ( ... "word" -- ... )
  ' execute ;
see bar
1 2 bar + .
@end example

You often want to parse a word during compilation and compile its XT so
it will be pushed on the stack at run-time.  @code{[']} does this:

@example
: xt-+ ( -- xt )
  ['] + ;
see xt-+
1 2 xt-+ execute .
@end example

Many programmers tend to see @code{'} and the word it parses as one
unit, and expect it to behave like @code{[']} when compiled, and are
confused by the actual behaviour.  If you are, just remember that the
Forth system just takes @code{'} as one unit and has no idea that it is
a parsing word (attempts to convenience programmers in this issue have
usually resulted in even worse pitfalls, see
@uref{https://www.complang.tuwien.ac.at/papers/ertl98.ps.gz,
@code{State}-smartness---Why it is evil and How to Exorcise it}).

Note that the state of the interpreter does not come into play when
creating and executing XTs.  I.e., even when you execute @code{'} in
compile state, it still gives you the interpretation semantics.  And
whatever that state is, @code{execute} performs the semantics
represented by the XT (i.e., for XTs produced with @code{'} the
interpretation semantics).

Reference: @ref{Tokens for Words}.


@node Exceptions Tutorial, Defining Words Tutorial, Execution Tokens Tutorial, Tutorial
@section Exceptions
@cindex exceptions tutorial

@code{throw ( n -- )} causes an exception unless n is zero.

@example
100 throw .s
0 throw .s
@end example

@code{catch ( ... xt -- ... n )} behaves similar to @code{execute}, but
it catches exceptions and pushes the number of the exception on the
stack (or 0, if the xt executed without exception).  If there was an
exception, the stacks have the same depth as when entering @code{catch}:

@example
.s
3 0 ' / catch .s
3 2 ' / catch .s
@end example

@quotation Assignment
Try the same with @code{execute} instead of @code{catch}.
@end quotation

@code{Throw} always jumps to the dynamically next enclosing
@code{catch}, even if it has to leave several call levels to achieve
this:

@example
: foo 100 throw ;
: foo1 foo ." after foo" ;
: bar ['] foo1 catch ;
bar .
@end example

It is often important to restore a value upon leaving a definition, even
if the definition is left through an exception.  You can ensure this
like this:

@example
: ...
   save-x
   ['] word-changing-x catch ( ... n )
   restore-x
   ( ... n ) throw ;
@end example

However, this is still not safe against, e.g., the user pressing
@kbd{Ctrl-C} when execution is between the @code{catch} and
@code{restore-x}.

Gforth provides an alternative exception handling syntax that is safe
against such cases: @code{try ... restore ... endtry}.  If the code
between @code{try} and @code{endtry} has an exception, the stack
depths are restored, the exception number is pushed on the stack, and
the execution continues right after @code{restore}.

The safer equivalent to the restoration code above is

@example
: ...
  save-x
  try
    word-changing-x 0
  restore
    restore-x
  endtry
  throw ;
@end example

Reference: @ref{Exception Handling}.


@node Defining Words Tutorial, Arrays and Records Tutorial, Exceptions Tutorial, Tutorial
@section Defining Words
@cindex defining words tutorial
@cindex does> tutorial
@cindex create...does> tutorial

@c before semantics?

@code{:}, @code{create}, and @code{variable} are definition words: They
define other words.  @code{Constant} is another definition word:

@example
5 constant foo
foo .
@end example

You can also use the prefixes @code{2} (double-cell) and @code{f}
(floating point) with @code{variable} and @code{constant}.

You can also define your own defining words.  E.g.:

@example
: variable ( "name" -- )
  create 0 , ;
@end example

You can also define defining words that create words that do something
other than just producing their address:

@example
: constant ( n "name" -- )
  create ,
does> ( -- n )
  ( addr ) @@ ;

5 constant foo
foo .
@end example

The definition of @code{constant} above ends at the @code{does>}; i.e.,
@code{does>} replaces @code{;}, but it also does something else: It
changes the last defined word such that it pushes the address of the
body of the word and then performs the code after the @code{does>}
whenever it is called.

In the example above, @code{constant} uses @code{,} to store 5 into the
body of @code{foo}.  When @code{foo} executes, it pushes the address of
the body onto the stack, then (in the code after the @code{does>})
fetches the 5 from there.

The stack comment near the @code{does>} reflects the stack effect of the
defined word, not the stack effect of the code after the @code{does>}
(the difference is that the code expects the address of the body that
the stack comment does not show).

You can use these definition words to do factoring in cases that involve
(other) definition words.  E.g., a field offset is always added to an
address.  Instead of defining

@example
2 cells constant offset-field1
@end example

and using this like

@example
( addr ) offset-field1 +
@end example

you can define a definition word

@example
: simple-field ( n "name" -- )
  create ,
does> ( n1 -- n1+n )
  ( addr ) @@ + ;
@end example

Definition and use of field offsets now look like this:

@example
2 cells simple-field field1
create mystruct 4 cells allot
mystruct .s field1 .s drop
@end example

If you want to do something with the word without performing the code
after the @code{does>}, you can access the body of a @code{create}d word
with @code{>body ( xt -- addr )}:

@example
: value ( n "name" -- )
  create ,
does> ( -- n1 )
  @@ ;
: to ( n "name" -- )
  ' >body ! ;

5 value foo
foo .
7 to foo
foo .
@end example

@quotation Assignment
Define @code{defer ( "name" -- )}, which creates a word that stores an
XT (at the start the XT of @code{abort}), and upon execution
@code{execute}s the XT.  Define @code{is ( xt "name" -- )} that stores
@code{xt} into @code{name}, a word defined with @code{defer}.  Indirect
recursion is one application of @code{defer}.
@end quotation

Reference: @ref{User-defined Defining Words}.


@node Arrays and Records Tutorial, POSTPONE Tutorial, Defining Words Tutorial, Tutorial
@section Arrays and Records
@cindex arrays tutorial
@cindex records tutorial
@cindex structs tutorial

Forth has no standard words for defining arrays, but you can build
them yourself based on address arithmetic.  You can also define words
for defining arrays and records (@pxref{Defining Words Tutorial,,
Defining Words}).

One of the first projects a Forth newcomer sets out upon when learning
about defining words is an array defining word (possibly for
n-dimensional arrays).  Go ahead and do it, I did it, too; you will
learn something from it.  However, don't be disappointed when you later
learn that you have little use for these words (inappropriate use would
be even worse).  I have not found a set of useful array words yet;
the needs are just too diverse, and named, global arrays (the result of
naive use of defining words) are often not flexible enough (e.g.,
consider how to pass them as parameters).  Another such project is a set
of words to help dealing with strings.

On the other hand, there is a useful set of record words, and it has
been defined in @file{compat/struct.fs}; these words are predefined in
Gforth.  They are explained in depth elsewhere in this manual (see
@pxref{Structures}).  The @code{simple-field} example above is
simplified variant of fields in this package.


@node POSTPONE Tutorial, Literal Tutorial, Arrays and Records Tutorial, Tutorial
@section @code{POSTPONE}
@cindex postpone tutorial

You can compile the compilation semantics (instead of compiling the
interpretation semantics) of a word with @code{POSTPONE}:

@example
: MY-+ ( Compilation: -- ; Run-time of compiled code: n1 n2 -- n )
 POSTPONE + ; immediate
: foo ( n1 n2 -- n )
 MY-+ ;
1 2 foo .
see foo
@end example

During the definition of @code{foo} the text interpreter performs the
compilation semantics of @code{MY-+}, which performs the compilation
semantics of @code{+}, i.e., it compiles @code{+} into @code{foo}.

This example also displays separate stack comments for the compilation
semantics and for the stack effect of the compiled code.  For words with
default compilation semantics these stack effects are usually not
displayed; the stack effect of the compilation semantics is always
@code{( -- )} for these words, the stack effect for the compiled code is
the stack effect of the interpretation semantics.

Note that the state of the interpreter does not come into play when
performing the compilation semantics in this way.  You can also perform
it interpretively, e.g.:

@example
: foo2 ( n1 n2 -- n )
 [ MY-+ ] ;
1 2 foo .
see foo
@end example

However, there are some broken Forth systems where this does not always
work, and therefore this practice was been declared non-standard in
1999.
@c !! repair.fs

Here is another example for using @code{POSTPONE}:

@example
: MY-- ( Compilation: -- ; Run-time of compiled code: n1 n2 -- n )
 POSTPONE negate POSTPONE + ; immediate compile-only
: bar ( n1 n2 -- n )
  MY-- ;
2 1 bar .
see bar
@end example

You can define @code{ENDIF} in this way:

@example
: ENDIF ( Compilation: orig -- )
  POSTPONE then ; immediate
@end example

@quotation Assignment
Write @code{MY-2DUP} that has compilation semantics equivalent to
@code{2dup}, but compiles @code{over over}.
@end quotation

@c !! @xref{Macros} for reference


@node Literal Tutorial, Advanced macros Tutorial, POSTPONE Tutorial, Tutorial
@section @code{Literal}
@cindex literal tutorial

You cannot @code{POSTPONE} numbers:

@example
: [FOO] POSTPONE 500 ; immediate
@end example

Instead, you can use @code{LITERAL (compilation: n --; run-time: -- n )}:

@example
: [FOO] ( compilation: --; run-time: -- n )
  500 POSTPONE literal ; immediate

: flip [FOO] ;
flip .
see flip
@end example

@code{LITERAL} consumes a number at compile-time (when it's compilation
semantics are executed) and pushes it at run-time (when the code it
compiled is executed).  A frequent use of @code{LITERAL} is to compile a
number computed at compile time into the current word:

@example
: bar ( -- n )
  [ 2 2 + ] literal ;
see bar
@end example

@quotation Assignment
Write @code{]L} which allows writing the example above as @code{: bar (
-- n ) [ 2 2 + ]L ;}
@end quotation

@c !! @xref{Macros} for reference


@node Advanced macros Tutorial, Compilation Tokens Tutorial, Literal Tutorial, Tutorial
@section Advanced macros
@cindex macros, advanced tutorial
@cindex run-time code generation, tutorial

Reconsider @code{map-array} from @ref{Execution Tokens Tutorial,,
Execution Tokens}.  It frequently performs @code{execute}, a relatively
expensive operation in some Forth implementations.  You can use
@code{compile,} and @code{POSTPONE} to eliminate these @code{execute}s
and produce a word that contains the word to be performed directly:

@c use ]] ... [[
@example
: compile-map-array ( compilation: xt -- ; run-time: ... addr u -- ... )
\ at run-time, execute xt ( ... x -- ... ) for each element of the
\ array beginning at addr and containing u elements
  @{ xt @}
  POSTPONE cells POSTPONE over POSTPONE + POSTPONE swap POSTPONE ?do
    POSTPONE i POSTPONE @@ xt compile,
  1 cells POSTPONE literal POSTPONE +loop ;

: sum-array ( addr u -- n )
 0 rot rot [ ' + compile-map-array ] ;
see sum-array
a 5 sum-array .
@end example

You can use the full power of Forth for generating the code; here's an
example where the code is generated in a loop:

@example
: compile-vmul-step ( compilation: n --; run-time: n1 addr1 -- n2 addr2 )
\ n2=n1+(addr1)*n, addr2=addr1+cell
  POSTPONE tuck POSTPONE @@
  POSTPONE literal POSTPONE * POSTPONE +
  POSTPONE swap POSTPONE cell+ ;

: compile-vmul ( compilation: addr1 u -- ; run-time: addr2 -- n )
\ n=v1*v2 (inner product), where the v_i are represented as addr_i u
  0 postpone literal postpone swap
  [ ' compile-vmul-step compile-map-array ]
  postpone drop ;
see compile-vmul

: a-vmul ( addr -- n )
\ n=a*v, where v is a vector that's as long as a and starts at addr
 [ a 5 compile-vmul ] ;
see a-vmul
a a-vmul .
@end example

This example uses @code{compile-map-array} to show off, but you could
also use @code{map-array} instead (try it now!).

You can use this technique for efficient multiplication of large
matrices.  In matrix multiplication, you multiply every row of one
matrix with every column of the other matrix.  You can generate the code
for one row once, and use it for every column.  The only downside of
this technique is that it is cumbersome to recover the memory consumed
by the generated code when you are done (and in more complicated cases
it is not possible portably).

@c !! @xref{Macros} for reference


@node Compilation Tokens Tutorial, Wordlists and Search Order Tutorial, Advanced macros Tutorial, Tutorial
@section Compilation Tokens
@cindex compilation tokens, tutorial
@cindex CT, tutorial

This section is Gforth-specific.  You can skip it.

@code{' word compile,} compiles the interpretation semantics.  For words
with default compilation semantics this is the same as performing the
compilation semantics.  To represent the compilation semantics of other
words (e.g., words like @code{if} that have no interpretation
semantics), Gforth has the concept of a compilation token (CT,
consisting of two cells), and words @code{comp'} and @code{[comp']}.
You can perform the compilation semantics represented by a CT with
@code{execute}:

@example
: foo2 ( n1 n2 -- n )
   [ comp' + execute ] ;
see foo
@end example

You can compile the compilation semantics represented by a CT with
@code{postpone,}:

@example
: foo3 ( -- )
  [ comp' + postpone, ] ;
see foo3
@end example

@code{[ comp' word postpone, ]} is equivalent to @code{POSTPONE word}.
@code{comp'} is particularly useful for words that have no
interpretation semantics:

@example
' if
comp' if .s 2drop
@end example

Reference: @ref{Tokens for Words}.


@node Wordlists and Search Order Tutorial,  , Compilation Tokens Tutorial, Tutorial
@section Wordlists and Search Order
@cindex wordlists tutorial
@cindex search order, tutorial

The dictionary is not just a memory area that allows you to allocate
memory with @code{allot}, it also contains the Forth words, arranged in
several wordlists.  When searching for a word in a wordlist,
conceptually you start searching at the youngest and proceed towards
older words (in reality most systems nowadays use hash-tables); i.e., if
you define a word with the same name as an older word, the new word
shadows the older word.

Which wordlists are searched in which order is determined by the search
order.  You can display the search order with @code{order}.  It displays
first the search order, starting with the wordlist searched first, then
it displays the wordlist that will contain newly defined words.

You can create a new, empty wordlist with @code{wordlist ( -- wid )}:

@example
wordlist constant mywords
@end example

@code{Set-current ( wid -- )} sets the wordlist that will contain newly
defined words (the @emph{current} wordlist):

@example
mywords set-current
order
@end example

Gforth does not display a name for the wordlist in @code{mywords}
because this wordlist was created anonymously with @code{wordlist}.

You can get the current wordlist with @code{get-current ( -- wid)}.  If
you want to put something into a specific wordlist without overall
effect on the current wordlist, this typically looks like this:

@example
get-current mywords set-current ( wid )
create someword
( wid ) set-current
@end example

You can write the search order with @code{set-order ( wid1 .. widn n --
)} and read it with @code{get-order ( -- wid1 .. widn n )}.  The first
searched wordlist is topmost.

@example
get-order mywords swap 1+ set-order
order
@end example

Yes, the order of wordlists in the output of @code{order} is reversed
from stack comments and the output of @code{.s} and thus unintuitive.

@quotation Assignment
Define @code{>order ( wid -- )} which adds @code{wid} as first searched
wordlist to the search order.  Define @code{previous ( -- )}, which
removes the first searched wordlist from the search order.  Experiment
with boundary conditions (you will see some crashes or situations that
are hard or impossible to leave).
@end quotation

The search order is a powerful foundation for providing features similar
to Modula-2 modules and C++ namespaces.  However, trying to modularize
programs in this way has disadvantages for debugging and reuse/factoring
that overcome the advantages in my experience (I don't do huge projects,
though).  These disadvantages are not so clear in other
languages/programming environments, because these languages are not so
strong in debugging and reuse.

@c !! example

Reference: @ref{Word Lists}.

@c ******************************************************************
@node Introduction, Literals in source code, Tutorial, Top
@comment node-name,     next,           previous, up
@chapter An Introduction to Standard Forth
@cindex Forth - an introduction

The difference of this chapter from the Tutorial (@pxref{Tutorial}) is
that it is slower-paced in its examples, but uses them to dive deep into
explaining Forth internals (not covered by the Tutorial).  Apart from
that, this chapter covers far less material.  It is suitable for reading
without using a computer.

The primary purpose of this manual is to document Gforth. However, since
Forth is not a widely-known language and there is a lack of up-to-date
teaching material, it seems worthwhile to provide some introductory
material.  For other sources of Forth-related
information, see @ref{Forth-related information}.

The examples in this section should work on any Standard Forth; the
output shown was produced using Gforth. Each example attempts to
reproduce the exact output that Gforth produces. If you try out the
examples (and you should), what you should type is shown @kbd{like this}
and Gforth's response is shown @code{like this}. The single exception is
that, where the example shows @key{RET} it means that you should
press the ``carriage return'' key. Unfortunately, some output formats for
this manual cannot show the difference between @kbd{this} and
@code{this} which will make trying out the examples harder (but not
impossible).

Forth is an unusual language. It provides an interactive development
environment which includes both an interpreter and compiler. Forth
programming style encourages you to break a problem down into many
@cindex factoring
small fragments (@dfn{factoring}), and then to develop and test each
fragment interactively. Forth advocates assert that breaking the
edit-compile-test cycle used by conventional programming languages can
lead to great productivity improvements.

@menu
* Introducing the Text Interpreter::
* Stacks and Postfix notation::
* Your first definition::
* How does that work?::
* Forth is written in Forth::
* Review - elements of a Forth system::
* Where to go next::
* Exercises::
@end menu

@comment ----------------------------------------------
@node Introducing the Text Interpreter, Stacks and Postfix notation, Introduction, Introduction
@section Introducing the Text Interpreter
@cindex text interpreter
@cindex outer interpreter

@c IMO this is too detailed and the pace is too slow for
@c an introduction.  If you know German, take a look at
@c https://www.complang.tuwien.ac.at/anton/lvas/skriptum-stack.html 
@c to see how I do it - anton 

@c nac-> Where I have accepted your comments 100% and modified the text
@c accordingly, I have deleted your comments. Elsewhere I have added a
@c response like this to attempt to rationalise what I have done. Of
@c course, this is a very clumsy mechanism for something that would be
@c done far more efficiently over a beer. Please delete any dialogue
@c you consider closed.

When you invoke the Forth image, you will see a startup banner printed
and nothing else (if you have Gforth installed on your system, try
invoking it now, by typing @kbd{gforth@key{RET}}). Forth is now running
its command line interpreter, which is called the @dfn{Text Interpreter}
(also known as the @dfn{Outer Interpreter}).  (You will learn a lot
about the text interpreter as you read through this chapter, for more
detail @pxref{The Text Interpreter}).

Although it's not obvious, Forth is actually waiting for your
input. Type a number and press the @key{RET} key:

@example
@kbd{45@key{RET}}  ok
@end example

Rather than give you a prompt to invite you to input something, the text
interpreter prints a status message @i{after} it has processed a line
of input. The status message in this case (``@code{ ok}'' followed by
carriage-return) indicates that the text interpreter was able to process
all of your input successfully. Now type something illegal:

@example
@kbd{qwer341@key{RET}}
*the terminal*:2: Undefined word
>>>qwer341<<<
Backtrace:
$2A95B42A20 throw 
$2A95B57FB8 no.extensions 
@end example

The exact text, other than the ``Undefined word'' may differ slightly
on your system, but the effect is the same; when the text interpreter
detects an error, it discards any remaining text on a line, resets
certain internal state and prints an error message. For a detailed
description of error messages see @ref{Error messages}.

The text interpreter waits for you to press carriage-return, and then
processes your input line. Starting at the beginning of the line, it
breaks the line into groups of characters separated by spaces. For each
group of characters in turn, it makes two attempts to do something:

@itemize @bullet
@item
@cindex name dictionary
It tries to treat it as a command. It does this by searching a @dfn{name
dictionary}. If the group of characters matches an entry in the name
dictionary, the name dictionary provides the text interpreter with
information that allows the text interpreter to perform some actions. In
Forth jargon, we say that the group
@cindex word
@cindex definition
@cindex execution token
@cindex xt
of characters names a @dfn{word}, that the dictionary search returns an
@dfn{execution token (xt)} corresponding to the @dfn{definition} of the
word, and that the text interpreter executes the xt. Often, the terms
@dfn{word} and @dfn{definition} are used interchangeably.
@item
If the text interpreter fails to find a match in the name dictionary, it
tries to treat the group of characters as a number in the current number
base (when you start up Forth, the current number base is base 10). If
the group of characters legitimately represents a number, the text
interpreter pushes the number onto a stack (we'll learn more about that
in the next section).
@end itemize

If the text interpreter is unable to do either of these things with any
group of characters, it discards the group of characters and the rest of
the line, then prints an error message. If the text interpreter reaches
the end of the line without error, it prints the status message ``@code{ ok}''
followed by carriage-return.

This is the simplest command we can give to the text interpreter:

@example
@key{RET}  ok
@end example

The text interpreter did everything we asked it to do (nothing) without
an error, so it said that everything is ``@code{ ok}''. Try a slightly longer
command:

@example
@kbd{12 dup fred dup@key{RET}}
*the terminal*:3: Undefined word
12 dup >>>fred<<< dup
Backtrace:
$2A95B42A20 throw 
$2A95B57FB8 no.extensions 
@end example

When you press the carriage-return key, the text interpreter starts to
work its way along the line:

@itemize @bullet
@item
When it gets to the space after the @code{2}, it takes the group of
characters @code{12} and looks them up in the name
dictionary@footnote{We can't tell if it found them or not, but assume
for now that it did not}. There is no match for this group of characters
in the name dictionary, so it tries to treat them as a number. It is
able to do this successfully, so it puts the number, 12, ``on the stack''
(whatever that means).
@item
The text interpreter resumes scanning the line and gets the next group
of characters, @code{dup}. It looks it up in the name dictionary and
(you'll have to take my word for this) finds it, and executes the word
@code{dup} (whatever that means).
@item
Once again, the text interpreter resumes scanning the line and gets the
group of characters @code{fred}. It looks them up in the name
dictionary, but can't find them. It tries to treat them as a number, but
they don't represent any legal number.
@end itemize

At this point, the text interpreter gives up and prints an error
message. The error message shows exactly how far the text interpreter
got in processing the line. In particular, it shows that the text
interpreter made no attempt to do anything with the final character
group, @code{dup}, even though we have good reason to believe that the
text interpreter would have no problem looking that word up and
executing it a second time.


@comment ----------------------------------------------
@node Stacks and Postfix notation, Your first definition, Introducing the Text Interpreter, Introduction
@section Stacks, postfix notation and parameter passing
@cindex text interpreter
@cindex outer interpreter

In procedural programming languages (like C and Pascal), the
building-block of programs is the @dfn{function} or @dfn{procedure}. These
functions or procedures are called with @dfn{explicit parameters}. For
example, in C we might write:

@example
total = total + new_volume(length,height,depth);
@end example

@noindent
where new_volume is a function-call to another piece of code, and total,
length, height and depth are all variables. length, height and depth are
parameters to the function-call.

In Forth, the equivalent of the function or procedure is the
@dfn{definition} and parameters are implicitly passed between
definitions using a shared stack that is visible to the
programmer. Although Forth does support variables, the existence of the
stack means that they are used far less often than in most other
programming languages. When the text interpreter encounters a number, it
will place (@dfn{push}) it on the stack. There are several stacks (the
actual number is implementation-dependent ...) and the particular stack
used for any operation is implied unambiguously by the operation being
performed. The stack used for all integer operations is called the @dfn{data
stack} and, since this is the stack used most commonly, references to
``the data stack'' are often abbreviated to ``the stack''.

The stacks have a last-in, first-out (LIFO) organisation. If you type:

@example
@kbd{1 2 3@key{RET}}  ok
@end example

Then this instructs the text interpreter to placed three numbers on the
(data) stack. An analogy for the behaviour of the stack is to take a
pack of playing cards and deal out the ace (1), 2 and 3 into a pile on
the table. The 3 was the last card onto the pile (``last-in'') and if
you take a card off the pile then, unless you're prepared to fiddle a
bit, the card that you take off will be the 3 (``first-out''). The
number that will be first-out of the stack is called the @dfn{top of
stack}, which
@cindex TOS definition
is often abbreviated to @dfn{TOS}.

To understand how parameters are passed in Forth, consider the
behaviour of the definition @code{+} (pronounced ``plus''). You will not
be surprised to learn that this definition performs addition. More
precisely, it adds two numbers together and produces a result. Where does
it get the two numbers from? It takes the top two numbers off the
stack. Where does it place the result? On the stack. You can act out the
behaviour of @code{+} with your playing cards like this:

@itemize @bullet
@item
Pick up two cards from the stack on the table
@item
Stare at them intently and ask yourself ``what @i{is} the sum of these two
numbers''
@item
Decide that the answer is 5
@item
Shuffle the two cards back into the pack and find a 5
@item
Put a 5 on the remaining ace that's on the table.
@end itemize

If you don't have a pack of cards handy but you do have Forth running,
you can use the definition @code{.s} to show the current state of the stack,
without affecting the stack. Type:

@example
@kbd{clearstacks 1 2 3@key{RET}} ok
@kbd{.s@key{RET}} <3> 1 2 3  ok
@end example

The text interpreter looks up the word @code{clearstacks} and executes
it; it tidies up the stacks (data and floating point stack) and
removes any entries that may have been left on them by earlier
examples. The text interpreter pushes each of the three numbers in
turn onto the stack. Finally, the text interpreter looks up the word
@code{.s} and executes it. The effect of executing @code{.s} is to
print the ``<3>'' (the total number of items on the stack) followed by
a list of all the items on the stack; the item on the far right-hand
side is the TOS.

You can now type:

@example
@kbd{+ .s@key{RET}} <2> 1 5  ok
@end example

@noindent
which is correct; there are now 2 items on the stack and the result of
the addition is 5.

If you're playing with cards, try doing a second addition: pick up the
two cards, work out that their sum is 6, shuffle them into the pack,
look for a 6 and place that on the table. You now have just one item on
the stack. What happens if you try to do a third addition? Pick up the
first card, pick up the second card -- ah! There is no second card. This
is called a @dfn{stack underflow} and consitutes an error. If you try to
do the same thing with Forth it often reports an error (probably a Stack
Underflow or an Invalid Memory Address error).

The opposite situation to a stack underflow is a @dfn{stack overflow},
which simply accepts that there is a finite amount of storage space
reserved for the stack. To stretch the playing card analogy, if you had
enough packs of cards and you piled the cards up on the table, you would
eventually be unable to add another card; you'd hit the ceiling. Gforth
allows you to set the maximum size of the stacks. In general, the only
time that you will get a stack overflow is because a definition has a
bug in it and is generating data on the stack uncontrollably.

There's one final use for the playing card analogy. If you model your
stack using a pack of playing cards, the maximum number of items on
your stack will be 52 (I assume you didn't use the Joker). The maximum
@i{value} of any item on the stack is 13 (the King). In fact, the only
possible numbers are positive integer numbers 1 through 13; you can't
have (for example) 0 or 27 or 3.52 or -2. If you change the way you
think about some of the cards, you can accommodate different
numbers. For example, you could think of the Jack as representing 0,
the Queen as representing -1 and the King as representing -2. Your
@i{range} remains unchanged (you can still only represent a total of 13
numbers) but the numbers that you can represent are -2 through 10.

In that analogy, the limit was the amount of information that a single
stack entry could hold, and Forth has a similar limit. In Forth, the
size of a stack entry is called a @dfn{cell}. The actual size of a cell is
implementation dependent and affects the maximum value that a stack
entry can hold. A Standard Forth provides a cell size of at least
16-bits, and most desktop systems use a cell size of 32-bits.

Forth does not do any type checking for you, so you are free to
manipulate and combine stack items in any way you wish. A convenient way
of treating stack items is as 2's complement signed integers, and that
is what Standard words like @code{+} do. Therefore you can type:

@example
@kbd{-5 12 + .s@key{RET}} <1> 7  ok
@end example

If you use numbers and definitions like @code{+} in order to turn Forth
into a great big pocket calculator, you will realise that it's rather
different from a normal calculator. Rather than typing 2 + 3 = you had
to type 2 3 + (ignore the fact that you had to use @code{.s} to see the
result). The terminology used to describe this difference is to say that
your calculator uses @dfn{Infix Notation} (parameters and operators are
mixed) whilst Forth uses @dfn{Postfix Notation} (parameters and
operators are separate), also called @dfn{Reverse Polish Notation}.

Whilst postfix notation might look confusing to begin with, it has
several important advantages:

@itemize @bullet
@item
it is unambiguous
@item
it is more concise
@item
it fits naturally with a stack-based system
@end itemize

To examine these claims in more detail, consider these sums:

@example
6 + 5 * 4 =
4 * 5 + 6 =
@end example

If you're just learning maths or your maths is very rusty, you will
probably come up with the answer 44 for the first and 26 for the
second. If you are a bit of a whizz at maths you will remember the
@i{convention} that multiplication takes precendence over addition, and
you'd come up with the answer 26 both times. To explain the answer 26
to someone who got the answer 44, you'd probably rewrite the first sum
like this:

@example
6 + (5 * 4) =
@end example

If what you really wanted was to perform the addition before the
multiplication, you would have to use parentheses to force it.

If you did the first two sums on a pocket calculator you would probably
get the right answers, unless you were very cautious and entered them using
these keystroke sequences:

6 + 5 = * 4 =
4 * 5 = + 6 =

Postfix notation is unambiguous because the order that the operators
are applied is always explicit; that also means that parentheses are
never required. The operators are @i{active} (the act of quoting the
operator makes the operation occur) which removes the need for ``=''.

The sum 6 + 5 * 4 can be written (in postfix notation) in two
equivalent ways:

@example
6 5 4 * +      or:
5 4 * 6 +
@end example

An important thing that you should notice about this notation is that
the @i{order} of the numbers does not change; if you want to subtract
2 from 10 you type @code{10 2 -}.

The reason that Forth uses postfix notation is very simple to explain: it
makes the implementation extremely simple, and it follows naturally from
using the stack as a mechanism for passing parameters. Another way of
thinking about this is to realise that all Forth definitions are
@i{active}; they execute as they are encountered by the text
interpreter. The result of this is that the syntax of Forth is trivially
simple.



@comment ----------------------------------------------
@node Your first definition, How does that work?, Stacks and Postfix notation, Introduction
@section Your first Forth definition
@cindex first definition

Until now, the examples we've seen have been trivial; we've just been
using Forth as a bigger-than-pocket calculator. Also, each calculation
we've shown has been a ``one-off'' -- to repeat it we'd need to type it in
again@footnote{That's not quite true. If you press the up-arrow key on
your keyboard you should be able to scroll back to any earlier command,
edit it and re-enter it.} In this section we'll see how to add new
words to Forth's vocabulary.

The easiest way to create a new word is to use a @dfn{colon
definition}. We'll define a few and try them out before worrying too
much about how they work. Try typing in these examples; be careful to
copy the spaces accurately:

@example
: add-two 2 + . ;
: greet ." Hello and welcome" ;
: demo 5 add-two ;
@end example

@noindent
Now try them out:

@example
@kbd{greet@key{RET}} Hello and welcome  ok
@kbd{greet greet@key{RET}} Hello and welcomeHello and welcome  ok
@kbd{4 add-two@key{RET}} 6  ok
@kbd{demo@key{RET}} 7  ok
@kbd{9 greet demo add-two@key{RET}} Hello and welcome7 11  ok
@end example

The first new thing that we've introduced here is the pair of words
@code{:} and @code{;}. These are used to start and terminate a new
definition, respectively. The first word after the @code{:} is the name
for the new definition.

As you can see from the examples, a definition is built up of words that
have already been defined; Forth makes no distinction between
definitions that existed when you started the system up, and those that
you define yourself.

The examples also introduce the words @code{.} (dot), @code{."}
(dot-quote) and @code{dup} (dewp). Dot takes the value from the top of
the stack and displays it. It's like @code{.s} except that it only
displays the top item of the stack and it is destructive; after it has
executed, the number is no longer on the stack. There is always one
space printed after the number, and no spaces before it. Dot-quote
defines a string (a sequence of characters) that will be printed when
the word is executed. The string can contain any printable characters
except @code{"}. A @code{"} has a special function; it is not a Forth
word but it acts as a delimiter (the way that delimiters work is
described in the next section). Finally, @code{dup} duplicates the value
at the top of the stack. Try typing @code{5 dup .s} to see what it does.

We already know that the text interpreter searches through the
dictionary to locate names. If you've followed the examples earlier, you
will already have a definition called @code{add-two}. Lets try modifying
it by typing in a new definition:

@example
@kbd{: add-two dup . ." + 2 = " 2 + . ;@key{RET}} redefined add-two  ok
@end example

Forth recognised that we were defining a word that already exists, and
printed a message to warn us of that fact. Let's try out the new
definition:

@example
@kbd{9 add-two@key{RET}} 9 + 2 = 11  ok
@end example

@noindent
All that we've actually done here, though, is to create a new
definition, with a particular name. The fact that there was already a
definition with the same name did not make any difference to the way
that the new definition was created (except that Forth printed a warning
message). The old definition of add-two still exists (try @code{demo}
again to see that this is true). Any new definition will use the new
definition of @code{add-two}, but old definitions continue to use the
version that already existed at the time that they were @code{compiled}.

Before you go on to the next section, try defining and redefining some
words of your own.

@comment ----------------------------------------------
@node How does that work?, Forth is written in Forth, Your first definition, Introduction
@section How does that work?
@cindex parsing words

@c That's pretty deep (IMO way too deep) for an introduction. - anton

@c Is it a good idea to talk about the interpretation semantics of a
@c number? We don't have an xt to go along with it. - anton

@c Now that I have eliminated execution semantics, I wonder if it would not
@c be better to keep them (or add run-time semantics), to make it easier to
@c explain what compilation semantics usually does. - anton

@c nac-> I removed the term ``default compilation sematics'' from the
@c introductory chapter. Removing ``execution semantics'' was making
@c everything simpler to explain, then I think the use of this term made
@c everything more complex again. I replaced it with ``default
@c semantics'' (which is used elsewhere in the manual) by which I mean
@c ``a definition that has neither the immediate nor the compile-only
@c flag set''.

@c anton: I have eliminated default semantics (except in one place where it
@c means "default interpretation and compilation semantics"), because it
@c makes no sense in the presence of combined words.  I reverted to
@c "execution semantics" where necessary.

@c nac-> I reworded big chunks of the ``how does that work''
@c section (and, unusually for me, I think I even made it shorter!).  See
@c what you think -- I know I have not addressed your primary concern
@c that it is too heavy-going for an introduction. From what I understood
@c of your course notes it looks as though they might be a good framework. 
@c Things that I've tried to capture here are some things that came as a
@c great revelation here when I first understood them. Also, I like the
@c fact that a very simple code example shows up almost all of the issues
@c that you need to understand to see how Forth works. That's unique and
@c worthwhile to emphasise.

@c anton: I think it's a good idea to present the details, especially those
@c that you found to be a revelation, and probably the tutorial tries to be
@c too superficial and does not get some of the things across that make
@c Forth special.  I do believe that most of the time these things should
@c be discussed at the end of a section or in separate sections instead of
@c in the middle of a section (e.g., the stuff you added in "User-defined
@c defining words" leads in a completely different direction from the rest
@c of the section).

Now we're going to take another look at the definition of @code{add-two}
from the previous section. From our knowledge of the way that the text
interpreter works, we would have expected this result when we tried to
define @code{add-two}:

@example
@kbd{: add-two 2 + . ;@key{RET}}
*the terminal*:4: Undefined word
: >>>add-two<<< 2 + . ;
@end example

The reason that this didn't happen is bound up in the way that @code{:}
works. The word @code{:} does two special things. The first special
thing that it does is to prevent the text interpreter from ever seeing the
characters @code{add-two}. The text interpreter uses a variable called
@cindex modifying >IN
@code{>IN} (pronounced ``to-in'') to keep track of where it is in the
input line. When it encounters the word @code{:} it behaves in exactly
the same way as it does for any other word; it looks it up in the name
dictionary, finds its xt and executes it. When @code{:} executes, it
looks at the input buffer, finds the word @code{add-two} and advances the
value of @code{>IN} to point past it. It then does some other stuff
associated with creating the new definition (including creating an entry
for @code{add-two} in the name dictionary). When the execution of @code{:}
completes, control returns to the text interpreter, which is oblivious
to the fact that it has been tricked into ignoring part of the input
line.

@cindex parsing words
Words like @code{:} -- words that advance the value of @code{>IN} and so
prevent the text interpreter from acting on the whole of the input line
-- are called @dfn{parsing words}.

@cindex @code{state} - effect on the text interpreter
@cindex text interpreter - effect of state
The second special thing that @code{:} does is change the value of a
variable called @code{state}, which affects the way that the text
interpreter behaves. When Gforth starts up, @code{state} has the value
0, and the text interpreter is said to be @dfn{interpreting}. During a
colon definition (started with @code{:}), @code{state} is set to -1 and
the text interpreter is said to be @dfn{compiling}.

In this example, the text interpreter is compiling when it processes the
string ``@code{2 + . ;}''. It still breaks the string down into
character sequences in the same way. However, instead of pushing the
number @code{2} onto the stack, it lays down (@dfn{compiles}) some magic
into the definition of @code{add-two} that will make the number @code{2} get
pushed onto the stack when @code{add-two} is @dfn{executed}. Similarly,
the behaviours of @code{+} and @code{.} are also compiled into the
definition.

Certain kinds of words do not get compiled. These so-called @dfn{immediate
words} get executed (performed @i{now}) regardless of whether the text
interpreter is interpreting or compiling. The word @code{;} is an
immediate word. Rather than being compiled into the definition, it
executes. Its effect is to terminate the current definition, which
includes changing the value of @code{state} back to 0.

When you execute @code{add-two}, it has a @dfn{run-time effect} that is
exactly the same as if you had typed @code{2 + . @key{RET}} outside of a
definition.

In Forth, every word or number can be described in terms of two
properties:

@itemize @bullet
@item
@cindex interpretation semantics
Its @dfn{interpretation semantics} describe how it will behave when the
text interpreter encounters it in @dfn{interpret} state. The
interpretation semantics of a word are represented by its @dfn{execution
token} (@pxref{Execution token}).
@item
@cindex compilation semantics
Its @dfn{compilation semantics} describe how it will behave when the
text interpreter encounters it in @dfn{compile} state. The compilation
semantics of a word are represented by its @dfn{compilation token}
(@pxref{Compilation token}).
@end itemize

@noindent
Numbers are always treated in a fixed way:

@itemize @bullet
@item
When the number is @dfn{interpreted}, its behaviour is to push the
number onto the stack.
@item
When the number is @dfn{compiled}, a piece of code is appended to the
current definition that pushes the number when it runs. (In other words,
the compilation semantics of a number are to postpone its interpretation
semantics until the run-time of the definition that it is being compiled
into.)
@end itemize

Words don't always behave in such a regular way, but most have @i{default
semantics} which means that they behave like this:

@itemize @bullet
@item
The @dfn{interpretation semantics} of the word are to do something useful.
@item
The @dfn{compilation semantics} of the word are to append its
@dfn{interpretation semantics} to the current definition (so that its
run-time behaviour is to do something useful).
@end itemize

@cindex immediate words
The actual behaviour of any particular word can be controlled by using
the words @code{immediate} and @code{compile-only} when the word is
defined. These words set flags in the name dictionary entry of the most
recently defined word, and these flags are retrieved by the text
interpreter when it finds the word in the name dictionary.

A word that is marked as @dfn{immediate} has compilation semantics that
are identical to its interpretation semantics. In other words, it
behaves like this:

@itemize @bullet
@item
The @dfn{interpretation semantics} of the word are to do something useful.
@item
The @dfn{compilation semantics} of the word are to do something useful
(and actually the same thing); i.e., it is executed during compilation.
@end itemize

Marking a word as @dfn{compile-only} means that the text interpreter
produces a warning when encountering this word in interpretation
state; ticking the word (with @code{'} or @code{[']} also produces a
warning.

It is never necessary to use @code{compile-only} (and it is not even
part of Standard Forth, though it is provided by many implementations)
but it is good etiquette to apply it to a word that will not behave
correctly (and might have unexpected side-effects) in interpret
state. For example, it is only legal to use the conditional word
@code{IF} within a definition. If you forget this and try to use it
elsewhere, the fact that (in Gforth) it is marked as
@code{compile-only} allows the text interpreter to generate a helpful
warning.

This example shows the difference between an immediate and a
non-immediate word:

@example
: show-state state @@ . ;
: show-state-now show-state ; immediate
: word1 show-state ;
: word2 show-state-now ;
@end example

The word @code{immediate} after the definition of @code{show-state-now}
makes that word an immediate word. These definitions introduce a new
word: @code{@@} (pronounced ``fetch''). This word fetches the value of a
variable, and leaves it on the stack. Therefore, the behaviour of
@code{show-state} is to print a number that represents the current value
of @code{state}.

When you execute @code{word1}, it prints the number 0, indicating that
the system is interpreting. When the text interpreter compiled the
definition of @code{word1}, it encountered @code{show-state} whose
compilation semantics are to append its interpretation semantics to the
current definition. When you execute @code{word1}, it performs the
interpretation semantics of @code{show-state}.  At the time that @code{word1}
(and therefore @code{show-state}) is executed, the system is
interpreting.

When you pressed @key{RET} after entering the definition of @code{word2},
you should have seen the number -1 printed, followed by ``@code{
ok}''. When the text interpreter compiled the definition of
@code{word2}, it encountered @code{show-state-now}, an immediate word,
whose compilation semantics are therefore to perform its interpretation
semantics. It is executed straight away (even before the text
interpreter has moved on to process another group of characters; the
@code{;} in this example). The effect of executing it is to display the
value of @code{state} @i{at the time that the definition of}
@code{word2} @i{is being defined}. Printing -1 demonstrates that the
system is compiling at this time. If you execute @code{word2} it does
nothing at all.

@cindex @code{."}, how it works
Before leaving the subject of immediate words, consider the behaviour of
@code{."} in the definition of @code{greet}, in the previous
section. This word is both a parsing word and an immediate word. Notice
that there is a space between @code{."} and the start of the text
@code{Hello and welcome}, but that there is no space between the last
letter of @code{welcome} and the @code{"} character. The reason for this
is that @code{."} is a Forth word; it must have a space after it so that
the text interpreter can identify it. The @code{"} is not a Forth word;
it is a @dfn{delimiter}. The examples earlier show that, when the string
is displayed, there is neither a space before the @code{H} nor after the
@code{e}. Since @code{."} is an immediate word, it executes at the time
that @code{greet} is defined. When it executes, its behaviour is to
search forward in the input line looking for the delimiter. When it
finds the delimiter, it updates @code{>IN} to point past the
delimiter. It also compiles some magic code into the definition of
@code{greet}; the xt of a run-time routine that prints a text string. It
compiles the string @code{Hello and welcome} into memory so that it is
available to be printed later. When the text interpreter gains control,
the next word it finds in the input stream is @code{;} and so it
terminates the definition of @code{greet}.


@comment ----------------------------------------------
@node Forth is written in Forth, Review - elements of a Forth system, How does that work?, Introduction
@section Forth is written in Forth
@cindex structure of Forth programs

When you start up a Forth compiler, a large number of definitions
already exist. In Forth, you develop a new application using bottom-up
programming techniques to create new definitions that are defined in
terms of existing definitions. As you create each definition you can
test and debug it interactively.

If you have tried out the examples in this section, you will probably
have typed them in by hand; when you leave Gforth, your definitions will
be lost. You can avoid this by using a text editor to enter Forth source
code into a file, and then loading code from the file using
@code{include} (@pxref{Forth source files}). A Forth source file is
processed by the text interpreter, just as though you had typed it in by
hand@footnote{Actually, there are some subtle differences -- see
@ref{The Text Interpreter}.}.

Gforth also supports the traditional Forth alternative to using text
files for program entry (@pxref{Blocks}).

In common with many, if not most, Forth compilers, most of Gforth is
actually written in Forth. All of the @file{.fs} files in the
installation directory@footnote{For example,
@file{/usr/local/share/gforth...}} are Forth source files, which you can
study to see examples of Forth programming.

Gforth maintains a history file that records every line that you type to
the text interpreter. This file is preserved between sessions, and is
used to provide a command-line recall facility. If you enter long
definitions by hand, you can use a text editor to paste them out of the
history file into a Forth source file for reuse at a later time
(for more information @pxref{Command-line editing}).


@comment ----------------------------------------------
@node Review - elements of a Forth system, Where to go next, Forth is written in Forth, Introduction
@section Review - elements of a Forth system
@cindex elements of a Forth system

To summarise this chapter:

@itemize @bullet
@item
Forth programs use @dfn{factoring} to break a problem down into small
fragments called @dfn{words} or @dfn{definitions}.
@item
Forth program development is an interactive process.
@item
The main command loop that accepts input, and controls both
interpretation and compilation, is called the @dfn{text interpreter}
(also known as the @dfn{outer interpreter}).
@item
Forth has a very simple syntax, consisting of words and numbers
separated by spaces or carriage-return characters. Any additional syntax
is imposed by @dfn{parsing words}.
@item
Forth uses a stack to pass parameters between words. As a result, it
uses postfix notation.
@item
To use a word that has previously been defined, the text interpreter
searches for the word in the @dfn{name dictionary}.
@item
Words have @dfn{interpretation semantics} and @dfn{compilation semantics}.
@item
The text interpreter uses the value of @code{state} to select between
the use of the @dfn{interpretation semantics} and the  @dfn{compilation
semantics} of a word that it encounters.
@item
The relationship between the @dfn{interpretation semantics} and
@dfn{compilation semantics} for a word
depends upon the way in which the word was defined (for example, whether
it is an @dfn{immediate} word).
@item
Forth definitions can be implemented in Forth (called @dfn{high-level
definitions}) or in some other way (usually a lower-level language and
as a result often called @dfn{low-level definitions}, @dfn{code
definitions} or @dfn{primitives}).
@item
Many Forth systems are implemented mainly in Forth.
@end itemize


@comment ----------------------------------------------
@node Where to go next, Exercises, Review - elements of a Forth system, Introduction
@section Where To Go Next
@cindex where to go next

Amazing as it may seem, if you have read (and understood) this far, you
know almost all the fundamentals about the inner workings of a Forth
system. You certainly know enough to be able to read and understand the
rest of this manual and the Standard Forth document, to learn more about the
facilities that Forth in general and Gforth in particular provide. Even
scarier, you know almost enough to implement your own Forth system.
However, that's not a good idea just yet... better to try writing some
programs in Gforth.

Forth has such a rich vocabulary that it can be hard to know where to
start in learning it. This section suggests a few sets of words that are
enough to write small but useful programs. Use the word index in this
document to learn more about each word, then try it out and try to write
small definitions using it. Start by experimenting with these words:

@itemize @bullet
@item
Arithmetic: @code{+ - * / /MOD */ ABS INVERT}
@item
Comparison: @code{MIN MAX =}
@item
Logic: @code{AND OR XOR NOT}
@item
Stack manipulation: @code{DUP DROP SWAP OVER}
@item
Loops and decisions: @code{IF ELSE ENDIF ?DO I LOOP}
@item
Input/Output: @code{. ." EMIT CR KEY}
@item
Defining words: @code{: ; CREATE}
@item
Memory allocation words: @code{ALLOT ,}
@item
Tools: @code{SEE WORDS .S MARKER}
@end itemize

When you have mastered those, go on to:

@itemize @bullet
@item
More defining words: @code{VARIABLE CONSTANT VALUE TO CREATE DOES>}
@item
Memory access: @code{@@ !}
@end itemize

When you have mastered these, there's nothing for it but to read through
the whole of this manual and find out what you've missed.

@comment ----------------------------------------------
@node Exercises,  , Where to go next, Introduction
@section Exercises
@cindex exercises

TODO: provide a set of programming excercises linked into the stuff done
already and into other sections of the manual. Provide solutions to all
the exercises in a .fs file in the distribution.

@c Get some inspiration from Starting Forth and Kelly&Spies.

@c excercises:
@c 1. take inches and convert to feet and inches.
@c 2. take temperature and convert from fahrenheight to celcius;
@c    may need to care about symmetric vs floored??
@c 3. take input line and do character substitution
@c    to encipher or decipher
@c 4. as above but work on a file for in and out
@c 5. take input line and convert to pig-latin 
@c
@c thing of sets of things to exercise then come up with
@c problems that need those things.


@c ******************************************************************
@node Literals in source code, Words, Introduction, Top
@chapter Literals in source code
@cindex Literals (in source code)

@cindex single-cell numbers, input format
@cindex input format for single-cell numbers
To push an integer number on the data stack, you write the number in
source code, e.g., @code{123}.  You can prefix the digits with
@code{-} to indicate a negative number, e.g. @code{-123}.  This works
both inside colon definitions and outside.  The number is interpreted
according to the value of @code{base} (@pxref{Number Conversion}).
The digits are @code{0} to @code{9} and @code{a} (decimal 10) to
@code{z} (decimal 35), but only digits smaller than the base are
recognized.  The conversion is case-insensitive, so @code{A} and
@code{a} are the same digit.

@cindex &-prefix for decimal numbers
@cindex #-prefix for decimal numbers
@cindex %-prefix for binary numbers
@cindex $-prefix for hexadecimal numbers
@cindex 0x-prefix for hexadecimal numbers
You can make the base explicit for the number by using a prefix:

@itemize @bullet
@item
@code{#} -- decimal
@item
@code{%} -- binary
@item
@code{$} -- hexadecimal
@item
@code{&} -- decimal (non-standard)
@item
@code{0x} -- hexadecimal, if base<33 (non-standard).
@end itemize

For combinations including base-prefix and sign, the standard order is
to have the base-prefix first (e.g., @code{#-123}); Gforth supports
both orders.

@cindex double-cell numbers, input format
@cindex input format for double-cell numbers
You can put a decimal point @code{.} at the end of a number (or,
non-standardly, anywhere else except before a prefix) to get a
double-cell integer (e.g., @code{#-123.} or @code{#-.123} (the same
number)).  If users experienced in another programming language see or
write such a number without base prefix (e.g., @code{-123.}), they may
expect that the number represents a floating-point value.  To clear up
the confusion early, Gforth warns of such usage; to avoid the
warnings, the best approach is to always write double numbers with a
base prefix (e.g., @code{#-123.})

Here are some examples, with the equivalent decimal number shown after
in braces:

@code{$-41} (-65), @code{%1001101} (205), @code{%1001.0001} (145 - a double-precision number),
@code{#905} (905), @code{$abc} (2478), @code{$ABC} (2478).

@cindex '-prefix for characters/code points
@cindex input format for characters/code points
You can get the numeric value of a (character) code point by
surrounding the character with @code{'} (e.g., @code{'a'}).  The
trailing @code{'} is required by the standard, but you can leave it
away in Gforth.  Note that this also works for non-ASCII characters.
For many uses, it is more useful to have the character as a string
rather than as a cell; see below for the string syntax.

@cindex floating-point numbers, input format
@cindex input format for floating-point numbers
For floating-point numbers in Forth, you recognize them due to their
exponent.  I.e. @code{1.} is a double-cell integer, and @code{1e0} is
a floating-point number; the latter can be (and usually is) shortened
to @code{1e}.  Both the significand (the part before the @code{e} or
@code{E}) and the exponent may have signs (including @code{+}); the
significand must contain at least one digit and may contain a decimal
point, the exponent can be empty.  Floating-point numbers always use
decimal base for both significand and exponent, and are only
recognized when the base is decimal.  Examples are: @code{1e 1e0 1.e
1.e0 +1e+0} (which all represent the same number) @code{+12.E-4}.

A Gforth extension (since 1.0) is to write a floating-point number in
scaled notation: It can optionally have a sign, then one or more
digits, then use one of the mostly SI-defined scaling symbols (aka
metric prefixes) or @code{%}, and then optionally more digits.  Here's
the full list of scaling symbols that Gforth accepts:

@itemize @bullet
@item
@code{Q} @code{e30}  quetta 
@item                
@code{R} @code{e27}  ronna  
@item                
@code{Y} @code{e24}  yotta  
@item                
@code{Z} @code{e21}  zetta  
@item                
@code{X} @code{e18}  exa    (not @code{E}) 
@item                
@code{P} @code{e15}  peta   
@item                
@code{T} @code{e12}  tera   
@item                
@code{G} @code{e9}   giga   
@item                
@code{M} @code{e6}   mega   
@item                
@code{k} @code{e3}   kilo   
@item                
@code{h} @code{e2}   hecto  
@item                
@code{d} @code{e-1}  deci   
@item                
@code{%} @code{e-2}  percent (not @code{c})
@item                
@code{m} @code{e-3}  milli  
@item                
@code{u} @code{e-6}  micro   (not @code{μ})
@item                
@code{n} @code{e-9}  nano   
@item                
@code{p} @code{e-12} pico   
@item                
@code{f} @code{e-15} femto  
@item                
@code{a} @code{e-18} atto   
@item                
@code{z} @code{e-21} zepto  
@item                
@code{y} @code{e-24} yocto  
@item                
@code{r} @code{e-27} ronto  
@item                
@code{q} @code{e-30} quecto 
@end itemize

Unlike most of the rest of Gforth, scaling symbols are treated
case-sensitively.  Using the scaled notation is equivalent to using a
decimal point instead of the scaling symbol and appending the
exponential notation at the end.  Examples of scaled notation:
@code{6k5} (6500e) @code{23%} (0.23e).

@cindex String input format
@cindex input format for strings
You can input a string by surrounding it with @code{"}
(e.g. @code{"abc"}, @code{"a b"}).  The result is the starting address
and byte (=char) count of the string on the data stack.

You have to escape any @code{"} inside the string with @code{\} (e.g.,
@code{"double-quote->\"<-"}).  In addition, this string syntax
supports all the ways to write control characters that are supported
by @code{s\"} (@pxref{String and character literals}).  A disadvantage
of this string syntax is that it is non-standard; for standard
programs, use @code{s\"} instead.

@cindex environment variable input format
@cindex input format for environment variables
You can input an environment variable by first loading
@file{rec-env.fs} and then prefixing the environment variable with
@code{$}, e.g., @code{$HOME}; the result is a string descriptor on the
data stack in the format described above.  This is equivalent to
@code{"HOME" getenv}, i.e., the environment variable is resolved at
run-time.

@cindex xt input format
@cindex execution token input format
@cindex input format for execution tokens
@cindex @code{`} prefix of word
You can input an execution token (xt) of a word by prefixing the name
of the word with @code{`} (e.g., @code{`dup}).  An advantage over
using @code{'} or @code{[']} is you do not need to switch between them
when copying and pasting code from inside to outside a colon
definition or vice versa.  A disadvantage is that this syntax is
non-standard.

@cindex nt input format
@cindex nt token input format
@cindex input format for name tokens
@cindex @code{``} prefix of word
You can input a name token (nt) of a word by prefixing the name of the
word with @code{``} (e.g., @code{``dup}).  This syntax is also
non-standard.

@cindex body-relative address input format
@cindex input format for body-relative addresses
You can input a body address of a word by surrounding it with @code{<}
and @code{>} (e.g., @code{<spaces>}).  You can also input an address
that is at a positive offset from the body address (typically an
address in that body), by putting @code{+} and a number (see syntax
above) between the word name and the closing @code{>} (e.g.,
@code{<spaces+$15>}, @code{spaces+-3}).  You will get the body address
plus the number.  This non-standard feature exists to allow copying
and pasting the output of @code{...} (@pxref{Examining data}).

@c !! rec-scope rec-meta rec-to

@c ******************************************************************
@node Words, Error messages, Literals in source code, Top
@chapter Forth Words
@cindex words

@menu
* Notation::
* Case insensitivity::
* Comments::
* Boolean Flags::
* Arithmetic::
* Stack Manipulation::
* Memory::
* Strings and Characters::
* Control Structures::
* Defining Words::
* Interpretation and Compilation Semantics::
* Tokens for Words::
* Compiling words::
* The Text Interpreter::
* The Input Stream::
* Word Lists::
* Environmental Queries::
* Files::
* Blocks::
* Other I/O::
* OS command line arguments::
* Locals::
* Structures::
* Object-oriented Forth::
* Regular Expressions::
* Programming Tools::
* Multitasker::
* C Interface::
* Assembler and Code Words::
* Carnal words::
* Passing Commands to the OS::
* Keeping track of Time::
* Miscellaneous Words::
@end menu

@node Notation, Case insensitivity, Words, Words
@section Notation
@cindex notation of glossary entries
@cindex format of glossary entries
@cindex glossary notation format
@cindex word glossary entry format

The Forth words are described in this section in the glossary notation
that has become a de-facto standard for Forth texts:

@format
@i{word}     @i{Stack effect}   @i{wordset}   @i{pronunciation}
@end format
@i{Description}

@table @var
@item word
The name of the word.

@item Stack effect
@cindex stack effect
The stack effect is written in the notation @code{@i{before} --
@i{after}}, where @i{before} and @i{after} describe the top of
stack entries before and after the execution of the word. The rest of
the stack is not touched by the word. The top of stack is rightmost,
i.e., a stack sequence is written as it is typed in. Note that Gforth
uses a separate floating point stack, but a unified stack
notation. Also, return stack effects are not shown in @i{stack
effect}, but in @i{Description}. The name of a stack item describes
the type and/or the function of the item. See below for a discussion of
the types.

All words have two stack effects: A compile-time stack effect and a
run-time stack effect. The compile-time stack-effect of most words is
@i{ -- }. If the compile-time stack-effect of a word deviates from
this standard behaviour, or the word does other unusual things at
compile time, both stack effects are shown; otherwise only the run-time
stack effect is shown.

Also note that in code templates or examples there can be comments in
parentheses that display the stack picture at this point; there is no
@code{--} in these places, because there is no before-after situation.

@cindex pronounciation of words
@item pronunciation
How the word is pronounced.

@cindex wordset
@cindex environment wordset
@item wordset
The wordset specifies whether a word has been standardized, it is an
environmental query string, or if it is a Gforth-specific word.  In
the latter case the wordset contains the string @code{gforth}, other
wordset names are either @code{environment} of refer to standard word
sets.

The Forth standard is divided into several word sets. In theory, a
standard system need not support all of them, but in practice, serious
systems on non-tiny machines support almost all standardized words
(some systems require explicit loading of some word sets, however), so
it does not increase portability in practice to be parsimonious in
using word sets.

For the Gforth-specific words, we have the following categories:

@table @code
@item gforth
@item gforth-@i{<version>}
We intend to permanently support this word in Gforth and it has been
available since Gforth @i{<version>} (possibly not as supported word at
that time).

@item gforth-experimental
This word is available in the present version and may turn into a
permanent word or may be removed in a future release of Gforth.
Feedback welcome.

@item gforth-internal
This word is an internal factor, not a supported word, and it may be
removed in a future release of Gforth.

@item gforth-obsolete
This word will be removed in a future release of Gforth.
@end table


@item Description
A description of the behaviour of the word.
@end table

@cindex types of stack items
@cindex stack item types
The type of a stack item is specified by the character(s) the name
starts with:

@table @code
@item f
@cindex @code{f}, stack item type
Boolean flags, i.e. @code{false} or @code{true}.
@item c
@cindex @code{c}, stack item type
Char
@item w
@cindex @code{w}, stack item type
Cell, can contain an integer or an address
@item n
@cindex @code{n}, stack item type
signed integer
@item u
@cindex @code{u}, stack item type
unsigned integer
@item d
@cindex @code{d}, stack item type
double sized signed integer
@item ud
@cindex @code{ud}, stack item type
double sized unsigned integer
@item r
@cindex @code{r}, stack item type
Float (on the FP stack)
@item a-
@cindex @code{a_}, stack item type
Cell-aligned address
@item c-
@cindex @code{c_}, stack item type
Char-aligned address (note that a Char may have two bytes in Windows NT)
@item f-
@cindex @code{f_}, stack item type
Float-aligned address
@item df-
@cindex @code{df_}, stack item type
Address aligned for IEEE double precision float
@item sf-
@cindex @code{sf_}, stack item type
Address aligned for IEEE single precision float
@item xt
@cindex @code{xt}, stack item type
Execution token, same size as Cell
@item wid
@cindex @code{wid}, stack item type
Word list ID, same size as Cell
@item ior, wior
@cindex ior type description
@cindex wior type description
I/O result code, cell-sized.  In Gforth, you can @code{throw} iors.
@item f83name
@cindex @code{f83name}, stack item type
Pointer to a name structure
@item "
@cindex @code{"}, stack item type
string in the input stream (not on the stack). The terminating character
is a blank by default. If it is not a blank, it is shown in @code{<>}
quotes.
@end table

@comment ----------------------------------------------
@node Case insensitivity, Comments, Notation, Words
@section Case insensitivity
@cindex case sensitivity
@cindex upper and lower case

Gforth is case-insensitive; you can enter definitions and invoke
Standard words using upper, lower or mixed case (however,
@pxref{core-idef, Implementation-defined options, Implementation-defined
options}).

Standard Forth only @i{requires} implementations to recognise Standard words
when they are typed entirely in upper case. Therefore, a Standard
program must use upper case for all Standard words. You can use whatever
case you like for words that you define, but in a Standard program you
have to use the words in the same case that you defined them.

Gforth supports case sensitivity through @code{cs-wordlist}s (case-sensitive
wordlists, @pxref{Word Lists}).

Two people have asked how to convert Gforth to be case-sensitive; while
we think this is a bad idea, you can change all wordlists into tables
like this:

@example
' table-find forth-wordlist wordlist-map @ !
@end example

Note that you now have to type the predefined words in the same case
that we defined them, which are varying.  You may want to convert them
to your favourite case before doing this operation (I won't explain how,
because if you are even contemplating doing this, you'd better have
enough knowledge of Forth systems to know this already).

@node Comments, Boolean Flags, Case insensitivity, Words
@section Comments
@cindex comments

Forth supports two styles of comment; the traditional @i{in-line} comment,
@code{(} and its modern cousin, the @i{comment to end of line}; @code{\}.


doc-(
doc-\
doc-\G


@node Boolean Flags, Arithmetic, Comments, Words
@section Boolean Flags
@cindex Boolean flags

A Boolean flag is cell-sized. A cell with all bits clear represents the
flag @code{false} and a flag with all bits set represents the flag
@code{true}. Words that check a flag (for example, @code{IF}) will treat
a cell that has @i{any} bit set as @code{true}.
@c on and off to Memory? 
@c true and false to "Bitwise operations" or "Numeric comparison"?

doc-true
doc-false
doc-on
doc-off
doc-select

@node Arithmetic, Stack Manipulation, Boolean Flags, Words
@section Arithmetic
@cindex arithmetic words

@cindex division with potentially negative operands
Forth arithmetic is not checked, i.e., you will not hear about integer
overflow on addition or multiplication, you may hear about division by
zero if you are lucky. The operator is written after the operands, but
the operands are still in the original order. I.e., the infix
@code{2-1} corresponds to @code{2 1 -}. Forth offers a variety of
division operators. If you perform division with potentially negative
operands, you do not want to use @code{/} or @code{/mod} with its
implementation-defined behaviour, but, e.g., @code{/f}, @code{/modf}
or @code{fm/mod} (@pxref{Integer division}).

@menu
* Single precision::
* Double precision::            Double-cell integer arithmetic
* Mixed precision::             Operations with single and double-cell integers
* Integer division::
* Two-stage integer division::
* Bitwise operations::
* Numeric comparison::
* Floating Point::
@end menu

@node Single precision, Double precision, Arithmetic, Arithmetic
@subsection Single precision
@cindex single precision arithmetic words

@c !! cell undefined

By default, numbers in Forth are single-precision integers that are one
cell in size. They can be signed or unsigned, depending upon how you
treat them. For the rules used by the text interpreter for recognising
single-precision integers see @ref{Number Conversion}.

These words are all defined for signed operands, but some of them also
work for unsigned numbers: @code{+}, @code{1+}, @code{-}, @code{1-},
@code{*}.

doc-+
doc-1+
doc-under+
doc--
doc-1-
doc-*
doc-negate
doc-abs
doc-min
doc-max
doc-umin
doc-umax

@node Double precision, Mixed precision, Single precision, Arithmetic
@subsection Double precision
@cindex double precision arithmetic words

For the rules used by the text interpreter for
recognising double-precision integers, see @ref{Number Conversion}.

A double precision number is represented by a cell pair, with the most
significant cell at the TOS. It is trivial to convert an unsigned single
to a double: simply push a @code{0} onto the TOS. Since numbers are
represented by Gforth using 2's complement arithmetic, converting a
signed single to a (signed) double requires sign-extension across the
most significant cell. This can be achieved using @code{s>d}. The moral
of the story is that you cannot convert a number without knowing whether
it represents an unsigned or a signed number.

These words are all defined for signed operands, but some of them also
work for unsigned numbers: @code{d+}, @code{d-}.

doc-s>d
doc-d>s
doc-d+
doc-d-
doc-dnegate
doc-dabs
doc-dmin
doc-dmax


@node Mixed precision, Integer division, Double precision, Arithmetic
@subsection Mixed precision
@cindex mixed precision arithmetic words

doc-m+
doc-m*
doc-um*

@node Integer division, Two-stage integer division, Mixed precision, Arithmetic
@subsection Integer division
@cindex dividing integers
@cindex modulus
@cindex remainder
@cindex Floating-point unidentified fault
@cindex Division by zero
@cindex Result out of range

Below you find a considerable number of words for dealing with
divisions.  A major difference between them is in dealing with signed
division: Do the words support signed division (those with the
@code{U} prefix do not)?

@cindex floored division
@cindex symmetric division
If they do, do they round towards negative infinity (floored division,
suffix @code{F}), or towards 0 (symmetric division, suffix @code{S}).
The standard leaves the issue implementation-defined for most standard
words (@code{/ mod /mod */ */mod m*/}), and different systems have
made different choices.  Gforth implements these words as floored
(since Gforth 0.7).  There is only a difference between floored and
symmetric division if the dividend and the divisor have different
signs, and the dividend is not a multiple of the divisor.  The
following table illustrates the results:

@example
                      floored          symmetric
dividend divisor remainder quotient remainder quotient
    10      7           3   1              3   1
   -10      7           4  -2             -3  -1
    10     -7          -4  -2              3  -1
   -10     -7          -3   1             -3   1
@end example

The common case where floored vs.@: symmetric makes a difference is when
dividends n1 with varying sign are divided by the same positive
divisor n2; in that case you usually want floored division, because
then the remainder is always positive and does not change sign
depending on the dividend; also, with floored division, the quotient
always increases by 1 when n1 increases by n2, while with symmetric
division there is no increase in the quotient for -n2<n1<n2 (the
quotient is 0 in this range).

In any case, if you divide numbers where floored vs.@: symmetric makes a
difference, you should think about which variant is the right one for
you, and then use either the appropriately suffixed Gforth words, or
the standard words @code{fm/mod} or @code{sm/rem}.

Single-by-single-cell division:

doc-/
doc-/s
doc-/f
doc-u/
doc-mod
doc-mods
doc-modf
doc-umod
doc-/mod
doc-/mods
doc-/modf
doc-u/mod

Double-by-single-cell division with single-cell results; these words
are roughly as fast as the words above on some architectures (e.g.,
AMD64), but much slower on others (e.g., an order of magnitude on
various Aarch64 CPUs).

doc-fm/mod
doc-sm/rem
doc-um/mod
doc-du/mod
doc-*/
doc-*/s
doc-*/f
doc-u*/
doc-*/mod
doc-*/mods
doc-*/modf
doc-u*/mod

Division with double-cell results; these words are much slower than
the words above.

doc-ud/mod
doc-m*/

You can use the following environmental query @code{floored}
(@pxref{Environmental Queries}) to learn whether @code{/ mod /mod */
*/mod m*/} use floored or symmetric division.

@cindex Floating-point unidentified fault (on integer division)
@cindex Division by zero
@cindex Result out of range (on integer division)
One other aspect of the integer division words is that most of them
can overflow, and division by zero is mathematically undefined.  What
happens if you hit one of these conditions depends on the engine, the
hardware, and the operating system: The engine @code{gforth} tries
hard to throw the appropriate error -10 (Division by zero) or -11
(Result out of range), but on some platforms throws -55
(Floating-point unidentified fault).  The engine @code{gforth-fast}
may produce an inappropriate throw code (and error message), or may
produce no error, just produce a bogus value.  I.e., you should not
bet on such conditions being thrown, but for quicker debugging
@code{gforth} catches more and produces more accurate errors than
@code{gforth-fast}.


@node Two-stage integer division, Bitwise operations, Integer division, Arithmetic
@subsection Two-stage integer division
@cindex dividing many integers with the same divisor
@cindex reciprocal of integer

On most hardware, multiplication is significantly faster than
division.  So if you have to divide many numbers by the same divisor,
it is usually faster to determine the reciprocal of the divisor once
and multiply the numbers with the reciprocal.  For integers, this is
tricky, so Gforth packages this work into the words described in this
section.

Let's start with an example: You want to divide all elements of an
array of cells by the same number n.  A straightforward way to
implement this is:

@example
: array/ ( addr u n -- )
  -rot cells bounds u+do
    i @@ over / i !
  1 cells +loop
  drop ;
@end example

A more efficient version looks like this:

@example
: array/ ( addr u n -- )
  @{: | reci[ staged/-size ] :@}
  reci[ /f-stage1m
  cells bounds u+do
    i @@ reci[ /f-stage2m i !
  1 cells +loop ;
@end example

This example first creates a local buffer @code{reci[} with size
@code{staged/-size} for storing the reciprocal data.  Then
@code{/f-stage1m} computes the reciprocal of @i{n} and stores it in
@code{reci[}.  Finally, inside the loop @code{/f-stage2m} uses the
data in @code{reci[} to compute the quotient.

There are some limitations: Only positive divisors are supported for
@code{/f-stage1m}; for @code{u/-stage1m} you can use a divisor of 2 or
higher.  You get an error if you try to use an unsupported divisor.
You must initalize the reciprocal buffer for the floored second-stage
words with @code{/f-stage1m} and for the unsigned second-stage words
with @code{u/-stage1m}.  You must not modify the reciprocal buffer
between the first stage and the second stage; basically, don't treat
it as a memory buffer, but as something that is only mutable by the
first stage; the point of this rule is that future versions of Gforth
will not consider aliasing of this buffer.

The words are:

doc-staged/-size
doc-/f-stage1m
doc-/f-stage2m
doc-modf-stage2m
doc-/modf-stage2m
doc-u/-stage1m
doc-u/-stage2m
doc-umod-stage2m
doc-u/mod-stage2m

Gforth currently does not support staged symmetrical division.

You can recover the divisor from (the address of) a reciprocal with
@code{staged/-divisor @@}:

doc-staged/-divisor

This can be useful when looking at the decompiler output of Gforth: a
division by a constant is often compiled to a literal containing the
address of a reciprocal followed by a second-stage word.

The performance impact of using these words strongly depends on the
architecture (does it have hardware division?) and the specific
implementation (how fast is hardware division?), but just to give you
an idea about the relative performance of these words, here are the
cycles per iteration of a microbenchmark (which performs the mentioned
word once per iteration) on two AMD64 implementations; the @i{norm}
column shows the normal division word (e.g., @code{u/}), while the
@i{stg2} column shows the corresponding stage2 word (e.g.,
@code{u/-stage2m}):

@example
 Skylake              Zen2
norm stg2           norm stg2
41.3 15.8 u/	    35.2 21.4 u/	   
39.8 19.7 umod	    36.9 25.8 umod	   
44.0 25.3 u/mod	    43.0 33.9 u/mod	   
48.7 16.9 /f	    36.2 22.5 /f	   
47.9 20.5 modf	    37.9 27.1 modf	   
53.0 24.6 /modf	    45.8 35.4 /modf	   
    227.2 u/stage1      101.9 u/stage1
    159.8 /fstage1       97.7 /fstage1
@end example


@node Bitwise operations, Numeric comparison, Two-stage integer division, Arithmetic
@subsection Bitwise operations
@cindex bitwise operation words


doc-and
doc-or
doc-xor
doc-invert
doc-mux
doc-lshift
doc-rshift
doc-arshift
doc-dlshift
doc-drshift
doc-darshift
doc-2*
doc-2/
doc-d2*
doc-d2/
doc->pow2
doc-log2
doc-pow2?
doc-ctz

Unlike most other operations, rotation of narrower units cannot easily
be synthesized from rotation of wider units, so using cell-wide and
double-wide rotation operations means that the results depend on the
cell width.  For published algorithms or cell-width-independent
results, you usually need to use a fixed-width rotation operation.

doc-wrol
doc-wror
doc-lrol
doc-lror
doc-rol
doc-ror
doc-drol
doc-dror

@node Numeric comparison, Floating Point, Bitwise operations, Arithmetic
@subsection Numeric comparison
@cindex numeric comparison words

Note that the words that compare for equality (@code{= <> 0= 0<> d= d<>
d0= d0<>}) work for for both signed and unsigned numbers.

doc-<
doc-<=
doc-<>
doc-=
doc->
doc->=

doc-0<
doc-0<=
doc-0<>
doc-0=
doc-0>
doc-0>=

doc-u<
doc-u<=
@c u<> and u= exist but are the same as <> and =
@c doc-u<>
@c doc-u=
doc-u>
doc-u>=

doc-within

doc-d<
doc-d<=
doc-d<>
doc-d=
doc-d>
doc-d>=

doc-d0<
doc-d0<=
doc-d0<>
doc-d0=
doc-d0>
doc-d0>=

doc-du<
doc-du<=
@c du<> and du= exist but are the same as d<> and d=
@c doc-du<>
@c doc-du=
doc-du>
doc-du>=


@node Floating Point,  , Numeric comparison, Arithmetic
@subsection Floating Point
@cindex floating point arithmetic words

For the rules used by the text interpreter for
recognising floating-point numbers see @ref{Number Conversion}.

Gforth has a separate floating point stack, but the documentation uses
the unified notation.@footnote{It's easy to generate the separate
notation from that by just separating the floating-point numbers out:
e.g. @code{( n r1 u r2 -- r3 )} becomes @code{( n u -- ) ( F: r1 r2 --
r3 )}.}

@cindex floating-point arithmetic, pitfalls
Floating point numbers have a number of unpleasant surprises for the
unwary (e.g., floating point addition is not associative) and even a
few for the wary. You should not use them unless you know what you are
doing or you don't care that the results you get are totally bogus. If
you want to learn about the problems of floating point numbers (and
how to avoid them), you might start with @cite{David Goldberg,
@uref{https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html,What Every
Computer Scientist Should Know About Floating-Point Arithmetic}, ACM
Computing Surveys 23(1):5@minus{}48, March 1991}.

Conversion between integers and floating-point:

doc-s>f
doc-d>f
doc-f>s
doc-f>d

Arithmetics:

doc-f+
doc-f-
doc-f*
doc-f/
doc-fnegate
doc-fabs
doc-fcopysign
doc-fmax
doc-fmin
doc-floor
doc-fround
doc-ftrunc
doc-f**
doc-fsqrt
doc-fexp
doc-fexpm1
doc-fln
doc-flnp1
doc-flog
doc-falog
doc-f2*
doc-f2/
doc-1/f

Vector arithmetics:

doc-v*
doc-faxpy

@cindex angles in trigonometric operations
@cindex trigonometric operations
Angles in floating point operations are given in radians (a full circle
has 2 pi radians).

doc-fsin
doc-fcos
doc-fsincos
doc-ftan
doc-fasin
doc-facos
doc-fatan
doc-fatan2
doc-fsinh
doc-fcosh
doc-ftanh
doc-fasinh
doc-facosh
doc-fatanh
doc-pi

@cindex equality of floats
@cindex floating-point comparisons
One particular problem with floating-point arithmetic is that comparison
for equality often fails when you would expect it to succeed.  For this
reason approximate equality is often preferred (but you still have to
know what you are doing).  Also note that IEEE NaNs may compare
differently from what you might expect.  The comparison words are:

doc-f~rel
doc-f~abs
doc-f~
doc-f=
doc-f<>

doc-f<
doc-f<=
doc-f>
doc-f>=

doc-f0<
doc-f0<=
doc-f0<>
doc-f0=
doc-f0>
doc-f0>=

@cindex floating-point constants
Special values in IEEE754 can be derived by for example dividing by zero.
The most common ones are defined as floating point constants for easy usage.

doc-infinity
doc--infinity
doc-nan

@node Stack Manipulation, Memory, Arithmetic, Words
@section Stack Manipulation
@cindex stack manipulation words

@cindex floating-point stack in the standard
Gforth maintains a number of separate stacks:

@cindex data stack
@cindex parameter stack
@itemize @bullet
@item
A data stack (also known as the @dfn{parameter stack}) -- for
characters, cells, addresses, and double cells.

@cindex floating-point stack
@item
A floating point stack -- for holding floating point (FP) numbers.

@cindex return stack
@item
A return stack -- for holding the return addresses of colon
definitions and other (non-FP) data.

@cindex locals stack
@item
A locals stack -- for holding local variables.
@end itemize

@menu
* Data stack::
* Floating point stack::
* Return stack::
* Locals stack::
* Stack pointer manipulation::
@end menu

@node Data stack, Floating point stack, Stack Manipulation, Stack Manipulation
@subsection Data stack
@cindex data stack manipulation words
@cindex stack manipulations words, data stack


doc-drop
doc-nip
doc-dup
doc-over
doc-third
doc-fourth
doc-tuck
doc-swap
doc-pick
doc-rot
doc--rot
doc-?dup
doc-roll
doc-2drop
doc-2nip
doc-2dup
doc-2over
doc-2tuck
doc-2swap
doc-2rot


@node Floating point stack, Return stack, Data stack, Stack Manipulation
@subsection Floating point stack
@cindex floating-point stack manipulation words
@cindex stack manipulation words, floating-point stack

doc-fdrop
doc-fnip
doc-fdup
doc-fover
doc-fthird
doc-ffourth
doc-ftuck
doc-fswap
doc-fpick
doc-frot
doc-f-rot


@node Return stack, Locals stack, Floating point stack, Stack Manipulation
@subsection Return stack
@cindex return stack manipulation words
@cindex stack manipulation words, return stack

@cindex return stack and locals
@cindex locals and return stack
A Forth system is allowed to keep local variables on the
return stack. This is reasonable, as local variables usually eliminate
the need to use the return stack explicitly. So, if you want to produce
a standard compliant program and you are using local variables in a
word, forget about return stack manipulations in that word (refer to the
standard document for the exact rules).

doc->r
doc-r>
doc-r@
doc-rdrop
doc-2>r
doc-2r>
doc-2r@
doc-2rdrop
doc-n>r
doc-nr>

@node Locals stack, Stack pointer manipulation, Return stack, Stack Manipulation
@subsection Locals stack

Gforth uses an extra locals stack.  It is described, along with the
reasons for its existence, in @ref{Locals implementation}.

@node Stack pointer manipulation,  , Locals stack, Stack Manipulation
@subsection Stack pointer manipulation
@cindex stack pointer manipulation words

@c removed s0 r0 l0 -- they are obsolete aliases for sp0 rp0 lp0
doc-sp0
doc-sp@
doc-sp!
doc-fp0
doc-fp@
doc-fp!
doc-rp0
doc-rp@
doc-rp!
doc-lp0
doc-lp@
doc-lp!


@node Memory, Strings and Characters, Stack Manipulation, Words
@section Memory
@cindex memory words

@menu
* Memory model::
* Dictionary allocation::
* Heap Allocation::
* Memory Access::
* Special Memory Accesses::
* Address arithmetic::
* Memory Blocks::
@end menu

In addition to the standard Forth memory allocation words, there is also
a @uref{https://www.complang.tuwien.ac.at/forth/garbage-collection.zip,
garbage collector}.

@node Memory model, Dictionary allocation, Memory, Memory
@subsection Memory model

@c The Standard Forth description is a mess (e.g., is the heap part of
@c the dictionary?), so let's not stick to closely with it.

Standard Forth considers a Forth system as consisting of several address
spaces, of which only @dfn{data space} is managed and accessible with
the memory words.  Memory not necessarily in data space includes the
stacks, the code (called code space) and the headers (called name
space). In Gforth everything is in data space, but the code for the
primitives is usually read-only.

Data space is divided into a number of areas: The (data space portion of
the) dictionary@footnote{Sometimes, the term @dfn{dictionary} is used to
refer to the search data structure embodied in word lists and headers,
because it is used for looking up names, just as you would in a
conventional dictionary.}, the heap, and a number of system-allocated
buffers.

Gforth provides one big address space, and address arithmetic can be
performed between any addresses. However, in the dictionary headers or
code are interleaved with data, so almost the only contiguous data space
regions there are those described by Standard Forth as contiguous; but you
can be sure that the dictionary is allocated towards increasing
addresses even between contiguous regions.  The memory order of
allocations in the heap is platform-dependent (and possibly different
from one run to the next).


@node Dictionary allocation, Heap Allocation, Memory model, Memory
@subsection Dictionary allocation
@cindex reserving data space
@cindex data space - reserving some

Dictionary allocation is a stack-oriented allocation scheme, i.e., if
you want to deallocate X, you also deallocate everything
allocated after X.

@cindex contiguous regions in dictionary allocation
The allocations using the words below are contiguous and grow the region
towards increasing addresses.  Other words that allocate dictionary
memory of any kind (i.e., defining words including @code{:noname}) end
the contiguous region and start a new one.

In Standard Forth only @code{create}d words are guaranteed to produce an
address that is the start of the following contiguous region.  In
particular, the cell allocated by @code{variable} is not guaranteed to
be contiguous with following @code{allot}ed memory.

You can deallocate memory by using @code{allot} with a negative argument
(with some restrictions, see @code{allot}). For larger deallocations use
@code{marker}.


doc-here
doc-unused
doc-allot
doc-->here
doc-c,
doc-f,
doc-,
doc-2,
doc-w,
doc-l,
doc-x,
doc-xd,
doc-a,
doc-mem,
doc-save-mem-dict

Memory accesses have to be aligned (@pxref{Address arithmetic}). So of
course you should allocate memory in an aligned way, too. I.e., before
allocating allocating a cell, @code{here} must be cell-aligned, etc.
The words below align @code{here} if it is not already.  Basically it is
only already aligned for a type, if the last allocation was a multiple
of the size of this type and if @code{here} was aligned for this type
before.

After freshly @code{create}ing a word, @code{here} is @code{align}ed in
Standard Forth (@code{maxalign}ed in Gforth).

doc-align
doc-falign
doc-sfalign
doc-dfalign
doc-maxalign
doc-cfalign




@node Heap Allocation, Memory Access, Dictionary allocation, Memory
@subsection Heap allocation
@cindex heap allocation
@cindex dynamic allocation of memory
@cindex memory-allocation word set

@cindex contiguous regions and heap allocation
Heap allocation supports deallocation of allocated memory in any
order. Dictionary allocation is not affected by it (i.e., it does not
end a contiguous region). In Gforth, these words are implemented using
the standard C library calls malloc(), free() and realloc().

The memory region produced by one invocation of @code{allocate} or
@code{resize} is internally contiguous.  There is no contiguity between
such a region and any other region (including others allocated from the
heap).

doc-allocate
doc-free
doc-resize

The following words are useful for dealing with memory blocks:

doc-save-mem
doc-free-mem-var
doc-extend-mem


The $tring words can also be used for dealing with memory blocks,
@xref{$tring words}.

For growable memory buffers you can use $trings, or the following
words.  Because the allocated memory of a buffer managed with
@code{adjust-buffer} does not shrink, there is no heap-management
overhead when adjusting the buffer for a size smaller than the largest
one seen yet.

doc-buffer%
doc-init-buffer
doc-adjust-buffer

You can get the current address and length of such a buffer with
@code{2@@}.

Typical usage:

@example
create mybuf  buffer% %size allot  mybuf init-buffer
s" frobnicate" mybuf adjust-buffer  mybuf 2@@ move
mybuf 2@@ type
s" foo"        mybuf adjust-buffer  mybuf 2@@ move
mybuf 2@@ type
@end example


@node Memory Access, Special Memory Accesses, Heap Allocation, Memory
@subsection Memory Access
@cindex memory access words

doc-@
doc-!
doc-+!
doc-c@
doc-c!
doc-2@
doc-2!
doc-f@
doc-f!
doc-sf@
doc-sf!
doc-df@
doc-df!

@node Special Memory Accesses, Address arithmetic, Memory Access, Memory
@subsection Special Memory Accesses
@cindex byte order
@cindex big-endian
@cindex little-endian
@cindex sign extension
@cindex unaligned memory access

This section is about memory accesses useful for communicating with
other software or other computers.  This means that the accesses are
of a certain bit width (independent of Gforth's cell width), are
possibly not naturally aligned and typically have a certain byte order
that may be different from the native byte order of the system that
Gforth runs on.

We use the following prefixes:

@table @code
@item c
8 bits (character)
@item w
16 bits
@item l
32 bits
@item x
64 bits represented as one cell
@item xd
64 bits represented as two cells
@end table

The @code{x}-prefix words do not work properly on 32-bit systems, so
for code that is intended to be portable to 32-bit systems you should
use @code{xd}-prefix words.  Note that @code{xd}-prefix words work on
64-bit systems: there the upper cell is just 0 (for unsigned values)
or a sign extension of the lower cell.

The memory-access words below all work with arbitrarily (un)aligned
addresses (unlike @code{@@}, @code{!}, @code{f@@}, @code{f!}, which
require alignment on some hardware).

doc-w@
doc-w!
doc-l@
doc-l!
doc-x@
doc-x!
doc-xd@
doc-xd!

For accesses with a specific byte order, you have to perform
byte-order adjustment immediately after a fetch (before the
sign-extension), or immediately before the store.  The results of these
byte-order adjustment words are always zero-extended.

doc-wbe
doc-wle
doc-lbe
doc-lle
doc-xbe
doc-xle
doc-xdbe
doc-xdle

For signed fetches with a specific byte order, you have to perform a
sign-extension word after an unsigned fetch and a byte-order correction:

doc-c>s
doc-w>s
doc-l>s
doc-x>s
doc-xd>s

Overall, this leads to sequences like

@example
w@@ wbe w>s   \ 16-bit unaligned signed big-endian fetch
>r lle r> l! \ 32-bit unaligned little-endian store
@end example

@node Address arithmetic, Memory Blocks, Special Memory Accesses, Memory
@subsection Address arithmetic
@cindex address arithmetic words

Address arithmetic is the foundation on which you can build data
structures like arrays, records (@pxref{Structures}) and objects
(@pxref{Object-oriented Forth}).

@cindex address unit
@cindex au (address unit)
Standard Forth does not specify the sizes of the data types. Instead, it
offers a number of words for computing sizes and doing address
arithmetic. Address arithmetic is performed in terms of address units
(aus); on most systems the address unit is one byte. Note that a
character may have more than one au, so @code{chars} is no noop (on
platforms where it is a noop, it compiles to nothing).

The basic address arithmetic words are @code{+} and @code{-}.  E.g., if
you have the address of a cell, perform @code{1 cells +}, and you will
have the address of the next cell.

@cindex alignment of addresses for types
Standard Forth also defines words for aligning addresses for specific
types. Many computers require that accesses to specific data types
must only occur at specific addresses; e.g., that cells may only be
accessed at addresses divisible by 4. Even if a machine allows unaligned
accesses, it can usually perform aligned accesses faster. 

For the performance-conscious: alignment operations are usually only
necessary during the definition of a data structure, not during the
(more frequent) accesses to it.

Standard Forth defines no words for character-aligning addresses; in
Forth-2012 all addresses are character-aligned.

@cindex @code{CREATE} and alignment
Standard Forth guarantees that addresses returned by @code{CREATE}d words
are cell-aligned; in addition, Gforth guarantees that these addresses
are aligned for all purposes.

Note that the Standard Forth word @code{char} has nothing to do with address
arithmetic.


doc-chars
doc-char+
doc-char-
doc-cells
doc-cell+
doc-cell-
doc-cell/
doc-cell
doc-aligned
doc-floats
doc-float+
doc-float
doc-float/
doc-faligned
doc-sfloats
doc-sfloat+
doc-sfloat/
doc-sfaligned
doc-dfloats
doc-dfloat+
doc-dfloat/
doc-dfaligned
doc-maxaligned
doc-cfaligned
doc-*aligned
doc-*align
doc-waligned
doc-walign
doc-laligned
doc-lalign
doc-xaligned
doc-xalign

The environmental query @code{address-unit-bits} (@pxref{Environmental
Queries}) and the following words may be useful to those who want to
write software portable to non-byte-addressed machines.

doc-/w
doc-/l
doc-/x

@node Memory Blocks,  , Address arithmetic, Memory
@subsection Memory Blocks
@cindex memory block words
@cindex character strings - moving and copying

Memory blocks often represent character strings; For ways of storing
character strings in memory see @ref{String representations}.  For other
string-processing words see @ref{Displaying characters and strings}.

A few of these words work on address unit blocks.  In that case, you
usually have to insert @code{CHARS} before the word when working on
character strings.  Most words work on character blocks, and expect a
char-aligned address.

When copying characters between overlapping memory regions, use
@code{move}.  @code{Cmove} and @code{cmove>} tend to be slower than a
well-implemented @code{move}.

doc-move
doc-cmove
doc-cmove>
doc-fill
doc-erase
doc-blank
doc-insert
doc-delete
doc-compare
doc-pad

@comment TODO examples

@node Strings and Characters, Control Structures, Memory, Words
@section Strings and Characters

@menu
* Characters::
* String representations::
* String and character literals::
* String words::
* $tring words::
* Counted string words::
@end menu

@node Characters, String representations, Strings and Characters, Strings and Characters
@subsection Characters
@cindex Characters - chars/bytes vs. extended characters
@cindex character encoding

Forth supports chars (aka bytes), used by words such as @code{c@@};
these can be used to represent an ASCII character.

Forth also supports extended characters, which may be represented by a
sequence of several bytes (i.e., several chars).  A common character
encoding is the UTF-8 representation of Unicode.

In general, most code does not have to worry about extended
characters: In the string representation it does not matter whether a
byte is a part of an extended character, or it is a character by
itself, and words that consume chars (like @code{emit}) also work when
the extended character is transferred as a sequence of chars.  Forth
still provides words for dealing with extended characters
(@pxref{Xchars and Unicode}).

In Unicode terms, chars are @i{code units}, whereas extended
characters are @i{code points}.  Note that an Unicode @i{abstract
character} can consist of a sequence of code points, but Forth (like
other programming languages) has no data type for individual abstract
characters; of course, they can be represented as strings.

You can use the usual integer words on chars and Xchars on the stack,
but Gforth also has some words for dealing with chars on the stack:

doc-toupper

@node String representations, String and character literals, Characters, Strings and Characters
@subsection String representations
@cindex strings - see character strings
@cindex character strings - representations
@cindex counted strings

Forth commonly represents strings as cell pair @i{c-addr u} on the
stack; @i{u} is the length of the string in bytes (aka chars), and
@i{c-addr} is the address of the first byte of the string.  Note that
a code point may be represented by a sequence of several chars in the
string (and a Unicode ``abstract character'' may consist of several
code points).  @xref{String words}.

Another string representation is used with the string library of words
containing @code{$}.  It represents the string on the stack through
the address of a cell-sized string handle, which can be located in,
e.g., a variable.  @xref{$tring words}.

A legacy string representation are @dfn{counted strings}, represented
on the stack by @i{c-addr}. The char addressed by @i{c-addr} contains
a character-count, @i{n}, of the string and the string occupies the
subsequent @i{n} char addresses in memory.  Counted strings are
limited to 255 bytes in length.  While counted strings may look
attractive due to needing only one stack item, due to their
limitations we recommend avoiding them, especially as input parameters
of words.  @xref{Counted string words}.

@node String and character literals, String words, String representations, Strings and Characters
@subsection String and Character literals
@cindex character literals
@cindex string literals
@cindex literals for characters and strings

The nicest way to write a string literal is to write it as
@code{"@i{STRING}"}.  You can use the same \-escapes inside as for
@code{s\"}.  However, this way is non-standard, so you may want
to use one of the following words for improved portability:

doc-s\"
doc-s"

Likewise, You can get the code @i{xc} of a character
@i{C} with @code{'@i{C}'}.  This way has been standardized since
Forth-2012.  An older way to get it is to use one of the following
words:

doc-char
doc-[char]

You usually use @code{char} outside and @code{[char]} inside colon
definitions, or you just use @code{'@i{C}'}.

Note that, e.g.,

@example
"C" type
@end example

@noindent
is (slightly) more efficient than

@example
'C' xemit
@end example

@noindent
because the latter converts the code point into a sequence of bytes
and individually @code{emit}s them.  Similarly, dealing with general
characters is usually more efficient when representing them as strings
rather than code points.

There are the following words for producing commonly-used characters
and strings that cannot be produced with @code{S"} or @code{'@i{C}'}:

doc-newline
doc-bl
doc-#tab
doc-#lf
doc-#cr
doc-#ff
doc-#bs
doc-#del
doc-#bell
doc-#esc
doc-#eof

@node String words, $tring words, String and character literals, Strings and Characters
@subsection String words

Words that are used for memory blocks are also useful for strings, so
for words that move, copy, compare and search strings, see @ref{Memory
Blocks}. For words that display characters and strings, see
@ref{Displaying characters and strings}.

The following words work on previously existing strings:

doc-str=
doc-str<
doc-string-prefix?
doc-string-suffix?
doc-search
doc-scan
doc-scan-back
doc-skip
doc--trailing
doc-/string
doc-safe/string
doc-cstring>sstring

The following words compare case-insensitively for ASCII characters,
but case-sensitively for non-ASCII characters (like in lookup in
wordlists).

doc-capscompare
doc-capsstring-prefix?
doc-capssearch

The following words create or extend strings on the heap:

doc-s+
doc-append
doc->string-execute

One could define @code{s+} using @code{>string-execute}, as follows:

: s+ ( c-addr1 u1 c-addr2 u2 -- c-addr u )
  [: 2swap type type ;] >string-execute ;

For concatenating just two strings @code{>string-execute} is
inefficient, but for concatenating many strings @code{>string-execute}
can be more efficient.

@node $tring words, Counted string words, String words, Strings and Characters
@subsection $tring words
@cindex string words with $

The following string library stores strings in ordinary cell-size
variables (string handles).  These handles contain a pointer to a
cell-counted string allocated from the heap.  The string library
originates from bigFORTH.

Because there is only one permanent reference to the contents (the one
in the handle), the string can be relocated or deleted without
worrying about dangling references; this requires that the programmer
uses references produced by, e.g., @code{$@@} only for temporary
purposes, i.e., these references are not passed out, e.g., as return
values or stored in global memory, and words that may change the
handle are not called while these references exist.

This library is complemented by the cell-pair representation: You use
the $tring words for variable strings which are cumbersome with the
c-addr u representation.  You use the cell-pair representation for
processing (e.g., inspecting) strings while they do not change.

doc-$!
doc-$@
doc-$@len
doc-$!len
doc-$+!len
doc-$del
doc-$ins
doc-$+!
doc-c$+!
doc-$free
doc-$init
doc-$split
doc-$iter
doc-$over
doc-$exec
doc-$tmp
doc-$.
doc-$slurp
doc-$slurp-file
doc-$+slurp
doc-$+slurp-file
doc-$[]
doc-$[]!
doc-$[]+!
doc-$+[]!
doc-$[]@
doc-$[]#
doc-$[]map
doc-$[]slurp
doc-$[]slurp-file
doc-$[].
doc-$[]free
doc-$save
doc-$[]save
doc-$boot
doc-$[]boot
doc-$saved
doc-$[]saved
doc-$variable
doc-$[]variable


@node Counted string words,  , $tring words, Strings and Characters
@subsection Counted string words

Counted strings store the length as byte at the address pointed to,
followed by the bytes of the string.  Their possible length is
severely limited, and you cannot create a substring in-place without
destroying the input string.  Therefore we recommend against using
counted strings.  Nevertheless, if you have to deal with counted
strings, here are some words for that:

doc-count

The following word has no useful interpretation semantics (unlike
@code{s"}) and no interpretive counterpart (unlike @code{[char]}), so
you should use it only inside colon definitions (if at all):

doc-c"

doc-place
doc-string,

@node Control Structures, Defining Words, Strings and Characters, Words
@section Control Structures
@cindex control structures

Control structures in Forth cannot be used interpretively, only in a
colon definition@footnote{To be precise, they have no interpretation
semantics (@pxref{Interpretation and Compilation Semantics}).}. We do
not like this limitation, but have not seen a satisfying way around it
yet, although many schemes have been proposed.

@menu
* Selection::                   IF ... ELSE ... ENDIF
* Simple Loops::                BEGIN ...
* Counted Loops::               DO
* BEGIN loops with multiple exits::
* General control structures with CASE::
* Arbitrary control structures::
* Calls and returns::
* Exception Handling::
@end menu

@node Selection, Simple Loops, Control Structures, Control Structures
@subsection Selection
@cindex selection control structures
@cindex control structures for selection

@cindex @code{IF} control structure
@example
@i{flag}
IF
  @i{code}
ENDIF
@end example
@noindent

If @i{flag} is non-zero (as far as @code{IF} etc. are concerned, a cell
with any bit set represents truth) @i{code} is executed.

@example
@i{flag}
IF
  @i{code1}
ELSE
  @i{code2}
ENDIF
@end example

If @var{flag} is true, @i{code1} is executed, otherwise @i{code2} is
executed.

You can use @code{THEN} instead of @code{ENDIF}. Indeed, @code{THEN} is
standard, and @code{ENDIF} is not, although it is quite popular. We
recommend using @code{ENDIF}, because it is less confusing for people
who also know other languages (and is not prone to reinforcing negative
prejudices against Forth in these people). Adding @code{ENDIF} to a
system that only supplies @code{THEN} is simple:
@example
: ENDIF   POSTPONE then ; immediate
@end example

[According to @cite{Webster's New Encyclopedic Dictionary}, @dfn{then
(adv.)}  has the following meanings:
@quotation
... 2b: following next after in order ... 3d: as a necessary consequence
(if you were there, then you saw them).
@end quotation
Forth's @code{THEN} has the meaning 2b, whereas @code{THEN} in Pascal
and many other programming languages has the meaning 3d.]

Gforth also provides the words @code{?DUP-IF} and @code{?DUP-0=-IF}, so
you can avoid using @code{?dup}. Using these alternatives is also more
efficient than using @code{?dup}. Definitions in Standard Forth
for @code{ENDIF}, @code{?DUP-IF} and @code{?DUP-0=-IF} are provided in
@file{compat/control.fs}.

@cindex @code{CASE} control structure
@example
@i{x}
CASE
  @i{x1} OF @i{code1} ENDOF
  @i{x2} OF @i{code2} ENDOF
  @dots{}
  ( x ) @i{default-code} ( x )
ENDCASE ( )
@end example

Executes the first @i{codei}, where the @i{xi} is equal to @i{x}.  If no
@i{xi} matches, the optional @i{default-code} is executed. The optional
default case can be added by simply writing the code after the last
@code{ENDOF}. It may use @i{x}, which is on top of the stack, but must
not consume it.  The value @i{x} is consumed by this construction
(either by an @code{OF} that matches, or by the @code{ENDCASE}, if no OF
matches).  Example:

@example
: num-name ( n -- c-addr u )
 case
   0 of s" zero " endof
   1 of s" one "  endof
   2 of s" two "  endof
   \ default case:
   s" other number" 
   rot \ get n on top so ENDCASE can drop it
 endcase ;
@end example

You can also use (the non-standard) @code{?of} to use @code{case} as a
general selection structure for more than two alternatives.
@code{?Of} takes a flag.  Example:

@example
: sgn ( n1 -- n2 )
    \ sign function
    case
	dup 0< ?of drop -1 endof
	dup 0> ?of drop 1 endof
	dup \ n1=0 -> n2=0; dup an item, to be consumed by ENDCASE
    endcase ;
@end example

@progstyle
To keep the code understandable, you should ensure that you change the
stack in the same way (wrt. number and types of stack items consumed
and pushed) on all paths through a selection structure.

@node Simple Loops, Counted Loops, Selection, Control Structures
@subsection Simple Loops
@cindex simple loops
@cindex loops without count 

@cindex @code{WHILE} loop
@example
BEGIN
  @i{code1}
  @i{flag}
WHILE
  @i{code2}
REPEAT
@end example

@i{code1} is executed and @i{flag} is computed. If it is true,
@i{code2} is executed and the loop is restarted; If @i{flag} is
false, execution continues after the @code{REPEAT}.

@cindex @code{UNTIL} loop
@example
BEGIN
  @i{code}
  @i{flag}
UNTIL
@end example

@i{code} is executed. The loop is restarted if @code{flag} is false.

@progstyle
To keep the code understandable, a complete iteration of the loop should
not change the number and types of the items on the stacks.

@cindex endless loop
@cindex loops, endless
@example
BEGIN
  @i{code}
AGAIN
@end example

This is an endless loop.


@node Counted Loops, BEGIN loops with multiple exits, Simple Loops, Control Structures
@subsection Counted Loops
@cindex counted loops
@cindex loops, counted
@cindex @code{DO} loops

The basic counted loop is:
@example
@i{limit} @i{start} ?DO
  @i{body}
LOOP
@end example

This performs one iteration for every integer, starting from @i{start}
and up to, but excluding @i{limit}. The counter, or @i{index}, can be
accessed with @code{i}. For example, the loop:
@example
10 0 ?DO
  i .
LOOP
@end example
@noindent
prints @code{0 1 2 3 4 5 6 7 8 9}

The index of the innermost loop can be accessed with @code{i}, the index
of the next loop with @code{j}, and the index of the third loop with
@code{k}.

You can access the limit of the innermost loop with @code{i'} and
@code{i'}-@code{i} with @code{delta-i}.  E.g., running

@example
: foo 7 5 ?do cr i . i' . delta-i . loop ; 
@end example

prints

@example
5 7 2 
6 7 1
@end example

The loop control data are kept on the return stack, so there are some
restrictions on mixing return stack accesses and counted loop words. In
particuler, if you put values on the return stack outside the loop, you
cannot read them inside the loop@footnote{well, not in a way that is
portable.}. If you put values on the return stack within a loop, you
have to remove them before the end of the loop and before accessing the
index of the loop.

There are several variations on the counted loop:

@itemize @bullet
@item
@code{LEAVE} leaves the innermost counted loop immediately; execution
continues after the associated @code{LOOP} or @code{NEXT}. For example:

@example
10 0 ?DO  i DUP . 3 = IF LEAVE THEN LOOP
@end example
prints @code{0 1 2 3}


@item
@code{UNLOOP} prepares for an abnormal loop exit, e.g., via
@code{EXIT}. @code{UNLOOP} removes the loop control parameters from the
return stack so @code{EXIT} can get to its return address. For example:

@example
: demo 10 0 ?DO i DUP . 3 = IF UNLOOP EXIT THEN LOOP ." Done" ;
@end example
prints @code{0 1 2 3}


@item
If @i{start} is greater than @i{limit}, a @code{?DO} loop is entered
(and @code{LOOP} iterates until they become equal by wrap-around
arithmetic). This behaviour is usually not what you want. Therefore,
Gforth offers @code{+DO} and @code{U+DO} (as replacements for
@code{?DO}), which do not enter the loop if @i{start} is greater than
@i{limit}; @code{+DO} is for signed loop parameters, @code{U+DO} for
unsigned loop parameters.

@item
@code{?DO} can be replaced by @code{DO}. @code{DO} always enters
the loop, independent of the loop parameters. Do not use @code{DO}, even
if you know that the loop is entered in any case. Such knowledge tends
to become invalid during maintenance of a program, and then the
@code{DO} will make trouble.

@item
@code{LOOP} can be replaced with @code{@i{n} +LOOP}; this updates the
index by @i{n} instead of by 1. The loop is terminated when the border
between @i{limit-1} and @i{limit} is crossed. E.g.:

@example
4 0 +DO  i .  2 +LOOP
@end example
@noindent
prints @code{0 2}

@example
4 1 +DO  i .  2 +LOOP
@end example
@noindent
prints @code{1 3}

@item
@cindex negative increment for counted loops
@cindex counted loops with negative increment
The behaviour of @code{@i{n} +LOOP} is peculiar when @i{n} is negative:

@example
-1 0 ?DO  i .  -1 +LOOP
@end example
@noindent
prints @code{0 -1}

@example
0 0 ?DO  i .  -1 +LOOP
@end example
prints nothing.

We recommend not combining @code{?DO} with @code{+LOOP}.  Gforth offers
several alternatives:

If you want @code{-1 +LOOP}'s behaviour of including an iteration
where @code{I}=limit, start the loop with @code{-[DO} or @code{U-[DO}
(where the @code{[} is inspired by the mathematical notation for
inclusive ranges, e.g., [1,n]):

@example
-1 0 -[DO  i .  -1 +LOOP
@end example

prints @code{0 -1}.

@example
0 0 -[DO  i .  -1 +LOOP
@end example

prints @code{0}.

@example
0 -1 -[DO  i .  -1 +LOOP
@end example

prints nothing.

If you want to exclude the limit, you instead use @code{1 -LOOP} (or
generally @code{@i{u} -LOOP}) and start the loop with @code{?DO},
@code{-DO} or @code{U-DO}.  @code{-LOOP} terminates the loop when the
border between @i{limit+1} and @i{limit} is crossed.  E.g.:

@example
-2 0 -DO  i .  1 -LOOP
@end example
@noindent
prints @code{0 -1}

@example
-1 0 -DO  i .  1 -LOOP
@end example
@noindent
prints @code{0}

@example
0 0 -DO  i .  1 -LOOP
@end example
@noindent
prints nothing.

Unfortunately, @code{+DO}, @code{U+DO}, @code{-DO}, @code{U-DO} and
@code{-LOOP} are not defined in Standard Forth. However, an implementation
for these words that uses only standard words is provided in
@file{compat/loops.fs}.

@item
@cindex iterate over array
@cindex array, iterating over
A common task is to iterate over the elements of an array, forwards or
backwards.  Iterating over the addresses of the elements has two
benefits: It avoids the need to keep the start address of the array
around, reducing the data stack load; and it avoids the need to
perform address computations in every iteration.  The disadvantage is
that, starting with the usual array representations @i{addr uelems} or
@i{addr ubytes}, some processing is required to produce a start and
limit address.  Gforth has @code{bounds} for getting there from the
@i{addr ubytes} representation, so you can write a forward loop
through a cell array @code{v} as:

@example
create v 1 , 3 , 7 ,
: foo v 3 cells bounds U+DO i @ . cell +LOOP ;
foo
@end example

which prints @code{1 3 7}.  Preprocessing the inputs for walking
backwards is more involved, so Gforth provide a loop construct of the
form @code{MEM-DO}...@code{LOOP} that does it for you: It takes an
array in @i{addr ubytes} representation and the element size, and
iterates over the addresses of the elements in backwards order:

@example
create v 1 , 3 , 7 ,
: foo1 v 3 cell array>mem MEM-DO i @ . LOOP ;
foo1
@end example

This prints @code{7 3 1}.  @code{ARRAY>MEM} converts the @i{addr
uelems uelemsize} representation into the @i{addr ubytes uelemsize}
representation expected by @code{MEM-DO}.  This loop is finished with
@code{LOOP} which decrements by @i{uelemsize} when it finishes a
@code{MEM-DO}.

Gforth also adds @code{MEM+DO} for completeness.  It takes the same
parameters as @code{MEM-DO}, but walks forwards through the array:

@example
create v 1 , 3 , 7 ,
: foo2 v 3 cell array>mem MEM+DO i @ . LOOP ;
foo2
@end example

prints @code{1 3 7}.

@item
@cindex @code{FOR} loops
Another counted loop is:
@example
@i{n}
FOR
  @i{body}
NEXT
@end example
This is the preferred loop of native code compiler writers who are too
lazy to optimize @code{?DO} loops properly. This loop structure is not
defined in Standard Forth. In Gforth, this loop iterates @i{n+1} times;
@code{i} produces values starting with @i{n} and ending with 0. Other
Forth systems may behave differently, even if they support @code{FOR}
loops. To avoid problems, don't use @code{FOR} loops.

@end itemize

The counted-loop words are:

doc-?do
doc-+do
doc-u+do
doc-bounds
doc--[do
doc-u-[do
doc--do
doc-u-do
doc-array>mem
doc-mem+do
doc-mem-do
doc-do
doc-for
doc-loop
doc-+loop
doc--loop
doc-next
doc-i
doc-j
doc-k
doc-i'
doc-delta-i
doc-leave
doc-?leave
doc-unloop
doc-done


The standard does not allow using @code{CS-PICK} and @code{CS-ROLL} on
@i{do-sys}. Gforth allows it, except for the do-sys produced by
@code{MEM+DO} and @code{MEM-DO}, but it's your job to ensure that for
every @code{?DO} etc. there is exactly one @code{UNLOOP} on any path
through the definition (@code{LOOP} etc. compile an @code{UNLOOP} on
the fall-through path). Also, you have to ensure that all
@code{LEAVE}s are resolved (by using one of the loop-ending words or
@code{DONE}).

@node BEGIN loops with multiple exits, General control structures with CASE, Counted Loops, Control Structures
@subsection @code{Begin} loops with multiple exits
@cindex Multiple exits from @code{begin}

For counted loops, you can use @code{leave} in several places.  For
@code{begin} loops, you have the following options:

Use @code{exit} (possibly several times) in the loop to leave not just
the loop, but the whole colon definition.  E.g.,:

@example
: foo
  begin
    condition1 while
      condition2 if
        exit-code2 exit then
      condition3 if
        exit-code3 exit then
    ...
  repeat
  exit-code1 ;
@end example
  
The disadvantage of this approach is that, if you want to have some
common code afterwards, you either have to wrap @code{foo} in another
word that contains the common code, or you have to call the common
code several times, from each exit-code.

Another approach is to use several @code{while}s in a @code{begin}
loop.  You have to append a @code{then} behind the loop for every
additional @code{while}.  E.g.,;

@example
begin
  condition1 while
    condition2 while
      condition3 while
again then then then
@end example

Here I used @code{again} at the end of the loop so that I would have a
@code{then} for each @code{while}; @code{repeat} would result in one
less @code{then}, but otherwise the same behaviour.  For an
explanation of why this works, @xref{Arbitrary control structures}.

We can have common code afterwards, but, as presented above, we cannot
have different exit-codes for the different exits.  You can have these
different exit-codes, as follows:

@example
begin
  condition1 while
    condition2 while
      condition3 while
again then exit-code3
else exit-code2 then
else exit-code1 then
@end example

This is relatively hard to comprehend, because the exit-codes are
relatively far from the exit conditions (it does not help that we are
not used to such control structures, either).

@node General control structures with CASE, Arbitrary control structures, BEGIN loops with multiple exits, Control Structures
@subsection  General control structures with @code{case}
@cindex @code{case} as generalized control structure
@cindex general control structures (@code{case})

Gforth provides an extended @code{case} that solves the problems of
the multi-exit loops discussed above, and offers additional options.
You can find a portable implementation of this extended @code{case} in
@file{compat/caseext.fs}.

There are three additional words in the extension.  The first is
@code{?of} which allows general tests (rather than just testing for
equality) in a @code{case};  e.g.,

@example
: sgn ( n -- -1|0|1 )
  ( n ) case
    dup 0 < ?of drop -1 endof
    dup 0 > ?of drop 1  endof
    \ otherwise leave the 0 on the stack
  0 endcase ;
@end example

Note that @code{endcase} drops a value, which works fine much of the
time with @code{of}, but usually not with @code{?of}, so we leave a 0
on the stack for @code{endcase} to drop.  The n that is passed into
@code{sgn} is also 0 if neither @code{?of} triggers, and that is then
passed out.

The second additional word is @code{next-case}, which allows turning
@code{case} into a loop.  Our triple-exit loop becomes:

@example
case
  condition1 ?of exit-code1 endof
  condition2 ?of exit-code2 endof
  condition3 ?of exit-code3 endof
  ...
next-case
common code afterwards
@end example

As you can see, this solves both problems of the variants discussed
above (@pxref{BEGIN loops with multiple exits}).  Note that
@code{next-case} does not drop a value, unlike
@code{endcase}.@footnote{@code{Next-case} has a @code{-}, unlike the
other @code{case} words, because VFX Forth contains a @code{nextcase}
that drops a value.}

The last additional word is @code{contof}, which is used instead of
@code{endof} and starts the next iteration instead of leaving the
loop.  This can be used in ways similar to Dijkstra's guarded command
@i{do}, e.g.:

@example
: gcd ( n1 n2 -- n )
    case
        2dup > ?of tuck - contof
        2dup < ?of over - contof
    endcase ;
@end example

Here the two @code{?of}s have different ways of continuing the loop;
when neither @code{?of} triggers, the two numbers are equal and are
the gcd.  @code{Endcase} drops one of them, leaving the other as n.

You can also combine these words.  Here's an example that uses each of
the @code{case} words once, except @code{endcase}:

@example
: collatz ( u -- )
    \ print the 3n+1 sequence starting at u until we reach 1
    case
        dup .
        1 of endof
        dup 1 and ?of 3 * 1+ contof
        2/
    next-case ;
@end example

This example keeps the current value of the sequence on the stack.  If
it is 1, the @code{of} triggers, drops the value, and leaves the
@code{case} structure.  For odd numbers, the @code{?of} triggers,
computes 3n+1, and starts the next iteration with @code{contof}.
Otherwise, if the number is even, it is divided by 2, and the loop is
restarted with @code{next-case}.

@node Arbitrary control structures, Calls and returns, General control structures with CASE, Control Structures
@subsection Arbitrary control structures
@cindex control structures, user-defined

@cindex control-flow stack
Standard Forth permits and supports using control structures in a non-nested
way. Information about incomplete control structures is stored on the
control-flow stack. This stack may be implemented on the Forth data
stack, and this is what we have done in Gforth.

@cindex @code{orig}, control-flow stack item
@cindex @code{dest}, control-flow stack item
An @i{orig} entry represents an unresolved forward branch, a @i{dest}
entry represents a backward branch target. A few words are the basis for
building any control structure possible (except control structures that
need storage, like calls, coroutines, and backtracking).


doc-if
doc-ahead
doc-then
doc-begin
doc-until
doc-again
doc-cs-pick
doc-cs-roll
doc-cs-drop

The Standard words @code{CS-PICK} and @code{CS-ROLL} allow you to
manipulate the control-flow stack in a portable way. Without them, you
would need to know how many stack items are occupied by a control-flow
entry (many systems use one cell. In Gforth they currently take three,
but this may change in the future).

@code{CS-PICK} can only pick a dest and @code{CS-DROP} can only drop a
dest, because an orig must be resolved exactly once.

Some standard control structure words are built from these words:


doc-else
doc-while
doc-repeat


@noindent
Gforth adds some more control-structure words:


doc-endif
doc-?dup-if
doc-?dup-0=-if

@noindent
Another group of control structure words are:


doc-case
doc-endcase
doc-next-case
doc-of
doc-?of
doc-endof
doc-contof

Internally, @i{of-sys} is an @code{orig}; and @i{case-sys} is a cell
and some stack-depth information, 0 or more @code{orig}s, and a
@code{dest}.

@subsubsection Programming Style
@cindex control structures programming style
@cindex programming style, arbitrary control structures

In order to ensure readability we recommend that you do not create
arbitrary control structures directly, but define new control structure
words for the control structure you want and use these words in your
program. For example, instead of writing:

@example
BEGIN
  ...
IF [ 1 CS-ROLL ]
  ...
AGAIN THEN
@end example

@noindent
we recommend defining control structure words, e.g.,

@example
: WHILE ( DEST -- ORIG DEST )
 POSTPONE IF
 1 CS-ROLL ; immediate

: REPEAT ( orig dest -- )
 POSTPONE AGAIN
 POSTPONE THEN ; immediate
@end example

@noindent
and then using these to create the control structure:

@example
BEGIN
  ...
WHILE
  ...
REPEAT
@end example

That's much easier to read, isn't it? Of course, @code{REPEAT} and
@code{WHILE} are predefined, so in this example it would not be
necessary to define them.

@node Calls and returns, Exception Handling, Arbitrary control structures, Control Structures
@subsection Calls and returns
@cindex calling a definition
@cindex returning from a definition

@cindex recursive definitions
A definition can be called simply be writing the name of the definition
to be called. Normally a definition is invisible during its own
definition. If you want to write a directly recursive definition, you
can use @code{recursive} to make the current definition visible, or
@code{recurse} to call the current definition directly.


doc-recursive
doc-recurse

@noindent For examples of using these words, @xref{Recursion Tutorial}.

@progstyle
I prefer using @code{recursive} to @code{recurse}, because calling the
definition by name is more descriptive (if the name is well-chosen) than
the somewhat cryptic @code{recurse}.  E.g., in a quicksort
implementation, it is much better to read (and think) ``now sort the
partitions'' than to read ``now do a recursive call''.

For mutual recursion, use @code{Defer}red words, like this:

@example
Defer foo

: bar ( ... -- ... )
 ... foo ... ;

:noname ( ... -- ... )
 ... bar ... ;
IS foo
@end example

Deferred words are discussed in more detail in @ref{Deferred Words}.

The current definition returns control to the calling definition when
the end of the definition is reached or @code{EXIT} is encountered.

doc-exit
doc-?exit
doc-;s


@node Exception Handling,  , Calls and returns, Control Structures
@subsection Exception Handling
@cindex exceptions

@c quit is a very bad idea for error handling, 
@c because it does not translate into a THROW
@c it also does not belong into this chapter

If a word detects an error condition that it cannot handle, it can
@code{throw} an exception.  In the simplest case, this will terminate
your program, and report an appropriate error.

doc-throw
doc-fast-throw

@code{Throw} consumes a cell-sized error number on the stack. There are
some predefined error numbers in Standard Forth (see @file{errors.fs}).  In
Gforth (and most other systems) you can use the iors produced by various
words as error numbers (e.g., a typical use of @code{allocate} is
@code{allocate throw}).  Gforth also provides the word @code{exception}
to define your own error numbers (with decent error reporting); a Standard
Forth version of this word (but without the error messages) is available
in @code{compat/except.fs}.  And finally, you can use your own error
numbers (anything outside the range -4095..0), but won't get nice error
messages, only numbers.  For example, try:

@example
-10 throw                    \ Standard defined
-267 throw                   \ system defined
s" my error" exception throw \ user defined
7 throw                      \ arbitrary number
@end example

doc---exception-exception

There are also cases where you have a word (typically modeled after
POSIX' @code{strerror}) for converting an error number into a string.
You can use the following word to get these strings into Gforth's
error handling:

doc-exceptions

As an example, if the @code{errno} errors (and the conversion using
@code{strerror}) was not already directly supported by Gforth, you
could tie @code{strerror} in as follows:

@example
' strerror 1536 exceptions constant errno-base
: errno-ior ( -- n )
\ n is the Gforth ior corresponding to the value in errno, so
\ we have to convert between the ranges here.
\ ERRNO is not a Gforth word, so you  would have to use the
\ C interface to access it.
  errno errno-base over - swap 0<> and ;
@end example

When you call a C function (with the C interface) and its return value
indictes that an error happened, you can then perform @code{errno-ior
throw} to produce an exception with the proper error message (such as
``Permission denied'').

A common idiom to @code{THROW} a specific @i{err#} if a
flag is true is this:

@example
@code{( flag ) 0<> @i{err#} and throw}
@end example

Your program can provide exception handlers to catch exceptions.  An
exception handler can be used to correct the problem, or to clean up
some data structures and just throw the exception to the next exception
handler.  Note that @code{throw} jumps to the dynamically innermost
exception handler.  The system's exception handler is outermost, and just
prints an error and restarts command-line interpretation (or, in batch
mode (i.e., while processing the shell command line), leaves Gforth).

The Standard Forth way to catch exceptions is @code{catch}:

doc-catch
doc-nothrow

The most common use of exception handlers is to clean up the state when
an error happens.  E.g.,

@example
base @@ >r hex \ actually the HEX should be inside foo to protect
              \ against exceptions between HEX and CATCH
['] foo catch ( nerror|0 )
r> base !
( nerror|0 ) throw \ pass it on
@end example

A use of @code{catch} for handling the error @code{myerror} might look
like this:

@example
['] foo catch
CASE
  myerror OF ... ( do something about it ) nothrow ENDOF
  dup throw \ default: pass other errors on, do nothing on non-errors
ENDCASE
@end example

Having to wrap the code into a separate word is often cumbersome,
therefore Gforth provides an alternative syntax:

@example
TRY
  @i{code1}
  IFERROR
    @i{code2}
  THEN
  @i{code3}
ENDTRY
@end example

This performs @i{code1}.  If @i{code1} completes normally, execution
continues with @i{code3}.  If there is an exception in @i{code1} or
before @code{endtry}, the stacks are reset to the depth during
@code{try}, the throw value is pushed on the data stack, and execution
continues at @i{code2}, and finally falls through to @i{code3}.

doc-try
doc-endtry
doc-iferror

If you don't need @i{code2}, you can write @code{restore} instead of
@code{iferror then}:

@example
TRY
  @i{code1}
RESTORE
  @i{code3}
ENDTRY
@end example

@cindex unwind-protect
The cleanup example from above in this syntax:

@example
base @@ @{ oldbase @}
TRY
  hex foo \ now the hex is placed correctly
  0       \ value for throw
RESTORE
  oldbase base !
ENDTRY
throw
@end example

An additional advantage of this variant is that an exception between
@code{restore} and @code{endtry} (e.g., from the user pressing
@kbd{Ctrl-C}) restarts the execution of the code after @code{restore},
so the base will be restored under all circumstances.

However, you have to ensure that this code does not cause an exception
itself, otherwise the @code{iferror}/@code{restore} code will loop.
Moreover, you should also make sure that the stack contents needed by
the @code{iferror}/@code{restore} code exist everywhere between
@code{try} and @code{endtry}; in our example this is achived by
putting the data in a local before the @code{try} (you cannot use the
return stack because the exception frame (@i{sys1}) is in the way
there).

This kind of usage corresponds to Lisp's @code{unwind-protect}.

@cindex @code{recover} (old Gforth versions)
If you do not want this exception-restarting behaviour, you achieve
this as follows:

@example
TRY
  @i{code1}
ENDTRY-IFERROR
  @i{code2}
THEN
@end example

If there is an exception in @i{code1}, then @i{code2} is executed,
otherwise execution continues behind the @code{then} (or in a possible
@code{else} branch).  This corresponds to the construct

@example
TRY
  @i{code1}
RECOVER
  @i{code2}
ENDTRY
@end example

in Gforth before version 0.7.  So you can directly replace
@code{recover}-using code; however, we recommend that you check if it
would not be better to use one of the other @code{try} variants while
you are at it.

To ease the transition, Gforth provides two compatibility files:
@file{endtry-iferror.fs} provides the @code{try ... endtry-iferror
... then} syntax (but not @code{iferror} or @code{restore}) for old
systems; @file{recover-endtry.fs} provides the @code{try ... recover
... endtry} syntax on new systems, so you can use that file as a
stopgap to run old programs.  Both files work on any system (they just
do nothing if the system already has the syntax it implements), so you
can unconditionally @code{require} one of these files, even if you use
a mix old and new systems.

doc-restore
doc-endtry-iferror

Here's the error handling example:

@example
TRY
  foo
ENDTRY-IFERROR
  CASE
    myerror OF ... ( do something about it ) nothrow ENDOF
    throw \ pass other errors on
  ENDCASE
THEN
@end example

@progstyle
As usual, you should ensure that the stack depth is statically known at
the end: either after the @code{throw} for passing on errors, or after
the @code{ENDTRY} (or, if you use @code{catch}, after the end of the
selection construct for handling the error).

There are two alternatives to @code{throw}: @code{Abort"} is conditional
and you can provide an error message.  @code{Abort} just produces an
``Aborted'' error.

The problem with these words is that exception handlers cannot
differentiate between different @code{abort"}s; they just look like
@code{-2 throw} to them (the error message cannot be accessed by
standard programs).  Similar @code{abort} looks like @code{-1 throw} to
exception handlers.

doc-abort"
doc-abort

For problems that are not that awful that you need to abort execution,
you can just display a warning.  The variable @code{warnings} allows
to tune how many warnings you see.

doc-warning"
doc-warnings

@c -------------------------------------------------------------
@node Defining Words, Interpretation and Compilation Semantics, Control Structures, Words
@section Defining Words
@cindex defining words

Defining words are used to extend Forth by creating new entries in the dictionary.

@menu
* CREATE::
* Variables::                   Variables and user variables
* Constants::
* Values::                      Initialised variables
* Varues::
* Colon Definitions::
* Anonymous Definitions::       Definitions without names
* Quotations::
* Supplying names::             Passing definition names as strings
* User-defined Defining Words::
* Deferred Words::              Allow forward references
* Forward::                     Auto-resolved forward references
* Aliases::
@end menu

@node CREATE, Variables, Defining Words, Defining Words
@subsection @code{CREATE}
@cindex simple defining words
@cindex defining words, simple

Defining words are used to create new entries in the dictionary. The
simplest defining word is @code{CREATE}. @code{CREATE} is used like
this:

@example
CREATE new-word1
@end example

@code{CREATE} is a parsing word, i.e., it takes an argument from the
input stream (@code{new-word1} in our example).  It generates a
dictionary entry for @code{new-word1}. When @code{new-word1} is
executed, all that it does is leave an address on the stack. The address
represents the value of the data space pointer (@code{HERE}) at the time
that @code{new-word1} was defined. Therefore, @code{CREATE} is a way of
associating a name with the address of a region of memory.

doc-create

Note that Standard Forth guarantees only for @code{create} that its body
is in dictionary data space (i.e., where @code{here}, @code{allot}
etc. work, @pxref{Dictionary allocation}).  Also, in Standard Forth only
@code{create}d words can be modified with @code{does>}
(@pxref{User-defined Defining Words}).  And in Standard Forth @code{>body}
can only be applied to @code{create}d words.

By extending this example to reserve some memory in data space, we end
up with something like a @i{variable}. Here are two different ways to do
it:

@example
CREATE new-word2 1 cells allot  \ reserve 1 cell - initial value undefined
CREATE new-word3 4 ,            \ reserve 1 cell and initialise it (to 4)
@end example

The variable can be examined and modified using @code{@@} (``fetch'') and
@code{!} (``store'') like this:

@example
new-word2 @@ .      \ get address, fetch from it and display
1234 new-word2 !   \ new value, get address, store to it
@end example

@cindex arrays
A similar mechanism can be used to create arrays. For example, an
80-character text input buffer:

@example
CREATE text-buf 80 chars allot

text-buf 0 chars + c@@ \ the 1st character (offset 0)
text-buf 3 chars + c@@ \ the 4th character (offset 3)
@end example

You can build arbitrarily complex data structures by allocating
appropriate areas of memory. For further discussions of this, and to
learn about some Gforth tools that make it easier,
@xref{Structures}.


@node Variables, Constants, CREATE, Defining Words
@subsection Variables
@cindex variables

The previous section showed how a sequence of commands could be used to
generate a variable.  As a final refinement, the whole code sequence can
be wrapped up in a defining word (pre-empting the subject of the next
section), making it easier to create new variables:

@example
: myvariableX ( "name" -- a-addr ) CREATE 1 cells allot ;
: myvariable0 ( "name" -- a-addr ) CREATE 0 , ;

myvariableX foo \ variable foo starts off with an unknown value
myvariable0 joe \ whilst joe is initialised to 0

45 3 * foo !   \ set foo to 135
1234 joe !     \ set joe to 1234
3 joe +!       \ increment joe by 3.. to 1237
@end example

Not surprisingly, there is no need to define @code{myvariable}, since
Forth already has a definition @code{Variable}. Standard Forth does not
guarantee that a @code{Variable} is initialised when it is created
(i.e., it may behave like @code{myvariableX}). In contrast, Gforth's
@code{Variable} initialises the variable to 0 (i.e., it behaves exactly
like @code{myvariable0}). Forth also provides @code{2Variable} and
@code{fvariable} for double and floating-point variables, respectively
-- they are initialised to 0. and 0e in Gforth. If you use a @code{Variable} to
store a boolean, you can use @code{on} and @code{off} to toggle its
state.

doc-variable
doc-avariable
doc-2variable
doc-fvariable

Finally, for buffers of arbitrary length there is

doc-buffer:

@comment TODO is that stuff about user variables strictly correct? Is it
@comment just terminal tasks that have user variables?
@comment should document tasker.fs (with some examples) elsewhere
@comment in this manual, then expand on user space and user variables.

@node Constants, Values, Variables, Defining Words
@subsection Constants
@cindex constants

@code{Constant} allows you to declare a fixed value and refer to it by
name. For example:

@example
12 Constant INCHES-PER-FOOT
3E+08 fconstant SPEED-O-LIGHT
@end example

A @code{Variable} can be both read and written, so its run-time
behaviour is to supply an address through which its current value can be
manipulated. In contrast, the value of a @code{Constant} cannot be
changed once it has been declared@footnote{Well, often it can be -- but
not in a Standard, portable way. It's safer to use a @code{Value} (read
on).} so it's not necessary to supply the address -- it is more
efficient to return the value of the constant directly. That's exactly
what happens; the run-time effect of a constant is to put its value on
the top of the stack (You can find one
way of implementing @code{Constant} in @ref{User-defined Defining Words}).

Forth also provides @code{2Constant} and @code{fconstant} for defining
double and floating-point constants, respectively.

doc-constant
doc-aconstant
doc-2constant
doc-fconstant

@c that's too deep, and it's not necessarily true for all ANS Forths. - anton
@c nac-> How could that not be true in an ANS Forth? You can't define a
@c constant, use it and then delete the definition of the constant..

@c anton->An ANS Forth system can compile a constant to a literal; On
@c decompilation you would see only the number, just as if it had been used
@c in the first place.  The word will stay, of course, but it will only be
@c used by the text interpreter (no run-time duties, except when it is 
@c POSTPONEd or somesuch).

@c nac:
@c I agree that it's rather deep, but IMO it is an important difference
@c relative to other programming languages.. often it's annoying: it
@c certainly changes my programming style relative to C.

@c anton: In what way?

Constants in Forth behave differently from their equivalents in other
programming languages. In other languages, a constant (such as an EQU in
assembler or a #define in C) only exists at compile-time; in the
executable program the constant has been translated into an absolute
number and, unless you are using a symbolic debugger, it's impossible to
know what abstract thing that number represents. In Forth a constant has
an entry in the header space and remains there after the code that uses
it has been defined. In fact, it must remain in the dictionary since it
has run-time duties to perform. For example:

@example
12 Constant INCHES-PER-FOOT
: FEET-TO-INCHES ( n1 -- n2 ) INCHES-PER-FOOT * ;
@end example

@cindex in-lining of constants
When @code{FEET-TO-INCHES} is executed, it will in turn execute the xt
associated with the constant @code{INCHES-PER-FOOT}. If you use
@code{see} to decompile the definition of @code{FEET-TO-INCHES}, you can
see that it makes a call to @code{INCHES-PER-FOOT}. Some Forth compilers
attempt to optimise constants by in-lining them where they are used. You
can force Gforth to in-line a constant like this:

@example
: FEET-TO-INCHES ( n1 -- n2 ) [ INCHES-PER-FOOT ] LITERAL * ;
@end example

If you use @code{see} to decompile @i{this} version of
@code{FEET-TO-INCHES}, you can see that @code{INCHES-PER-FOOT} is no
longer present. To understand how this works, read
@ref{Interpret/Compile states}, and @ref{Literals}.

In-lining constants in this way might improve execution time
fractionally, and can ensure that a constant is now only referenced at
compile-time. However, the definition of the constant still remains in
the dictionary. Some Forth compilers provide a mechanism for controlling
a second dictionary for holding transient words such that this second
dictionary can be deleted later in order to recover memory
space. However, there is no standard way of doing this.


@node Values, Varues, Constants, Defining Words
@subsection Values
@cindex values

A @code{Value} behaves like a @code{Constant}, but it can be changed.
@code{TO} is a parsing word that changes a @code{Values}.  In Gforth
(not in Standard Forth) you can access (and change) a @code{value} also with
@code{>body}.

Here are some
examples:

@example
12 Value APPLES     \ Define APPLES with an initial value of 12
34 TO APPLES        \ Change the value of APPLES. TO is a parsing word
1 ' APPLES >body +! \ Increment APPLES.  Non-standard usage.
APPLES              \ puts 35 on the top of the stack.
@end example

doc-value
doc-avalue
doc-2value
doc-fvalue
doc-to
doc-+to

@node Varues, Colon Definitions, Values, Defining Words
@subsection Varues
@cindex varues

Sometimes you want to take the address of a value-like word.  Because
this has some disadvantages, Gforth asks you to be explicit about it,
and use @code{varue} (named that way because it combines
characteristics of a variable and a value) to declare the name.

doc-varue
doc-2varue
doc-fvarue
doc-addr

@node Colon Definitions, Anonymous Definitions, Varues, Defining Words
@subsection Colon Definitions
@cindex colon definitions

@example
: name ( ... -- ... )
    word1 word2 word3 ;
@end example

@noindent
Creates a word called @code{name} that, upon execution, executes
@code{word1 word2 word3}. @code{name} is a @dfn{(colon) definition}.

The explanation above is somewhat superficial. For simple examples of
colon definitions see @ref{Your first definition}.  For an in-depth
discussion of some of the issues involved, @xref{Interpretation and
Compilation Semantics}.

doc-:
doc-;

We plan to to perform automatic inlining eventually, but for now you
can perform inlining with

doc-inline:
doc-;inline

As an example, you can define an inlined word and use it with

@example
inline: my2dup ( a b -- a b a b )
    ]] over over [[ ;inline

#1. my2dup d. d.
: foo my2dup ;
#1. foo d. d.
see foo
@end example

Inline words are related to macros (@pxref{Macros}); the difference is
that a macro has immediate compilation semantics while an
@code{inline:}-defined word has default compilation semantics; this
means that you normally use a macro only inside a colon definition,
while you can use an @code{inline:} word also interpretively.  But
that also means that you can do some things with macros that you
cannot do as an @code{inline:} word.  E.g.,

@example
\ Doesn't work:
\   inline: endif ]] then [[ ;inline
\ Instead, write a macro:
: endif ]] then [[ ; immediate
@end example

Conversely, for words that would be fine as non-immediate colon
definitions, define them as non-immediate colon definitions or (if
utmost performance is required) as @code{inline:} words; don't define
them as macros, because then you cannot properly use them
interpretively:

@example
: another2dup ]] over over [[ ; immediate
\ Doesn't work:
\   #1. another2dup d. d.
@end example

You may wonder why you have to write compiling code between
@code{inline:} and @code{;inline}.  That's because the implementation
of an inline word like @code{my2dup} above works similar to:

@example
: compile-my2dup ( xt -- )
    drop ]] over over [[ ;

: my2dup [ 0 compile-my2dup ] ;
' compile-my2dup set-optimizer
@end example

The @code{DROP} and @code{0} are there because @code{compile-my2dup}
is the implementation of @code{compile,} for @code{my2dup}, and
@code{compile,} expects an xt (@pxref{User-defined compile-comma}).

@node Anonymous Definitions, Quotations, Colon Definitions, Defining Words
@subsection Anonymous Definitions
@cindex colon definitions
@cindex defining words without name

Sometimes you want to define an @dfn{anonymous word}; a word without a
name. You can do this with:

doc-:noname

This leaves the execution token for the word on the stack after the
closing @code{;}. Here's an example in which a deferred word is
initialised with an @code{xt} from an anonymous colon definition:

@example
Defer deferred
:noname ( ... -- ... )
  ... ;
IS deferred
@end example

@noindent
Gforth provides an alternative way of doing this, using two separate
words:

doc-noname
@cindex execution token of last defined word
doc-latestxt

@noindent
The previous example can be rewritten using @code{noname} and
@code{latestxt}:

@example
Defer deferred
noname : ( ... -- ... )
  ... ;
latestxt IS deferred
@end example

@noindent
@code{noname} works with any defining word, not just @code{:}.

@code{latestxt} also works when the last word was not defined as
@code{noname}.  It does not work for combined words, though.  It also has
the useful property that is is valid as soon as the header for a
definition has been built. Thus:

@example
latestxt . : foo [ latestxt . ] ; ' foo .
@end example

@noindent
prints 3 numbers; the last two are the same.


@node Quotations, Supplying names, Anonymous Definitions, Defining Words
@subsection Quotations
@cindex quotations
@cindex nested colon definitions
@cindex colon definitions, nesting

A quotation is an anonymous colon definition inside another colon
definition.  Quotations are useful when dealing with words that
consume an execution token, like @code{catch} or
@code{outfile-execute}.  E.g. consider the following example of using
@code{outfile-execute} (@pxref{Redirection}):

@example
: some-warning ( n -- )
    cr ." warning# " . ;

: print-some-warning ( n -- )
    ['] some-warning stderr outfile-execute ;
@end example

Here we defined @code{some-warning} as a helper word whose xt we could
pass to outfile-execute.  Instead, we can use a quotation to define
such a word anonymously inside @code{print-some-warning}:

@example
: print-some-warning ( n -- )
  [: cr ." warning# " . ;] stderr outfile-execute ;
@end example

The quotation is bouded by @code{[:} and @code{;]}.  It produces an
execution token at run-time.

doc-[:
doc-;]


@node Supplying names, User-defined Defining Words, Quotations, Defining Words
@subsection Supplying the name of a defined word
@cindex names for defined words
@cindex defining words, name given in a string

By default, a defining word takes the name for the defined word from the
input stream. Sometimes you want to supply the name from a string. You
can do this with:

doc-nextname

For example:

@example
s" foo" nextname create
@end example

@noindent
is equivalent to:

@example
create foo
@end example

@noindent
@code{nextname} works with any defining word.


@node User-defined Defining Words, Deferred Words, Supplying names, Defining Words
@subsection User-defined Defining Words
@cindex user-defined defining words
@cindex defining words, user-defined
@cindex defining defining words

You can define new defining words in terms of any existing defining
word, but @code{:} and @code{create}...@code{does>}/@code{set-does>}
are particularly flexible, whereas the children of, e.g.,
@code{constant} are all just constants.

@menu
* User-defined defining words with colon definitions::
* User-defined defining words using CREATE::
* CREATE..DOES> applications::
* CREATE..DOES> details::
* Advanced does> usage example::
* User-defined TO and DEFER@@::
* User-defined compile-comma::
* Creating from a prototype::
* Making a word current::
* Const-does>::
@end menu

@node User-defined defining words with colon definitions, User-defined defining words using CREATE, User-defined Defining Words, User-defined Defining Words
@subsubsection User-defined defining words with colon definitions

You can create a new defining word by wrapping defining-time code around
an existing defining word and putting the sequence in a colon
definition. 

@c anton: This example is very complex and leads in a quite different
@c direction from the CREATE-DOES> stuff that follows.  It should probably
@c be done elsewhere, or as a subsubsection of this subsection (or as a
@c subsection of Defining Words)

For example, suppose that you have a word @code{stats} that
gathers statistics about colon definitions given the @i{xt} of the
definition, and you want every colon definition in your application to
make a call to @code{stats}. You can define and use a new version of
@code{:} like this:

@example
: stats
  ( xt -- ) DUP ." (Gathering statistics for " . ." )"
  ... ;  \ other code

: my: : latestxt postpone literal ['] stats compile, ;

my: foo + - ;
@end example

When @code{foo} is defined using @code{my:} these steps occur:

@itemize @bullet
@item
@code{my:} is executed.
@item
The @code{:} within the definition (the one between @code{my:} and
@code{latestxt}) is executed, and does just what it always does; it parses
the input stream for a name, builds a dictionary header for the name
@code{foo} and switches @code{state} from interpret to compile.
@item
The word @code{latestxt} is executed. It puts the @i{xt} for the word that is
being defined -- @code{foo} -- onto the stack.
@item
The code that was produced by @code{postpone literal} is executed; this
causes the value on the stack to be compiled as a literal in the code
area of @code{foo}.
@item
The code @code{['] stats} compiles a literal into the definition of
@code{my:}. When @code{compile,} is executed, that literal -- the
execution token for @code{stats} -- is layed down in the code area of
@code{foo} , following the literal@footnote{Strictly speaking, the
mechanism that @code{compile,} uses to convert an @i{xt} into something
in the code area is implementation-dependent. A threaded implementation
might spit out the execution token directly whilst another
implementation might spit out a native code sequence.}.
@item
At this point, the execution of @code{my:} is complete, and control
returns to the text interpreter. The text interpreter is in compile
state, so subsequent text @code{+ -} is compiled into the definition of
@code{foo} and the @code{;} terminates the definition as always.
@end itemize

You can use @code{see} to decompile a word that was defined using
@code{my:} and see how it is different from a normal @code{:}
definition. For example:

@example
: bar + - ;  \ like foo but using : rather than my:
see bar
: bar
  + - ;
see foo
: foo
  `foo stats + - ;
@end example

@code{`foo} is another way of writing @code{['] foo}.



@node User-defined defining words using CREATE, CREATE..DOES> applications, User-defined defining words with colon definitions, User-defined Defining Words
@subsubsection User-defined defining words using create
@cindex @code{CREATE} ... @code{DOES>}

If you want the words defined with your defining words to behave
differently from words defined with standard defining words, you can
write your defining word like this:

@example
: def-word ( "name" -- )
    CREATE @i{code1}
DOES> ( ... -- ... )
    @i{code2} ;

def-word name
@end example

@cindex child words
This fragment defines a @dfn{defining word} @code{def-word} and then
executes it.  When @code{def-word} executes, it @code{CREATE}s a new
word @code{name}, and executes the code @i{code1}. The code @i{code2}
is not executed at this time. The word @code{name} is sometimes called a
@dfn{child} of @code{def-word}.

When you execute @code{name}, the address of the body of @code{name} is
put on the data stack and @i{code2} is executed (the address of the body
of @code{name} is the address @code{HERE} returns immediately after the
@code{CREATE}, i.e., the address a @code{create}d word returns by
default).

You can use @code{def-word} to define a set of child words that behave
similarly; they all have a common run-time behaviour determined by
@i{code2}. Typically, the @i{code1} sequence builds a data area in the
body of the child word. The structure of the data is common to all
children of @code{def-word}, but the data values are specific -- and
private -- to each child word. When a child word is executed, the
address of its private data area is passed as a parameter on TOS to be
used and manipulated@footnote{It is legitimate both to read and write to
this data area.} by @i{code2}.

The two fragments of code that make up the defining words act (are
executed) at two completely separate times:

@itemize @bullet
@item
At @i{define time}, the defining word executes @i{code1} to generate a
child word
@item
At @i{child execution time}, when a child word is invoked, @i{code2}
is executed, using parameters (data) that are private and specific to
the child word.
@end itemize

Another way of understanding the behaviour of @code{def-word} and
@code{name} is to say that, if you make the following definitions:
@example
: def-word1 ( "name" -- )
    CREATE @i{code1} ;

: action1 ( ... -- ... )
    @i{code2} ;

def-word1 name1
@end example

@noindent
Then using @code{name1 action1} is equivalent to using @code{name}.

@cindex @code{CREATE} ... @code{SET-DOES>}
Another way of writing @code{def-word} is (@pxref{Quotations}):

@example
: def-word ( "name" -- ; name execution: ... -- ... )
    create @i{code1}
    [: @i{code2} ;] set-does> ;
@end example

Gforth actually compiles the code using @code{does>} into code
equivalent to the latter code.  An advantage of the @code{set-does>}
approach is that you can put other code behind it and you can use it
inside control structures without needing workarounds.  A disadvantage
is that it is Gforth-specific.

A classic example is that you can define @code{CONSTANT} in this way:

@example
: CONSTANT ( w "name" -- )
    CREATE ,
DOES> ( -- w )
    @@ ;
@end example

@noindent
or equivalently

@example
: CONSTANT ( w "name" -- ; name execution: -- w )
    create ,
    ['] @@ set-does> ;
@end example

@comment There is a beautiful description of how this works and what
@comment it does in the Forthwrite 100th edition.. as well as an elegant
@comment commentary on the Counting Fruits problem.

When you create a constant with @code{5 CONSTANT five}, a set of
define-time actions take place; first a new word @code{five} is
created, then the value 5 is laid down in the body of @code{five} with
@code{,}. When @code{five} is executed, the address of the body is put
on the stack, and @code{@@} retrieves the value 5. The word
@code{five} has no code of its own; it simply contains a data field
and the xt of the quotation or of @code{@@}.

The final example in this section is intended to remind you that space
reserved in @code{CREATE}d words is @i{data} space and therefore can be
both read and written by a Standard program@footnote{Exercise: use this
example as a starting point for your own implementation of @code{Value}
and @code{TO} -- if you get stuck, investigate the behaviour of @code{'} and
@code{[']}.}:

@example
: foo ( "name" -- )
    CREATE -1 ,
DOES> ( -- )
    @@ . ;

foo first-word
foo second-word

123 ' first-word >BODY !
@end example

If @code{first-word} had been a @code{CREATE}d word, we could simply
have executed it to get the address of its data field. However, since it
was defined to have @code{DOES>} actions, its execution semantics are to
perform those @code{DOES>} actions. To get the address of its data field
it's necessary to use @code{'} to get its xt, then @code{>BODY} to
translate the xt into the address of the data field.  When you execute
@code{first-word}, it will display @code{123}. When you execute
@code{second-word} it will display @code{-1}.

@cindex stack effect of @code{DOES>}-parts
@cindex @code{DOES>}-parts, stack effect
In the examples above the stack comment after the @code{DOES>} specifies
the stack effect of the defined words, not the stack effect of the
following code (the following code expects the address of the body on
the top of stack, which is not reflected in the stack comment). This is
the convention that I use and recommend (it clashes a bit with using
locals declarations for stack effect specification, though).


@node CREATE..DOES> applications, CREATE..DOES> details, User-defined defining words using CREATE, User-defined Defining Words
@subsubsection Applications of @code{CREATE..DOES>}
@cindex @code{CREATE} ... @code{DOES>}, applications

You may wonder how to use this feature. Here are some usage patterns:

@cindex factoring similar colon definitions
When you see a sequence of code occurring several times, and you can
identify a meaning, you will factor it out as a colon definition. When
you see similar colon definitions, you can factor them using
@code{CREATE..DOES>}. E.g., an assembler usually defines several words
that look very similar:
@example
: ori, ( reg-target reg-source n -- )
    0 asm-reg-reg-imm ;
: andi, ( reg-target reg-source n -- )
    1 asm-reg-reg-imm ;
@end example

@noindent
This could be factored with:
@example
: reg-reg-imm ( op-code -- )
    CREATE ,
DOES> ( reg-target reg-source n -- )
    @@ asm-reg-reg-imm ;

0 reg-reg-imm ori,
1 reg-reg-imm andi,
@end example

@cindex currying
Another view of @code{CREATE..DOES>} is to consider it as a crude way to
supply a part of the parameters for a word (known as @dfn{currying} in
the functional language community). E.g., @code{+} needs two
parameters. Creating versions of @code{+} with one parameter fixed can
be done like this:

@example
: curry+ ( n1 "name" -- )
    CREATE ,
DOES> ( n2 -- n1+n2 )
    @@ + ;

 3 curry+ 3+
-2 curry+ 2-
@end example


@node CREATE..DOES> details, Advanced does> usage example, CREATE..DOES> applications, User-defined Defining Words
@subsubsection The gory details of @code{CREATE..DOES>}
@cindex @code{CREATE} ... @code{DOES>}, details

doc-does>

@cindex @code{DOES>} in a separate definition
This means that you need not use @code{CREATE} and @code{DOES>} in the
same definition; you can put the @code{DOES>}-part in a separate
definition. This allows us to, e.g., select among different @code{DOES>}-parts:
@example
: does1 
DOES> ( ... -- ... )
    code1 ;

: does2
DOES> ( ... -- ... )
    code2 ;

: def-word ( ... -- ... )
    create ...
    IF
       does1
    ELSE
       does2
    ENDIF ;
@end example

In this example, the selection of whether to use @code{does1} or
@code{does2} is made at definition-time; at the time that the child word is
@code{CREATE}d.

Note that the property of @code{does>} to end the definition makes it
necessary to introduce extra definitions @code{does1} and
@code{does2}.  You can avoid that with @code{set-does>}:

@example
: def-word ( ... -- ... )
    create ...
    IF
       [: code1 ;] set-does>
    ELSE
       [: code2 ;] set-does>
    ENDIF ;
@end example

@cindex @code{DOES>} in interpretation state
In a standard program you can apply a @code{DOES>}-part only if the last
word was defined with @code{CREATE}. In Gforth, the @code{DOES>}-part
will override the behaviour of the last word defined in any case. In a
standard program, you can use @code{DOES>} only in a colon
definition. In Gforth, you can also use it in interpretation state, in a
kind of one-shot mode; for example:
@example
CREATE name ( ... -- ... )
  @i{initialization}
DOES>
  @i{code} ;
@end example

@noindent
is equivalent to the standard:
@example
:noname
DOES>
    @i{code} ;
CREATE name EXECUTE ( ... -- ... )
    @i{initialization}
@end example

Gforth also supports quotations in interpreted code, and quotations
save and restore the current definition, so you can also write the
example above also as:

@example
CREATE name ( ... -- ... )
  @i{initialization}
[: @i{code} ;] set-does>
@end example

doc-set-does>
doc->body

@node Advanced does> usage example, User-defined TO and DEFER@@, CREATE..DOES> details, User-defined Defining Words
@subsubsection Advanced does> usage example

The MIPS disassembler (@file{arch/mips/disasm.fs}) contains many words
for disassembling instructions, that follow a very repetetive scheme:

@example
:noname @var{disasm-operands} s" @var{inst-name}" type ;
@var{entry-num} cells @var{table} + !
@end example

Of course, this inspires the idea to factor out the commonalities to
allow a definition like

@example
@var{disasm-operands} @var{entry-num} @var{table} define-inst @var{inst-name}
@end example

The parameters @var{disasm-operands} and @var{table} are usually
correlated.  Moreover, before I wrote the disassembler, there already
existed code that defines instructions like this:

@example
@var{entry-num} @var{inst-format} @var{inst-name}
@end example

This code comes from the assembler and resides in
@file{arch/mips/insts.fs}.

So I had to define the @var{inst-format} words that performed the scheme
above when executed.  At first I chose to use run-time code-generation:

@example
: @var{inst-format} ( entry-num "name" -- ; compiled code: addr w -- )
  :noname Postpone @var{disasm-operands}
  name Postpone sliteral Postpone type Postpone ;
  swap cells @var{table} + ! ;
@end example

Note that this supplies the other two parameters of the scheme above.

An alternative would have been to write this using
@code{create}/@code{does>}:

@example
: @var{inst-format} ( entry-num "name" -- )
  here name string, ( entry-num c-addr ) \ parse and save "name"
  noname create , ( entry-num )
  latestxt swap cells @var{table} + !
does> ( addr w -- )
  \ disassemble instruction w at addr
  @@ >r 
  @var{disasm-operands}
  r> count type ;
@end example

Somehow the first solution is simpler, mainly because it's simpler to
shift a string from definition-time to use-time with @code{sliteral}
than with @code{string,} and friends.

I wrote a lot of words following this scheme and soon thought about
factoring out the commonalities among them.  Note that this uses a
two-level defining word, i.e., a word that defines ordinary defining
words.

This time a solution involving @code{postpone} and friends seemed more
difficult (try it as an exercise), so I decided to use a
@code{create}/@code{does>} word; since I was already at it, I also used
@code{create}/@code{does>} for the lower level (try using
@code{postpone} etc. as an exercise), resulting in the following
definition:

@example
: define-format ( disasm-xt table-xt -- )
    \ define an instruction format that uses disasm-xt for
    \ disassembling and enters the defined instructions into table
    \ table-xt
    create 2,
does> ( u "inst" -- )
    \ defines an anonymous word for disassembling instruction inst,
    \ and enters it as u-th entry into table-xt
    2@@ swap here name string, ( u table-xt disasm-xt c-addr ) \ remember string
    noname create 2,      \ define anonymous word
    execute latestxt swap ! \ enter xt of defined word into table-xt
does> ( addr w -- )
    \ disassemble instruction w at addr
    2@@ >r ( addr w disasm-xt R: c-addr )
    execute ( R: c-addr ) \ disassemble operands
    r> count type ; \ print name 
@end example

Note that the tables here (in contrast to above) do the @code{cells +}
by themselves (that's why you have to pass an xt).  This word is used in
the following way:

@example
' @var{disasm-operands} ' @var{table} define-format @var{inst-format}
@end example

As shown above, the defined instruction format is then used like this:

@example
@var{entry-num} @var{inst-format} @var{inst-name}
@end example

In terms of currying, this kind of two-level defining word provides the
parameters in three stages: first @var{disasm-operands} and @var{table},
then @var{entry-num} and @var{inst-name}, finally @code{addr w}, i.e.,
the instruction to be disassembled.  

Of course this did not quite fit all the instruction format names used
in @file{insts.fs}, so I had to define a few wrappers that conditioned
the parameters into the right form.

If you have trouble following this section, don't worry.  First, this is
involved and takes time (and probably some playing around) to
understand; second, this is the first two-level
@code{create}/@code{does>} word I have written in seventeen years of
Forth; and if I did not have @file{insts.fs} to start with, I may well
have elected to use just a one-level defining word (with some repeating
of parameters when using the defining word). So it is not necessary to
understand this, but it may improve your understanding of Forth.



@node User-defined TO and DEFER@@, User-defined compile-comma, Advanced does> usage example, User-defined Defining Words
@subsubsection User-defined @code{to} and @code{defer@@}

Gforth's values have several operators: @code{to} (to which @code{is} is
an alias, and @code{defer!} takes an xt instead of a name in the input
stream), @code{+to}, @code{addr}, and @code{action-of} (where
@code{defer@@} takes an xt instead of a name in the input stream).

Gforth allows you to change the @code{(to)} action of a word.

doc-(to)
doc-to-table:
doc-to-method:
doc-set-to
doc-n/a

@code{(to)} is a word used inside @code{to}: it stores the value at
run-time.  The general stack effect of @code{(to)} method is @code{(
val operation xt -- )}, where @i{xt} identifies the word stored into,
@i{operation} the actual variant of @code{to}-like operations, and
@i{val} is the value (of appropriate type) stored there.

You shall implement the @code{to}-method using @code{to-table:} to
create a type-specific table of operations (not specified slots at the
end of the table are filled up with @code{n/a}), and using
@code{to-method:} together with the operation to get from the @i{xt} of
the value to its data field (usually @code{>body} for values in the
dictionary, but value-style data can also reside in structures or user
areas).

E.g., one can implement @code{fvalue} as follows:

@example
to-table: f!-table f! f+!
' >body f!-table to-method: fvalue-to

: fvalue ( r "name" -- ; name: -- r )
  create f,
  ['] f@@ set-does>
  ['] fvalue-to set-to ;

5e fvalue foo
: bar foo 1e f+ to foo ;
see bar
@end example

@node User-defined compile-comma, Creating from a prototype, User-defined TO and DEFER@@, User-defined Defining Words
@subsubsection User-defined @code{compile,}

You can also change the implementation of @code{compile,} for a word,
with

doc-set-optimizer
doc-opt:

Note that the resulting @code{compile,} must still be equivalent to
@code{postpone literal postpone execute}, so @code{set-optimizer} is
useful for efficiency, not for changing the behaviour.  There is
nothing that prevents you from shooting yourself in the foot, however.
You can check whether your uses of @code{set-optimizer} are correct by
comparing the results when you use it with the results you get when
you disable your uses by first defining

@example
: set-optimizer drop ;
@end example

As an example of the use of @code{set-optimizer}, we can enhance one
of the definitions of @code{CONSTANT} above as follows.

@example
: CONSTANT ( n "name" -- ; name: -- n )
  create ,
  ['] @@ set-does>
  [: >body @@ postpone literal ;] set-optimizer
;
@end example

The only change is the addition of the @code{set-optimizer} line.
When you define a constant and compile it:

@example
5 constant five
: foo five ;
@end example

the compiled @code{five} in @code{foo} is now compiled to the literal
5 instead of a generic invocation of @code{five}.  The quotation has
the same stack effect as @code{compile,}: @code{( xt -- )}.  The
passed xt belongs to the @code{compile,}d word, i.e., @code{five} in
the example.  In the example the xt is first converted to the body
address, then the value 5 at that place is fetched, and that value is
compiled with the @code{postpone literal} (@pxref{Literals}).

This use of @code{set-optimizer} assumes that the user does not change
the value of a constant with, e.g., @code{6 ' five >body !}.  While
@code{five} has been defined with @code{create}, that is an
implementation detail of @code{CONSTANT}, and if you don't document
it, the user must not rely on it.  And if you use @code{set-optimizer}
in a way that assumes that the body does not change (like is done
here), you must not document that @code{create} is used; and
conversely, if you document it, you have to write the @code{compile,}
implementation such that it can deal with changing bodies.

Another example is a better-optimized variant of the @code{fvalue}
example above:

@example
: compile-fvalue-to ( xt-value-to -- )
  drop ]] >body f! [[ ;
  
: fvalue-to ( r xt -- )
  >body f! ;
' compile-fvalue-to set-optimizer

: fvalue ( r "name" -- ; name: -- r )
  create f,
  ['] f@@ set-does>
  [: >body ]] literal f@@ [[ ;] set-optimizer
  ['] fvalue-to set-to ;

5e fvalue foo
: bar foo 1e f+ to foo ;
see bar
@end example

Compare the code for @code{bar} with the one for the earlier
definition.  Here we see the optimization of both the code for reading
the fvalue (coming from the @code{set-optimizer} in @code{fvalue}) and
for writing the fvalue (coming from the @code{set-optimizer} applied
to @code{fvalue-to}.  Because an fvalue can change (unlike a
constant), the reading part (inside @code{fvalue}) compiles the
address and a @code{f@@} that is performed at run-time.

For @code{fvalue-to}, the @code{compile,} implementation basically
just compiles the code executed by @code{fvalue} inline.  The
compilation semantics of @code{to} compiles the address as literal and
then the @code{(to)} implementation (i.e., @code{fvalue-to}).  In this
process the @code{>body} is optimized away.
@c !! reference the appropriate sections that explain these
@c optimizations

In practice Gforth's @code{fvalue} includes a few additional twists,
e.g., to support @code{+TO}.

Note that the call to @code{set-optimizer} has to be performed after
the call to @code{set-does>} (or @code{does>}, because
@code{set-does>} overwrites the @code{compile,} implementation itself.

As we can see in the @code{fvalue-to} example, we can also apply
@code{set-optimizer} to individual words rather than inside a defining
word like @code{constant} or @code{fvalue}.  In this case, the xt of
the word passed to optimizer is usually unnecessary and is
@code{drop}ped, as in @code{compile-fvalue-to}.

The engine @code{gforth-itc} uses @code{,} for @code{compile,} and
@code{set-optimizer} has no effect there.


@node Creating from a prototype, Making a word current, User-defined compile-comma, User-defined Defining Words
@subsubsection Creating from a prototype

In the above we show how to define a word by first using
@code{create}, and then modifying it with @code{set-does>},
@code{set-to}, @code{set-optimizer} etc.

An alternative way is to create a prototype using these words, and
then create a new word from that prototype.  This kind of copying does
not cover the body, so that has to be allocated and initialized
explicitly.  Taking @code{fvalue} above, we could instead define it
as:

@example
create fvalue-prototype ( -- r )
['] f@@ set-does>
[: >body ]] literal f@@ [[ ;] set-optimizer
['] fvalue-to set-to

: fvalue ( r "name" -- ; name: -- r )
  ``fvalue-prototype create-from f, reveal ;
@end example

An advantage of this approach is that creating @code{fvalue} words is
now faster, because it does not need to first duplicate the header
methods of @code{create}, modify them, and eventually deduplicate
them.  But this advantage is only relevant if the number of words
created with this defining word is huge.

doc-create-from
doc-reveal

The performance advantage does not extend to using @code{noname} with
the defining word.  Therefore we also have

doc-noname-from

Here's a usage example:

@example
``fvalue-prototype noname create-from
latestnt constant noname-fvalue-prototype

: noname-fvalue ( r -- xt ; xt execution: -- r )
  noname-fvalue-prototype noname-from f,
  latestxt ;
@end example


@node Making a word current, Const-does>, Creating from a prototype, User-defined Defining Words
@subsubsection Making a word current
@cindex Modifying a word defined earlier

Many words mentioned above, such as @code{immediate} or
@code{set-optimizer} change the ``current'' or ``most recently
defined'' word.  Sometimes you want to change an earlier word.  You
can do this with

doc-make-latest


@node Const-does>,  , Making a word current, User-defined Defining Words
@subsubsection @code{Const-does>}

A frequent use of @code{create}...@code{does>} is for transferring some
values from definition-time to run-time.  Gforth supports this use with

doc-const-does>

A typical use of this word is:

@example
: curry+ ( n1 "name" -- )
1 0 CONST-DOES> ( n2 -- n1+n2 )
    + ;

3 curry+ 3+
@end example

Here the @code{1 0} means that 1 cell and 0 floats are transferred from
definition to run-time.

The advantages of using @code{const-does>} are:

@itemize

@item
You don't have to deal with storing and retrieving the values, i.e.,
your program becomes more writable and readable.

@item
When using @code{does>}, you have to introduce a @code{@@} that cannot
be optimized away (because you could change the data using
@code{>body}...@code{!}); @code{const-does>} avoids this problem.

@end itemize

A Standard Forth implementation of @code{const-does>} is available in
@file{compat/const-does.fs}.


@node Deferred Words, Forward, User-defined Defining Words, Defining Words
@subsection Deferred Words
@cindex deferred words

The defining word @code{Defer} allows you to define a word by name
without defining its behaviour; the definition of its behaviour is
deferred. Here are two situation where this can be useful:

@itemize @bullet
@item
Where you want to allow the behaviour of a word to be altered later, and
for all precompiled references to the word to change when its behaviour
is changed.
@item
For mutual recursion; @xref{Calls and returns}.
@end itemize

In the following example, @code{foo} always invokes the version of
@code{greet} that prints ``@code{Good morning}'' whilst @code{bar}
always invokes the version that prints ``@code{Hello}''. There is no way
of getting @code{foo} to use the later version without re-ordering the
source code and recompiling it.

@example
: greet ." Good morning" ;
: foo ... greet ... ;
: greet ." Hello" ;
: bar ... greet ... ;
@end example

This problem can be solved by defining @code{greet} as a @code{Defer}red
word. The behaviour of a @code{Defer}red word can be defined and
redefined at any time by using @code{IS} to associate the xt of a
previously-defined word with it. The previous example becomes:

@example
Defer greet ( -- )
: foo ... greet ... ;
: bar ... greet ... ;
: greet1 ( -- ) ." Good morning" ;
: greet2 ( -- ) ." Hello" ;
' greet2 IS greet  \ make greet behave like greet2
@end example

@progstyle
You should write a stack comment for every deferred word, and put only
XTs into deferred words that conform to this stack effect.  Otherwise
it's too difficult to use the deferred word.

A deferred word can be used to improve the statistics-gathering example
from @ref{User-defined Defining Words}; rather than edit the
application's source code to change every @code{:} to a @code{my:}, do
this:

@example
: real: : ;     \ retain access to the original
defer :         \ redefine as a deferred word
' my: IS :      \ use special version of :
\
\ load application here
\
' real: IS :    \ go back to the original
@end example


One thing to note is that @code{IS} has special compilation semantics,
such that it parses the name at compile time (like @code{TO}):

@example
: set-greet ( xt -- )
  IS greet ;

' greet1 set-greet
@end example

In situations where @code{IS} does not fit, use @code{defer!} instead.

A deferred word can only inherit execution semantics from the xt
(because that is all that an xt can represent -- for more discussion of
this @pxref{Tokens for Words}); by default it will have default
interpretation and compilation semantics deriving from this execution
semantics.  However, you can change the interpretation and compilation
semantics of the deferred word in the usual ways:

@example
: bar .... ; immediate
Defer fred immediate
Defer jim

' bar IS jim  \ jim has default semantics
' bar IS fred \ fred is immediate
@end example

doc-defer
doc-defer!
doc-is
doc-defer@
doc-action-of

@c Use @code{words-deferred} to see a list of deferred words.

Definitions of these Forth-2012 words in Forth-94 are provided in
@file{compat/defer.fs}.  In addition, Gforth provides:

doc-defers
doc-wrap-xt
doc-preserve


@node Forward, Aliases, Deferred Words, Defining Words
@subsection Forward

The defining word @code{Forward} in @code{forward.fs} allows you to
create forward references, which are resolved automatically, and do
not incur additional costs like the indirection of @code{Defer}.
However, these forward definitions only work for colon definitions.

doc-forward
doc-.unresolved

@node Aliases,  , Forward, Defining Words
@subsection Aliases
@cindex aliases
@cindex synonyms

The defining word @code{synonym} allows you to define
a word by name that has the same behaviour as some other word. Here
are two situation where this can be useful:

@itemize @bullet
@item
When you want access to a word's definition from a different word list
(for an example of this, see the definition of the @code{Root} word list
in the Gforth source).
@item
When you want to create a synonym; a definition that can be known by
either of two names (for example, @code{THEN} and @code{ENDIF} are
synonyms).
@end itemize

doc-synonym

Gforth also offers the non-standard @code{alias}, that does not
inherit the compilation semantics, @code{to}/@code{defer!} or
@code{defer@@} semantics from its parent.  You can then change, e.g.,
the compilation semantics with, e.g., @code{immediate}.

doc-alias

Example:

@example
: foo ... ; immediate

' foo Alias bar1           \ bar1 is not an immediate word
' foo Alias bar2 immediate \ bar2 is an immediate word
synonym bar3 foo           \ bar3 is an immediate word
@end example

Both synonyms and aliases have a different nt than the original, but
ticking it (or using @code{name>interpret}) produces the same xt as
the original (@pxref{Tokens for Words}).

@node Interpretation and Compilation Semantics, Tokens for Words, Defining Words, Words
@section Interpretation and Compilation Semantics
@cindex semantics, interpretation and compilation

@c !! state and ' are used without explanation
@c example for immediate/compile-only? or is the tutorial enough

@cindex interpretation semantics
The @dfn{interpretation semantics} of a (named) word are what the text
interpreter does when it encounters the word in interpret state. It also
appears in some other contexts, e.g., the execution token returned by
@code{' @i{word}} identifies the interpretation semantics of @i{word}
(in other words, @code{' @i{word} execute} is equivalent to
interpret-state text interpretation of @code{@i{word}}).

@cindex compilation semantics
The @dfn{compilation semantics} of a (named) word are what the text
interpreter does when it encounters the word in compile state. It also
appears in other contexts, e.g, @code{POSTPONE @i{word}}
compiles@footnote{In standard terminology, ``appends to the current
definition''.} the compilation semantics of @i{word}.

Most words have default compilation semantics: compile the execution
semantics (stack effect @code{( -- )}).  But a number of words have
other compilation semantics, documented for the individual word
(including its stack effect).

@cindex execution semantics
The standard also talks about @dfn{execution semantics}.  In the
standard it never differs from the interpretation semantics if both
are defined, but one or both of them may not be defined.  Gforth makes
no difference between interpretation and execution semantics, so these
terms are used interchangeably.

In Gforth (since 1.0) all words have defined interpretation/execution
semantics.  For many words that have no defined interpretation nor
execution semantics in the standard (e.g., @code{if}), the
interpretation/execution semantics in Gforth are to perform the
compilation semantics.

In the standard, execution semantics are used to define interpretation
and compilation semantics by default: By default, the interpretation
semantics of a word are to @code{execute} its execution semantics, and
the compilation semantics of a word are to @code{compile,} its
execution semantics.@footnote{In standard terminology: The default
interpretation semantics are its execution semantics; the default
compilation semantics are to append its execution semantics to the
execution semantics of the current definition.}

Unnamed words (@pxref{Anonymous Definitions}) cannot be encountered by
the text interpreter, ticked, or @code{postpone}d.  Such a word is
represented by its xt (@pxref{Tokens for Words}), and the behaviour
when this xt is @code{execute}d is called its execution semantics.

@comment TODO expand, make it co-operate with new sections on text interpreter.

@cindex immediate words
@cindex compile-only words
You can change the semantics of the most-recently defined word:


doc-immediate
doc-compile-only
doc-restrict

By convention, words with non-default compilation semantics (e.g.,
immediate words) often have names surrounded with brackets (e.g.,
@code{[']}, @pxref{Execution token}).

Note that ticking (@code{'}) a compile-only word gives a warning
(``<word> is compile-only'').

@menu
* Combined words::
@end menu


@node Combined words,  , Interpretation and Compilation Semantics, Interpretation and Compilation Semantics
@subsection Combined Words
@cindex combined words

Gforth allows you to define @dfn{combined words} -- words that have an
arbitrary combination of interpretation and compilation semantics.

doc-interpret/compile:

This feature was introduced for implementing @code{TO} and @code{S"}. I
recommend that you do not define such words, as cute as they may be:
they make it hard to get at both parts of the word in some contexts.
E.g., assume you want to get an execution token for the compilation
part. Instead, define two words, one that embodies the interpretation
part, and one that embodies the compilation part.  Once you have done
that, you can define a combined word with @code{interpret/compile:} for
the convenience of your users.

You might try to use this feature to provide an optimizing
implementation of the default compilation semantics of a word. For
example, by defining:
@example
:noname
   foo bar ;
:noname
   POSTPONE foo POSTPONE bar ;
interpret/compile: opti-foobar
@end example

@noindent
as an optimizing version of:

@example
: foobar
    foo bar ;
@end example

Unfortunately, this does not work correctly with @code{[compile]},
because @code{[compile]} assumes that the compilation semantics of all
@code{interpret/compile:} words are non-default. I.e., @code{[compile]
opti-foobar} would compile compilation semantics, whereas
@code{[compile] foobar} would compile interpretation semantics.

@cindex state-smart words (are a bad idea)
@anchor{state-smartness}
Some people try to use @dfn{state-smart} words to emulate the feature provided
by @code{interpret/compile:} (words are state-smart if they check
@code{STATE} during execution). E.g., they would try to code
@code{foobar} like this:

@example
: foobar
  STATE @@
  IF ( compilation state )
    POSTPONE foo POSTPONE bar
  ELSE
    foo bar
  ENDIF ; immediate
@end example

Although this works if @code{foobar} is only processed by the text
interpreter, it does not work in other contexts (like @code{'} or
@code{POSTPONE}). E.g., @code{' foobar} will produce an execution token
for a state-smart word, not for the interpretation semantics of the
original @code{foobar}; when you execute this execution token (directly
with @code{EXECUTE} or indirectly through @code{COMPILE,}) in compile
state, the result will not be what you expected (i.e., it will not
perform @code{foo bar}). State-smart words are a bad idea. Simply don't
write them@footnote{For a more detailed discussion of this topic, see
M. Anton Ertl,
@cite{@uref{https://www.complang.tuwien.ac.at/papers/ertl98.ps.gz,@code{State}-smartness---Why
it is Evil and How to Exorcise it}}, EuroForth '98.}!

@c -------------------------------------------------------------
@node Tokens for Words, Compiling words, Interpretation and Compilation Semantics, Words
@section Tokens for Words
@cindex tokens for words

This section describes the creation and use of tokens that represent
words.

@menu
* Execution token::             represents execution/interpretation semantics
* Name token::                  represents named words
* Compilation token::           represents compilation semantics
@end menu

@node Execution token, Name token, Tokens for Words, Tokens for Words
@subsection Execution token

@cindex xt
@cindex execution token
An @dfn{execution token} (@emph{xt}) represents some behaviour of a word.
You can use @code{execute} to invoke the behaviour represented by the
xt and @code{compile,} (@pxref{Macros}) to compile it into the current
definition.  Other uses include deferred words (@pxref{Deferred
Words}).

In particular, there is @emph{the} execution token of a word that
represents its interpretation semantics aka execution
semantics.@footnote{The Forth standard has words with undefined
interpretation semantics (e.g., @code{r@@}) and words without defined
execution semantics (e.g., @code{s"}) and words with neither (e.g.,
@code{if}), but in cases where both interpretation and execution
semantics are defined, they are the same; so we treat them as being
the same.}

@cindex prefix @code{`}
@cindex @code{`} prefix
For a named word @i{x}, you can use @code{`@i{x}} to get its execution
token:

@example
5 `. ( n xt )
execute ( )      \ execute the xt (i.e., ".")
: foo `. execute ;
5 foo
@end example

@cindex tick (')
However, the @code{`} prefix is a Gforth extension, so you may prefer
to use the standard Forth words:

doc-'
doc-[']

These are parsing words (whereas @code{`@i{x}} is treated as a literal
by a recognizer), and you may find the behaviour in interpreted and
compiled code unintuitive:

@example
5 ' .   ( n xt ) 
execute ( )      \ execute the xt of .
\ does not work as intended:
\ : foo ' . ;
\ 5 foo execute
\ instead:
: foo ['] . ;
5 foo execute    \ execute the xt of .
\ Usage of ' in colon definition:
: bar ' execute ;
5 bar .          \ execute the xt of .
@end example

@code{'} parses at run-time, so if you put it in a colon definition,
as in @code{bar}, it does not consume the next word in the colon
definition, but the next word at run-time (i.e., the @code{.} in the
invocation of @code{bar}).  If you want to put a literal xt in a colon
definition without writing @code{`@i{x}}, write @code{['] @i{x}}.

Gforth's @code{`@i{x}}, @code{'} and @code{[']} warn when you use them
on compile-only words, because such usage may be non-portable between
different Forth systems.

You can avoid that warning as well as the portability problems by
defining an immediate variant of the word, e.g.:

@example
: if postpone if ; immediate
: test [ ' if execute ] ." test" then ;
@end example

The resulting execution token performs the compilation semantics of
@code{if} when @code{execute}d.

Another way to get an xt is @code{:noname} or @code{latestxt}
(@pxref{Anonymous Definitions}).  For anonymous words this gives an xt
for the only behaviour the word has (the execution semantics), but you
can also use it after defining a named word to get its xt.

@example
:noname ." hello" ;
execute
@end example

An xt occupies one cell and can be manipulated like any other cell.

In Standard Forth the xt is just an abstract data type (i.e., defined
by the operations that produce or consume it).  The concrete
implementation (since Gforth 1.0) is the body address (for old
hands: PFA) of the word; in Gforth 0.7 and earlier, the xt was
implemented as code field addres (CFA, 2 cells before the PFA).

doc-execute
doc-execute-exit
doc-perform

@code{Noop} is sometimes used to have a placeholder execution token:

doc-noop


@node Name token, Compilation token, Execution token, Tokens for Words
@subsection Name token

@cindex name token
@cindex nt (name token)
Gforth represents named words by the @dfn{name token}, (@i{nt}).  The
name token is a cell-sized abstract data type that occurs as argument
or result of the words below.

Since Gforth 1.0, for most words the concrete implementation of their
nt is the same address as its xt (this is the primary nt for the xt).
However, synonyms, aliases, and words defined with
@code{interpret/compile:} get their xt from another word, but still
have an nt of their own (that is different from the xt).  Therefore,
you cannot use xts and nts interchangeably, even if you are prepared
to write code specific to Gforth 1.0.  You do not get these alternate
nts for the xt with @code{>name}.

You get the nt of a word @i{x} with @code{``@i{x}} (since Gforth 1.0)
or with

doc-find-name
doc-find-name-in
doc-latest
doc-latestnt
doc->name
doc-xt>name

You can get all the nts in a wordlist with

doc-traverse-wordlist

You can use the nt to access the interpretation and compilation
semantics of a word, its name, and the next word in the wordlist:

doc-name>interpret
doc-name>compile
doc-name>string
doc-id.
doc-.id
doc-name>link

A nameless word usually has no interpretation nor compilation
semantics, no name, and it's not in a wordlist.  But in Gforth (since
1.0) all words are equal, so even nameless words have an nt (but they
are in no wordlist).  You can get that nt with @code{latestnt}, and
the words above that consume nts do something reasonable for these
nts.

As a usage example, the following code lists all the words in
@code{forth-wordlist} with non-default compilation semantics:

@example
: ndcs-words ( wid -- )
  [: dup name>compile ['] compile, <> if over id. then 2drop true ;]
  swap traverse-wordlist ;

forth-wordlist ndcs-words
@end example

This code assumes that a word has default compilation semantics if the
xt part of its compilation token is the xt of @code{compile,}.
@c which
@c holds for Gforth.  There are a few standard ways for a Forth system to
@c deviate from this assumption (although it's more likely that a system
@c that deviates gets it wrong and is non-standard).


@cindex name field address
@cindex NFA
The closest thing to the nt in older Forth systems is the name field
address (NFA), but there are significant differences: in older Forth
systems each word had a unique NFA, LFA, CFA and PFA (in this order,
or LFA, NFA, CFA, PFA) and there were words for getting from one to
the next.  In contrast, in Gforth several nts can get the same xt from
@code{name>interpret} xt; there is a link field in the structure
identified by the name token, but searching usually uses a hash table
external to these structures; the name in Gforth has a cell-wide
count-and-flags field, and the nt is not implemented as the address of
that count field.


@node Compilation token,  , Name token, Tokens for Words
@subsection Compilation token

@cindex compilation token
@cindex ct (compilation token)
The compilation semantics of a named word is represented by a
@dfn{compilation token} consisting of two cells: @i{w xt}. The top
cell @i{xt} is an execution token. The compilation semantics
represented by the compilation token can be performed with
@code{execute}, which consumes the whole compilation token, with an
additional stack effect determined by the represented compilation
semantics.  

At present, the @i{w} part of a compilation token is an execution token,
and the @i{xt} part represents either @code{execute} or
@code{compile,}@footnote{Depending upon the compilation semantics of the
word. If the word has default compilation semantics, the @i{xt} will
represent @code{compile,}. Otherwise (e.g., for immediate words), the
@i{xt} will represent @code{execute}.}. However, don't rely on that
knowledge, unless necessary; future versions of Gforth may introduce
unusual compilation tokens (e.g., a compilation token that represents
the compilation semantics of a literal).

You get the compilation token of, e.g., @code{if} in a standard way
with @code{name>compile}, e.g., @code{`if name>compile}, but there are
also parsing words to get the compilation token of a word:

doc-[comp']
doc-comp'

You can perform the compilation semantics represented by the compilation
token with @code{execute}.  You can compile the compilation semantics
with @code{postpone,}. I.e., @code{``@i{x} name>compile postpone,} is
equivalent to @code{postpone @i{x}}.

doc-postpone,

@c ----------------------------------------------------------
@node Compiling words, The Text Interpreter, Tokens for Words, Words
@section Compiling words
@cindex compiling words
@cindex macros

In contrast to most other languages, Forth has no strict boundary
between compilation and run-time.  E.g., you can run arbitrary code
between defining words (or for computing data used by defining words
like @code{constant}). Moreover, @code{Immediate} (@pxref{Interpretation
and Compilation Semantics} and @code{[}...@code{]} (see below) allow
running arbitrary code while compiling a colon definition (exception:
you must not allot dictionary space).

@menu
* Literals::                    Compiling data values
* Macros::                      Compiling words
@end menu

@node Literals, Macros, Compiling words, Compiling words
@subsection Literals
@cindex Literals

The simplest and most frequent example is to compute a literal during
compilation.  E.g., the following definition prints an array of strings,
one string per line:

@example
: .strings ( addr u -- ) \ gforth
    2* cells bounds U+DO
	cr i 2@@ type
    2 cells +LOOP ;  
@end example

With a simple-minded compiler like Gforth's, this computes @code{2
cells} on every loop iteration.  You can compute this value once and for
all at compile time and compile it into the definition like this:

@example
: .strings ( addr u -- ) \ gforth
    2* cells bounds U+DO
	cr i 2@@ type
    [ 2 cells ] literal +LOOP ;  
@end example

@code{[} switches the text interpreter to interpret state (you will get
an @code{ok} prompt if you type this example interactively and insert a
newline between @code{[} and @code{]}), so it performs the
interpretation semantics of @code{2 cells}; this computes a number.
@code{]} switches the text interpreter back into compile state.  It then
performs @code{Literal}'s compilation semantics, which are to compile
this number into the current word.  You can decompile the word with
@code{see .strings} to see the effect on the compiled code.

You can also optimize the @code{2* cells} into @code{[ 2 cells ] literal
*} in this way.

doc-[
doc-]
doc-literal
doc-aliteral
doc-]L

There are also words for compiling other data types than single cells as
literals:

doc-2literal
doc-fliteral
doc-sliteral

@cindex colon-sys, passing data across @code{:}
@cindex @code{:}, passing data across
You might be tempted to pass data from outside a colon definition to the
inside on the data stack.  This does not work, because @code{:} puhes a
colon-sys, making stuff below unaccessible.  E.g., this does not work:

@example
5 : foo literal ; \ error: "unstructured"
@end example

Instead, you have to pass the value in some other way, e.g., through a
variable:

@example
variable temp
5 temp !
: foo [ temp @@ ] literal ;
@end example


@node Macros,  , Literals, Compiling words
@subsection Macros
@cindex Macros
@cindex compiling compilation semantics

@code{Literal} and friends compile data values into the current
definition.  You can also write words that compile other words into the
current definition.  E.g.,

@example
: compile-+ ( -- ) \ compiled code: ( n1 n2 -- n )
  POSTPONE + ;

: foo ( n1 n2 -- n )
  [ compile-+ ] ;
1 2 foo .
@end example

This is equivalent to @code{: foo + ;} (@code{see foo} to check this).
What happens in this example?  @code{Postpone} compiles the compilation
semantics of @code{+} into @code{compile-+}; later the text interpreter
executes @code{compile-+} and thus the compilation semantics of +, which
compile (the execution semantics of) @code{+} into
@code{foo}.@footnote{A recent RFI answer requires that compiling words
should only be executed in compile state, so this example is not
guaranteed to work on all standard systems, but on any decent system it
will work.}

doc-postpone

Compiling words like @code{compile-+} are usually immediate (or similar)
so you do not have to switch to interpret state to execute them;
modifying the last example accordingly produces:

@example
: [compile-+] ( compilation: --; interpretation: -- )
  \ compiled code: ( n1 n2 -- n )
  POSTPONE + ; immediate

: foo ( n1 n2 -- n )
  [compile-+] ;
1 2 foo .
@end example

You will occassionally find the need to POSTPONE several words;
putting POSTPONE before each such word is cumbersome, so Gforth
provides a more convenient syntax: @code{]] ... [[}.  This
allows us to write @code{[compile-+]} as:

@example
: [compile-+] ( compilation: --; interpretation: -- )
  ]] + [[ ; immediate
@end example

doc-]]

The unusual direction of the brackets indicates their function:
@code{]]} switches from compilation to postponing (i.e., compilation
of compilation), just like @code{]} switches from immediate execution
(interpretation) to compilation.  Conversely, @code{[[} switches from
postponing to compilation, ananlogous to @code{[} which switches from
compilation to immediate execution.

The real advantage of @code{]] }...@code{ [[} becomes apparent when
there are many words to POSTPONE.  E.g., the word
@code{compile-map-array} (@pxref{Advanced macros Tutorial}) can be
written much shorter as follows:

@example
: compile-map-array ( compilation: xt -- ; run-time: ... addr u -- ... )
\ at run-time, execute xt ( ... x -- ... ) for each element of the
\ array beginning at addr and containing u elements
  @{: xt: xt :@}
  ]] cells over + swap ?do
    i @@ xt 1 cells +loop [[ ;

: sum-array ( addr u -- n )
  0 [ ' + compile-map-array ] ;
@end example

If you then say @code{see sum-array}, it shows the following code:

@example
: sum-array
  #0 over + swap ?do
    i @ + #8 +LOOP
;
@end example

In addition to @code{]]}...@code{[[}, this example shows off some
other features:

@itemize

@item
It uses a defer-flavoured (defined with @code{xt:} local @code{xt};
mentioning the local inside @code{]]}...@code{[[} results in
@code{compile,}ing the xt in the local.

@item
It uses the literal @code{1} inside @code{]]}...@code{[[}.  This
results in @code{postpone}ing the @code{1}, i.e. compiling it when
@code{compile-map-array} is run.

@item
When @code{compile-map-array} is run, @code{1 cells} is compiled and
optimized into @code{#8} by Gforth's constant folding.

@end itemize

Note that parsing words such as @code{s\"} don't parse at postpone
time and therefore not inside @code{]]}...@code{[[}.  Instead of
@code{s\" mystring\n"} you can use the string recognizer and write
@code{"mystring\n"}, which works inside @code{]]}...@code{[[}.
Likewise for the parsing word @code{[']} and the recognizer notation
starting with @code{`}.

But if you prefer to use @code{s\"} (or have a parsing word that has
no recognizer replacement), you can do it by switching back to
compilation:

@example
]] ... [[ s\" mystring\n" ]] 2literal ... [[
@end example

Definitions of @code{]]} and friends in Standard Forth are provided in
@file{compat/macros.fs}.

Immediate compiling words are similar to macros in other languages (in
particular, Lisp).  The important differences to macros in, e.g., C are:

@itemize @bullet

@item
You use the same language for defining and processing macros, not a
separate preprocessing language and processor.

@item
Consequently, the full power of Forth is available in macro definitions.
E.g., you can perform arbitrarily complex computations, or generate
different code conditionally or in a loop (e.g., @pxref{Advanced macros
Tutorial}).  This power is very useful when writing a parser generators
or other code-generating software.

@item
Macros defined using @code{postpone} etc. deal with the language at a
higher level than strings; name binding happens at macro definition
time, so you can avoid the pitfalls of name collisions that can happen
in C macros.  Of course, Forth is a liberal language and also allows to
shoot yourself in the foot with text-interpreted macros like

@example
: [compile-+] s" +" evaluate ; immediate
@end example

Apart from binding the name at macro use time, using @code{evaluate}
also makes your definition @code{state}-smart (@pxref{state-smartness}).
@end itemize

You may want the macro to compile a number into a word.  The word to do
it is @code{literal}, but you have to @code{postpone} it, so its
compilation semantics take effect when the macro is executed, not when
it is compiled:

@example
: [compile-5] ( -- ) \ compiled code: ( -- n )
  5 POSTPONE literal ; immediate

: foo [compile-5] ;
foo .
@end example

You may want to pass parameters to a macro, that the macro should
compile into the current definition.  If the parameter is a number, then
you can use @code{postpone literal} (similar for other values).

If you want to pass a word that is to be compiled, the usual way is to
pass an execution token and @code{compile,} it:

@example
: twice1 ( xt -- ) \ compiled code: ... -- ...
  dup compile, compile, ;

: 2+ ( n1 -- n2 )
  [ ' 1+ twice1 ] ;
@end example

doc-compile,

An alternative available in Gforth, that allows you to pass the
compilation semantics as parameters is to use the compilation token
(@pxref{Compilation token}).  The same example in this technique:

@example
: twice ( ... ct -- ... ) \ compiled code: ... -- ...
  2dup 2>r execute 2r> execute ;

: 2+ ( n1 -- n2 )
  [ comp' 1+ twice ] ;
@end example

In the example above @code{2>r} and @code{2r>} ensure that @code{twice}
works even if the executed compilation semantics has an effect on the
data stack.

You can also define complete definitions with these words; this provides
an alternative to using @code{does>} (@pxref{User-defined Defining
Words}).  E.g., instead of

@example
: curry+ ( n1 "name" -- )
    CREATE ,
DOES> ( n2 -- n1+n2 )
    @@ + ;
@end example

you could define

@example
: curry+ ( n1 "name" -- )
  \ name execution: ( n2 -- n1+n2 )
  >r : r> POSTPONE literal POSTPONE + POSTPONE ; ;

-3 curry+ 3-
see 3-
@end example

The sequence @code{>r : r>} is necessary, because @code{:} puts a
colon-sys on the data stack that makes everything below it unaccessible.

This way of writing defining words is sometimes more, sometimes less
convenient than using @code{does>} (@pxref{Advanced does> usage
example}).  One advantage of this method is that it can be optimized
better, because the compiler knows that the value compiled with
@code{literal} is fixed, whereas the data associated with a
@code{create}d word can be changed.

doc-[compile]

@c ----------------------------------------------------------
@node The Text Interpreter, The Input Stream, Compiling words, Words
@section  The Text Interpreter
@cindex interpreter - outer
@cindex text interpreter
@cindex outer interpreter

@c Should we really describe all these ugly details?  IMO the text
@c interpreter should be much cleaner, but that may not be possible within
@c ANS Forth. - anton
@c nac-> I wanted to explain how it works to show how you can exploit
@c it in your own programs. When I was writing a cross-compiler, figuring out
@c some of these gory details was very helpful to me. None of the textbooks
@c I've seen cover it, and the most modern Forth textbook -- Forth Inc's,
@c seems to positively avoid going into too much detail for some of
@c the internals.

@c anton: ok.  I wonder, though, if this is the right place; for some stuff
@c it is; for the ugly details, I would prefer another place.  I wonder
@c whether we should have a chapter before "Words" that describes some
@c basic concepts referred to in words, and a chapter after "Words" that
@c describes implementation details.

The text interpreter@footnote{This is an expanded version of the
material in @ref{Introducing the Text Interpreter}.} is an endless loop
that processes input from the current input device. It is also called
the outer interpreter, in contrast to the inner interpreter
(@pxref{Engine}) which executes the compiled Forth code on interpretive
implementations.

@cindex interpret state
@cindex compile state
The text interpreter operates in one of two states: @dfn{interpret
state} and @dfn{compile state}. The current state is defined by the
aptly-named variable @code{state}.

This section starts by describing how the text interpreter behaves when
it is in interpret state, processing input from the user input device --
the keyboard. This is the mode that a Forth system is in after it starts
up.

@cindex input buffer
@cindex terminal input buffer
The text interpreter works from an area of memory called the @dfn{input
buffer}@footnote{When the text interpreter is processing input from the
keyboard, this area of memory is called the @dfn{terminal input buffer}
(TIB) and is addressed by the (obsolescent) words @code{TIB} and
@code{#TIB}.}, which stores your keyboard input when you press the
@key{RET} key. Starting at the beginning of the input buffer, it skips
leading spaces (called @dfn{delimiters}) then parses a string (a
sequence of non-space characters) until it reaches either a space
character or the end of the buffer. Having parsed a string, it makes two
attempts to process it:

@cindex dictionary
@itemize @bullet
@item
It looks for the string in a @dfn{dictionary} of definitions. If the
string is found, the string names a @dfn{definition} (also known as a
@dfn{word}) and the dictionary search returns information that allows
the text interpreter to perform the word's @dfn{interpretation
semantics}. In most cases, this simply means that the word will be
executed.
@item
If the string is not found in the dictionary, the text interpreter
attempts to treat it as a number, using the rules described in
@ref{Number Conversion}. If the string represents a legal number in the
current radix, the number is pushed onto a parameter stack (the data
stack for integers, the floating-point stack for floating-point
numbers).
@end itemize

If both attempts fail, the text interpreter discards the
remainder of the input buffer, issues an error message and waits for
more input. If one of the attempts succeeds, the text interpreter
repeats the parsing process until the whole of the input buffer has been
processed, at which point it prints the status message ``@code{ ok}''
and waits for more input.

@c anton: this should be in the input stream subsection (or below it)

@cindex parse area
The text interpreter keeps track of its position in the input buffer by
updating a variable called @code{>IN} (pronounced ``to-in''). The value
of @code{>IN} starts out as 0, indicating an offset of 0 from the start
of the input buffer. The region from offset @code{>IN @@} to the end of
the input buffer is called the @dfn{parse area}@footnote{In other words,
the text interpreter processes the contents of the input buffer by
parsing strings from the parse area until the parse area is empty.}.
This example shows how @code{>IN} changes as the text interpreter parses
the input buffer:

@example
: remaining source >in @@ /string
  cr ." ->" type ." <-" ; immediate 

1 2 3 remaining + remaining . 

: foo 1 2 3 remaining swap remaining ;
@end example

@noindent
The result is:

@example
->+ remaining .<-
->.<-5  ok

->SWAP remaining ;-<
->;<-  ok
@end example

@cindex parsing words
The value of @code{>IN} can also be modified by a word in the input
buffer that is executed by the text interpreter.  This means that a word
can ``trick'' the text interpreter into either skipping a section of the
input buffer@footnote{This is how parsing words work.} or into parsing a
section twice. For example:

@example
: lat ." <<foo>>" ;
: flat ." <<bar>>" >IN DUP @@ 3 - SWAP ! ;
@end example

@noindent
When @code{flat} is executed, this output is produced@footnote{Exercise
for the reader: what would happen if the @code{3} were replaced with
@code{4}?}:

@example
<<bar>><<foo>>
@end example

This technique can be used to work around some of the interoperability
problems of parsing words.  Of course, it's better to avoid parsing
words where possible.

@noindent
Two important notes about the behaviour of the text interpreter:

@itemize @bullet
@item
It processes each input string to completion before parsing additional
characters from the input buffer.
@item
It treats the input buffer as a read-only region (and so must your code).
@end itemize

@noindent
When the text interpreter is in compile state, its behaviour changes in
these ways:

@itemize @bullet
@item
If a parsed string is found in the dictionary, the text interpreter will
perform the word's @dfn{compilation semantics}. In most cases, this
simply means that the execution semantics of the word will be appended
to the current definition.
@item
When a number is encountered, it is compiled into the current definition
(as a literal) rather than being pushed onto a parameter stack.
@item
If an error occurs, @code{state} is modified to put the text interpreter
back into interpret state.
@item
Each time a line is entered from the keyboard, Gforth prints
``@code{ compiled}'' rather than `` @code{ok}''.
@end itemize

@cindex text interpreter - input sources
When the text interpreter is using an input device other than the
keyboard, its behaviour changes in these ways:

@itemize @bullet
@item
When the parse area is empty, the text interpreter attempts to refill
the input buffer from the input source. When the input source is
exhausted, the input source is set back to the previous input source.
@item
It doesn't print out ``@code{ ok}'' or ``@code{ compiled}'' messages each
time the parse area is emptied.
@item
If an error occurs, the input source is set back to the user input
device.
@end itemize

You can read about this in more detail in @ref{Input Sources}.

doc->in
doc-source

doc-tib
doc-#tib

doc-interpret

@menu
* Input Sources::
* Number Conversion::
* Interpret/Compile states::
* Interpreter Directives::
* Recognizers::
* Text Interpreter Hooks::
@end menu

@node Input Sources, Number Conversion, The Text Interpreter, The Text Interpreter
@subsection Input Sources
@cindex input sources
@cindex text interpreter - input sources

By default, the text interpreter processes input from the user input
device (the keyboard) when Forth starts up. The text interpreter can
process input from any of these sources:

@itemize @bullet
@item
The user input device -- the keyboard.
@item
A file, using the words described in @ref{Forth source files}.
@item
A block, using the words described in @ref{Blocks}.
@item
A text string, using @code{evaluate}.
@end itemize

A program can identify the current input device from the values of
@code{source-id} and @code{blk}.


doc-source-id
doc-blk

doc-save-input
doc-restore-input

doc-evaluate
doc-query



@node Number Conversion, Interpret/Compile states, Input Sources, The Text Interpreter
@subsection Number Conversion
@cindex number conversion

You get an overview of how the text interpreter converts its numeric
input in @ref{Literals in source code}.  This section describes some
related words.

By default, the number base used for integer number conversion is
given by the contents of the variable @code{base}.  Note that a lot of
confusion can result from unexpected values of @code{base}.  If you
change @code{base} anywhere, make sure to save the old value and
restore it afterwards; better yet, use @code{base-execute}, which does
this for you.  In general I recommend keeping @code{base} decimal, and
using the prefixes described in @ref{Literals in source code} for the
popular non-decimal bases.

doc-base-execute
doc-base
doc-hex
doc-decimal
doc-dpl

@cindex number conversion - traps for the unwary
@noindent
Number conversion has a number of traps for the unwary:

@itemize @bullet
@item
You cannot determine the current number base using the code sequence
@code{base @@ .} -- the number base is always 10 in the current number
base. Instead, use something like @code{base @@ dec.}
@item
There is a word @code{bin} but it does @i{not} set the number base!
(@pxref{General files}).
@item
Standard Forth requires the @code{.} of a double-precision number to be the
final character in the string.  Gforth allows the @code{.} to be
anywhere.
@item
The number conversion process does not check for overflow.
@end itemize

You can read numbers into your programs with the words described in
@ref{Line input and conversion}.

@node Interpret/Compile states, Interpreter Directives, Number Conversion, The Text Interpreter
@subsection Interpret/Compile states
@cindex Interpret/Compile states

A standard program is not permitted to change @code{state}
explicitly. However, it can change @code{state} implicitly, using the
words @code{[} and @code{]}. When @code{[} is executed it switches
@code{state} to interpret state, and therefore the text interpreter
starts interpreting. When @code{]} is executed it switches @code{state}
to compile state and therefore the text interpreter starts
compiling. The most common usage for these words is for switching into
interpret state and back from within a colon definition; this technique
can be used to compile a literal (for an example, @pxref{Literals}) or
for conditional compilation (for an example, @pxref{Interpreter
Directives}).


@c This is a bad example: It's non-standard, and it's not necessary.
@c However, I can't think of a good example for switching into compile
@c state when there is no current word (@code{state}-smart words are not a
@c good reason).  So maybe we should use an example for switching into
@c interpret @code{state} in a colon def. - anton
@c nac-> I agree. I started out by putting in the example, then realised
@c that it was non-ANS, so wrote more words around it. I hope this
@c re-written version is acceptable to you. I do want to keep the example
@c as it is helpful for showing what is and what is not portable, particularly
@c where it outlaws a style in common use.

@c anton: it's more important to show what's portable.  After we have done
@c that, we can also show what's not.  In any case, I have written a
@c section Compiling Words which also deals with [ ].

@c  !! The following example does not work in Gforth 0.5.9 or later.

@c  @code{[} and @code{]} also give you the ability to switch into compile
@c  state and back, but we cannot think of any useful Standard application
@c  for this ability. Pre-ANS Forth textbooks have examples like this:

@c  @example
@c  : AA ." this is A" ;
@c  : BB ." this is B" ;
@c  : CC ." this is C" ;

@c  create table ] aa bb cc [

@c  : go ( n -- ) \ n is offset into table.. 0 for 1st entry
@c    cells table + @@ execute ;
@c  @end example

@c  This example builds a jump table; @code{0 go} will display ``@code{this
@c  is A}''. Using @code{[} and @code{]} in this example is equivalent to
@c  defining @code{table} like this:

@c  @example
@c  create table ' aa COMPILE, ' bb COMPILE, ' cc COMPILE,
@c  @end example

@c  The problem with this code is that the definition of @code{table} is not
@c  portable -- it @i{compile}s execution tokens into code space. Whilst it
@c  @i{may} work on systems where code space and data space co-incide, the
@c  Standard only allows data space to be assigned for a @code{CREATE}d
@c  word. In addition, the Standard only allows @code{@@} to access data
@c  space, whilst this example is using it to access code space. The only
@c  portable, Standard way to build this table is to build it in data space,
@c  like this:

@c  @example
@c  create table ' aa , ' bb , ' cc ,
@c  @end example

@c  doc-state


@node Interpreter Directives, Recognizers, Interpret/Compile states, The Text Interpreter
@subsection Interpreter Directives
@cindex interpreter directives
@cindex conditional compilation

These words are usually used in interpret state; typically to control
which parts of a source file are processed by the text
interpreter. There are only a few Standard Forth Standard words, but Gforth
supplements these with a rich set of immediate control structure words
to compensate for the fact that the non-immediate versions can only be
used in compile state (@pxref{Control Structures}). Typical usage:

@example
[undefined] \G [if]
  : \G ... ; immediate
[endif]
@end example

So if the system does not define @code{\G}, compile some replacement
code (with possibly reduced functionality).


doc-[IF]
doc-[ELSE]
doc-[THEN]
doc-[ENDIF]

doc-[DEFINED]
doc-[UNDEFINED]

doc-[IFDEF]
doc-[IFUNDEF]

doc-[?DO]
doc-[DO]
doc-[LOOP]
doc-[+LOOP]
doc-[FOR]
doc-[NEXT]
doc-[I]
doc-INT-[I]

doc-[BEGIN]
doc-[UNTIL]
doc-[AGAIN]
doc-[WHILE]
doc-[REPEAT]

You can use @code{#line} to change Gforth's idea about the current
source line number and source file.  This is useful in cases where the
Forth file is generated from some other source code file, and you want
to get, e.g. error messages etc. that refer to the original source
code; then the Forth-code generator needs to insert @code{#line} lines
in the Forth code wherever appropriate.

doc-#line

@node Recognizers, Text Interpreter Hooks, Interpreter Directives, The Text Interpreter
@subsection Recognizers
@cindex recongizers

When the text interpreter processes source code, it divides the code
into blank-delimited strings, and then calls recognizers to identify
them as words, numbers, etc., until one recognizer identifies
(recognizes) the string; if the string is not recognized, the text
interpreter reports an error (@code{undefined word}).

The usual way to deal with recognizers is to just write code that one
of them identifies (@pxref{Default Recognizers}); however, you can
also manipulate them (@pxref{Dealing with existing Recognizers}) or
even define new ones (@pxref{Defining Recognizers}).

@menu
* Default Recognizers::
* Dealing with existing Recognizers::
* Defining Recognizers::
@end menu

@node Default Recognizers, Dealing with existing Recognizers, Recognizers, Recognizers
@subsubsection Default Recognizers
@cindex Recognizers normal usage

The standard Forth text interpreter recognizes words in the search
order (@code{rec-nt}), integer numbers (@code{rec-num}), and floating
point numbers (@code{rec-float}).  By default Gforth also recognizes
syntaxes for

@itemize @bullet

@item
strings, e.g., @code{"mystring"}, with @code{rec-string}

@item
complex numbers, e.g., @code{0e+1ei}, with @code{rec-complex}

@item
storing a value or changing a defered word, e.g., @code{->myvalue},
with @code{rec-to}

@item
the xt representing the interpretation semantics of a word, e.g.,
@code{`dup}, with @code{rec-tick}

@item
the nt of a word, e.g., @code{``mysynonym}, with @code{rec-dtick}

@item
an address in the body of a word, e.g., @code{<myarray+8>}, with
@code{rec-body}

@item
an access to an environment variable of the operating system, e.g.,
@code{$@{HOME@}}, with @code{rec-env}

@item
a word in a vocabulary, e.g., @code{myvoc1:myvoc2:myword}, with @code{rec-scope}

@item
using a specific recognizer to recognize something, e.g., @code{float?1.}, , with @code{rec-meta}

@end itemize

You can use @code{locate} (@pxref{Locating source code definitions}) to
determine which recognizer recognizes a piece of source code.  E.g.:

@example
defer mydefer
locate ->mydefer
@end example

will show that @code{rec-to} recognizes @code{->mydefer}.  However, if
the recognizer recognizes a dictionary word (e.g., the scope
recognizer), locate will show that word.

You can see which recognizers are used and the order of recognizers with

doc-.recognizers

Recognizers are typically designed to avoid matching the same strings
as other recognizers.  E.g., @code{rec-env} (the environment variable
recognizer) requires braces to avoid a conflict with the number
recognizer for input strings like @code{$ADD}.  There are a few
exceptions to this policy, however:

@itemize @bullet

@item
Word names can be anything, so they can conflict with any other
recognizer (and the search order is searched before other
recognizers).

However, they tend not to start with @code{0} (and if they do, they
contain special characters), so if your base is @code{hex}, it is a
good practice to let your numbers start with @code{0}.

In the code bases we have looked at, starting words with @code{'}
(quote aka tick) is much more common than starting them with @code{`}
(backquote aka backtick), so the recognizers for the xt and the nt use
@code{`} to reduce the number of conflicts.

@item
Both the integer recognizer @code{rec-num} and the floating-point
recognizer @code{rec-float} recognize, e.g., @code{1.}.  Because
@code{rec-num} is (by default) first, @code{1.} is recognized as a
double-cell integer.  If you change the recognizer order to use
@code{rec-float} first, @code{1.} is recognized as a floating-point
number, but loading code written in standard Forth may behave in a
non-standard way.

In any case, it's a good practice to avoid that conflict in your own
code as follows: Always write double-cell integers with a number
prefix, e.g., @code{#1.}; and always write floating-point numbers with
an @code{e}, e.g., @code{1e}.

@item
We have seen a few word names that start with @code{->}.  You can
avoid a conflict by using @code{to myvalue} or @code{to?->myvalue}
(the latter works with @code{postpone}).

@end itemize

@node Dealing with existing Recognizers, Defining Recognizers, Default Recognizers, Recognizers
@subsubsection Dealing with existing Recognizers
@cindex Recognizers, dealing with

A recognizer is a word to which you pass a string.  If the recognizer
recognizes the string, it typically returns some data and the xt of a
word for processing the data; this word is called the translator.  If
the recognizer does not recognize the string, it returns the xt of
@code{notfound}.

All recognizers have the stack effect ( c-addr u -- ... xt ).



Recognizers take a string and return some data and a translator for
interpreting that data.  Gforth implements that translator as xt
(executing it will perform the appropriate action to handle the token in
the current state), but other Forth systems may implement it as actual
table, with three xts inside.  The first xt is the
interpretation/run-time xt, it performs the interpretation semantics on
the data (usually, this means it just leaves the data on the stack).
The second xt performs the compilation semantics, it gets the data and
the run-time semantics xt.  The third xt perfoms the postpone semantics,
it also gets the data and the run-time semantics xt.  You can use
@code{>postpone} to postpone the run-time xt.

Recognizers are organized as stack, so you can arrange the sequence of
recognizers in the same way as the vocabulary stack.  Recognizer stacks
are themselves recognizers, i.e. they are executable, take a string and
return a translator.

doc-notfound
doc-rec-nt
doc-rec-num
doc-rec-float
doc-rec-string
doc-rec-to
doc-rec-tick
doc-rec-dtick
doc-rec-body
doc-get-recognizers
doc-set-recognizers
doc-recognize
doc-recognizer-sequence:
doc-forth-recognize
doc-forth-recognizer
doc-set-forth-recognize
doc-translate:
doc-translate-nt
doc-translate-num
doc-translate-dnum
doc-translate-float
doc-try-recognize
doc->interpret
doc->compile
doc->postpone
doc-translate-method:
doc-translate-state

@node Defining Recognizers,  , Dealing with existing Recognizers, Recognizers
@subsubsection Defining Recognizers



@node Text Interpreter Hooks,  , Recognizers, The Text Interpreter
@subsection Text Interpreter Hooks
@cindex hooks in the text interpreter

doc-before-line
doc-before-word
doc-line-end-hook

@c -------------------------------------------------------------
@node The Input Stream, Word Lists, The Text Interpreter, Words
@section The Input Stream
@cindex input stream

@c !! integrate this better with the "Text Interpreter" section
The text interpreter reads from the input stream, which can come from
several sources (@pxref{Input Sources}).  Some words, in particular
defining words, but also words like @code{'}, read parameters from the
input stream instead of from the stack.

Such words are called parsing words, because they parse the input
stream.  Parsing words are hard to use in other words, because it is
hard to pass program-generated parameters through the input stream.
They also usually have an unintuitive combination of interpretation and
compilation semantics when implemented naively, leading to various
approaches that try to produce a more intuitive behaviour
(@pxref{Combined words}).

It should be obvious by now that parsing words are a bad idea.  If you
want to implement a parsing word for convenience, also provide a factor
of the word that does not parse, but takes the parameters on the stack.
To implement the parsing word on top if it, you can use the following
words:

doc-parse
doc-string-parse
doc-parse-name
doc-parse-word
doc-name
doc-word
doc-refill

If you have to deal with a parsing word that does not have a
non-parsing factor, you can use @code{execute-parsing} to pass a
string to it:

doc-execute-parsing

Example:

@example
5 s" foo" ' constant execute-parsing
\ equivalent to
5 constant foo
@end example

A definition of this word in Standard Forth is provided in
@file{compat/execute-parsing.fs}.

If you want to run a parsing word on a file, the following word should
help:

doc-execute-parsing-file

@c -------------------------------------------------------------
@node Word Lists, Environmental Queries, The Input Stream, Words
@section Word Lists
@cindex word lists
@cindex header space

A wordlist is a list of named words; you can add new words and look up
words by name (and you can remove words in a restricted way with
markers).  Every named (and @code{reveal}ed) word is in one wordlist.

@cindex search order stack
The text interpreter searches the wordlists present in the search order
(a stack of wordlists), from the top to the bottom.  Within each
wordlist, the search starts conceptually at the newest word; i.e., if
two words in a wordlist have the same name, the newer word is found.

@cindex compilation word list
New words are added to the @dfn{compilation wordlist} (aka current
wordlist).

@cindex wid
A word list is identified by a cell-sized word list identifier (@i{wid})
in much the same way as a file is identified by a file handle. The
numerical value of the wid has no (portable) meaning, and might change
from session to session.

The Standard Forth ``Search order'' word set is intended to provide a set of
low-level tools that allow various different schemes to be
implemented. Gforth also provides @code{vocabulary}, a traditional Forth
word.  @file{compat/vocabulary.fs} provides an implementation in Standard
Forth.

@comment TODO: locals section refers to here, saying that every word list (aka
@comment vocabulary) has its own methods for searching etc. Need to document that.
@c anton: but better in a separate subsection on wordlist internals

@comment TODO: document markers, reveal, tables, mappedwordlist

@comment the gforthman- prefix is used to pick out the true definition of a
@comment word from the source files, rather than some alias.

doc-forth-wordlist
doc-definitions
doc-get-current
doc-set-current
doc-get-order
doc-set-order
doc-wordlist
doc-table
doc-cs-wordlist
doc-cs-vocabulary
doc->order
doc-previous
doc-also
doc-forth
doc-only
doc-order
doc-.voc

doc-find
doc-search-wordlist

doc-words
doc-vlist
doc-wordlist-words
doc-mwords
@c doc-words-deferred

@c doc-mappedwordlist @c map-structure undefined, implemantation-specific
doc-root
doc-vocabulary
doc-seal
doc-vocs
doc-current
doc-context

doc-map-vocs

@menu
* Vocabularies::
* Why use word lists?::
* Word list example::
@end menu

@node Vocabularies, Why use word lists?, Word Lists, Word Lists
@subsection Vocabularies
@cindex Vocabularies, detailed explanation

Here is an example of creating and using a new wordlist using Standard
Forth words:

@example
wordlist constant my-new-words-wordlist
: my-new-words get-order nip my-new-words-wordlist swap set-order ;

\ add it to the search order
also my-new-words

\ alternatively, add it to the search order and make it
\ the compilation word list
also my-new-words definitions
\ type "order" to see the problem
@end example

The problem with this example is that @code{order} has no way to
associate the name @code{my-new-words} with the wid of the word list (in
Gforth, @code{order} and @code{vocs} will display @code{???}  for a wid
that has no associated name). There is no Standard way of associating a
name with a wid.

In Gforth, this example can be re-coded using @code{vocabulary}, which
associates a name with a wid:

@example
vocabulary my-new-words

\ add it to the search order
also my-new-words

\ alternatively, add it to the search order and make it
\ the compilation word list
my-new-words definitions
\ type "order" to see that the problem is solved
@end example


@node Why use word lists?, Word list example, Vocabularies, Word Lists
@subsection Why use word lists?
@cindex word lists - why use them?

Here are some reasons why people use wordlists:

@itemize @bullet

@c anton: Gforth's hashing implementation makes the search speed
@c independent from the number of words.  But it is linear with the number
@c of wordlists that have to be searched, so in effect using more wordlists
@c actually slows down compilation.

@c @item
@c To improve compilation speed by reducing the number of header space
@c entries that must be searched. This is achieved by creating a new
@c word list that contains all of the definitions that are used in the
@c definition of a Forth system but which would not usually be used by
@c programs running on that system. That word list would be on the search
@c list when the Forth system was compiled but would be removed from the
@c search list for normal operation. This can be a useful technique for
@c low-performance systems (for example, 8-bit processors in embedded
@c systems) but is unlikely to be necessary in high-performance desktop
@c systems.

@item
To prevent a set of words from being used outside the context in which
they are valid. Two classic examples of this are an integrated editor
(all of the edit commands are defined in a separate word list; the
search order is set to the editor word list when the editor is invoked;
the old search order is restored when the editor is terminated) and an
integrated assembler (the op-codes for the machine are defined in a
separate word list which is used when a @code{CODE} word is defined).

@item
To organize the words of an application or library into a user-visible
set (in @code{forth-wordlist} or some other common wordlist) and a set
of helper words used just for the implementation (hidden in a separate
wordlist).  This keeps @code{words}' output smaller, separates
implementation and interface, and reduces the chance of name conflicts
within the common wordlist.

@item
To prevent a name-space clash between multiple definitions with the same
name. For example, when building a cross-compiler you might have a word
@code{IF} that generates conditional code for your target system. By
placing this definition in a different word list you can control whether
the host system's @code{IF} or the target system's @code{IF} get used in
any particular context by controlling the order of the word lists on the
search order stack.

@end itemize

The downsides of using wordlists are:

@itemize

@item
Debugging becomes more cumbersome.

@item
Name conflicts worked around with wordlists are still there, and you
have to arrange the search order carefully to get the desired results;
if you forget to do that, you get hard-to-find errors (as in any case
where you read the code differently from the compiler; @code{see} can
help seeing which of several possible words the name resolves to in such
cases).  @code{See} displays just the name of the words, not what
wordlist they belong to, so it might be misleading.  Using unique names
is a better approach to avoid name conflicts.

@item
You have to explicitly undo any changes to the search order.  In many
cases it would be more convenient if this happened implicitly.  Gforth
currently does not provide such a feature, but it may do so in the
future.
@end itemize


@node Word list example,  , Why use word lists?, Word Lists
@subsection Word list example
@cindex word lists - example

The following example is from the
@uref{https://www.complang.tuwien.ac.at/forth/garbage-collection.zip,
garbage collector} and uses wordlists to separate public words from
helper words:

@example
get-current ( wid )
vocabulary garbage-collector also garbage-collector definitions
... \ define helper words
( wid ) set-current \ restore original (i.e., public) compilation wordlist
... \ define the public (i.e., API) words
    \ they can refer to the helper words
previous \ restore original search order (helper words become invisible)
@end example

@c -------------------------------------------------------------
@node Environmental Queries, Files, Word Lists, Words
@section Environmental Queries
@cindex environmental queries

Forth-94 introduced the idea of ``environmental queries'' as a way
for a program running on a system to determine certain characteristics
of the system.  The Standard specifies a number of strings that might
be recognised by a system, and a way of querying them:

doc-environment?

Note that, whilst the documentation for (e.g.)
@code{ADDRESS-UNIT-BITS} shows it returning one cell on the stack,
querying it using @code{environment?}  will return an additional item;
the @code{true} flag that shows that the string was recognised; so for
querying @code{ADDRESS-UNIT-BITS} the stack effect of
@code{environment?} is @code{( c-addr u -- n true )}.

@c Get all the words in environment-wordlist:
@c gforth -e "environment-wordlist wordlist-words cr bye"|sed 's/[ ]/\n/g'|sed '/^$/d'|tac

Several environmental queries deal with the system's limits:

doc-ADDRESS-UNIT-BITS
doc-MAX-CHAR
doc-/COUNTED-STRING
doc-/HOLD
doc-/PAD
doc-CORE
doc-CORE-EXT
doc-FLOORED
doc-MAX-N
doc-MAX-U
doc-MAX-D
doc-MAX-UD
doc-return-stack-cells
doc-stack-cells
doc-floating-stack
doc-#locals
doc-wordlists
doc-max-float
doc-xchar-encoding
doc-max-xchar
doc-xchar-maxmem

Several environemtal queries are there for determining the presence of
the Forth-94 version of a wordset; they all have the stack effect
@code{( -- f )} if the string is present (so the @code{environment?}
stack effect for these queries is @code{( c-addr u -- false / f true
)}.

@code{block
block-ext
double
double-ext
exception
exception-ext
facility
facility-ext
file
file-ext
floating
floating-ext
locals
locals-ext
memory-alloc
memory-alloc-ext
tools
tools-ext
search-order
search-order-ext
string
string-ext}

These wordset queries were rarely used and implemented, so Forth-2012
did not introduce a way to query for the Forth-2012 variants of the
wordsets.  Instead, the idea is that you use @code{[defined]}
(@pxref{Interpreter Directives}) instead.

Forth-200x (a group that works on the next standard; the documents
that they produce are also called Forth-200x) defines extension
queries for the extension proposals once they finish changing (CfV
stage), so programs using these proposals can check whether a system
has them, and maybe load the reference implementation (if one exists).
If @code{environment?} finds such a query, then the corresponding
proposal on @url{www.forth200x.org} is implemented on the system (but
the absence tells you nothing, as usual with @code{environment?}).
These queries have the stack effect @code{( -- )}, which means that
for them @code{environment?} has the stack effect @code{( c-addr u --
false / true )}, which is more convenient than that of wordset
queries.  A number of these proposals have been incorporated into
Forth-2012.  The extension queries are also not particularly popular
among Forth system implementors, so going for @code{[defined]} may be
the better approach.  Anyway, Gforth implements the following
extension queries:

@code{X:2value
X:buffer
X:deferred
X:defined
X:ekeys
X:escaped-strings
X:extension-query
X:fp-stack
X:ftrunc
X:fvalue
X:locals
X:n-to-r
X:number-prefixes
X:parse-name
X:required
X:s-escape-quote
X:s-to-f
X:structures
X:synonym
X:text-substitution
X:throw-iors
X:traverse-wordlist
X:xchar}

In addition, Gforth implements the following Gforth-specific queries:

doc-gforth
doc-os-class
doc-os-type

The Standard requires that the header space used for environmental queries
be distinct from the header space used for definitions.

Typically, a Forth system supports environmental queries by creating a
set of definitions in a wordlist that is @i{only} used for
environmental queries; that is what Gforth does. There is no Standard
way of adding definitions to the set of recognised environmental
queries, but in Gforth and other systems that use the wordlist
mechanism, the wordlist used to honour environmental queries can be
manipulated just like any other word list.

doc-environment-wordlist
doc-environment

@comment TODO Document the standard strings or note where they are documented herein

Here are some examples of using environmental queries:

@example
s" address-unit-bits" environment? 0=
[IF]
     cr .( environmental attribute address-units-bits unknown... ) cr
[ELSE]
     drop \ ensure balanced stack effect
[THEN]

\ this might occur in the prelude of a standard program that uses THROW
s" exception" environment? [IF]
   0= [IF]
      : throw abort" exception thrown" ;
   [THEN]
[ELSE] \ we don't know, so make sure
   : throw abort" exception thrown" ;
[THEN]

s" gforth" environment? [IF] .( Gforth version ) TYPE
                        [ELSE] .( Not Gforth..) [THEN]

\ a program using v*
s" gforth" environment? [IF]
  s" 0.5.0" compare 0< [IF] \ v* is a primitive since 0.5.0
   : v* ( f_addr1 nstride1 f_addr2 nstride2 ucount -- r )
     >r swap 2swap swap 0e r> 0 ?DO
       dup f@@ over + 2swap dup f@@ f* f+ over + 2swap
     LOOP
     2drop 2drop ; 
  [THEN]
[ELSE] \ 
  : v* ( f_addr1 nstride1 f_addr2 nstride2 ucount -- r )
  ...
[THEN]
@end example

Here is an example of adding a definition to the environment word list:

@example
get-current environment-wordlist set-current
true constant block
true constant block-ext
set-current
@end example

You can see what definitions are in the environment word list like this:

@example
environment-wordlist wordlist-words
@end example


@c -------------------------------------------------------------
@node Files, Blocks, Environmental Queries, Words
@section Files
@cindex files
@cindex I/O - file-handling

Gforth provides facilities for accessing files that are stored in the
host operating system's file-system. Files that are processed by Gforth
can be divided into two categories:

@itemize @bullet
@item
Files that are processed by the Text Interpreter (@dfn{Forth source files}).
@item
Files that are processed by some other program (@dfn{general files}).
@end itemize

@menu
* Forth source files::
* General files::
* Redirection::
* Directories::
* Search Paths::
@end menu

@c -------------------------------------------------------------
@node Forth source files, General files, Files, Files
@subsection Forth source files
@cindex including files
@cindex Forth source files

The simplest way to interpret the contents of a file is to use one of
these two formats:

@example
include mysource.fs
s" mysource.fs" included
@end example

You usually want to include a file only if it is not included already
(by, say, another source file). In that case, you can use one of these
three formats:

@example
require mysource.fs
needs mysource.fs
s" mysource.fs" required
@end example

@cindex stack effect of included files
@cindex including files, stack effect
It is good practice to write your source files such that interpreting them
does not change the stack. Source files designed in this way can be used with
@code{required} and friends without complications. For example:

@example
1024 require foo.fs drop
@end example

Here you want to pass the argument 1024 (e.g., a buffer size) to
@file{foo.fs}.  Interpreting @file{foo.fs} has the stack effect ( n -- n
), which allows its use with @code{require}.  Of course with such
parameters to required files, you have to ensure that the first
@code{require} fits for all uses (i.e., @code{require} it early in the
master load file).

doc-include-file
doc-included
doc-included?
doc-include
doc-required
doc-require
doc-needs
doc-\\\
doc-.included
@c doc-init-included-files @c internal
doc-sourcefilename
doc-sourceline#

A definition in Standard Forth for @code{required} is provided in
@file{compat/required.fs}.

@c -------------------------------------------------------------
@node General files, Redirection, Forth source files, Files
@subsection General files
@cindex general files
@cindex file-handling

Files are opened/created by name and type. The following file access
methods (FAMs) are recognised:

@cindex fam (file access method)
doc-r/o
doc-r/w
doc-w/o
doc-bin
doc-+fmode


When a file is opened/created, it returns a file identifier,
@i{wfileid} that is used for all other file commands. All file
commands also return a status value, @i{wior}, that is 0 for a
successful operation and an implementation-defined non-zero value in the
case of an error.


doc-open-file
doc-create-file

doc-close-file
doc-delete-file
doc-rename-file
doc-read-file
doc-read-line
doc-key-file
doc-key?-file
doc-file-eof?
doc-write-file
doc-write-line
doc-emit-file
doc-flush-file

doc-file-status
doc-file-position
doc-reposition-file
doc-file-size
doc-resize-file

doc-slurp-file
doc-slurp-fid
doc-stdin
doc-stdout
doc-stderr

@c ---------------------------------------------------------
@node Redirection, Directories, General files, Files
@subsection Redirection
@cindex Redirection
@cindex Input Redirection
@cindex Output Redirection

You can redirect the output of @code{type} and @code{emit} and all the
words that use them (all output words that don't have an explicit
target file) to an arbitrary file with the @code{outfile-execute},
used like this:

@example
: some-warning ( n -- )
    cr ." warning# " . ;

: print-some-warning ( n -- )
    ['] some-warning stderr outfile-execute ;
@end example

After @code{some-warning} is executed, the original output direction
is restored; this construct is safe against exceptions.  Similarly,
there is @code{infile-execute} for redirecting the input of @code{key}
and its users (any input word that does not take a file explicitly).

doc-outfile-execute
doc-outfile-id
doc-infile-execute
doc-infile-id

If you do not want to redirect the input or output to a file, you can
also make use of the fact that @code{key}, @code{emit} and @code{type}
are deferred words (@pxref{Deferred Words}).  However, in that case
you have to worry about the restoration and the protection against
exceptions yourself; also, note that for redirecting the output in
this way, you have to redirect both @code{emit} and @code{type}.

@c ---------------------------------------------------------
@node Directories, Search Paths, Redirection, Files
@subsection Directories
@cindex Directories

You can split a file name into a directory and base component:

doc-basename
doc-dirname

You can open and read directories similar to files.  Reading gives you
one directory entry at a time; you can match that to a filename (with
wildcards).

doc-open-dir
doc-read-dir
doc-close-dir
doc-filename-match
doc-get-dir
doc-set-dir
doc-=mkdir
doc-mkdir-parents

@c ---------------------------------------------------------
@node Search Paths,  , Directories, Files
@subsection Search Paths
@cindex path for @code{included}
@cindex file search path
@cindex @code{include} search path
@cindex search path for files

If you specify an absolute filename (i.e., a filename starting with
@file{/} or @file{~}, or with @file{:} in the second position (as in
@samp{C:...})) for @code{included} and friends, that file is included
just as you would expect.

If the filename starts with @file{./}, this refers to the directory that
the present file was @code{included} from.  This allows files to include
other files relative to their own position (irrespective of the current
working directory or the absolute position).  This feature is essential
for libraries consisting of several files, where a file may include
other files from the library.  It corresponds to @code{#include "..."}
in C. If the current input source is not a file, @file{.} refers to the
directory of the innermost file being included, or, if there is no file
being included, to the current working directory.

For relative filenames (not starting with @file{./}), Gforth uses a
search path similar to Forth's search order (@pxref{Word Lists}). It
tries to find the given filename in the directories present in the path,
and includes the first one it finds. There are separate search paths for
Forth source files and general files.  If the search path contains the
directory @file{.}, this refers to the directory of the current file, or
the working directory, as if the file had been specified with @file{./}.

Use @file{~+} to refer to the current working directory (as in the
@code{bash}).

doc-absolute-file?

@c anton: fold the following subsubsections into this subsection?

@menu
* Source Search Paths::
* General Search Paths::
@end menu

@c ---------------------------------------------------------
@node Source Search Paths, General Search Paths, Search Paths, Search Paths
@subsubsection Source Search Paths
@cindex search path control, source files

The search path is initialized when you start Gforth (@pxref{Invoking
Gforth}). You can display it and change it using @code{fpath} in
combination with the general path handling words.

doc-fpath
doc-.fpath
doc-file>fpath
@c the functionality of the following words is easily available through
@c   fpath and the general path words.  The may go away.
@c doc-fpath+
@c doc-fpath=
@c doc-open-fpath-file

@noindent
Here is an example of using @code{fpath} and @code{require}:

@example
fpath path= /usr/lib/forth/|./
require timer.fs
@end example


@c ---------------------------------------------------------
@node General Search Paths,  , Source Search Paths, Search Paths
@subsubsection General Search Paths
@cindex search path control, source files

Your application may need to search files in several directories, like
@code{included} does. To facilitate this, Gforth allows you to define
and use your own search paths, by providing generic equivalents of the
Forth search path words:

doc-open-path-file
doc-file>path
doc-clear-path
doc-also-path
doc-.path
doc-path+
doc-path=

Here's an example of creating a custom search path:
@c
@example
variable mypath \ no special allocation required, just a variable
mypath path= /lib|/usr/lib \ assign initial directories
mypath path+ /usr/local/lib \ append directory
mypath .path \ output:"/lib /usr/lib /usr/local/lib"
@end example

Search file and show resulting path:
@example
s" libm.so" mypath open-path-file throw type close-file \ output:"/lib/libm.so"
@end example

@c -------------------------------------------------------------
@node Blocks, Other I/O, Files, Words
@section Blocks
@cindex I/O - blocks
@cindex blocks

When you run Gforth on a modern desk-top computer, it runs under the
control of an operating system which provides certain services.  One of
these services is @var{file services}, which allows Forth source code
and data to be stored in files and read into Gforth (@pxref{Files}).

Traditionally, Forth has been an important programming language on
systems where it has interfaced directly to the underlying hardware with
no intervening operating system. Forth provides a mechanism, called
@dfn{blocks}, for accessing mass storage on such systems.

A block is a 1024-byte data area, which can be used to hold data or
Forth source code. No structure is imposed on the contents of the
block. A block is identified by its number; blocks are numbered
contiguously from 1 to an implementation-defined maximum.

A typical system that used blocks but no operating system might use a
single floppy-disk drive for mass storage, with the disks formatted to
provide 256-byte sectors. Blocks would be implemented by assigning the
first four sectors of the disk to block 1, the second four sectors to
block 2 and so on, up to the limit of the capacity of the disk. The disk
would not contain any file system information, just the set of blocks.

@cindex blocks file
On systems that do provide file services, blocks are typically
implemented by storing a sequence of blocks within a single @dfn{blocks
file}.  The size of the blocks file will be an exact multiple of 1024
bytes, corresponding to the number of blocks it contains. This is the
mechanism that Gforth uses.

@cindex @file{blocks.fb}
Only one blocks file can be open at a time. If you use block words without
having specified a blocks file, Gforth defaults to the blocks file
@file{blocks.fb}. Gforth uses the Forth search path when attempting to
locate a blocks file (@pxref{Source Search Paths}).

@cindex block buffers
When you read and write blocks under program control, Gforth uses a
number of @dfn{block buffers} as intermediate storage. These buffers are
not used when you use @code{load} to interpret the contents of a block.

The behaviour of the block buffers is analagous to that of a cache.
Each block buffer has three states:

@itemize @bullet
@item
Unassigned
@item
Assigned-clean
@item
Assigned-dirty
@end itemize

Initially, all block buffers are @i{unassigned}. In order to access a
block, the block (specified by its block number) must be assigned to a
block buffer.

The assignment of a block to a block buffer is performed by @code{block}
or @code{buffer}. Use @code{block} when you wish to modify the existing
contents of a block. Use @code{buffer} when you don't care about the
existing contents of the block@footnote{The Standard Forth definition of
@code{buffer} is intended not to cause disk I/O; if the data associated
with the particular block is already stored in a block buffer due to an
earlier @code{block} command, @code{buffer} will return that block
buffer and the existing contents of the block will be
available. Otherwise, @code{buffer} will simply assign a new, empty
block buffer for the block.}.

Once a block has been assigned to a block buffer using @code{block} or
@code{buffer}, that block buffer becomes the @i{current block
buffer}. Data may only be manipulated (read or written) within the
current block buffer.

When the contents of the current block buffer has been modified it is
necessary, @emph{before calling @code{block} or @code{buffer} again}, to
either abandon the changes (by doing nothing) or mark the block as
changed (assigned-dirty), using @code{update}. Using @code{update} does
not change the blocks file; it simply changes a block buffer's state to
@i{assigned-dirty}.  The block will be written implicitly when it's
buffer is needed for another block, or explicitly by @code{flush} or
@code{save-buffers}.

word @code{Flush} writes all @i{assigned-dirty} blocks back to the
blocks file on disk. Leaving Gforth with @code{bye} also performs a
@code{flush}.

In Gforth, @code{block} and @code{buffer} use a @i{direct-mapped}
algorithm to assign a block buffer to a block. That means that any
particular block can only be assigned to one specific block buffer,
called (for the particular operation) the @i{victim buffer}. If the
victim buffer is @i{unassigned} or @i{assigned-clean} it is allocated to
the new block immediately. If it is @i{assigned-dirty} its current
contents are written back to the blocks file on disk before it is
allocated to the new block.

Although no structure is imposed on the contents of a block, it is
traditional to display the contents as 16 lines each of 64 characters.  A
block provides a single, continuous stream of input (for example, it
acts as a single parse area) -- there are no end-of-line characters
within a block, and no end-of-file character at the end of a
block. There are two consequences of this:

@itemize @bullet
@item
The last character of one line wraps straight into the first character
of the following line
@item
The word @code{\} -- comment to end of line -- requires special
treatment; in the context of a block it causes all characters until the
end of the current 64-character ``line'' to be ignored.
@end itemize

In Gforth, when you use @code{block} with a non-existent block number,
the current blocks file will be extended to the appropriate size and the
block buffer will be initialised with spaces.

Gforth includes a simple block editor (type @code{use blocked.fb 0 list}
for details) but doesn't encourage the use of blocks; the mechanism is
only provided for backward compatibility.

Common techniques that are used when working with blocks include:

@itemize @bullet
@item
A screen editor that allows you to edit blocks without leaving the Forth
environment.
@item
Shadow screens; where every code block has an associated block
containing comments (for example: code in odd block numbers, comments in
even block numbers). Typically, the block editor provides a convenient
mechanism to toggle between code and comments.
@item
Load blocks; a single block (typically block 1) contains a number of
@code{thru} commands which @code{load} the whole of the application.
@end itemize

See Frank Sergeant's Pygmy Forth to see just how well blocks can be
integrated into a Forth programming environment.

@comment TODO what about errors on open-blocks?

doc-open-blocks
doc-use
doc-block-offset
doc-get-block-fid
doc-block-position

doc-list
doc-scr

doc-block
doc-buffer

doc-empty-buffers
doc-empty-buffer
doc-update
doc-updated?
doc-save-buffers
doc-save-buffer
doc-flush

doc-load
doc-thru
doc-+load
doc-+thru
doc---gforthman--->
doc-block-included


@c -------------------------------------------------------------
@node Other I/O, OS command line arguments, Blocks, Words
@section Other I/O
@cindex I/O - keyboard and display

@menu
* Simple numeric output::       Predefined formats
* Formatted numeric output::    Formatted (pictured) output
* Floating-point output::
* Miscellaneous output::
* Displaying characters and strings::  Other stuff
* Terminal output::             Cursor positioning etc.
* Single-key input::
* Line input and conversion::
* Pipes::                       How to create your own pipes
* Xchars and Unicode::          Non-ASCII characters
* i18n and l10n::               Internationalization and Localization
* Substitute::                  Text macro substitution
* CSV Reader::                  Data import facility
@end menu

@node Simple numeric output, Formatted numeric output, Other I/O, Other I/O
@subsection Simple numeric output
@cindex numeric output - simple/free-format

The simplest output functions are those that display numbers from the
data stack.  Numbers are displayed in the base (aka radix) stored in
@code{base} (@pxref{Number Conversion}).

doc-.
doc-dec.
doc-hex.
doc-u.
doc-.r
doc-u.r
doc-dec.r
doc-d.
doc-ud.
doc-d.r
doc-ud.r


@node Formatted numeric output, Floating-point output, Simple numeric output, Other I/O
@subsection Formatted numeric output
@cindex formatted numeric output
@cindex pictured numeric output
@cindex numeric output - formatted

Forth traditionally uses a technique called @dfn{pictured numeric
output} for formatted printing of integers.  In this technique, digits
are extracted from the number (using the current output radix defined
by @code{base}, @pxref{Number Conversion}), converted to ASCII codes
and prepended to a string that is built in a scratch-pad area of
memory (@pxref{core-idef, Implementation-defined options,
Implementation-defined options}). Arbitrary characters can be
prepended to the string during the extraction process. The completed
string is specified by an address and length and can be manipulated
(@code{TYPE}ed, copied, modified) under program control.

All of the integer output words described in the previous section
(@pxref{Simple numeric output}) are implemented in Gforth using pictured
numeric output.

Three important things to remember about pictured numeric output:

@itemize @bullet
@item
It always operates on double-precision numbers; to display a
single-precision number, convert it first (for ways of doing this
@pxref{Double precision}).
@item
It always treats the double-precision number as though it were
unsigned. The examples below show ways of printing signed numbers.
@item
The string is built up from right to left; least significant digit first.
@end itemize

Standard Forth supports a single output buffer (aka hold area) that
you empty and initialize with @code{<#} and for which you get the
result string with @code{#>}.

Gforth additionally supports nested usage of this buffer, allowing,
e.g., to nest output from the debugging tracer @code{~~} inside code
dealing with the hold area: @code{<<#} starts a new nest, @code{#>}
produces the result string, and @code{#>>} unnests: the hold area for
the nest is reclaimed, and @code{#>} now produces the string for the
next-outer nest.  All of Gforth's higher-level numeric output words
use @code{<<#} ... @code{#>} ... @code{#>>} and can be nested inside
other users of the hold area.

doc-<#
doc-<<#
doc-#
doc-#s
doc-hold
doc-holds
doc-sign
doc-#>
doc-#>>

@noindent
Here are some examples of using pictured numeric output:

@example
: my-u. ( u -- )
  \ Simplest use of pns.. behaves like Standard u. 
  0              \ convert to unsigned double
  <<#            \ start conversion
  #s             \ convert all digits
  #>             \ complete conversion
  TYPE SPACE     \ display, with trailing space
  #>> ;          \ release hold area

: cents-only ( u -- )
  0              \ convert to unsigned double
  <<#            \ start conversion
  # #            \ convert two least-significant digits
  #>             \ complete conversion, discard other digits
  TYPE SPACE     \ display, with trailing space
  #>> ;          \ release hold area

: dollars-and-cents ( u -- )
  0              \ convert to unsigned double
  <<#            \ start conversion
  # #            \ convert two least-significant digits
  '.' hold       \ insert decimal point
  #s             \ convert remaining digits
  '$' hold       \ append currency symbol
  #>             \ complete conversion
  TYPE SPACE     \ display, with trailing space
  #>> ;          \ release hold area

: my-. ( n -- )
  \ handling negatives.. behaves like Standard .
  s>d            \ convert to signed double
  swap over dabs \ leave sign byte followed by unsigned double
  <<#            \ start conversion
  #s             \ convert all digits
  rot sign       \ get at sign byte, append "-" if needed
  #>             \ complete conversion
  TYPE SPACE     \ display, with trailing space
  #>> ;          \ release hold area

: account. ( n -- )
  \ accountants don't like minus signs, they use parentheses
  \ for negative numbers
  s>d            \ convert to signed double
  swap over dabs \ leave sign byte followed by unsigned double
  <<#            \ start conversion
  2 pick         \ get copy of sign byte
  0< IF ')' hold THEN \ right-most character of output
  #s             \ convert all digits
  rot            \ get at sign byte
  0< IF '(' hold THEN
  #>             \ complete conversion
  TYPE SPACE     \ display, with trailing space
  #>> ;          \ release hold area

@end example

Here are some examples of using these words:

@example
1 my-u. 1
hex -1 my-u. decimal FFFFFFFF
1 cents-only 01
1234 cents-only 34
2 dollars-and-cents $0.02
1234 dollars-and-cents $12.34
123 my-. 123
-123 my. -123
123 account. 123
-456 account. (456)
@end example

@node Floating-point output, Miscellaneous output, Formatted numeric output, Other I/O
@subsection Floating-point output
@cindex numeric output, FP
@cindex FP output
@cindex floating-point output

Floating-point output is always displayed using base 10.

doc-f.
doc-fe.
doc-fs.
doc-fp.

Examples of printing the number 1234.5678E23 in the different
floating-point output formats are shown below.

@example
f. 123456780000000000000000000.
fe. 123.456780000000E24
fs. 1.23456780000000E26
fp. 123.456780000000Y
@end example

The length of the output is influenced by:

doc-precision
doc-set-precision

You can control the output in more detail with:

doc-f.rdp

To give you a better intuition of how they influence the output, here
are some examples of parameter combinations; in each line the same
number is printed, in each column the same parameter combination is
used for printing:

@example
    12 13 0    7 3 4   7 3 0   7 3 1   7 5 1   7 7 1   7 0 2  4 2 1
|-1.234568E-6|-1.2E-6| -0.000|-1.2E-6|-1.2E-6|-1.2E-6|-1.2E-6|****|
|-1.234568E-5|-1.2E-5| -0.000|-1.2E-5|-.00001|-1.2E-5|-1.2E-5|****|
|-1.234568E-4|-1.2E-4| -0.000|-1.2E-4|-.00012|-1.2E-4|-1.2E-4|****|
|-1.234568E-3|-1.2E-3| -0.001| -0.001|-.00123|-1.2E-3|-1.2E-3|****|
|-1.234568E-2|-1.2E-2| -0.012| -0.012|-.01235|-1.2E-2|-1.2E-2|-.01|
|-1.234568E-1|-1.2E-1| -0.123| -0.123|-.12346|-1.2E-1|-1.2E-1|-.12|
|-1.2345679E0| -1.235| -1.235| -1.235|-1.23E0|-1.23E0|-1.23E0|-1E0|
|-1.2345679E1|-12.346|-12.346|-12.346|-1.23E1|-1.23E1|   -12.|-1E1|
|-1.2345679E2|-1.23E2|-1.23E2|-1.23E2|-1.23E2|-1.23E2|  -123.|-1E2|
|-1.2345679E3|-1.23E3|-1.23E3|-1.23E3|-1.23E3|-1.23E3| -1235.|-1E3|
|-1.2345679E4|-1.23E4|-1.23E4|-1.23E4|-1.23E4|-1.23E4|-12346.|-1E4|
|-1.2345679E5|-1.23E5|-1.23E5|-1.23E5|-1.23E5|-1.23E5|-1.23E5|-1E5|
@end example

You can generate a string instead of displaying the number with:

doc-f>str-rdp
doc-f>buf-rdp

There is also a primitive used for implementing higher-level
FP-to-string words:

doc-represent

@node Miscellaneous output, Displaying characters and strings, Floating-point output, Other I/O
@subsection Miscellaneous output


doc-cr
doc-space
doc-spaces
doc-out
doc-.\"
doc-."
doc-.(

If you don't want to worry about wether to use @code{.( hello)} or
@code{." hello"}, you can write @code{"hello" type}, which gives you
what you usually want (but is less portable to other Forth systems).

@noindent
As an example, consider the following text, stored in a file @file{test.fs}:

@example
.( text-1)
: my-word
  ." text-2" cr
  .( text-3)
  "text-4" type
;

." text-5"
"text-6" type
@end example

When you load this code into Gforth, the following output is generated:

@example
@kbd{include test.fs @key{RET}} text-1text-3text-5text-6 ok
@end example

@itemize @bullet
@item
Messages @code{text-1} and @code{text-3} are displayed because @code{.(} 
is an immediate word; it behaves in the same way whether it is used inside
or outside a colon definition.
@item
Message @code{text-5} is displayed because of Gforth's added interpretation
semantics for @code{."}.
@item
Message @code{text-6} is displayed because @code{"text-6" type} is
interpreted.
@item
Message @code{text-2} is @i{not} displayed, because the text interpreter
performs the compilation semantics for @code{."} within the definition of
@code{my-word}.
@item
Message @code{text-4} is @i{not} displayed, because @code{"text-4"
type} is compiled into @code{my-word}.
@end itemize


@node Displaying characters and strings, Terminal output, Miscellaneous output, Other I/O
@subsection Displaying characters and strings
@cindex characters - displaying
@cindex character strings - displaying

doc-type
doc-xemit
doc-emit
doc-typewhite


@node Terminal output, Single-key input, Displaying characters and strings, Other I/O
@subsection Terminal output
@cindex output to terminal
@cindex terminal output

If you are outputting to a terminal, you may want to control the
positioning of the cursor:
@cindex cursor positioning

doc-at-xy
doc-at-deltaxy

In order to know where to position the cursor, it is often helpful to
know the size of the screen:
@cindex terminal size 

doc-form

And sometimes you want to use:
@cindex clear screen

doc-page

Note that on non-terminals you should use @code{12 emit}, not
@code{page}, to get a form feed.

@subsubsection Color output
The following words are used to create (semantic) colorful output;
further output is produced in the color and style given by the word;
the actual color and style depends on the theme (see below).

doc-default-color
doc-error-color
doc-error-hl-inv
doc-error-hl-ul
doc-warning-color
doc-info-color
doc-success-color
doc-input-color
doc-status-color
@c !! Commented out, because it makes no sense to document attr! without
@c    documenting how to produce an attr
@c doc-attr!

@subsubsection Color themes
Depending on wether you prefer bright or dark background the
foreground colors-theme can be changed by:

doc-light-mode
doc-dark-mode
doc-uncolored-mode
doc-magenta-input

@node Single-key input, Line input and conversion, Terminal output, Other I/O
@subsection Single-key input
@cindex single-key input
@cindex input, single-key

If you want to get a single printable character, you can use
@code{key}; to check whether a character is available for @code{key},
you can use @code{key?}.

doc-key
doc-key-ior
doc-key?
doc-xkey?

If you want to process a mix of printable and non-printable
characters, you can do that with @code{ekey} and friends.  @code{Ekey}
produces a keyboard event that you have to convert into a character
with @code{ekey>char} or into a key identifier with @code{ekey>fkey}.

Typical code for using EKEY looks like this:

@example
ekey ekey>xchar if ( xc )
  ... \ do something with the character
else ekey>fkey if ( key-id )
  case
    k-up                                  of ... endof
    k-f1                                  of ... endof
    k-left k-shift-mask or k-ctrl-mask or of ... endof
    ...
  endcase
else ( keyboard-event )
  drop \ just ignore an unknown keyboard event type
then then
@end example

doc-ekey
doc-ekey>xchar
doc-ekey>char
doc-ekey>fkey
doc-ekey?

The key identifiers for cursor keys are:

doc-k-left
doc-k-right
doc-k-up
doc-k-down
doc-k-home
doc-k-end
doc-k-prior
doc-k-next
doc-k-insert
doc-k-delete

The key identifiers for function keys (aka keypad keys) are:

doc-k-f1
doc-k-f2
doc-k-f3
doc-k-f4
doc-k-f5
doc-k-f6
doc-k-f7
doc-k-f8
doc-k-f9
doc-k-f10
doc-k-f11
doc-k-f12

Note that @code{k-f11} and @code{k-f12} are not as widely available.

You can combine these key identifiers with masks for various shift keys:

doc-k-shift-mask
doc-k-ctrl-mask
doc-k-alt-mask

There are a number of keys that have ASCII values, and therefore are
unlikely to be reported as special keys, but the combination of these
keys with shift keys may be reported as a special key:

doc-k-enter
doc-k-backspace
doc-k-tab

Moreover, there the following key codes for keys and other events:

doc-k-winch
doc-k-pause
doc-k-mute
doc-k-volup
doc-k-voldown
doc-k-sel
doc-k-eof

Note that, even if a Forth system has @code{ekey>fkey} and the key
identifier words, the keys are not necessarily available or it may not
necessarily be able to report all the keys and all the possible
combinations with shift masks.  Therefore, write your programs in such
a way that they are still useful even if the keys and key combinations
cannot be pressed or are not recognized.

Examples: Older keyboards often do not have an F11 and F12 key.  If
you run Gforth in an xterm, the xterm catches a number of combinations
(e.g., @key{Shift-Up}), and never passes it to Gforth.  Finally,
Gforth currently does not recognize and report combinations with
multiple shift keys (so the @key{shift-ctrl-left} case in the example
above would never be entered).

Gforth recognizes various keys available on ANSI terminals (in MS-DOS
you need the ANSI.SYS driver to get that behaviour); it works by
recognizing the escape sequences that ANSI terminals send when such a
key is pressed.  If you have a terminal that sends other escape
sequences, you will not get useful results on Gforth.  Other Forth
systems may work in a different way.

Gforth also provides a few words for outputting names of function
keys:

doc-fkey.
doc-simple-fkey-string


@node  Line input and conversion, Pipes, Single-key input, Other I/O
@subsection Line input and conversion
@cindex line input from terminal
@cindex input, linewise from terminal
@cindex convertin strings to numbers
@cindex I/O - see input

For ways of storing character strings in memory see @ref{String
representations}.

@comment TODO examples for >number >float accept key key? pad parse word refill
@comment then index them

Words for inputting one line from the keyboard:

doc-accept
doc-edit-line

Conversion words:

doc-s>number?
doc-s>unumber?
doc->number
doc->float
doc->float1

@comment obsolescent words..
Obsolescent input and conversion words:

doc-convert
doc-expect
doc-span


@node Pipes, Xchars and Unicode, Line input and conversion, Other I/O
@subsection Pipes
@cindex pipes, creating your own

In addition to using Gforth in pipes created by other processes
(@pxref{Gforth in pipes}), you can create your own pipe with
@code{open-pipe}, and read from or write to it.

doc-open-pipe
doc-close-pipe

If you write to a pipe, Gforth can throw a @code{broken-pipe-error}; if
you don't catch this exception, Gforth will catch it and exit, usually
silently (@pxref{Gforth in pipes}).  Since you probably do not want
this, you should wrap a @code{catch} or @code{try} block around the code
from @code{open-pipe} to @code{close-pipe}, so you can deal with the
problem yourself, and then return to regular processing.

doc-broken-pipe-error

@node Xchars and Unicode, i18n and l10n, Pipes, Other I/O
@subsection Xchars and Unicode

ASCII is only appropriate for the English language. Most western
languages however fit somewhat into the Forth frame, since a byte is
sufficient to encode the few special characters in each (though not
always the same encoding can be used; latin-1 is most widely used,
though). For other languages, different char-sets have to be used,
several of them variable-width.  To deal with this problem, characters
are often represented as Unicode codepoints on the stack, and as UTF-8
byte strings in memory.  An Unicode codepoint often represents one
application-level character, but Unicode also supports decomposed
characters that consist of several code points, e.g., a base letter
and a combining diacritical mark.

An Unicode codepoint can consume more than one byte in memory, so we
adjust our terminology: A char is a raw byte in memory or a value in
the range 0-255 on the stack.  An xchar (for extended char) stands for
one codepoint; it is represented by one or more bytes in memory and
may have larger values on the stack.  ASCII characters are the same as
chars and as xchars: values in the range 0-127, and a single byte with
that value in memory.

When using UTF-8 encoding, all other codepoints take more than one
byte/char.  In most cases, you can just treat such characters as
strings in memory and don't need to use the following words, but if
you want to deal with individual codepoints, the following words are
useful.  We currently have no words for dealing with decomposed
characters.

The xchar words add a few data types:

@itemize

@item
@var{xc} is an extended char (xchar) on the stack. It occupies one
cell, and is a subset of unsigned cell.  On 16 bit systems, only the
BMP subset of the Unicode character set (i.e., codepoints <65536) can
be represented on the stack.  If you represent your application
characters as strings at all times, you can avoid this limitation.

@item
@var{xc-addr} is the address of an xchar in memory. Alignment
requirements are the same as @var{c-addr}. The memory representation of an
xchar differs from the stack representation, and depends on the
encoding used. An xchar may use a variable number of chars in memory.

@item
@var{xc-addr} @var{u} is a buffer of xchars in memory, starting at
@var{xc-addr}, @var{u} chars (i.e., bytes, not xchars) long.

@end itemize

doc-xc-size
doc-x-size
doc-xc@
doc-xc@+
doc-xc@+?
doc-xc!+?
doc-xc!+
doc-xchar+
doc-xchar-
doc-+x/string
doc-x\string-
doc--trailing-garbage
doc-x-width
doc-xkey
doc-xc-width
doc-xhold
doc-xc,

@node i18n and l10n, Substitute, Xchars and Unicode, Other I/O
@subsection Internationalization and Localization

Programs for end users require to address those in their native
language.  There is a decades old proposal for such a facility that has
been split from other proposals for international character sets like
Xchars (@pxref{Xchars and Unicode}) and Substitute (@pxref{Substitute}).
Messages displayed on the screen need to be translated from the native
language of the developers to the local languages of the user.

Strings subject to translation are declared with
@code{L" }@var{string}@code{"}.  This returns a locale string identifier
(LSID).  LSIDs are opaque types, taking a cell on the stack.  LSIDs can
be translated into a locale; locales are languages and country-specific
variants of that language.

doc-l"
doc-lu"
doc-native@
doc-locale@
doc-locale!
doc-language
doc-country
doc-locale-file
doc-included-locale
doc-include-locale
doc-locale-csv
doc-.locale-csv
doc-locale-csv-out

@node Substitute, CSV Reader, i18n and l10n, Other I/O
@subsection Substitute

This is a simple text macro replacement facility.  Texts in the form
@code{"text %macro% text"} are processed, and the macro variables enclosed
in @code{'%'} are replaced with their associated strings.  Two
consecutive @code{%} are replaced by one @code{%}.  Macros are defined
in a specific wordlist, and return a string upon execution; the standard
defines only one way to declare macros, @code{replaces}, which creates a
macro that just returns a string.

doc-macros-wordlist
doc-replaces
doc-.substitute
doc-$substitute
doc-substitute
doc-unescape
doc-$unescape

@node CSV Reader,  , Substitute, Other I/O
@subsection CSV Reader

Comma-separated values (CSV) are a popular text format to interchange
data.  Gforth provides an reader in @file{csv.fs}.

doc-read-csv

@node OS command line arguments, Locals, Other I/O, Words
@section OS command line arguments
@cindex OS command line arguments
@cindex command line arguments, OS
@cindex arguments, OS command line

The usual way to pass arguments to Gforth programs on the command line
is via the @option{-e} option, e.g.

@example
gforth -e "123 456" foo.fs -e bye
@end example

However, you may want to interpret the command-line arguments directly.
In that case, you can access the (image-specific) command-line arguments
through @code{next-arg}:

doc-next-arg

Here's an example program @file{echo.fs} for @code{next-arg}:

@example
: echo ( -- )
    begin
	next-arg 2dup 0 0 d<> while
	    type space
    repeat
    2drop ;

echo cr bye
@end example

This can be invoked with

@example
gforth echo.fs hello world
@end example

and it will print

@example
hello world
@end example

The next lower level of dealing with the OS command line are the
following words:

doc-arg
doc-shift-args

Finally, at the lowest level Gforth provides the following words:

doc-argc
doc-argv

@c -------------------------------------------------------------
@node Locals, Structures, OS command line arguments, Words
@section Locals
@cindex locals

Local variables can make Forth programming more enjoyable and Forth
programs easier to read. Unfortunately, the locals of Standard Forth are
laden with restrictions. Therefore, we provide not only the Standard Forth
locals wordset, but also our own, more powerful locals wordset (we
implemented the Standard Forth locals wordset through our locals wordset).

The ideas in this section have also been published in M. Anton Ertl,
@cite{@uref{https://www.complang.tuwien.ac.at/papers/ertl94l.ps.gz,
Automatic Scoping of Local Variables}}, EuroForth '94.

@menu
* Gforth locals::
* Standard Forth locals::
@end menu

@node Gforth locals, Standard Forth locals, Locals, Locals
@subsection Gforth locals
@cindex Gforth locals
@cindex locals, Gforth style

Locals can be defined with

@example
@{: local1 local2 ... -- comment :@}
@end example
or
@example
@{: local1 local2 ... :@}
@end example
or
@example
@{: local1 local2 ... | ulocal0 ulocal1 -- comment :@}
@end example

E.g.,
@example
: max @{: n1 n2 -- n3 :@}
 n1 n2 > if
   n1
 else
   n2
 endif ;
@end example

The similarity of locals definitions with stack comments is intended. A
locals definition often replaces the stack comment of a word. The order
of the locals corresponds to the order in a stack comment and everything
after the @code{--} is really a comment.

This similarity has one disadvantage: It is too easy to confuse locals
declarations with stack comments, causing bugs and making them hard to
find. However, this problem can be avoided by appropriate coding
conventions: Do not use both notations in the same program. If you do,
they should be distinguished using additional means, e.g. by position.

@cindex types of locals
@cindex locals types
The name of the local may be preceded by a type specifier, e.g.,
@code{F:} for a floating point value:

@example
: CX* @{: F: Ar F: Ai F: Br F: Bi -- Cr Ci :@}
\ complex multiplication
 Ar Br f* Ai Bi f* f-
 Ar Bi f* Ai Br f* f+ ;
@end example

@cindex flavours of locals
@cindex locals flavours
@cindex value-flavoured locals
@cindex varue-flavoured locals
@cindex variable-flavoured locals
Gforth currently supports cells (@code{W:}, @code{WA:}, @code{W^}),
doubles (@code{D:}, @code{DA:}, @code{D^}), floats (@code{F:},
@code{FA:}, @code{F^}), characters (@code{C:}, @code{CA:},
@code{C^}), and xts (@code{xt:}, @code{xta:}) in several flavours:

@table @dfn

@item value-flavoured
(@pxref{Values}) A value-flavoured local (defined with @code{W:},
@code{D:} etc.) produces its value and can be changed with @code{TO}.

@item varue-flavoured
(@pxref{Varues}) A varue-flavoured local @i{l} (defined with
@code{WA:} etc.) behaves exactly like a value-flavoured local, except
that you can use @code{addr @i{l}} to get its address (which becomes
invalid when the variable's scope is left).  Currently there is no
performance difference, but in the long run value-flavoured locals
will be significantly faster, because they can reside in registers.

@item variable-flavoured
(@pxref{Variables}) A variable-flavoured local (defined with @code{W^}
etc.)  produces its address (which becomes invalid when the variable's
scope is left).  E.g., the standard word @code{emit} can be defined in
terms of @code{type} like this:

@example
: emit @{: C^ char* -- :@}
    char* 1 type ;
@end example

@item defer-flavoured
(@pxref{Deferred Words}) A defer-flavoured local (defined with
@code{XT:} or @code{XTA:}) @code{execute}s the xt; you can use
@code{action-of} (@pxref{Deferred Words}) to get the xt out of a
defer-flavoured local.  If the local is defined with @code{xta:}, you
can use @code{addr} to get the address (valid until the end of the
scope of the local) where the xt is stored.  E.g., the standard word
@code{execute} can be defined with a defer-flavoured local like this:

@example
: execute @{: xt: x -- :@}
  x ;
@end example

@end table

@cindex default type of locals
@cindex locals, default type
A local without type specifier is a @code{W:} local.  You can allow or
disallow the use of @code{addr} with:

doc-default-wa:
doc-default-w:

@cindex locals initialization
@cindex initialization of locals
All flavours of locals are initialized with values from the data or
(for FP locals) FP stack, with the exception being locals defined
behind @code{|}: Gforth initializes them to 0; some Forth systems leave
them uninitialized.

@cindex data structure locals
Gforth supports the square bracket notation for local buffers and data
structures.  These locals are similar to variable-flavored locals, the
size is specified as a constant expression.  A declaration looks
@code{@i{name}[ @i{size} ]}.  The Forth expression @code{@i{size}} is
evaluated during declaration, it must have the stack effect @code{( --
+n )}, giving the size in bytes.  The square bracket @code{[} is part
of the defined name.

Local data structures are initialized by copying @i{size} bytes from
an address passed on the stack; uninitialized local data structures
(after @code{|} in the declaration) are not erased, they just contain
whatever data there was on the locals stack before.

Example:

@example
begin-structure test-struct
  field: a1
  field: a2
end-structure

: test-local @{: foo[ test-struct ] :@}
    foo[ a1 !  foo[ a2 !
    foo[ test-struct dump ;
@end example

Gforth allows defining locals everywhere in a colon definition. This
poses the following questions:

@menu
* Locals definition words::
* Where are locals visible by name?::
* How long do locals live?::
* Locals programming style::
* Locals implementation::
* Closures::
@end menu

@node Locals definition words, Where are locals visible by name?, Gforth locals, Gforth locals
@subsubsection Locals definitions words

This section documents the words used for defining locals.  Note that
the run-times for the words (like @code{W:}) that define a local are
performed from the rightmost defined local to the leftmost defined
local, such that the rightmost local gets the top of stack.

doc-{:
doc---locals---
doc-|
doc-:}
doc-{
doc-}
doc-w:
doc-wa:
doc-w^
doc-d:
doc-da:
doc-d^
doc-c:
doc-ca:
doc-c^
doc-f:
doc-fa:
doc-f^
doc-xt:
doc-xta:

Note that @code{|}, @code{--}, @code{:@}} and @code{@}} are not
normally in the search order (they are in the vocabulary
@code{locals-types}), and they are not necessarily words in all Forth
systems; therefore they are documented as Gforth words.

@node Where are locals visible by name?, How long do locals live?, Locals definition words, Gforth locals
@subsubsection Where are locals visible by name?
@cindex locals visibility
@cindex visibility of locals
@cindex scope of locals

Basically, the answer is that locals are visible where you would expect
it in block-structured languages, and sometimes a little longer. If you
want to restrict the scope of a local, enclose its definition in
@code{SCOPE}...@code{ENDSCOPE}.


doc-scope
doc-endscope


These words behave like control structure words, so you can use them
with @code{CS-PICK} and @code{CS-ROLL} to restrict the scope in
arbitrary ways.

If you want a more exact answer to the visibility question, here's the
basic principle: A local is visible in all places that can only be
reached through the definition of the local@footnote{In compiler
construction terminology, all places dominated by the definition of the
local.}. In other words, it is not visible in places that can be reached
without going through the definition of the local. E.g., locals defined
in @code{IF}...@code{ENDIF} are visible until the @code{ENDIF}, locals
defined in @code{BEGIN}...@code{UNTIL} are visible after the
@code{UNTIL} (until, e.g., a subsequent @code{ENDSCOPE}).

The reasoning behind this solution is: We want to have the locals
visible as long as it is meaningful. The user can always make the
visibility shorter by using explicit scoping. In a place that can
only be reached through the definition of a local, the meaning of a
local name is clear. In other places it is not: How is the local
initialized at the control flow path that does not contain the
definition? Which local is meant, if the same name is defined twice in
two independent control flow paths?

This should be enough detail for nearly all users, so you can skip the
rest of this section. If you really must know all the gory details and
options, read on.

In order to implement this rule, the compiler has to know which places
are unreachable. It knows this automatically after @code{AHEAD},
@code{AGAIN}, @code{EXIT} and @code{LEAVE}; in other cases (e.g., after
most @code{THROW}s), you can use the word @code{UNREACHABLE} to tell the
compiler that the control flow never reaches that place. If
@code{UNREACHABLE} is not used where it could, the only consequence is
that the visibility of some locals is more limited than the rule above
says. If @code{UNREACHABLE} is used where it should not (i.e., if you
lie to the compiler), buggy code will be produced.


doc-unreachable


Another problem with this rule is that at @code{BEGIN}, the compiler
does not know which locals will be visible on the incoming
back-edge. All problems discussed in the following are due to this
ignorance of the compiler (we discuss the problems using @code{BEGIN}
loops as examples; the discussion also applies to @code{?DO} and other
loops). Perhaps the most insidious example is:
@example
AHEAD
BEGIN
  x
[ 1 CS-ROLL ] THEN
  @{: x :@}
  ...
UNTIL
@end example

This should be legal according to the visibility rule. The use of
@code{x} can only be reached through the definition; but that appears
textually below the use.

From this example it is clear that the visibility rules cannot be fully
implemented without major headaches. Our implementation treats common
cases as advertised and the exceptions are treated in a safe way: The
compiler makes a reasonable guess about the locals visible after a
@code{BEGIN}; if it is too pessimistic, the
user will get a spurious error about the local not being defined; if the
compiler is too optimistic, it will notice this later and issue a
warning. In the case above the compiler would complain about @code{x}
being undefined at its use. You can see from the obscure examples in
this section that it takes quite unusual control structures to get the
compiler into trouble, and even then it will often do fine.

If the @code{BEGIN} is reachable from above, the most optimistic guess
is that all locals visible before the @code{BEGIN} will also be
visible after the @code{BEGIN}. This guess is valid for all loops that
are entered only through the @code{BEGIN}, in particular, for normal
@code{BEGIN}...@code{WHILE}...@code{REPEAT} and
@code{BEGIN}...@code{UNTIL} loops and it is implemented in our
compiler. When the branch to the @code{BEGIN} is finally generated by
@code{AGAIN} or @code{UNTIL}, the compiler checks the guess and
warns the user if it was too optimistic:
@example
IF
  @{: x :@}
BEGIN
  \ x ? 
[ 1 cs-roll ] THEN
  ...
UNTIL
@end example

Here, @code{x} lives only until the @code{BEGIN}, but the compiler
optimistically assumes that it lives until the @code{THEN}. It notices
this difference when it compiles the @code{UNTIL} and issues a
warning. The user can avoid the warning, and make sure that @code{x}
is not used in the wrong area by using explicit scoping:
@example
IF
  SCOPE
  @{: x :@}
  ENDSCOPE
BEGIN
[ 1 cs-roll ] THEN
  ...
UNTIL
@end example

Since the guess is optimistic, there will be no spurious error messages
about undefined locals.

If the @code{BEGIN} is not reachable from above (e.g., after
@code{AHEAD} or @code{EXIT}), the compiler cannot even make an
optimistic guess, as the locals visible after the @code{BEGIN} may be
defined later.

It pessimistically assumes that all locals are visible that were
visible at the latest place outside any control structure (i.e., where
nothing is on the control-flow stack).  This means that in:

@example
: foo
  IF @{: z :@} THEN
  @{: x :@}
  AHEAD
    BEGIN
      ( * )
    [ 1 CS-ROLL ] THEN
    @{: y :@}
    ...
  UNTIL ;
@end example

At the place marked with @code{( * )}, @code{x} is visible, but
@code{y} is not (although, according to the reachability rule it
should); @code{z} is not and should not be visible there.

However, you can use @code{ASSUME-LIVE} to make the compiler assume
that the same locals are visible at the BEGIN as at the point where
the top control-flow stack item was created.


doc-assume-live


@noindent
E.g.,
@example
IF
  @{: x :@}
  AHEAD
    ASSUME-LIVE
    BEGIN
      x
    [ 1 CS-ROLL ] THEN
    ...
  UNTIL
THEN
@end example

Here @code{x} would not be visible at the use of @code{x}, because its
definition is inside a control structure, but by using ASSUME-LIVE the
programmer tells the compiler that the locals visible at the
@code{AHEAD} should be visible at the @code{BEGIN}.

Other cases where the locals are defined before the @code{BEGIN} can be
handled by inserting an appropriate @code{CS-ROLL} before the
@code{ASSUME-LIVE} (and changing the control-flow stack manipulation
behind the @code{ASSUME-LIVE}).

Cases where locals are defined after the @code{BEGIN} (but should be
visible immediately after the @code{BEGIN}) can only be handled by
rearranging the loop. E.g., the ``most insidious'' example above can be
arranged into:
@example
BEGIN
  @{: x :@}
  ... 0=
WHILE
  x
REPEAT
@end example

@node How long do locals live?, Locals programming style, Where are locals visible by name?, Gforth locals
@subsubsection How long do locals live?
@cindex locals lifetime
@cindex lifetime of locals

The right answer for the lifetime question would be: A local lives at
least as long as it can be accessed. For a value-flavoured local this
means: until the end of its visibility. However, a variable-flavoured
local could be accessed through its address far beyond its visibility
scope. Ultimately, this would mean that such locals would have to be
garbage collected. Since this entails un-Forth-like implementation
complexities, I adopted the same cowardly solution as some other
languages (e.g., C): The local lives only as long as it is visible;
afterwards its address is invalid (and programs that access it
afterwards are erroneous).

@node Locals programming style, Locals implementation, How long do locals live?, Gforth locals
@subsubsection Locals programming style
@cindex locals programming style
@cindex programming style, locals

The freedom to define locals anywhere has the potential to change
programming styles dramatically. In particular, the need to use the
return stack for intermediate storage vanishes. Moreover, all stack
manipulations (except @code{PICK}s and @code{ROLL}s with run-time
determined arguments) can be eliminated: If the stack items are in the
wrong order, just write a locals definition for all of them; then
write the items in the order you want.

This seems a little far-fetched and eliminating stack manipulations is
unlikely to become a conscious programming objective. Still, the number
of stack manipulations will be reduced dramatically if local variables
are used liberally (e.g., compare @code{max} (@pxref{Gforth locals}) with
a traditional implementation of @code{max}).

This shows one potential benefit of locals: making Forth programs more
readable. Of course, this benefit will only be realized if the
programmers continue to honour the principle of factoring instead of
using the added latitude to make the words longer.

@cindex single-assignment style for locals
Using @code{TO} can and should be avoided.  Without @code{TO},
every value-flavoured local has only a single assignment and many
advantages of functional languages apply to Forth. I.e., programs are
easier to analyse, to optimize and to read: It is clear from the
definition what the local stands for, it does not turn into something
different later.

E.g., a definition using @code{TO} might look like this:
@example
: strcmp @{: addr1 u1 addr2 u2 -- n :@}
 u1 u2 min 0
 ?do
   addr1 c@@ addr2 c@@ -
   ?dup-if
     unloop exit
   then
   addr1 char+ TO addr1
   addr2 char+ TO addr2
 loop
 u1 u2 - ;
@end example
Here, @code{TO} is used to update @code{addr1} and @code{addr2} at
every loop iteration. @code{strcmp} is a typical example of the
readability problems of using @code{TO}. When you start reading
@code{strcmp}, you think that @code{addr1} refers to the start of the
string. Only near the end of the loop you realize that it is something
else.

This can be avoided by defining two locals at the start of the loop that
are initialized with the right value for the current iteration.
@example
: strcmp @{: addr1 u1 addr2 u2 -- n :@}
 addr1 addr2
 u1 u2 min 0 
 ?do @{: s1 s2 :@}
   s1 c@@ s2 c@@ -
   ?dup-if
     unloop exit
   then
   s1 char+ s2 char+
 loop
 2drop
 u1 u2 - ;
@end example
Here it is clear from the start that @code{s1} has a different value
in every loop iteration.

@node Locals implementation, Closures, Locals programming style, Gforth locals
@subsubsection Locals implementation
@cindex locals implementation
@cindex implementation of locals

@cindex locals stack
Gforth uses an extra locals stack. The most compelling reason for this
is that the return stack is not float-aligned; using an extra stack
also eliminates the problems and restrictions of using the return
stack as locals stack. Like the other stacks, the locals stack grows
toward lower addresses. A few primitives allow an efficient
implementation; you should not use them directly, but they appear in
the output of @code{see}, so they are documented here:

doc-@localn
doc-f@localn
doc-lp@
doc-lp+!#
doc-lp!
doc->l
doc-f>l


In addition to these primitives, some specializations of these
primitives for commonly occurring inline arguments are provided for
efficiency reasons, e.g., @code{@@local0} as specialization of @code{0
@@localn}. The following compiling words compile the right specialized
version, or the general version, as appropriate:


@c doc-compile-@local
@c doc-compile-f@local
doc-compile-lp+!


Combinations of conditional branches and @code{lp+!#} like
@code{?branch-lp+!#} (the locals pointer is only changed if the branch
is taken) are provided for efficiency and correctness in loops.

A special area in the dictionary space is reserved for keeping the
local variable names. @code{@{:} switches the dictionary pointer to this
area and @code{:@}} switches it back and generates the locals
initializing code. @code{W:} etc.@ are normal defining words. This
special area is cleared at the start of every colon definition.

@cindex word list for defining locals
A special feature of Gforth's dictionary is used to implement the
definition of locals without type specifiers: every word list (aka
vocabulary) has its own methods for searching
etc. (@pxref{Word Lists}). For the present purpose we defined a word list
with a special search method: When it is searched for a word, it
actually creates that word using @code{W:}. @code{@{:} changes the search
order to first search the word list containing @code{:@}}, @code{W:} etc.,
and then the word list for defining locals without type specifiers.

The lifetime rules support a stack discipline within a colon
definition: The lifetime of a local is either nested with other locals
lifetimes or it does not overlap them.

At @code{BEGIN}, @code{IF}, and @code{AHEAD} no code for locals stack
pointer manipulation is generated. Between control structure words
locals definitions can push locals onto the locals stack. @code{AGAIN}
is the simplest of the other three control flow words. It has to
restore the locals stack depth of the corresponding @code{BEGIN}
before branching. The code looks like this:
@format
@code{lp+!#} current-locals-size @minus{} dest-locals-size
@code{branch} <begin>
@end format

@code{UNTIL} is a little more complicated: If it branches back, it
must adjust the stack just like @code{AGAIN}. But if it falls through,
the locals stack must not be changed. The compiler generates the
following code:
@format
@code{?branch-lp+!#} <begin> current-locals-size @minus{} dest-locals-size
@end format
The locals stack pointer is only adjusted if the branch is taken.

@code{THEN} can produce somewhat inefficient code:
@format
@code{lp+!#} current-locals-size @minus{} orig-locals-size
<orig target>:
@code{lp+!#} orig-locals-size @minus{} new-locals-size
@end format
The second @code{lp+!#} adjusts the locals stack pointer from the
level at the @i{orig} point to the level after the @code{THEN}. The
first @code{lp+!#} adjusts the locals stack pointer from the current
level to the level at the orig point, so the complete effect is an
adjustment from the current level to the right level after the
@code{THEN}.

@cindex locals information on the control-flow stack
@cindex control-flow stack items, locals information
In a conventional Forth implementation a dest control-flow stack entry
is just the target address and an orig entry is just the address to be
patched. Our locals implementation adds a word list to every orig or dest
item. It is the list of locals visible (or assumed visible) at the point
described by the entry. Our implementation also adds a tag to identify
the kind of entry, in particular to differentiate between live and dead
(reachable and unreachable) orig entries.

A few unusual operations have to be performed on locals word lists:

doc-common-list
doc-sub-list?
doc-list-size


Several features of our locals word list implementation make these
operations easy to implement: The locals word lists are organised as
linked lists; the tails of these lists are shared, if the lists
contain some of the same locals; and the address of a name is greater
than the address of the names behind it in the list.

Another important implementation detail is the variable
@code{dead-code}. It is used by @code{BEGIN} and @code{THEN} to
determine if they can be reached directly or only through the branch
that they resolve. @code{dead-code} is set by @code{UNREACHABLE},
@code{AHEAD}, @code{EXIT} etc., and cleared at the start of a colon
definition, by @code{BEGIN} and usually by @code{THEN}.

Counted loops are similar to other loops in most respects, but
@code{LEAVE} requires special attention: It performs basically the same
service as @code{AHEAD}, but it does not create a control-flow stack
entry. Therefore the information has to be stored elsewhere;
traditionally, the information was stored in the target fields of the
branches created by the @code{LEAVE}s, by organizing these fields into a
linked list. Unfortunately, this clever trick does not provide enough
space for storing our extended control flow information. Therefore, we
introduce another stack, the leave stack. It contains the control-flow
stack entries for all unresolved @code{LEAVE}s.

Local names are kept until the end of the colon definition, even if
they are no longer visible in any control-flow path. In a few cases
this may lead to increased space needs for the locals name area, but
usually less than reclaiming this space would cost in code size.

@node Closures,  , Locals implementation, Gforth locals
@subsubsection Closures
@cindex closures

Gforth also provides basic closures.  A closure is a combination of a
quotation (@pxref{Quotations}) and locals.  Gforth's closures have
locals which are filled with values at the closure's run-time,
producing a trampoline xt.  When executing that trampoline xt, the
closure's code is executed, with access to the closure's locals on the
locals stack.  Modifications of the closure's locals aren't
persistent, i.e. when the closure @code{EXIT}s, the modified values
are lost.

doc-[{:
doc-:}l
doc-:}d
doc-:}h
doc-:}h1
doc-:}xt
doc->addr
doc-free-closure

@example
: foo [@{: a f: b d: c xt: d :@}d a . b f. c d. d ;] ;
5 3.3e #1234. ' cr foo execute
@end example

@code{foo} creates a closure in the dictionary with a single cell, a
floating point, a double, and an xt, and prints the first three values
before executing the xt on invocation.

This allows to implement Donald Knuth's ``Man or boy test'' proposed
in 1964 to test Algol compilers.

@example
: A @{: w^ k x1 x2 x3 xt: x4 xt: x5 | w^ B :@} recursive
    k @ 0<= IF  x4 x5 f+  ELSE
        B k x1 x2 x3 action-of x4 [@{: B k x1 x2 x3 x4 :@}L
            -1 k +!
            k @ B @ x1 x2 x3 x4 A ;] dup B !
        execute  THEN ;
: man-or-boy? ( n -- ) [: 1e ;] [: -1e ;] 2dup swap [: 0e ;] A f. ;
@end example

Sometimes, closures need a permanent storage to be modified; it is
even possible that more than one closure shares that permanent
storage.  In the example above, local variables of the outer procedure
are used for this, but in some cases, the closure lives longer than
the outer procedure; especially closures allocated in the dictionary
or on the heap are designed to outlive their parent procedure.

For those, we have home locations, which are allocated like closures,
but their code is directly executed at creation and should provide us
with the addresses of the home locations.

@example
: bar ( a b c -- aaddr baddr caddr hl-addr )
    <@{: w^ a w^ b w^ c :@}h a b c ;> ;
@end example

This example creates a home location with three cells on the heap, and
returns the addresses of the three locations and the address of the
home location.  This address can be used to @code{free} the home
location when it is no longer needed.

doc-<{:
doc-;>

@node Standard Forth locals,  , Gforth locals, Locals
@subsection Standard Forth locals
@cindex locals, Standard Forth style

The Forth-2012 standard defines a syntax for locals is restricted
version of Gforth's locals:

@itemize @bullet
@item
Locals can only be cell-sized values (no type specifiers are allowed).
@item
Locals can be defined only outside control structures.
@item
Only one locals definition per definition is allowed.
@item
Locals can interfere with explicit usage of the return stack. For the
exact (and long) rules, see the standard. If you don't use return stack
accessing words in a definition using locals, you will be all right. The
purpose of this rule is to make locals implementation on the return
stack easier.
@item
The whole locals definition must be in one line.
@end itemize

The Standard Forth locals wordset itself consists of two words:
@code{@{:} and:

doc-(local)

The ANS Forth locals extension wordset defines a syntax using
@code{locals|}, but it is so awful that we strongly recommend not to
use it. We have implemented this syntax to make porting to Gforth
easy, but do not document it here. The problem with this syntax is
that the locals are defined in an order reversed with respect to the
standard stack comment notation, making programs harder to read, and
easier to misread and miswrite. The only merit of this syntax is that
it is easy to implement using the ANS Forth locals wordset, but then,
so is the @code{@{:} syntax.

@c ----------------------------------------------------------
@node Structures, Object-oriented Forth, Locals, Words
@section  Structures
@cindex structures
@cindex records

This section presents the structure package that comes with Gforth. A
version of the package implemented in Standard Forth is available in
@file{compat/struct.fs}. This package was inspired by a posting on
comp.lang.forth in 1989 (unfortunately I don't remember, by whom;
possibly John Hayes). A version of this section has been published in
M. Anton Ertl,
@uref{https://www.complang.tuwien.ac.at/forth/objects/structs.html, Yet
Another Forth Structures Package}, Forth Dimensions 19(3), pages
13--16. Marcel Hendrix provided helpful comments.

@menu
* Why explicit structure support?::
* Structure Usage::
* Structure Naming Convention::
* Structure Implementation::
* Structure Glossary::
* Forth200x Structures::
@end menu

@node Why explicit structure support?, Structure Usage, Structures, Structures
@subsection Why explicit structure support?

@cindex address arithmetic for structures
@cindex structures using address arithmetic
If we want to use a structure containing several fields, we could simply
reserve memory for it, and access the fields using address arithmetic
(@pxref{Address arithmetic}). As an example, consider a structure with
the following fields

@table @code
@item a
is a float
@item b
is a cell
@item c
is a float
@end table

Given the (float-aligned) base address of the structure we get the
address of the field

@table @code
@item a
without doing anything further.
@item b
with @code{float+}
@item c
with @code{float+ cell+ faligned}
@end table

It is easy to see that this can become quite tiring. 

Moreover, it is not very readable, because seeing a
@code{cell+} tells us neither which kind of structure is
accessed nor what field is accessed; we have to somehow infer the kind
of structure, and then look up in the documentation, which field of
that structure corresponds to that offset.

Finally, this kind of address arithmetic also causes maintenance
troubles: If you add or delete a field somewhere in the middle of the
structure, you have to find and change all computations for the fields
afterwards.

So, instead of using @code{cell+} and friends directly, how
about storing the offsets in constants:

@example
0 constant a-offset
0 float+ constant b-offset
0 float+ cell+ faligned c-offset
@end example

Now we can get the address of field @code{x} with @code{x-offset
+}. This is much better in all respects. Of course, you still
have to change all later offset definitions if you add a field. You can
fix this by declaring the offsets in the following way:

@example
0 constant a-offset
a-offset float+ constant b-offset
b-offset cell+ faligned constant c-offset
@end example

Since we always use the offsets with @code{+}, we could use a defining
word @code{cfield} that includes the @code{+} in the action of the
defined word:

@example
: cfield ( n "name" -- )
    create ,
does> ( name execution: addr1 -- addr2 )
    @@ + ;

0 cfield a
0 a float+ cfield b
0 b cell+ faligned cfield c
@end example

Instead of @code{x-offset +}, we now simply write @code{x}.

The structure field words now can be used quite nicely. However,
their definition is still a bit cumbersome: We have to repeat the
name, the information about size and alignment is distributed before
and after the field definitions etc.  The structure package presented
here addresses these problems.

@node Structure Usage, Structure Naming Convention, Why explicit structure support?, Structures
@subsection Structure Usage
@cindex structure usage

@cindex @code{field} usage
@cindex @code{struct} usage
@cindex @code{end-struct} usage
You can define a structure for a (data-less) linked list with:
@example
struct
    cell% field list-next
end-struct list%
@end example

With the address of the list node on the stack, you can compute the
address of the field that contains the address of the next node with
@code{list-next}. E.g., you can determine the length of a list
with:

@example
: list-length ( list -- n )
\ "list" is a pointer to the first element of a linked list
\ "n" is the length of the list
    0 BEGIN ( list1 n1 )
        over
    WHILE ( list1 n1 )
        1+ swap list-next @@ swap
    REPEAT
    nip ;
@end example

You can reserve memory for a list node in the dictionary with
@code{list% %allot}, which leaves the address of the list node on the
stack. For the equivalent allocation on the heap you can use @code{list%
%alloc} (or, for an @code{allocate}-like stack effect (i.e., with ior),
use @code{list% %allocate}). You can get the the size of a list
node with @code{list% %size} and its alignment with @code{list%
%alignment}.

Note that in Standard Forth the body of a @code{create}d word is
@code{aligned} but not necessarily @code{faligned};
therefore, if you do a:

@example
create @emph{name} foo% %allot drop
@end example

@noindent
then the memory alloted for @code{foo%} is guaranteed to start at the
body of @code{@emph{name}} only if @code{foo%} contains only character,
cell and double fields.  Therefore, if your structure contains floats,
better use

@example
foo% %allot constant @emph{name}
@end example

@cindex structures containing structures
You can include a structure @code{foo%} as a field of
another structure, like this:
@example
struct
...
    foo% field ...
...
end-struct ...
@end example

@cindex structure extension
@cindex extended records
Instead of starting with an empty structure, you can extend an
existing structure. E.g., a plain linked list without data, as defined
above, is hardly useful; You can extend it to a linked list of integers,
like this:@footnote{This feature is also known as @emph{extended
records}. It is the main innovation in the Oberon language; in other
words, adding this feature to Modula-2 led Wirth to create a new
language, write a new compiler etc.  Adding this feature to Forth just
required a few lines of code.}

@example
list%
    cell% field intlist-int
end-struct intlist%
@end example

@code{intlist%} is a structure with two fields:
@code{list-next} and @code{intlist-int}.

@cindex structures containing arrays
You can specify an array type containing @emph{n} elements of
type @code{foo%} like this:

@example
foo% @emph{n} *
@end example

You can use this array type in any place where you can use a normal
type, e.g., when defining a @code{field}, or with
@code{%allot}.

@cindex first field optimization
The first field is at the base address of a structure and the word for
this field (e.g., @code{list-next}) actually does not change the address
on the stack. You may be tempted to leave it away in the interest of
run-time and space efficiency. This is not necessary, because the
structure package optimizes this case: If you compile a first-field
words, no code is generated. So, in the interest of readability and
maintainability you should include the word for the field when accessing
the field.


@node Structure Naming Convention, Structure Implementation, Structure Usage, Structures
@subsection Structure Naming Convention
@cindex structure naming convention

The field names that come to (my) mind are often quite generic, and,
if used, would cause frequent name clashes. E.g., many structures
probably contain a @code{counter} field. The structure names
that come to (my) mind are often also the logical choice for the names
of words that create such a structure.

Therefore, I have adopted the following naming conventions: 

@itemize @bullet
@cindex field naming convention
@item
The names of fields are of the form
@code{@emph{struct}-@emph{field}}, where
@code{@emph{struct}} is the basic name of the structure, and
@code{@emph{field}} is the basic name of the field. You can
think of field words as converting the (address of the)
structure into the (address of the) field.

@cindex structure naming convention
@item
The names of structures are of the form
@code{@emph{struct}%}, where
@code{@emph{struct}} is the basic name of the structure.
@end itemize

This naming convention does not work that well for fields of extended
structures; e.g., the integer list structure has a field
@code{intlist-int}, but has @code{list-next}, not
@code{intlist-next}.

@node Structure Implementation, Structure Glossary, Structure Naming Convention, Structures
@subsection Structure Implementation
@cindex structure implementation
@cindex implementation of structures

The central idea in the implementation is to pass the data about the
structure being built on the stack, not in some global
variable. Everything else falls into place naturally once this design
decision is made.

The type description on the stack is of the form @emph{align
size}. Keeping the size on the top-of-stack makes dealing with arrays
very simple.

@code{field} is a defining word that uses @code{Create}
and @code{DOES>}. The body of the field contains the offset
of the field, and the normal @code{DOES>} action is simply:

@example
@@ +
@end example

@noindent
i.e., add the offset to the address, giving the stack effect
@i{addr1 -- addr2} for a field.

@cindex first field optimization, implementation
This simple structure is slightly complicated by the optimization
for fields with offset 0, which requires a different
@code{DOES>}-part (because we cannot rely on there being
something on the stack if such a field is invoked during
compilation). Therefore, we put the different @code{DOES>}-parts
in separate words, and decide which one to invoke based on the
offset. For a zero offset, the field is basically a noop; it is
immediate, and therefore no code is generated when it is compiled.

@node Structure Glossary, Forth200x Structures, Structure Implementation, Structures
@subsection Structure Glossary
@cindex structure glossary


doc-%align
doc-%alignment
doc-%alloc
doc-%allocate
doc-%allot
doc-cell%
doc-char%
doc-dfloat%
doc-double%
doc-end-struct
doc-field
doc-float%
doc-sfloat%
doc-%size
doc-struct


@node Forth200x Structures,  , Structure Glossary, Structures
@subsection Forth200x Structures
@cindex Structures in Forth200x

The Forth 2012 standard defines a slightly less convenient form of
structures.  In general (when using @code{field+}, you have to perform
the alignment yourself, but there are a number of convenience words
(e.g., @code{field:}) that include alignment.

A typical usage example is:

@example
0
  field:                   s-a
  faligned 2 floats +field s-b
constant s-struct
@end example

An alternative way of writing this structure is:

@example
begin-structure s-struct
  field:                   s-a
  faligned 2 floats +field s-b
end-structure
@end example

You can define a structure that has the same fields and additional
fields as follows:

@example
s-struct
  cfield: t-c
  cfield: t-d
constant t-struct
@end example

or alternatively

@example
s-struct extend-structure t-struct
  cfield: t-c
  cfield: t-d
end-structure
@end example

doc-begin-structure
doc-extend-structure
doc-end-structure
doc-+field
doc-cfield:
doc-field:
doc-2field:
doc-ffield:
doc-sffield:
doc-dffield:
doc-wfield:
doc-lfield:
doc-xfield:

@c -------------------------------------------------------------
@node Object-oriented Forth, Regular Expressions, Structures, Words
@section Object-oriented Forth

Gforth comes with three packages for object-oriented programming:
@file{objects.fs}, @file{oof.fs}, and @file{mini-oof.fs}; none of them
is preloaded, so you have to @code{include} them before use. The most
important differences between these packages (and others) are discussed
in @ref{Comparison with other object models}. All packages are written
in Standard Forth and can be used with any other Standard Forth.

@menu
* Why object-oriented programming?::
* Object-Oriented Terminology::
* Objects::
* OOF::
* Mini-OOF::
* Mini-OOF2::
* Comparison with other object models::
@end menu

@c ----------------------------------------------------------------
@node Why object-oriented programming?, Object-Oriented Terminology, Object-oriented Forth, Object-oriented Forth
@subsection Why object-oriented programming?
@cindex object-oriented programming motivation
@cindex motivation for object-oriented programming

Often we have to deal with several data structures (@emph{objects}),
that have to be treated similarly in some respects, but differently in
others. Graphical objects are the textbook example: circles, triangles,
dinosaurs, icons, and others, and we may want to add more during program
development. We want to apply some operations to any graphical object,
e.g., @code{draw} for displaying it on the screen. However, @code{draw}
has to do something different for every kind of object.
@comment TODO add some other operations eg perimeter, area
@comment and tie in to concrete examples later..

We could implement @code{draw} as a big @code{CASE}
control structure that executes the appropriate code depending on the
kind of object to be drawn. This would be not be very elegant, and,
moreover, we would have to change @code{draw} every time we add
a new kind of graphical object (say, a spaceship).

What we would rather do is: When defining spaceships, we would tell
the system: ``Here's how you @code{draw} a spaceship; you figure
out the rest''.

This is the problem that all systems solve that (rightfully) call
themselves object-oriented; the object-oriented packages presented here
solve this problem (and not much else).
@comment TODO ?list properties of oo systems.. oo vs o-based?

@c ------------------------------------------------------------------------
@node Object-Oriented Terminology, Objects, Why object-oriented programming?, Object-oriented Forth
@subsection Object-Oriented Terminology
@cindex object-oriented terminology
@cindex terminology for object-oriented programming

This section is mainly for reference, so you don't have to understand
all of it right away.  The terminology is mainly Smalltalk-inspired.  In
short:

@table @emph
@cindex class
@item class
a data structure definition with some extras.

@cindex object
@item object
an instance of the data structure described by the class definition.

@cindex instance variables
@item instance variables
fields of the data structure.

@cindex selector
@cindex method selector
@cindex virtual function
@item selector
(or @emph{method selector}) a word (e.g.,
@code{draw}) that performs an operation on a variety of data
structures (classes). A selector describes @emph{what} operation to
perform. In C++ terminology: a (pure) virtual function.

@cindex method
@item method
the concrete definition that performs the operation
described by the selector for a specific class. A method specifies
@emph{how} the operation is performed for a specific class.

@cindex selector invocation
@cindex message send
@cindex invoking a selector
@item selector invocation
a call of a selector. One argument of the call (the TOS (top-of-stack))
is used for determining which method is used. In Smalltalk terminology:
a message (consisting of the selector and the other arguments) is sent
to the object.

@cindex receiving object
@item receiving object
the object used for determining the method executed by a selector
invocation. In the @file{objects.fs} model, it is the object that is on
the TOS when the selector is invoked. (@emph{Receiving} comes from
the Smalltalk @emph{message} terminology.)

@cindex child class
@cindex parent class
@cindex inheritance
@item child class
a class that has (@emph{inherits}) all properties (instance variables,
selectors, methods) from a @emph{parent class}. In Smalltalk
terminology: The subclass inherits from the superclass. In C++
terminology: The derived class inherits from the base class.

@end table

@c If you wonder about the message sending terminology, it comes from
@c a time when each object had it's own task and objects communicated via
@c message passing; eventually the Smalltalk developers realized that
@c they can do most things through simple (indirect) calls. They kept the
@c terminology.

@c --------------------------------------------------------------
@node Objects, OOF, Object-Oriented Terminology, Object-oriented Forth
@subsection The @file{objects.fs} model
@cindex objects
@cindex object-oriented programming

@cindex @file{objects.fs}
@cindex @file{oof.fs}

This section describes the @file{objects.fs} package. This material also
has been published in M. Anton Ertl,
@cite{@uref{https://www.complang.tuwien.ac.at/forth/objects/objects.html,
Yet Another Forth Objects Package}}, Forth Dimensions 19(2), pages
37--43.
@c McKewan's and Zsoter's packages

This section assumes that you have read @ref{Structures}.

The techniques on which this model is based have been used to implement
the parser generator, Gray, and have also been used in Gforth for
implementing the various flavours of word lists (hashed or not,
case-sensitive or not, special-purpose word lists for locals etc.).


@menu
* Properties of the Objects model::
* Basic Objects Usage::
* The Objects base class::
* Creating objects::
* Object-Oriented Programming Style::
* Class Binding::
* Method conveniences::
* Classes and Scoping::
* Dividing classes::
* Object Interfaces::
* Objects Implementation::
* Objects Glossary::
@end menu

Marcel Hendrix provided helpful comments on this section.

@node Properties of the Objects model, Basic Objects Usage, Objects, Objects
@subsubsection Properties of the @file{objects.fs} model
@cindex @file{objects.fs} properties

@itemize @bullet
@item
It is straightforward to pass objects on the stack. Passing
selectors on the stack is a little less convenient, but possible.

@item
Objects are just data structures in memory, and are referenced by their
address. You can create words for objects with normal defining words
like @code{constant}. Likewise, there is no difference between instance
variables that contain objects and those that contain other data.

@item
Late binding is efficient and easy to use.

@item
It avoids parsing, and thus avoids problems with state-smartness
and reduced extensibility; for convenience there are a few parsing
words, but they have non-parsing counterparts. There are also a few
defining words that parse. This is hard to avoid, because all standard
defining words parse (except @code{:noname}); however, such
words are not as bad as many other parsing words, because they are not
state-smart.

@item
It does not try to incorporate everything. It does a few things and does
them well (IMO). In particular, this model was not designed to support
information hiding (although it has features that may help); you can use
a separate package for achieving this.

@item
It is layered; you don't have to learn and use all features to use this
model. Only a few features are necessary (@pxref{Basic Objects Usage},
@pxref{The Objects base class}, @pxref{Creating objects}.), the others
are optional and independent of each other.

@item
An implementation in Standard Forth is available.

@end itemize


@node Basic Objects Usage, The Objects base class, Properties of the Objects model, Objects
@subsubsection Basic @file{objects.fs} Usage
@cindex basic objects usage
@cindex objects, basic usage

You can define a class for graphical objects like this:

@cindex @code{class} usage
@cindex @code{end-class} usage
@cindex @code{selector} usage
@example
object class \ "object" is the parent class
  selector draw ( x y graphical -- )
end-class graphical
@end example

This code defines a class @code{graphical} with an
operation @code{draw}.  We can perform the operation
@code{draw} on any @code{graphical} object, e.g.:

@example
100 100 t-rex draw
@end example

@noindent
where @code{t-rex} is a word (say, a constant) that produces a
graphical object.

@comment TODO add a 2nd operation eg perimeter.. and use for
@comment a concrete example

@cindex abstract class
How do we create a graphical object? With the present definitions,
we cannot create a useful graphical object. The class
@code{graphical} describes graphical objects in general, but not
any concrete graphical object type (C++ users would call it an
@emph{abstract class}); e.g., there is no method for the selector
@code{draw} in the class @code{graphical}.

For concrete graphical objects, we define child classes of the
class @code{graphical}, e.g.:

@cindex @code{overrides} usage
@cindex @code{field} usage in class definition
@example
graphical class \ "graphical" is the parent class
  cell% field circle-radius

:noname ( x y circle -- )
  circle-radius @@ draw-circle ;
overrides draw

:noname ( n-radius circle -- )
  circle-radius ! ;
overrides construct

end-class circle
@end example

Here we define a class @code{circle} as a child of @code{graphical},
with field @code{circle-radius} (which behaves just like a field
(@pxref{Structures}); it defines (using @code{overrides}) new methods
for the selectors @code{draw} and @code{construct} (@code{construct} is
defined in @code{object}, the parent class of @code{graphical}).

Now we can create a circle on the heap (i.e.,
@code{allocate}d memory) with:

@cindex @code{heap-new} usage
@example
50 circle heap-new constant my-circle
@end example

@noindent
@code{heap-new} invokes @code{construct}, thus
initializing the field @code{circle-radius} with 50. We can draw
this new circle at (100,100) with:

@example
100 100 my-circle draw
@end example

@cindex selector invocation, restrictions
@cindex class definition, restrictions
Note: You can only invoke a selector if the object on the TOS
(the receiving object) belongs to the class where the selector was
defined or one of its descendents; e.g., you can invoke
@code{draw} only for objects belonging to @code{graphical}
or its descendents (e.g., @code{circle}).  Immediately before
@code{end-class}, the search order has to be the same as
immediately after @code{class}.

@node The Objects base class, Creating objects, Basic Objects Usage, Objects
@subsubsection The @file{object.fs} base class
@cindex @code{object} class

When you define a class, you have to specify a parent class.  So how do
you start defining classes? There is one class available from the start:
@code{object}. It is ancestor for all classes and so is the
only class that has no parent. It has two selectors: @code{construct}
and @code{print}.

@node Creating objects, Object-Oriented Programming Style, The Objects base class, Objects
@subsubsection Creating objects
@cindex creating objects
@cindex object creation
@cindex object allocation options

@cindex @code{heap-new} discussion
@cindex @code{dict-new} discussion
@cindex @code{construct} discussion
You can create and initialize an object of a class on the heap with
@code{heap-new} ( ... class -- object ) and in the dictionary
(allocation with @code{allot}) with @code{dict-new} (
... class -- object ). Both words invoke @code{construct}, which
consumes the stack items indicated by "..." above.

@cindex @code{init-object} discussion
@cindex @code{class-inst-size} discussion
If you want to allocate memory for an object yourself, you can get its
alignment and size with @code{class-inst-size 2@@} ( class --
align size ). Once you have memory for an object, you can initialize
it with @code{init-object} ( ... class object -- );
@code{construct} does only a part of the necessary work.

@node Object-Oriented Programming Style, Class Binding, Creating objects, Objects
@subsubsection Object-Oriented Programming Style
@cindex object-oriented programming style
@cindex programming style, object-oriented

This section is not exhaustive.

@cindex stack effects of selectors
@cindex selectors and stack effects
In general, it is a good idea to ensure that all methods for the
same selector have the same stack effect: when you invoke a selector,
you often have no idea which method will be invoked, so, unless all
methods have the same stack effect, you will not know the stack effect
of the selector invocation.

One exception to this rule is methods for the selector
@code{construct}. We know which method is invoked, because we
specify the class to be constructed at the same place. Actually, I
defined @code{construct} as a selector only to give the users a
convenient way to specify initialization. The way it is used, a
mechanism different from selector invocation would be more natural
(but probably would take more code and more space to explain).

@node Class Binding, Method conveniences, Object-Oriented Programming Style, Objects
@subsubsection Class Binding
@cindex class binding
@cindex early binding

@cindex late binding
Normal selector invocations determine the method at run-time depending
on the class of the receiving object. This run-time selection is called
@i{late binding}.

Sometimes it's preferable to invoke a different method. For example,
you might want to use the simple method for @code{print}ing
@code{object}s instead of the possibly long-winded @code{print} method
of the receiver class. You can achieve this by replacing the invocation
of @code{print} with:

@cindex @code{[bind]} usage
@example
[bind] object print
@end example

@noindent
in compiled code or:

@cindex @code{bind} usage
@example
bind object print
@end example

@cindex class binding, alternative to
@noindent
in interpreted code. Alternatively, you can define the method with a
name (e.g., @code{print-object}), and then invoke it through the
name. Class binding is just a (often more convenient) way to achieve
the same effect; it avoids name clutter and allows you to invoke
methods directly without naming them first.

@cindex superclass binding
@cindex parent class binding
A frequent use of class binding is this: When we define a method
for a selector, we often want the method to do what the selector does
in the parent class, and a little more. There is a special word for
this purpose: @code{[parent]}; @code{[parent]
@emph{selector}} is equivalent to @code{[bind] @emph{parent
selector}}, where @code{@emph{parent}} is the parent
class of the current class. E.g., a method definition might look like:

@cindex @code{[parent]} usage
@example
:noname
  dup [parent] foo \ do parent's foo on the receiving object
  ... \ do some more
; overrides foo
@end example

@cindex class binding as optimization
In @cite{Object-oriented programming in ANS Forth} (Forth Dimensions,
March 1997), Andrew McKewan presents class binding as an optimization
technique. I recommend not using it for this purpose unless you are in
an emergency. Late binding is pretty fast with this model anyway, so the
benefit of using class binding is small; the cost of using class binding
where it is not appropriate is reduced maintainability.

While we are at programming style questions: You should bind
selectors only to ancestor classes of the receiving object. E.g., say,
you know that the receiving object is of class @code{foo} or its
descendents; then you should bind only to @code{foo} and its
ancestors.

@node Method conveniences, Classes and Scoping, Class Binding, Objects
@subsubsection Method conveniences
@cindex method conveniences

In a method you usually access the receiving object pretty often.  If
you define the method as a plain colon definition (e.g., with
@code{:noname}), you may have to do a lot of stack
gymnastics. To avoid this, you can define the method with @code{m:
... ;m}. E.g., you could define the method for
@code{draw}ing a @code{circle} with

@cindex @code{this} usage
@cindex @code{m:} usage
@cindex @code{;m} usage
@example
m: ( x y circle -- )
  ( x y ) this circle-radius @@ draw-circle ;m
@end example

@cindex @code{exit} in @code{m: ... ;m}
@cindex @code{exitm} discussion
@cindex @code{catch} in @code{m: ... ;m}
When this method is executed, the receiver object is removed from the
stack; you can access it with @code{this} (admittedly, in this
example the use of @code{m: ... ;m} offers no advantage). Note
that I specify the stack effect for the whole method (i.e. including
the receiver object), not just for the code between @code{m:}
and @code{;m}. You cannot use @code{exit} in
@code{m:...;m}; instead, use
@code{exitm}.@footnote{Moreover, for any word that calls
@code{catch} and was defined before loading
@code{objects.fs}, you have to redefine it like I redefined
@code{catch}: @code{: catch this >r catch r> to-this ;}}

@cindex @code{inst-var} usage
You will frequently use sequences of the form @code{this
@emph{field}} (in the example above: @code{this
circle-radius}). If you use the field only in this way, you can
define it with @code{inst-var} and eliminate the
@code{this} before the field name. E.g., the @code{circle}
class above could also be defined with:

@example
graphical class
  cell% inst-var radius

m: ( x y circle -- )
  radius @@ draw-circle ;m
overrides draw

m: ( n-radius circle -- )
  radius ! ;m
overrides construct

end-class circle
@end example

@code{radius} can only be used in @code{circle} and its
descendent classes and inside @code{m:...;m}.

@cindex @code{inst-value} usage
You can also define fields with @code{inst-value}, which is
to @code{inst-var} what @code{value} is to
@code{variable}.  You can change the value of such a field with
@code{[to-inst]}.  E.g., we could also define the class
@code{circle} like this:

@example
graphical class
  inst-value radius

m: ( x y circle -- )
  radius draw-circle ;m
overrides draw

m: ( n-radius circle -- )
  [to-inst] radius ;m
overrides construct

end-class circle
@end example

@c !! :m is easy to confuse with m:.  Another name would be better.

@c Finally, you can define named methods with @code{:m}.  One use of this
@c feature is the definition of words that occur only in one class and are
@c not intended to be overridden, but which still need method context
@c (e.g., for accessing @code{inst-var}s).  Another use is for methods that
@c would be bound frequently, if defined anonymously.


@node Classes and Scoping, Dividing classes, Method conveniences, Objects
@subsubsection Classes and Scoping
@cindex classes and scoping
@cindex scoping and classes

Inheritance is frequent, unlike structure extension. This exacerbates
the problem with the field name convention (@pxref{Structure Naming
Convention}): One always has to remember in which class the field was
originally defined; changing a part of the class structure would require
changes for renaming in otherwise unaffected code.

@cindex @code{inst-var} visibility
@cindex @code{inst-value} visibility
To solve this problem, I added a scoping mechanism (which was not in my
original charter): A field defined with @code{inst-var} (or
@code{inst-value}) is visible only in the class where it is defined and in
the descendent classes of this class.  Using such fields only makes
sense in @code{m:}-defined methods in these classes anyway.

This scoping mechanism allows us to use the unadorned field name,
because name clashes with unrelated words become much less likely.

@cindex @code{protected} discussion
@cindex @code{private} discussion
Once we have this mechanism, we can also use it for controlling the
visibility of other words: All words defined after
@code{protected} are visible only in the current class and its
descendents. @code{public} restores the compilation
(i.e. @code{current}) word list that was in effect before. If you
have several @code{protected}s without an intervening
@code{public} or @code{set-current}, @code{public}
will restore the compilation word list in effect before the first of
these @code{protected}s.

@node Dividing classes, Object Interfaces, Classes and Scoping, Objects
@subsubsection Dividing classes
@cindex Dividing classes
@cindex @code{methods}...@code{end-methods}

You may want to do the definition of methods separate from the
definition of the class, its selectors, fields, and instance variables,
i.e., separate the implementation from the definition.  You can do this
in the following way:

@example
graphical class
  inst-value radius
end-class circle

... \ do some other stuff

circle methods \ now we are ready

m: ( x y circle -- )
  radius draw-circle ;m
overrides draw

m: ( n-radius circle -- )
  [to-inst] radius ;m
overrides construct

end-methods
@end example

You can use several @code{methods}...@code{end-methods} sections.  The
only things you can do to the class in these sections are: defining
methods, and overriding the class's selectors.  You must not define new
selectors or fields.

Note that you often have to override a selector before using it.  In
particular, you usually have to override @code{construct} with a new
method before you can invoke @code{heap-new} and friends.  E.g., you
must not create a circle before the @code{overrides construct} sequence
in the example above.

@node Object Interfaces, Objects Implementation, Dividing classes, Objects
@subsubsection Object Interfaces
@cindex object interfaces
@cindex interfaces for objects

In this model you can only call selectors defined in the class of the
receiving objects or in one of its ancestors. If you call a selector
with a receiving object that is not in one of these classes, the
result is undefined; if you are lucky, the program crashes
immediately.

@cindex selectors common to hardly-related classes
Now consider the case when you want to have a selector (or several)
available in two classes: You would have to add the selector to a
common ancestor class, in the worst case to @code{object}. You
may not want to do this, e.g., because someone else is responsible for
this ancestor class.

The solution for this problem is interfaces. An interface is a
collection of selectors. If a class implements an interface, the
selectors become available to the class and its descendents. A class
can implement an unlimited number of interfaces. For the problem
discussed above, we would define an interface for the selector(s), and
both classes would implement the interface.

As an example, consider an interface @code{storage} for
writing objects to disk and getting them back, and a class
@code{foo} that implements it. The code would look like this:

@cindex @code{interface} usage
@cindex @code{end-interface} usage
@cindex @code{implementation} usage
@example
interface
  selector write ( file object -- )
  selector read1 ( file object -- )
end-interface storage

bar class
  storage implementation

... overrides write
... overrides read1
...
end-class foo
@end example

@noindent
(I would add a word @code{read} @i{( file -- object )} that uses
@code{read1} internally, but that's beyond the point illustrated
here.)

Note that you cannot use @code{protected} in an interface; and
of course you cannot define fields.

In the Neon model, all selectors are available for all classes;
therefore it does not need interfaces. The price you pay in this model
is slower late binding, and therefore, added complexity to avoid late
binding.

@node Objects Implementation, Objects Glossary, Object Interfaces, Objects
@subsubsection @file{objects.fs} Implementation
@cindex @file{objects.fs} implementation

@cindex @code{object-map} discussion
An object is a piece of memory, like one of the data structures
described with @code{struct...end-struct}. It has a field
@code{object-map} that points to the method map for the object's
class.

@cindex method map
@cindex virtual function table
The @emph{method map}@footnote{This is Self terminology; in C++
terminology: virtual function table.} is an array that contains the
execution tokens (@i{xt}s) of the methods for the object's class. Each
selector contains an offset into a method map.

@cindex @code{selector} implementation, class
@code{selector} is a defining word that uses
@code{CREATE} and @code{DOES>}. The body of the
selector contains the offset; the @code{DOES>} action for a
class selector is, basically:

@example
( object addr ) @@ over object-map @@ + @@ execute
@end example

Since @code{object-map} is the first field of the object, it
does not generate any code. As you can see, calling a selector has a
small, constant cost.

@cindex @code{current-interface} discussion
@cindex class implementation and representation
A class is basically a @code{struct} combined with a method
map. During the class definition the alignment and size of the class
are passed on the stack, just as with @code{struct}s, so
@code{field} can also be used for defining class
fields. However, passing more items on the stack would be
inconvenient, so @code{class} builds a data structure in memory,
which is accessed through the variable
@code{current-interface}. After its definition is complete, the
class is represented on the stack by a pointer (e.g., as parameter for
a child class definition).

A new class starts off with the alignment and size of its parent,
and a copy of the parent's method map. Defining new fields extends the
size and alignment; likewise, defining new selectors extends the
method map. @code{overrides} just stores a new @i{xt} in the method
map at the offset given by the selector.

@cindex class binding, implementation
Class binding just gets the @i{xt} at the offset given by the selector
from the class's method map and @code{compile,}s (in the case of
@code{[bind]}) it.

@cindex @code{this} implementation
@cindex @code{catch} and @code{this}
@cindex @code{this} and @code{catch}
I implemented @code{this} as a @code{value}. At the
start of an @code{m:...;m} method the old @code{this} is
stored to the return stack and restored at the end; and the object on
the TOS is stored @code{TO this}. This technique has one
disadvantage: If the user does not leave the method via
@code{;m}, but via @code{throw} or @code{exit},
@code{this} is not restored (and @code{exit} may
crash). To deal with the @code{throw} problem, I have redefined
@code{catch} to save and restore @code{this}; the same
should be done with any word that can catch an exception. As for
@code{exit}, I simply forbid it (as a replacement, there is
@code{exitm}).

@cindex @code{inst-var} implementation
@code{inst-var} is just the same as @code{field}, with
a different @code{DOES>} action:
@example
@@ this +
@end example
Similar for @code{inst-value}.

@cindex class scoping implementation
Each class also has a word list that contains the words defined with
@code{inst-var} and @code{inst-value}, and its protected
words. It also has a pointer to its parent. @code{class} pushes
the word lists of the class and all its ancestors onto the search order stack,
and @code{end-class} drops them.

@cindex interface implementation
An interface is like a class without fields, parent and protected
words; i.e., it just has a method map. If a class implements an
interface, its method map contains a pointer to the method map of the
interface. The positive offsets in the map are reserved for class
methods, therefore interface map pointers have negative
offsets. Interfaces have offsets that are unique throughout the
system, unlike class selectors, whose offsets are only unique for the
classes where the selector is available (invokable).

This structure means that interface selectors have to perform one
indirection more than class selectors to find their method. Their body
contains the interface map pointer offset in the class method map, and
the method offset in the interface method map. The
@code{does>} action for an interface selector is, basically:

@example
( object selector-body )
2dup selector-interface @@ ( object selector-body object interface-offset )
swap object-map @@ + @@ ( object selector-body map )
swap selector-offset @@ + @@ execute
@end example

where @code{object-map} and @code{selector-offset} are
first fields and generate no code.

As a concrete example, consider the following code:

@example
interface
  selector if1sel1
  selector if1sel2
end-interface if1

object class
  if1 implementation
  selector cl1sel1
  cell% inst-var cl1iv1

' m1 overrides construct
' m2 overrides if1sel1
' m3 overrides if1sel2
' m4 overrides cl1sel2
end-class cl1

create obj1 object dict-new drop
create obj2 cl1    dict-new drop
@end example

The data structure created by this code (including the data structure
for @code{object}) is shown in the
@uref{objects-implementation.eps,figure}, assuming a cell size of 4.
@comment TODO add this diagram..

@node Objects Glossary,  , Objects Implementation, Objects
@subsubsection @file{objects.fs} Glossary
@cindex @file{objects.fs} Glossary


doc---objects-bind
doc---objects-<bind>
doc---objects-bind'
doc---objects-[bind]
doc---objects-class
doc---objects-class->map
doc---objects-class-inst-size
doc---objects-class-override!
doc---objects-class-previous
doc---objects-class>order
doc---objects-construct
doc---objects-current'
doc---objects-[current]
doc---objects-current-interface
doc---objects-dict-new
doc---objects-end-class
doc---objects-end-class-noname
doc---objects-end-interface
doc---objects-end-interface-noname
doc---objects-end-methods
doc---objects-exitm
doc---objects-heap-new
doc---objects-implementation
doc---objects-init-object
doc---objects-inst-value
doc---objects-inst-var
doc---objects-interface
doc---objects-m:
doc---objects-:m
doc---objects-;m
doc---objects-method
doc---objects-methods
doc---objects-object
doc---objects-overrides
doc---objects-[parent]
doc---objects-print
doc---objects-protected
doc---objects-public
doc---objects-selector
doc---objects-this
doc---objects-<to-inst>
doc---objects-[to-inst]
doc---objects-to-this
doc---objects-xt-new


@c -------------------------------------------------------------
@node OOF, Mini-OOF, Objects, Object-oriented Forth
@subsection The @file{oof.fs} model
@cindex oof
@cindex object-oriented programming

@cindex @file{objects.fs}
@cindex @file{oof.fs}

This section describes the @file{oof.fs} package.

The package described in this section has been used in bigFORTH since 1991, and
used for two large applications: a chromatographic system used to
create new medicaments, and a graphic user interface library (MINOS).

You can find a description (in German) of @file{oof.fs} in @cite{Object
oriented bigFORTH} by Bernd Paysan, published in @cite{Vierte Dimension}
10(2), 1994.

@menu
* Properties of the OOF model::
* Basic OOF Usage::
* The OOF base class::
* Class Declaration::
* Class Implementation::
@end menu

@node Properties of the OOF model, Basic OOF Usage, OOF, OOF
@subsubsection Properties of the @file{oof.fs} model
@cindex @file{oof.fs} properties

@itemize @bullet
@item
This model combines object oriented programming with information
hiding. It helps you writing large application, where scoping is
necessary, because it provides class-oriented scoping.

@item
Named objects, object pointers, and object arrays can be created,
selector invocation uses the ``object selector'' syntax. Selector invocation
to objects and/or selectors on the stack is a bit less convenient, but
possible.

@item
Selector invocation and instance variable usage of the active object is
straightforward, since both make use of the active object.

@item
Late binding is efficient and easy to use.

@item
State-smart objects parse selectors. However, extensibility is provided
using a (parsing) selector @code{postpone} and a selector @code{'}.

@item
An implementation in Standard Forth is available.

@end itemize


@node Basic OOF Usage, The OOF base class, Properties of the OOF model, OOF
@subsubsection Basic @file{oof.fs} Usage
@cindex @file{oof.fs} usage

This section uses the same example as for @code{objects} (@pxref{Basic Objects Usage}).

You can define a class for graphical objects like this:

@cindex @code{class} usage
@cindex @code{class;} usage
@cindex @code{method} usage
@example
object class graphical \ "object" is the parent class
  method draw ( x y -- )
class;
@end example

This code defines a class @code{graphical} with an
operation @code{draw}.  We can perform the operation
@code{draw} on any @code{graphical} object, e.g.:

@example
100 100 t-rex draw
@end example

@noindent
where @code{t-rex} is an object or object pointer, created with e.g.
@code{graphical : t-rex}.

@cindex abstract class
How do we create a graphical object? With the present definitions,
we cannot create a useful graphical object. The class
@code{graphical} describes graphical objects in general, but not
any concrete graphical object type (C++ users would call it an
@emph{abstract class}); e.g., there is no method for the selector
@code{draw} in the class @code{graphical}.

For concrete graphical objects, we define child classes of the
class @code{graphical}, e.g.:

@example
graphical class circle \ "graphical" is the parent class
  cell var circle-radius
how:
  : draw ( x y -- )
    circle-radius @@ draw-circle ;

  : init ( n-radius -- )
    circle-radius ! ;
class;
@end example

Here we define a class @code{circle} as a child of @code{graphical},
with a field @code{circle-radius}; it defines new methods for the
selectors @code{draw} and @code{init} (@code{init} is defined in
@code{object}, the parent class of @code{graphical}).

Now we can create a circle in the dictionary with:

@example
50 circle : my-circle
@end example

@noindent
@code{:} invokes @code{init}, thus initializing the field
@code{circle-radius} with 50. We can draw this new circle at (100,100)
with:

@example
100 100 my-circle draw
@end example

@cindex selector invocation, restrictions
@cindex class definition, restrictions
Note: You can only invoke a selector if the receiving object belongs to
the class where the selector was defined or one of its descendents;
e.g., you can invoke @code{draw} only for objects belonging to
@code{graphical} or its descendents (e.g., @code{circle}). The scoping
mechanism will check if you try to invoke a selector that is not
defined in this class hierarchy, so you'll get an error at compilation
time.


@node The OOF base class, Class Declaration, Basic OOF Usage, OOF
@subsubsection The @file{oof.fs} base class
@cindex @file{oof.fs} base class

When you define a class, you have to specify a parent class.  So how do
you start defining classes? There is one class available from the start:
@code{object}. You have to use it as ancestor for all classes. It is the
only class that has no parent. Classes are also objects, except that
they don't have instance variables; class manipulation such as
inheritance or changing definitions of a class is handled through
selectors of the class @code{object}.

@code{object} provides a number of selectors:

@itemize @bullet
@item
@code{class} for subclassing, @code{definitions} to add definitions
later on, and @code{class?} to get type informations (is the class a
subclass of the class passed on the stack?).

doc---oof-object-class
doc---oof-object-definitions
doc---oof-object-class?


@item
@code{init} and @code{dispose} as constructor and destructor of the
object. @code{init} is invocated after the object's memory is allocated,
while @code{dispose} also handles deallocation. Thus if you redefine
@code{dispose}, you have to call the parent's dispose with @code{super
dispose}, too.

doc---oof-object-init
doc---oof-object-dispose


@item
@code{new}, @code{new[]}, @code{:}, @code{ptr}, @code{asptr}, and
@code{[]} to create named and unnamed objects and object arrays or
object pointers.

doc---oof-object-new
doc---oof-object-new[]
doc---oof-object-:
doc---oof-object-ptr
doc---oof-object-asptr
doc---oof-object-[]


@item
@code{::} and @code{super} for explicit scoping. You should use explicit
scoping only for super classes or classes with the same set of instance
variables. Explicitly-scoped selectors use early binding.

doc---oof-object-::
doc---oof-object-super


@item
@code{self} to get the address of the object

doc---oof-object-self


@item
@code{bind}, @code{bound}, @code{link}, and @code{is} to assign object
pointers and instance defers.

doc---oof-object-bind
doc---oof-object-bound
doc---oof-object-link
doc---oof-object-is


@item
@code{'} to obtain selector tokens, @code{send} to invocate selectors
form the stack, and @code{postpone} to generate selector invocation code.

doc---oof-object-'
doc---oof-object-postpone


@item
@code{with} and @code{endwith} to select the active object from the
stack, and enable its scope. Using @code{with} and @code{endwith}
also allows you to create code using selector @code{postpone} without being
trapped by the state-smart objects.

doc---oof-object-with
doc---oof-object-endwith


@end itemize

@node Class Declaration, Class Implementation, The OOF base class, OOF
@subsubsection Class Declaration
@cindex class declaration

@itemize @bullet
@item
Instance variables

doc---oof-var


@item
Object pointers

doc---oof-ptr
doc---oof-asptr


@item
Instance defers

doc---oof-defer


@item
Method selectors

doc---oof-early
doc---oof-method


@item
Class-wide variables

doc---oof-static


@item
End declaration

doc---oof-how:
doc---oof-class;


@end itemize

@c -------------------------------------------------------------
@node Class Implementation,  , Class Declaration, OOF
@subsubsection Class Implementation
@cindex class implementation

@c -------------------------------------------------------------
@node Mini-OOF, Mini-OOF2, OOF, Object-oriented Forth
@subsection The @file{mini-oof.fs} model
@cindex mini-oof

Gforth's third object oriented Forth package is a 12-liner. It uses a
mixture of the @file{objects.fs} and the @file{oof.fs} syntax,
and reduces to the bare minimum of features. This is based on a posting
of Bernd Paysan in comp.lang.forth.

@menu
* Basic Mini-OOF Usage::
* Mini-OOF Example::
* Mini-OOF Implementation::
@end menu

@c -------------------------------------------------------------
@node Basic Mini-OOF Usage, Mini-OOF Example, Mini-OOF, Mini-OOF
@subsubsection Basic @file{mini-oof.fs} Usage
@cindex mini-oof usage

There is a base class (@code{class}, which allocates one cell for the
object pointer) plus seven other words: to define a method, a variable,
a class; to end a class, to resolve binding, to allocate an object and
to compile a class method.
@comment TODO better description of the last one


doc-object
doc-method
doc-var
doc-class
doc-end-class
doc-defines
doc-new
doc-::



@c -------------------------------------------------------------
@node Mini-OOF Example, Mini-OOF Implementation, Basic Mini-OOF Usage, Mini-OOF
@subsubsection Mini-OOF Example
@cindex mini-oof example

A short example shows how to use this package. This example, in slightly
extended form, is supplied as @file{moof-exm.fs}
@comment TODO could flesh this out with some comments from the Forthwrite article

@example
object class
  method init
  method draw
end-class graphical
@end example

This code defines a class @code{graphical} with an
operation @code{draw}.  We can perform the operation
@code{draw} on any @code{graphical} object, e.g.:

@example
100 100 t-rex draw
@end example

where @code{t-rex} is an object or object pointer, created with e.g.
@code{graphical new Constant t-rex}.

For concrete graphical objects, we define child classes of the
class @code{graphical}, e.g.:

@example
graphical class
  cell var circle-radius
end-class circle \ "graphical" is the parent class

:noname ( x y -- )
  circle-radius @@ draw-circle ; circle defines draw
:noname ( r -- )
  circle-radius ! ; circle defines init
@end example

There is no implicit init method, so we have to define one. The creation
code of the object now has to call init explicitely.

@example
circle new Constant my-circle
50 my-circle init
@end example

It is also possible to add a function to create named objects with
automatic call of @code{init}, given that all objects have @code{init}
on the same place:

@example
: new: ( .. o "name" -- )
    new dup Constant init ;
80 circle new: large-circle
@end example

We can draw this new circle at (100,100) with:

@example
100 100 my-circle draw
@end example

@node Mini-OOF Implementation,  , Mini-OOF Example, Mini-OOF
@subsubsection @file{mini-oof.fs} Implementation

Object-oriented systems with late binding typically use a
``vtable''-approach: the first variable in each object is a pointer to a
table, which contains the methods as function pointers. The vtable
may also contain other information.

So first, let's declare selectors:

@example
: method ( m v "name" -- m' v ) Create  over , swap cell+ swap
  DOES> ( ... o -- ... ) @@ over @@ + @@ execute ;
@end example

During selector declaration, the number of selectors and instance
variables is on the stack (in address units). @code{method} creates one
selector and increments the selector number. To execute a selector, it
takes the object, fetches the vtable pointer, adds the offset, and
executes the method @i{xt} stored there. Each selector takes the object
it is invoked with as top of stack parameter; it passes the parameters
(including the object) unchanged to the appropriate method which should
consume that object.

Now, we also have to declare instance variables

@example
: var ( m v size "name" -- m v' ) Create  over , +
  DOES> ( o -- addr ) @@ + ;
@end example

As before, a word is created with the current offset. Instance
variables can have different sizes (cells, floats, doubles, chars), so
all we do is take the size and add it to the offset. If your machine
has alignment restrictions, put the proper @code{aligned} or
@code{faligned} before the variable, to adjust the variable
offset. That's why it is on the top of stack.

We need a starting point (the base object) and some syntactic sugar:

@example
Create object  1 cells , 2 cells ,
: class ( class -- class selectors vars ) dup 2@@ ;
@end example

For inheritance, the vtable of the parent object has to be
copied when a new, derived class is declared. This gives all the
methods of the parent class, which can be overridden, though.

@example
: end-class  ( class selectors vars "name" -- )
  Create  here >r , dup , 2 cells ?DO ['] noop , 1 cells +LOOP
  cell+ dup cell+ r> rot @@ 2 cells /string move ;
@end example

The first line creates the vtable, initialized with
@code{noop}s. The second line is the inheritance mechanism, it
copies the xts from the parent vtable.

We still have no way to define new methods, let's do that now:

@example
: defines ( xt class "name" -- ) ' >body @@ + ! ;
@end example

To allocate a new object, we need a word, too:

@example
: new ( class -- o )  here over @@ allot swap over ! ;
@end example

Sometimes derived classes want to access the method of the
parent object. There are two ways to achieve this with Mini-OOF:
first, you could use named words, and second, you could look up the
vtable of the parent object.

@example
: :: ( class "name" -- ) ' >body @@ + @@ compile, ;
@end example


Nothing can be more confusing than a good example, so here is
one. First let's declare a text object (called
@code{button}), that stores text and position:

@example
object class
  cell var text
  cell var len
  cell var x
  cell var y
  method init
  method draw
end-class button
@end example

@noindent
Now, implement the two methods, @code{draw} and @code{init}:

@example
:noname ( o -- )
 >r r@@ x @@ r@@ y @@ at-xy  r@@ text @@ r> len @@ type ;
 button defines draw
:noname ( addr u o -- )
 >r 0 r@@ x ! 0 r@@ y ! r@@ len ! r> text ! ;
 button defines init
@end example

@noindent
To demonstrate inheritance, we define a class @code{bold-button}, with no
new data and no new selectors:

@example
button class
end-class bold-button

: bold   27 emit ." [1m" ;
: normal 27 emit ." [0m" ;
@end example

@noindent
The class @code{bold-button} has a different draw method to
@code{button}, but the new method is defined in terms of the draw method
for @code{button}:

@example
:noname bold [ button :: draw ] normal ; bold-button defines draw
@end example

@noindent
Finally, create two objects and apply selectors:

@example
button new Constant foo
s" thin foo" foo init
page
foo draw
bold-button new Constant bar
s" fat bar" bar init
1 bar y !
bar draw
@end example

@node Mini-OOF2, Comparison with other object models, Mini-OOF, Object-oriented Forth
@subsection Mini-OOF2

Mini-OOF2 is very similar to Mini-OOF in many respects, but differs
significantly in a few aspects.  In particular, Mini-OOF2 has a current
object variable, and uses the primitives @code{>o} and @code{o>} to
manipulate that object stack.  All method invocations and instance
variable accesses refer to the current object.

doc->o
doc-o>

To ease passing an object pointer to method invocation or instance
variable accesses, the additional recognizer @code{rec-moof2} is
activated.

doc-rec-moof2

To assign methods to selectors, use @var{xt class} @code{is}
@var{selector}, so no @code{defines} necessary.  For early binding of
methods, @code{[} @var{class} @code{] defers} @var{selector} is used, no
need for @code{::}.

@node Comparison with other object models,  , Mini-OOF2, Object-oriented Forth
@subsection Comparison with other object models
@cindex comparison of object models
@cindex object models, comparison

Many object-oriented Forth extensions have been proposed (@cite{A survey
of object-oriented Forths} (SIGPLAN Notices, April 1996) by Bradford
J. Rodriguez and W. F. S. Poehlman lists 17). This section discusses the
relation of the object models described here to two well-known and two
closely-related (by the use of method maps) models.  Andras Zsoter
helped us with this section.

@cindex Neon model
The most popular model currently seems to be the Neon model (see
@cite{Object-oriented programming in ANS Forth} (Forth Dimensions, March
1997) by Andrew McKewan) but this model has a number of limitations
@footnote{A longer version of this critique can be
found in @cite{On Standardizing Object-Oriented Forth Extensions} (Forth
Dimensions, May 1997) by Anton Ertl.}:

@itemize @bullet
@item
It uses a @code{@emph{selector object}} syntax, which makes it unnatural
to pass objects on the stack.

@item
It requires that the selector parses the input stream (at
compile time); this leads to reduced extensibility and to bugs that are
hard to find.

@item
It allows using every selector on every object; this eliminates the
need for interfaces, but makes it harder to create efficient
implementations.
@end itemize

@cindex Pountain's object-oriented model
Another well-known publication is @cite{Object-Oriented Forth} (Academic
Press, London, 1987) by Dick Pountain. However, it is not really about
object-oriented programming, because it hardly deals with late
binding. Instead, it focuses on features like information hiding and
overloading that are characteristic of modular languages like Ada (83).

@cindex Zsoter's object-oriented model
In @uref{http://www.forth.org/oopf.html, Does late binding have to be
slow?} (Forth Dimensions 18(1) 1996, pages 31-35) Andras Zsoter
describes a model that makes heavy use of an active object (like
@code{this} in @file{objects.fs}): The active object is not only used
for accessing all fields, but also specifies the receiving object of
every selector invocation; you have to change the active object
explicitly with @code{@{ ... @}}, whereas in @file{objects.fs} it
changes more or less implicitly at @code{m: ... ;m}. Such a change at
the method entry point is unnecessary with Zsoter's model, because the
receiving object is the active object already. On the other hand, the
explicit change is absolutely necessary in that model, because otherwise
no one could ever change the active object. An Standard Forth implementation
of this model is available through
@uref{http://www.forth.org/oopf.html}.

@cindex @file{oof.fs}, differences to other models
The @file{oof.fs} model combines information hiding and overloading
resolution (by keeping names in various word lists) with object-oriented
programming. It sets the active object implicitly on method entry, but
also allows explicit changing (with @code{>o...o>} or with
@code{with...endwith}). It uses parsing and state-smart objects and
classes for resolving overloading and for early binding: the object or
class parses the selector and determines the method from this. If the
selector is not parsed by an object or class, it performs a call to the
selector for the active object (late binding), like Zsoter's model.
Fields are always accessed through the active object. The big
disadvantage of this model is the parsing and the state-smartness, which
reduces extensibility and increases the opportunities for subtle bugs;
essentially, you are only safe if you never tick or @code{postpone} an
object or class (Bernd disagrees, but I (Anton) am not convinced).

@cindex @file{mini-oof.fs}, differences to other models
The @file{mini-oof.fs} model is quite similar to a very stripped-down
version of the @file{objects.fs} model, but syntactically it is a
mixture of the @file{objects.fs} and @file{oof.fs} models.

@c -------------------------------------------------------------
@node Regular Expressions, Programming Tools, Object-oriented Forth, Words
@section Regular Expressions
@cindex regexps

Regular expressions are pattern matching algorithms for strings found in
many contemporary languages.  You can add regular expression
functionality to Gforth with @code{require regexp.fs}.

The classical implementation for this pattern matching is a backtracking
algorithm, which is also necessary if you want to have features like
backreferencing.  Gforth implements regular expressions by providing a
language to define backtracking programs for pattern matching.  Basic
element is the control structure @code{FORK} @dots{} @code{JOIN}, which
is a forward call within a word, and therefore allows to code a
lightweight try and fail control structure.

doc-FORK
doc-JOIN

You can program any sort of arbitrary checks yourself by computing a
flag and @code{?LEAVE} when the check fails.  Your regular expression
code is enclosed in @code{((} and @code{))}.

doc-((
doc-))

Pattern matching in regular expressions have character sets as elements,
so a number of functions allow you to create and modify character sets
(called @code{charclass}).  All characters here are bytes, so this
doesn't extend to unicode characters.

doc-charclass
doc-+char
doc--char
doc-..char
doc-+chars
doc-+class
doc--class

There are predefined charclasses and tests for them, and generic checks.
If a check fails, the next possible alternative of the regular
expression is tried, or a loop is terminated.

doc-c?
doc--c?
doc-\d
doc-\s
doc-.?
doc--\d
doc--\s
doc-`
doc-`?
doc--`

You can certainly also check for start and end of the string, and for
whole string constants.

doc-\^
doc-\$
doc-str=?
doc-=''

Loops that check for repeated character sets can be greedy or
non-greedy.

doc-{**
doc-**}
doc-{++
doc-++}
doc-{*
doc-*}
doc-{+
doc-+}

Example: Searching for a substring really is a non-greedy match of
anything in front of it.

doc-//

Alternatives are written with

doc-{{
doc-||
doc-}}

You can use up to 9 variables named @code{\1} to @code{\9} to refer to
matched substrings

doc-\(
doc-\)
doc-\0

Certainly, you can also write code to replace patterns you found.

doc-s>>
doc->>
doc-<<
doc-<<"
doc-s//
doc-//s
doc-//o
doc-//g

Examples can be found in @code{test/regexp-test.fs}.

@c -------------------------------------------------------------
@node Programming Tools, Multitasker, Regular Expressions, Words
@section Programming Tools
@cindex programming tools

@c !! move this and assembler down below OO stuff.

@menu
* Locating source code definitions::  and editing
* Locating uses of a word::     @code{where}.
* Locating exception source::   Examining backtraces.
* Examining compiled code::     @code{see} and friends.
* Examining data::              Stacks and Memory.
* Forgetting words::            Usually before reloading.
* Debugging::                   By inserting @code{~~} tracers.
* Assertions::                  Making your programs self-checking.
* Singlestep Debugger::         Executing your program word by word.
* Code Coverage::               Measure execution frequency.
@end menu

@node Locating source code definitions, Locating uses of a word, Programming Tools, Programming Tools
@subsection Locating source code definitions
@cindex source code of a word
@cindex @code{view} (called @code{locate} in Gforth)
@cindex integrated development environment
@cindex IDE (integrated development environment)

Many programming systems are organized as an integrated development
environment (IDE) where the editor is the hub of the system, and
allows building and running programs.  If you want that, Gforth has
it, too (@pxref{Emacs and Gforth}).

However, several Forth systems have a different kind of IDE: The Forth
command line is the hub of the environment; you can view the source
from there in various ways, and call an editor if needed.

Gforth also implements such an IDE.  It mostly follows the conventions
of SwiftForth where they exist, but implements features beyond them.

An advantage of this approach is that it allows you to use your
favourite editor: set the environment variable @code{EDITOR} to your
favourite editor, and the editing commands will call that editor;
Gforth invokes some GUI editors in the background (so you do not need
to finish editing to continue with your Forth session), terminal
editors in the foreground (default for editors not known to Gforth is
foreground).  If you have not set @code{EDITOR}, the default editor is
@file{vi}.

doc-locate

The @emph{current location} is set by a number of other words in
addition to @code{locate}.  Also, when an error happens while loading
a file, the location of the error becomes the current location.

A number of words work with the current location:

doc-l
doc-n
doc-b
doc-g

You can control how many lines @code{l}, @code{n} and @code{b} show by
changing the values:

doc-before-locate
doc-after-locate

Finally, you can directly go to the source code of a word in the
editor with

doc-edit

You can see the definitions of similarly-named words with

doc-browse


@node Locating uses of a word, Locating exception source, Locating source code definitions, Programming Tools
@subsection Locating uses of a word
@cindex Uses of a word

doc-where
doc-ww
doc-nw
doc-bw
doc-gg
doc-ll
doc-whereg
doc-short-where
doc-expand-where
doc-prepend-where

The data we have on word usage also allows us to show which words have
no uses:

doc-unused-words

@node Locating exception source, Examining compiled code, Locating uses of a word, Programming Tools
@subsection Locating exception source
@cindex exception source code
@cindex source code for exception
@cindex backtrace examination

doc-tt
doc-nt
doc-bt

@node Examining compiled code, Examining data, Locating exception source, Programming Tools
@subsection Examining compiled code
@cindex compiled code examination

And finally, @code{see} and friends show compiled code.  Some of the
things in the source code are not present in the compiled code (e.g.,
formatting and comments), but this is useful to see what threaded code
or native code is produced by macros and Gforth's optimization
features.

doc-see
doc-xt-see
doc-simple-see
doc-xt-simple-see
doc-simple-see-range
doc-see-code
doc-xt-see-code
doc-see-code-range

As an example, consider:

@example
: foo x f@@ fsin drop over ;
@end example

This is not particularly useful, but it demonstrates the various code
generation differences.  Compiling this on @code{gforth-fast} on AMD64
and then using @code{see-code foo} outputs:

@example
$7FD0CEE8C510 lit f@@     1->1 
$7FD0CEE8C518 x
$7FD0CEE8C520 f@@
7FD0CEB51697:   movsd   [r12],xmm15
7FD0CEB5169D:   mov     rax,$00[r13]
7FD0CEB516A1:   sub     r12,$08
7FD0CEB516A5:   add     r13,$18
7FD0CEB516A9:   movsd   xmm15,[rax]
7FD0CEB516AE:   mov     rcx,-$08[r13]
7FD0CEB516B2:   jmp     ecx
$7FD0CEE8C528 fsin
$7FD0CEE8C530 drop    1->0 
7FD0CEB516B4:   add     r13,$08
$7FD0CEE8C538 over    0->1 
7FD0CEB516B8:   mov     r8,$10[r15]
7FD0CEB516BC:   add     r13,$08
$7FD0CEE8C540 ;s    1->1 
7FD0CEB516C0:   mov     r10,[rbx]
7FD0CEB516C3:   add     rbx,$08
7FD0CEB516C7:   lea     r13,$08[r10]
7FD0CEB516CB:   mov     rcx,-$08[r13]
7FD0CEB516CF:   jmp     ecx
@end example

First, you see a threaded-code cell for a static superinstruction with
the components @code{lit} and @code{f@@}, starting and ending with one
data stack item in a register (@code{1->1}); this is followed by the
cell for the argument @code{x} of @code{lit}, and the cell for the
@code{f@@} component of the superinstruction; the latter cell is not
used, but is there for Gforth-internal reasons.

Next, the dynamically generated native code for the superinstruction
@code{lit f@@} is shown; note that this native code is not mixed with
the threaded code in memory, as you can see by comparing the
addresses.

If you want to understand the native code shown here: the
threaded-code instruction pointer is in @code{r13}, the data stack
pointer in @code{r15}; the first data stack register is @code{r8}
(i.e., the top of stack resides there if there is one data stack item
in a register); the return stack pointer is in @code{rbx}, the FP
stack pointer in @code{r12}, and the top of the floating-pont stack in
@code{xmm15}.  Note that the register assignments vary between
engines, so you may see a different register assignment for this code.

The dynamic native code for @code{lit f@@} ends with a dispatch jump
(aka NEXT), because the code for the next word @code{fsin} in the
definition is not dynamically generated.

Next, you see the threaded-code cell for @code{fsin}.  There is no
dynamically-generated native code for this word, and @code{see-code}
does not show the static native code for it (you can look at it with
@code{see fsin}).  Like all words with static native code in
@code{gforth-fast}, the effect on the data stack representation is
@code{1->1} (for @code{gforth}, @code{0->0}), but this is not shown.

Next, you see the threaded-code cell for @code{drop}; the native-code
variant used here starts with one data stack item in registers, and
ends with zero data stack items in registers (@code{1->0}).  This is
followed by the native code for this variant of @code{drop}.  There is
no NEXT here, because the native code falls through to the code for
the next word.

Next, you see the threaded-code cell for @code{over} followed by the
dynamically-generated native code in the @code{0->1} variant.

Finally, you see the threaded and native code for @code{;s} (the
primitive compiled for @code{;} in @code{foo}).  @code{;s} performs
control flow (it returns), so it has to end with a NEXT.

@node Examining data, Forgetting words, Examining compiled code, Programming Tools
@subsection Examining data and code
@cindex examining data and code

The following words inspect the stack non-destructively:

doc-...
doc-.s
doc-f.s
doc-f.s-precision
doc-maxdepth-.s

There is a word @code{.r} but it does @i{not} display the return stack!
It is used for formatted numeric output (@pxref{Simple numeric output}).

The following words work on the stack as a whole, either by determining
the depth or by clearing them:

doc-depth
doc-fdepth
doc-clearstack
doc-fclearstack
doc-clearstacks

The following words inspect memory.

doc-?
doc-dump

@node Forgetting words, Debugging, Examining data, Programming Tools
@subsection Forgetting words
@cindex words, forgetting
@cindex forgeting words

@c  anton: other, maybe better places for this subsection: Defining Words;
@c  Dictionary allocation.  At least a reference should be there.

Forth allows you to forget words (and everything that was alloted in the
dictonary after them) in a LIFO manner.

doc-marker

The most common use of this feature is during progam development: when
you change a source file, forget all the words it defined and load it
again (since you also forget everything defined after the source file
was loaded, you have to reload that, too).  Note that effects like
storing to variables and destroyed system words are not undone when you
forget words.  With a system like Gforth, that is fast enough at
starting up and compiling, I find it more convenient to exit and restart
Gforth, as this gives me a clean slate.

Here's an example of using @code{marker} at the start of a source file
that you are debugging; it ensures that you only ever have one copy of
the file's definitions compiled at any time:

@example
[IFDEF] my-code
    my-code
[ENDIF]

marker my-code
init-included-files

\ .. definitions start here
\ .
\ .
\ end
@end example


@node Debugging, Assertions, Forgetting words, Programming Tools
@subsection Debugging
@cindex debugging

Languages with a slow edit/compile/link/test development loop tend to
require sophisticated tracing/stepping debuggers to facilate debugging.

A much better (faster) way in fast-compiling languages is to add
printing code at well-selected places, let the program run, look at
the output, see where things went wrong, add more printing code, etc.,
until the bug is found.

The simple debugging aids provided in @file{debugs.fs}
are meant to support this style of debugging.

The word @code{~~} prints debugging information (by default the source
location and the stack contents). It is easy to insert. If you use Emacs
it is also easy to remove (@kbd{C-x ~} in the Emacs Forth mode to
query-replace them with nothing). The deferred words
@code{printdebugdata} and @code{.debugline} control the output of
@code{~~}. The default source location output format works well with
Emacs' compilation mode, so you can step through the program at the
source level using @kbd{C-x `} (the advantage over a stepping debugger
is that you can step in any direction and you know where the crash has
happened or where the strange data has occurred).

doc-~~
doc-printdebugdata
doc-.debugline
doc-debug-fid

@cindex filenames in @code{~~} output
@code{~~} (and assertions) will usually print the wrong file name if a
marker is executed in the same file after their occurance.  They will
print @samp{*somewhere*} as file name if a marker is executed in the
same file before their occurance.

doc-once
doc-~~bt
doc-~~1bt
doc-???
doc-WTF??
doc-!!FiXME!!
doc-replace-word
doc-~~Variable
doc-~~Value
doc-+ltrace
doc--ltrace
doc-#loc

@node Assertions, Singlestep Debugger, Debugging, Programming Tools
@subsection Assertions
@cindex assertions

It is a good idea to make your programs self-checking, especially if you
make an assumption that may become invalid during maintenance (for
example, that a certain field of a data structure is never zero). Gforth
supports @dfn{assertions} for this purpose. They are used like this:

@example
assert( @i{flag} )
@end example

The code between @code{assert(} and @code{)} should compute a flag, that
should be true if everything is alright and false otherwise. It should
not change anything else on the stack. The overall stack effect of the
assertion is @code{( -- )}. E.g.

@example
assert( 1 1 + 2 = ) \ what we learn in school
assert( dup 0<> ) \ assert that the top of stack is not zero
assert( false ) \ this code should not be reached
@end example

The need for assertions is different at different times. During
debugging, we want more checking, in production we sometimes care more
for speed. Therefore, assertions can be turned off, i.e., the assertion
becomes a comment. Depending on the importance of an assertion and the
time it takes to check it, you may want to turn off some assertions and
keep others turned on. Gforth provides several levels of assertions for
this purpose:


doc-assert0(
doc-assert1(
doc-assert2(
doc-assert3(
doc-assert(
doc-)


The variable @code{assert-level} specifies the highest assertions that
are turned on. I.e., at the default @code{assert-level} of one,
@code{assert0(} and @code{assert1(} assertions perform checking, while
@code{assert2(} and @code{assert3(} assertions are treated as comments.

The value of @code{assert-level} is evaluated at compile-time, not at
run-time. Therefore you cannot turn assertions on or off at run-time;
you have to set the @code{assert-level} appropriately before compiling a
piece of code. You can compile different pieces of code at different
@code{assert-level}s (e.g., a trusted library at level 1 and
newly-written code at level 3).


doc-assert-level


If an assertion fails, a message compatible with Emacs' compilation mode
is produced and the execution is aborted (currently with @code{ABORT"}.
If there is interest, we will introduce a special throw code. But if you
intend to @code{catch} a specific condition, using @code{throw} is
probably more appropriate than an assertion).

@cindex filenames in assertion output
Assertions (and @code{~~}) will usually print the wrong file name if a
marker is executed in the same file after their occurance.  They will
print @samp{*somewhere*} as file name if a marker is executed in the
same file before their occurance.

Definitions in Standard Forth for these assertion words are provided
in @file{compat/assert.fs}.


@node Singlestep Debugger, Code Coverage, Assertions, Programming Tools
@subsection Singlestep Debugger
@cindex singlestep Debugger
@cindex debugging Singlestep

The singlestep debugger works only with the engine @code{gforth-itc}.

When you create a new word there's often the need to check whether it
behaves correctly or not. You can do this by typing @code{dbg
badword}. A debug session might look like this:

@example
: badword 0 DO i . LOOP ;  ok
2 dbg badword 
: badword  
Scanning code...

Nesting debugger ready!

400D4738  8049BC4 0              -> [ 2 ] 00002 00000 
400D4740  8049F68 DO             -> [ 0 ] 
400D4744  804A0C8 i              -> [ 1 ] 00000 
400D4748 400C5E60 .              -> 0 [ 0 ] 
400D474C  8049D0C LOOP           -> [ 0 ] 
400D4744  804A0C8 i              -> [ 1 ] 00001 
400D4748 400C5E60 .              -> 1 [ 0 ] 
400D474C  8049D0C LOOP           -> [ 0 ] 
400D4758  804B384 ;              ->  ok
@end example

Each line displayed is one step. You always have to hit return to
execute the next word that is displayed. If you don't want to execute
the next word in a whole, you have to type @kbd{n} for @code{nest}. Here is
an overview what keys are available:

@table @i

@item @key{RET}
Next; Execute the next word.

@item n
Nest; Single step through next word.

@item u
Unnest; Stop debugging and execute rest of word. If we got to this word
with nest, continue debugging with the calling word.

@item d
Done; Stop debugging and execute rest.

@item s
Stop; Abort immediately.

@end table

Debugging large application with this mechanism is very difficult, because
you have to nest very deeply into the program before the interesting part
begins. This takes a lot of time. 

To do it more directly put a @code{BREAK:} command into your source code.
When program execution reaches @code{BREAK:} the single step debugger is
invoked and you have all the features described above.

If you have more than one part to debug it is useful to know where the
program has stopped at the moment. You can do this by the 
@code{BREAK" string"} command. This behaves like @code{BREAK:} except that
string is typed out when the ``breakpoint'' is reached.


doc-dbg
doc-break:
doc-break"

@c -------------------------------------------------------------
@node Code Coverage,  , Singlestep Debugger, Programming Tools
@subsection Code Coverage and Execution Frequency
@cindex code coverage
@cindex execution frequency

If you run extensive tests on your code, you often want to figure out
if the tests exercise all parts of the code.  This is called (test)
coverage.  The file @file{coverage.fs} contains tools for measuring
the coverage as well as execution frequency.

Code coverage inserts counting code in every basic block
(straight-line code sequence) loaded after @file{coverage.fs}.  Each
time that code is run, it increments the counter for that basic block.
Later you can show the source file with the counts inserted in these
basic blocks.

doc-nocov[
doc-]nocov
doc-coverage?
doc-cov+
doc-?cov+
doc-.coverage
doc-annotate-cov
doc-cov%
doc-.cover-raw
@c commented out because the included-file index is not a
@c   user-friendly way to pass a file (yet).
@c doc-.cover-file
@c doc-covered?

By default, the counts are shown in colour (using ANSI escape
sequences), but you can use @code{bw-cover} to show them in
parenthesized form without escape sequences.

doc-bw-cover
doc-color-cover

You can save and reload the coverage counters in binary format, to
aggregate coverage counters across several test runs.

doc-save-cov
doc-load-cov

@c -------------------------------------------------------------
@node Multitasker, C Interface, Programming Tools, Words
@section Multitasker
@cindex multitasker

@menu
* Pthreads::                    Native Unix multitasker
* Cilk::                        Implements Cilk-like spawn+sync
@end menu

Gforth offers two multitaskers: a traditional, cooperative round-robin
multitasker, and a pthread-based multitasker which allows to run several
threads concurrently on multi-core machines.  The pthread-based is now
marked as experimental feature, as standardization of Forth
multitaskers will likely change the names of words without changing
their semantics.

@node Pthreads, Cilk, Multitasker, Multitasker
@subsection Pthreads
@cindex pthread

@c You need to include @file{unix/pthread.fs} to use the following words.

Posix threads can run in parallel on several cores, or with
pre-emptive multitasking on onecore.  However, many of the following
words are the same as in the traditional cooperative multi-tasker.

In addition, there are words that allow you to make sure that only one
task at a time changes something, and for communicating between tasks.
These words are necessary for pre-emptive and multi-core
multi-tasking, because the cooperative-multitasking way of performing
transactions between calls to @code{pause} does not work in this
environment.

@menu
* Basic multi-tasking::
* Task-local data::
* Semaphores::
* Hardware operations for multi-tasking::
* Message queues::
@end menu

@node Basic multi-tasking, Task-local data, Pthreads, Pthreads
@subsubsection Basic multi-tasking

Tasks can be created with @code{newtask} or @code{newtask4} with a
given amount of stack space (either all the same or each stack's size
specified).

doc-newtask
doc-task
doc-newtask4

If you don't know which stack sizes to use for the task, you can use
the size(s) of the main task:

doc-stacksize
doc-stacksize4

A task is created in an inactive state.  To let it run, you have to
activate it with one of the following words:

doc-initiate

The following legacy words provide the same functionality as
@code{initiate}, but with a different interface: Like @code{does>}, they
split their containing colon definition in two parts: The part before
@code{activate}/@code{pass} runs in the activating task, and returns to
its caller after activating the task.  The part behind
@code{activate}/@code{pass} is executed in the activated target task.

doc-activate
doc-pass

You can also do creation and activation in one step:

doc-execute-task

Apart from terminating by running to the end, a task can terminate
itself with @code{kill-task}.  Other tasks can terminate it with
@code{kill}.

doc-kill-task
doc-kill

Tasks can also temporarily stop themselves or be stopped:

doc-halt
doc-stop
doc-stop-ns

A task restarts when the timeout is over or when another task wakes it
with:

doc-restart

There is also:

doc-pause

@node Task-local data, Semaphores, Basic multi-tasking, Pthreads
@subsubsection Task-local data
@cindex user variables
@cindex user space
@cindex task-local data

In Forth every task has essentially the same task-local data, called
``user'' area (early Forth systems were multi-user systems and there
often was one user per task).  The @i{task} result of,
e.g. @code{newtask} is the start address of its user area.  Each task
gets the user data defined by the system (e.g., @code{base}).  You can
define additional user data with:

doc-user
doc-auser
doc-uallot
doc-uvalue
doc-udefer

There are also the following words for dealing with user data.

doc-up@
doc-user'
doc-'s

@c doc-udp

@node Semaphores, Hardware operations for multi-tasking, Task-local data, Pthreads
@subsubsection Semaphores

A cooperative multitasker can ensure that there is no other task
interacting between two invocations of @code{pause}.  Pthreads however
are really concurrent tasks (at least on a multi-core CPU), and
therefore, several techniques to avoid conflicts when accessing the same
resources.

Semaphores can only be aquired by one thread, all other threads have to
wait until the semapohre is released.

doc-semaphore
doc-lock
doc-unlock

The other approach to prevent concurrent access is the critical section.
Here, we implement a critical section with a semaphore, so you have to
specify the semaphore which is used for the critical section.  Only
those critical sections which use the same semaphore are mutually
exclusive.

doc-critical-section

@node Hardware operations for multi-tasking, Message queues, Semaphores, Pthreads
@subsubsection Hardware operations for multi-tasking

Atomic hardware operations perform the whole operation, without any
other task seeing an intermediate state.  These operations can be used
to synchronize tasks without using slow OS primitives, but compared to
the non-atomic sequences of operations they tend to be slow.  Atomic
operations only work correctly on aligned addresses, even on hardware
that otherwise does not require alignment.

doc-!@
doc-+!@
doc-?!@

Another hardware operation is the memory barrier.  Unfortunately
modern hardware often can reorder memory operations relative to other
memory operations (as seen by a different core), and the memory
barrier suppresses this reordering for one point in the execution of
the task.

doc-barrier

@node Message queues,  , Hardware operations for multi-tasking, Pthreads
@subsubsection Message queues

Gforth's message queues are a variant of the actor model.

An event is an xt; the sending task tells the receiving task to run an
event, and the receiving task performs one event from its message queue
when it is ready.  Execute-once closures can be used for events that
pass parameters.

doc-send-event

When several tasks send event to one task, the events may arrive in any
order.  In order to receive events, you put one of the following words
in the receiving task:

doc-?events
doc-event-loop

Alternatively, when a task is @code{stop}ped, it is also ready for
receiving event, and receiving an event will wake it up.

@c @node Conditions,  , Message queues, Pthreads
@c @subsubsection Conditions
@c 
@c The pthreads library also provides conditional variables, which allow to
@c wait for a condition.  Using the message queue is generally preferred.
@c 
@c doc-cond
@c doc-pthread_cond_signal
@c doc-pthread_cond_broadcast
@c doc-pthread_cond_wait
@c doc-pthread_cond_timedwait

@node Cilk,  , Pthreads, Multitasker
@subsection Cilk

Gforth's Cilk is a framework for dividing work between multiple tasks
running on several cores, inspired by the programming language of the
same name.  Use @code{require cilk.fs} if you want to use Cilk.

The idea is that you identify subproblems that can be solved in
parallel, and the framework assigns worker tasks to these subproblems.
In particular, you use one of the @code{spawn} words for each subtask.
Eventually you need to wait with @code{cilk-sync} for the subproblems
to be solved.

Currently all the spawning has to happen from one task, and
@code{cilk-sync} waits for all subproblems to complete, so using the
current Gforth Cilk for recursive algorithms is not straightforward.

Do not divide the subproblems too finely, in order to avoid overhead;
how fine is too fine depends on how uniform the run-time for the
subproblems is, but for problems with substantial run-time, having
5*@code{cores} subproblems is probably a good starting point.

doc-cores
doc-cilk-init
doc-spawn
doc-spawn1
doc-spawn2
doc-cilk-sync
doc-cilk-bye

@c ------------------------------------------------------------
@node C Interface, Assembler and Code Words, Multitasker, Words
@section C Interface
@cindex C interface
@cindex foreign language interface
@cindex interface to C functions

Gforth's C interface works by compiling a wrapper library that contains
C functions which take parameters from the Forth stacks and calls the C
functions.  This wrapper library is compiled by the C compiler.
Compilation results are cached, so that Gforth only needs to rerun the C
compilation if the wrapper library has to change.  This build process is
automatic, and done at the end of a interface declaration.  Gforth uses
libtool and GCC for that process.

The C interface is now mostly complete, callbacks have been added, but
for structs, we use Forth2012 structs, which don't have independent
scopes.  The offsets of those structs are extracted from header files
with a SWIG plugin.

@menu
* Calling C Functions::
* Declaring C Functions::
* Calling C function pointers::
* Defining library interfaces::
* Declaring OS-level libraries::
* Callbacks::
* C interface internals::
* Low-Level C Interface Words::
* Automated interface generation using SWIG::
* Migrating the C interface from earlier Gforth::
@end menu

@node Calling C Functions, Declaring C Functions, C Interface, C Interface
@subsection Calling C functions
@cindex C functions, calls to
@cindex calling C functions

Once a C function is declared (see @pxref{Declaring C Functions}), you
can call it as follows: You push the arguments on the stack(s), and
then call the word for the C function.  The arguments have to be
pushed in the same order as the arguments appear in the C
documentation (i.e., the first argument is deepest on the stack).
Integer and pointer arguments have to be pushed on the data stack,
floating-point arguments on the FP stack; these arguments are consumed
by the called C function.

On returning from the C function, the return value, if any, resides on
the appropriate stack: an integer return value is pushed on the data
stack, an FP return value on the FP stack, and a void return value
results in not pushing anything.  Note that most C functions have a
return value, even if that is often not used in C; in Forth, you have
to @code{drop} this return value explicitly if you do not use it.

The C interface automatically converts between the C type and the
Forth type as necessary, on a best-effort basis (in some cases, there
may be some loss).

As an example, consider the POSIX function @code{lseek()}:

@example
off_t lseek(int fd, off_t offset, int whence);
@end example

This function takes three integer arguments, and returns an integer
argument, so a Forth call for setting the current file offset to the
start of the file could look like this:

@example
fd @@ 0 SEEK_SET lseek -1 = if
  ... \ error handling
then
@end example

You might be worried that an @code{off_t} does not fit into a cell, so
you could not pass larger offsets to lseek, and might get only a part
of the return values.  In that case, in your declaration of the
function (@pxref{Declaring C Functions}) you should declare it to use
double-cells for the off_t argument and return value, and maybe give
the resulting Forth word a different name, like @code{dlseek}; the
result could be called like this:

@example
fd @@ 0. SEEK_SET dlseek -1. d= if
  ... \ error handling
then
@end example

Passing and returning structs or unions is currently not supported by
our interface@footnote{If you know the calling convention of your C
compiler, you usually can call such functions in some way, but that
way is usually not portable between platforms, and sometimes not even
between C compilers.}.

Calling functions with a variable number of arguments (@emph{variadic}
functions, e.g., @code{printf()}) is only supported by having you
declare one function-calling word for each argument pattern, and
calling the appropriate word for the desired pattern.



@node Declaring C Functions, Calling C function pointers, Calling C Functions, C Interface
@subsection Declaring C Functions
@cindex C functions, declarations
@cindex declaring C functions

Before you can call @code{lseek} or @code{dlseek}, you have to declare
it.  The declaration consists of two parts: 

@table @b

@item The C part
is the C declaration of the function, or more typically and portably,
a C-style @code{#include} of a file that contains the declaration of
the C function.

@item The Forth part
declares the Forth types of the parameters and the Forth word name
corresponding to the C function.

@end table

For the words @code{lseek} and @code{dlseek} mentioned earlier, the
declarations are:

@example
\c #define _FILE_OFFSET_BITS 64
\c #include <sys/types.h>
\c #include <unistd.h>
c-function lseek lseek n n n -- n
c-function dlseek lseek n d n -- d
@end example

The C part of the declarations is prefixed by @code{\c}, and the rest
of the line is ordinary C code.  You can use as many lines of C
declarations as you like, and they are visible for all further
function declarations.

The Forth part declares each interface word with @code{c-function},
followed by the Forth name of the word, the C name of the called
function, and the stack effect of the word.  The stack effect contains
an arbitrary number of types of parameters, then @code{--}, and then
exactly one type for the return value.  The possible types are:

@table @code

@item n
single-cell integer

@item a
address (single-cell)

@item d
double-cell integer

@item r
floating-point value

@item func
C function pointer

@item void
no value (used as return type for void functions)

@end table

@cindex variadic C functions

To deal with variadic C functions, you can declare one Forth word for
every pattern you want to use, e.g.:

@example
\c #include <stdio.h>
c-function printf-nr printf a n r -- n
c-function printf-rn printf a r n -- n
@end example

Note that with C functions declared as variadic (or if you don't
provide a prototype), the C interface has no C type to convert to, so
no automatic conversion happens, which may lead to portability
problems in some cases.  You can add the C type cast in curly braces
after the Forth type.  This also allows to pass e.g. structs to C
functions, which in Forth cannot live on the stack.

@example
c-function printfll printf a n@{(long long)@} -- n
c-function pass-struct pass_struct a@{*(struct foo *)@} -- n
@end example

This typecasting is not available to return values, as C does not
allow typecasts for lvalues.

doc-\c
doc-c-function
doc-c-value
doc-c-variable

In order to work, this C interface invokes GCC at run-time and uses
dynamic linking.  If these features are not available, there are
other, less convenient and less portable C interfaces in @file{lib.fs}
and @file{oldlib.fs}.  These interfaces are mostly undocumented and
mostly incompatible with each other and with the documented C
interface; you can find some examples for the @file{lib.fs} interface
in @file{lib.fs}.


@node Calling C function pointers, Defining library interfaces, Declaring C Functions, C Interface
@subsection Calling C function pointers from Forth
@cindex C function pointers, calling from Forth

If you come across a C function pointer (e.g., in some C-constructed
structure) and want to call it from your Forth program, you could use
the structures as described above by defining a macro.  Or you use
@code{c-funptr}.

doc-c-funptr

Let us assume that there is a C function pointer type @code{func1}
defined in some header file @file{func1.h}, and you know that these
functions take one integer argument and return an integer result; and
you want to call functions through such pointers.  Just define

@example
\c #include <func1.h>
c-funptr call-func1 @{((func1)ptr)@} n -- n
@end example

and then you can call a function pointed to by, say @code{func1a} as
follows:

@example
-5 func1a call-func1 .
@end example

The Forth word @code{call-func1} is similar to @code{execute}, except
that it takes a C @code{func1} pointer instead of a Forth execution
token, and it is specific to @code{func1} pointers.  For each type of
function pointer you want to call from Forth, you have to define
a separate calling word.

@node Defining library interfaces, Declaring OS-level libraries, Calling C function pointers, C Interface
@subsection Defining library interfaces
@cindex giving a name to a library interface
@cindex library interface names

You can give a name to a bunch of C function declarations (a library
interface), as follows:

@example
c-library lseek-lib
\c #define _FILE_OFFSET_BITS 64
...
end-c-library
@end example

The effect of giving such a name to the interface is that the names of
the generated files will contain that name, and when you use the
interface a second time, it will use the existing files instead of
generating and compiling them again, saving you time.  The generated
file contains a 128 bit hash (not cryptographically safe, but good
enough for that purpose) of the source code, so changing the
declarations will cause a new compilation.  Normally these files are
cached in @file{$HOME/.gforth/}@var{architecture}@file{/libcc-named}, so
if you experience problems or have other reasons to force a
recompilation, you can delete the files there.

Note that you should use @code{c-library} before everything else having
anything to do with that library, as it resets some setup stuff.  The
idea is that the typical use is to put each
@code{c-library}...@code{end-c-library} unit in its own file, and to be
able to include these files in any order.  All other words dealing with
the C interface are hidden in the vocabulary @code{c-lib}, which is put on top o the search stack by @code{c-library} and removed by @code{end-c-library}.

Note that the library name is not allocated in the dictionary and
therefore does not shadow dictionary names.  It is used in the file
system, so you have to use naming conventions appropriate for file
systems.  The name is also used as part of the C symbols, but characters
outside the legal C symbol names are replaced with underscores.  Also,
you shall not call a function you declare after @code{c-library} before
you perform @code{end-c-library}.

A major benefit of these named library interfaces is that, once they are
generated, the tools used to generated them (in particular, the C
compiler and libtool) are no longer needed, so the interface can be used
even on machines that do not have the tools installed.  The build system
of Gforth can even cross-compile these libraries, so that the libraries
are available for plattforms on which build tools aren't installed.

doc-c-library-name
doc-c-library
doc-end-c-library


@node Declaring OS-level libraries, Callbacks, Defining library interfaces, C Interface
@subsection Declaring OS-level libraries
@cindex Shared libraries in C interface
@cindex Dynamically linked libraries in C interface
@cindex Libraries in C interface

For calling some C functions, you need to link with a specific
OS-level library that contains that function.  E.g., the @code{sin}
function requires linking a special library by using the command line
switch @code{-lm}.  In our C iterface you do the equivalent thing by
calling @code{add-lib} as follows:

@example
clear-libs
s" m" add-lib
\c #include <math.h>
c-function sin sin r -- r
@end example

First, you clear any libraries that may have been declared earlier
(you don't need them for @code{sin}); then you add the @code{m}
library (actually @code{libm.so} or somesuch) to the currently
declared libraries; you can add as many as you need.  Finally you
declare the function as shown above.  Typically you will use the same
set of library declarations for many function declarations; you need
to write only one set for that, right at the beginning.

Note that you must not call @code{clear-libs} inside
@code{c-library...end-c-library}; however, @code{c-library} performs
the function of @code{clear-libs}, so @code{clear-libs} is not
necessary, and you usually want to put @code{add-lib} calls inside
@code{c-library...end-c-library}.

doc-clear-libs
doc-add-lib
doc-add-libpath
doc-add-framework
doc-add-incdir
doc-add-cflags
doc-add-ldflags

@node Callbacks, C interface internals, Declaring OS-level libraries, C Interface
@subsection Callbacks
@cindex Callback functions written in Forth
@cindex C function pointers to Forth words

In some cases you have to pass a function pointer to a C function,
i.e., the library wants to call back to your application (and the
pointed-to function is called a callback function).  You can pass the
address of an existing C function (that you get with @code{lib-sym},
@pxref{Low-Level C Interface Words}), but if there is no appropriate C
function, you probably want to define the function as a Forth word.
Then you need to generate a callback as described below:

You can generate C callbacks from Forth code with @code{c-callback}.

doc-c-callback
doc-c-callback-thread

This precompiles a number of callback functions (up to the value
@code{callback#}).  The prototype of the C function is deduced from
its Forth signature.  If this is not sufficient, you can add types in
curly braces after the Forth type.

@example
c-callback vector4double: f f f f -- void
c-callback vector4single: f@{float@} f@{float@} f@{float@} f@{float@} -- void
@end example

@c I don't understand the existing callback interface from the example - anton


@c > > Und dann gibt's noch die fptr-Deklaration, die einem
@c > > C-Funktionspointer entspricht (Deklaration gleich wie bei
@c > > Library-Funktionen, nur ohne den C-Namen, Aufruf mit der
@c > > C-Funktionsadresse auf dem TOS).
@c >
@c > Ja, da bin ich dann ausgestiegen, weil ich aus dem Beispiel nicht
@c > gesehen habe, wozu das gut ist.
@c 
@c Irgendwie muss ich den Callback ja testen. Und es soll ja auch 
@c vorkommen, dass man von irgendwelchen kranken Interfaces einen 
@c Funktionspointer übergeben bekommt, den man dann bei Gelegenheit 
@c aufrufen muss. Also kann man den deklarieren, und das damit deklarierte 
@c Wort verhält sich dann wie ein EXECUTE für alle C-Funktionen mit 
@c demselben Prototyp.


@node C interface internals, Low-Level C Interface Words, Callbacks, C Interface
@subsection How the C interface works

The documented C interface works by generating a C code out of the
declarations.

In particular, for every Forth word declared with @code{c-function},
it generates a wrapper function in C that takes the Forth data from
the Forth stacks, and calls the target C function with these data as
arguments.  The C compiler then performs an implicit conversion
between the Forth type from the stack, and the C type for the
parameter, which is given by the C function prototype.  After the C
function returns, the return value is likewise implicitly converted to
a Forth type and written back on the stack.

The @code{\c} lines are literally included in the C code (but without
the @code{\c}), and provide the necessary declarations so that the C
compiler knows the C types and has enough information to perform the
conversion.

These wrapper functions are eventually compiled and dynamically linked
into Gforth, and then they can be called.

The libraries added with @code{add-lib} are used in the compile
command line to specify dependent libraries with @code{-l@var{lib}},
causing these libraries to be dynamically linked when the wrapper
function is linked.


@node Low-Level C Interface Words, Automated interface generation using SWIG, C interface internals, C Interface
@subsection Low-Level C Interface Words

doc-open-lib
doc-lib-sym
doc-lib-error
doc-call-c

@node Automated interface generation using SWIG, Migrating the C interface from earlier Gforth, Low-Level C Interface Words, C Interface
@subsection Automated interface generation using SWIG

SWIG, the Simple Wrapper Interface Generator, is used to create C interfaces for a lot of programming languages.
The SWIG version extended with a Forth module can be found on @uref{https://github.com/GeraldWodni/swig,github}.

@subsubsection Basic operation
C-headers are parsed and converted to Forth-Sourcecode which uses the previously describe C interface functions.

@subsubsection Detailed operation:
@enumerate 1
@item Select a target, in this example we are using @code{example.h}
@item Create an interface file for the header. This can be used to pass options, switches and define variables.
    In the simplest case it just instructs to translate all of @code{example.h}:@/
@example
%module example
%insert("include")
@{
    #include "example.h"
@}
%include "example.h"
@end example
@item Use SWIG to create a @code{.fsi-c} file. @*
    @code{swig -forth -stackcomments -use-structs -enumcomments -o example-fsi.c example.i}. @*
    FSI stands ``Forth Source Independent'' meaning it can be transferred to any host having a C-compiler. SWIG is not required past this point.
@item On the target machine compile the @code{.fsi-c} file to a @code{.fsx} (x stands for executable) @*
    @code{gcc -o example.fsx example-fsi.c} @*
    The compilation will resolve all constants to the values on the target.
@item The last step is to run the executable and capture its output to a @code{.fs} ``Forth Source'' file. @*
    @code{./example.fsx -gforth > example.fs} @*
    This code can now be used on the target platform.
@end enumerate

@subsubsection Examples
You can find some examples in SWIG's @uref{https://github.com/GeraldWodni/swig/tree/master/Examples/forth, Forth Example section}.

A lot of interface files can be found in @uref{https://github.com/GeraldWodni/posix, Forth Posix C-Interface}
and @uref{https://github.com/GeraldWodni/forth-c-interfaces, Forth C-Interface Modules}.

Contribution to the @uref{https://github.com/GeraldWodni/forth-c-interfaces, Forth C-Interface Module repository} is always welcome.

@node Migrating the C interface from earlier Gforth,  , Automated interface generation using SWIG, C Interface
@subsection Migrating from Gforth 0.7
@cindex Must now be used inside C-LIBRARY, see C interface doc

In this version, you can use @code{\c}, @code{c-function} and
@code{add-lib} only inside @code{c-library}...@code{end-c-library}.
@code{add-lib} now always starts from a clean slate inside a
@code{c-library}, so you don't need to use @code{clear-libs} in most
cases.

If you have a program that uses these words outside
@code{c-library}...@code{end-c-library}, just wrap them in
@code{c-library}...@code{end-c-library}.  You may have to add some
instances of @code{add-lib}, however.

@c -------------------------------------------------------------
@node Assembler and Code Words, Carnal words, C Interface, Words
@section Assembler and Code Words
@cindex assembler
@cindex code words

@menu
* Assembler Definitions::       Definitions in assembly language
* Common Assembler::            Assembler Syntax
* Common Disassembler::
* 386 Assembler::               Deviations and special cases
* AMD64 Assembler::
* Alpha Assembler::             Deviations and special cases
* MIPS assembler::              Deviations and special cases
* PowerPC assembler::           Deviations and special cases
* ARM Assembler::               Deviations and special cases
* Other assemblers::            How to write them
@end menu

@node Assembler Definitions, Common Assembler, Assembler and Code Words, Assembler and Code Words
@subsection Definitions in assembly language

Gforth provides ways to implement words in assembly language (using
@code{abi-code}...@code{end-code}), and also ways to define defining
words with arbitrary run-time behaviour (like @code{does>}), where
(unlike @code{does>}) the behaviour is not defined in Forth, but in
assembly language (with @code{;code}).

However, the machine-independent nature of Gforth poses a few
problems: First of all, Gforth runs on several architectures, so it
can provide no standard assembler. It does provide assemblers for
several of the architectures it runs on, though.  Moreover, you can
use a system-independent assembler in Gforth, or compile machine code
directly with @code{,} and @code{c,}.

Another problem is that the virtual machine registers of Gforth (the
stack pointers and the virtual machine instruction pointer) depend on
the installation and engine.  Also, which registers are free to use
also depend on the installation and engine.  So any code written to
run in the context of the Gforth virtual machine is essentially
limited to the installation and engine it was developed for (it may
run elsewhere, but you cannot rely on that).

Fortunately, you can define @code{abi-code} words in Gforth that are
portable to any Gforth running on a platform with the same calling
convention (ABI); typically this means portability to the same
architecture/OS combination, sometimes crossing OS boundaries).

doc-assembler
doc-init-asm
doc-abi-code
doc-;abi-code
doc-end-code
doc-code
doc-;code
doc-flush-icache


If @code{flush-icache} does not work correctly, @code{abi-code} words
etc. will not work (reliably), either.

The typical usage of these words can be shown most easily by analogy
to the equivalent high-level defining words:

@example
: foo                              abi-code foo
   <high-level Forth words>              <assembler>
;                                  end-code
                                
: bar                              : bar
   <high-level Forth words>           <high-level Forth words>
   CREATE                             CREATE
      <high-level Forth words>           <high-level Forth words>
   DOES>                              ;code
      <high-level Forth words>           <assembler>
;                                  end-code
@end example

For using @code{abi-code}, take a look at the ABI documentation of
your platform to see how the parameters are passed (so you know where
you get the stack pointers) and how the return value is passed (so you
know where the data stack pointer is returned).  The ABI documentation
also tells you which registers are saved by the caller (caller-saved),
so you are free to destroy them in your code, and which registers have
to be preserved by the called word (callee-saved), so you have to save
them before using them, and restore them afterwards.  For some
architectures and OSs we give short summaries of the parts of the
calling convention in the appropriate sections.  More
reverse-engineering oriented people can also find out about the
passing and returning of the stack pointers through @code{see
abi-call}.

Most ABIs pass the parameters through registers, but some (in
particular the most common 386 (aka IA-32) calling conventions) pass
them on the architectural stack.  The common ABIs all pass the return
value in a register.

Other things you need to know for using @code{abi-code} is that both
the data and the FP stack grow downwards (towards lower addresses) in
Gforth, with @code{1 cells} size per cell, and @code{1 floats} size
per FP value.

Here's an example of using @code{abi-code} on the 386 architecture:

@example
abi-code my+ ( n1 n2 -- n )
4 sp d) ax mov \ sp into return reg
ax )    cx mov \ tos
4 #     ax add \ update sp (pop)
cx    ax ) add \ sec = sec+tos
ret            \ return from my+
end-code
@end example

An AMD64 variant of this example can be found in @ref{AMD64 Assembler}.

Here's a 386 example that deals with FP values:

@example
abi-code my-f+ ( r1 r2 -- r )
8 sp d) cx mov  \ load address of fp
cx )    dx mov  \ load fp
.fl dx )   fld  \ r2
8 #     dx add  \ update fp
.fl dx )   fadd \ r1+r2
.fl dx )   fstp \ store r
dx    cx ) mov  \ store new fp
4 sp d) ax mov  \ sp into return reg
ret             \ return from my-f+
end-code
@end example


@node Common Assembler, Common Disassembler, Assembler Definitions, Assembler and Code Words
@subsection Common Assembler

The assemblers in Gforth generally use a postfix syntax, i.e., the
instruction name follows the operands.

The operands are passed in the usual order (the same that is used in the
manual of the architecture).  Since they all are Forth words, they have
to be separated by spaces; you can also use Forth words to compute the
operands.

The instruction names usually end with a @code{,}.  This makes it easier
to visually separate instructions if you put several of them on one
line; it also avoids shadowing other Forth words (e.g., @code{and}).

Registers are usually specified by number; e.g., (decimal) @code{11}
specifies registers R11 and F11 on the Alpha architecture (which one,
depends on the instruction).  The usual names are also available, e.g.,
@code{s2} for R11 on Alpha.

Control flow is specified similar to normal Forth code (@pxref{Arbitrary
control structures}), with @code{if,}, @code{ahead,}, @code{then,},
@code{begin,}, @code{until,}, @code{again,}, @code{cs-roll},
@code{cs-pick}, @code{else,}, @code{while,}, and @code{repeat,}.  The
conditions are specified in a way specific to each assembler.

The rest of this section is of interest mainly for those who want to
define @code{code} words (instead of the more portable @code{abi-code}
words).

Note that the register assignments of the Gforth engine can change
between Gforth versions, or even between different compilations of the
same Gforth version (e.g., if you use a different GCC version).  If
you are using @code{CODE} instead of @code{ABI-CODE}, and you want to
refer to Gforth's registers (e.g., the stack pointer or TOS), I
recommend defining your own words for refering to these registers, and
using them later on; then you can adapt to a changed register
assignment.

The most common use of these registers is to end a @code{code}
definition with a dispatch to the next word (the @code{next} routine).
A portable way to do this is to jump to @code{' noop >code-address}
(of course, this is less efficient than integrating the @code{next}
code and scheduling it well).  When using @code{ABI-CODE}, you can
just assemble a normal subroutine return (but make sure you return the
data stack pointer).

Another difference between Gforth versions is that the top of stack is
kept in memory in @code{gforth} and, on most platforms, in a register
in @code{gforth-fast}.  For @code{ABI-CODE} definitions, any stack
caching registers are guaranteed to be flushed to the stack, allowing
you to reliably access the top of stack in memory.

@node  Common Disassembler, 386 Assembler, Common Assembler, Assembler and Code Words
@subsection Common Disassembler
@cindex disassembler, general
@cindex gdb disassembler

You can disassemble a @code{code} word with @code{see}
(@pxref{Debugging}).  You can disassemble a section of memory with

doc-discode

There are two kinds of disassembler for Gforth: The Forth disassembler
(available on some CPUs) and the gdb disassembler (available on
platforms with @command{gdb} and @command{mktemp}).  If both are
available, the Forth disassembler is used by default.  If you prefer
the gdb disassembler, say

@example
' disasm-gdb is discode
@end example

If neither is available, @code{discode} performs @code{dump}.

The Forth disassembler generally produces output that can be fed into the
assembler (i.e., same syntax, etc.).  It also includes additional
information in comments.  In particular, the address of the instruction
is given in a comment before the instruction.

The gdb disassembler produces output in the same format as the gdb
@code{disassemble} command (@pxref{Machine Code,,Source and machine
code,gdb,Debugging with GDB}), in the default flavour (AT&T syntax for
the 386 and AMD64 architectures).

@code{See} may display more or less than the actual code of the word,
because the recognition of the end of the code is unreliable.  You can
use @code{discode} if it did not display enough.  It may display more, if
the code word is not immediately followed by a named word.  If you have
something else there, you can follow the word with @code{align latest ,}
to ensure that the end is recognized.

@node 386 Assembler, AMD64 Assembler, Common Disassembler, Assembler and Code Words
@subsection 386 Assembler

The 386 assembler included in Gforth was written by Bernd Paysan, it's
available under GPL, and originally part of bigFORTH.

The 386 disassembler included in Gforth was written by Andrew McKewan
and is in the public domain.

The disassembler displays code in an Intel-like prefix syntax.

The assembler uses a postfix syntax with AT&T-style parameter order
(i.e., destination last).

The assembler includes all instruction of the Athlon, i.e. 486 core
instructions, Pentium and PPro extensions, floating point, MMX, 3Dnow!,
but not ISSE. It's an integrated 16- and 32-bit assembler. Default is 32
bit, you can switch to 16 bit with .86 and back to 32 bit with .386.

There are several prefixes to switch between different operation sizes,
@code{.b} for byte accesses, @code{.w} for word accesses, @code{.d} for
double-word accesses. Addressing modes can be switched with @code{.wa}
for 16 bit addresses, and @code{.da} for 32 bit addresses. You don't
need a prefix for byte register names (@code{AL} et al).

For floating point operations, the prefixes are @code{.fs} (IEEE
single), @code{.fl} (IEEE double), @code{.fx} (extended), @code{.fw}
(word), @code{.fd} (double-word), and @code{.fq} (quad-word).  The
default is @code{.fx}, so you need to specify @code{.fl} explicitly
when dealing with Gforth FP values.

The MMX opcodes don't have size prefixes, they are spelled out like in
the Intel assembler. Instead of move from and to memory, there are
PLDQ/PLDD and PSTQ/PSTD.

The registers lack the 'e' prefix; even in 32 bit mode, eax is called
ax.  Immediate values are indicated by postfixing them with @code{#},
e.g., @code{3 #}.  Here are some examples of addressing modes in various
syntaxes:

@example
Gforth          Intel (NASM)   AT&T (gas)      Name
.w ax           ax             %ax             register (16 bit)
ax              eax            %eax            register (32 bit)
3 #             offset 3       $3              immediate
1000 #)         byte ptr 1000  1000            displacement
bx )            [ebx]          (%ebx)          base
100 di d)       100[edi]       100(%edi)       base+displacement
20 ax *4 i#)    20[eax*4]      20(,%eax,4)     (index*scale)+displacement
di ax *4 i)     [edi][eax*4]   (%edi,%eax,4)   base+(index*scale)
4 bx cx di)     4[ebx][ecx]    4(%ebx,%ecx)    base+index+displacement
12 sp ax *2 di) 12[esp][eax*2] 12(%esp,%eax,2) base+(index*scale)+displacement
@end example

You can use @code{L)} and @code{LI)} instead of @code{D)} and
@code{DI)} to enforce 32-bit displacement fields (useful for
later patching).

Some example of instructions are:

@example
ax bx mov             \ move ebx,eax
3 # ax mov            \ mov eax,3
100 di d) ax mov      \ mov eax,100[edi]
4 bx cx di) ax mov    \ mov eax,4[ebx][ecx]
.w ax bx mov          \ mov bx,ax
@end example

The following forms are supported for binary instructions:

@example
<reg> <reg> <inst>
<n> # <reg> <inst>
<mem> <reg> <inst>
<reg> <mem> <inst>
<n> # <mem> <inst>
@end example

The shift/rotate syntax is:

@example
<reg/mem> 1 # shl \ shortens to shift without immediate
<reg/mem> 4 # shl
<reg/mem> cl shl
@end example

Precede string instructions (@code{movs} etc.) with @code{.b} to get
the byte version.

The control structure words @code{IF} @code{UNTIL} etc. must be preceded
by one of these conditions: @code{vs vc u< u>= 0= 0<> u<= u> 0< 0>= ps
pc < >= <= >}. (Note that most of these words shadow some Forth words
when @code{assembler} is in front of @code{forth} in the search path,
e.g., in @code{code} words).  Currently the control structure words use
one stack item, so you have to use @code{roll} instead of @code{cs-roll}
to shuffle them (you can also use @code{swap} etc.).

Based on the Intel ABI (used in Linux), @code{abi-code} words can find
the data stack pointer at @code{4 sp d)}, and the address of the FP
stack pointer at @code{8 sp d)}; the data stack pointer is returned in
@code{ax}; @code{Ax}, @code{cx}, and @code{dx} are caller-saved, so
you do not need to preserve their values inside the word.  You can
return from the word with @code{ret}, the parameters are cleaned up by
the caller.

For examples of 386 @code{abi-code} words, see @ref{Assembler Definitions}.


@node AMD64 Assembler, Alpha Assembler, 386 Assembler, Assembler and Code Words
@subsection AMD64 (x86_64) Assembler

The AMD64 assembler is a slightly modified version of the 386
assembler, and as such shares most of the syntax.  Two new prefixes,
@code{.q} and @code{.qa}, are provided to select 64-bit operand and
address sizes respectively.  64-bit sizes are the default, so normally
you only have to use the other prefixes.  Also there are additional
register operands @code{R8}-@code{R15}.

The registers lack the 'e' or 'r' prefix; even in 64 bit mode,
@code{rax} is called @code{ax}.  Additional register operands are
available to refer to the lowest-significant byte of all registers:
@code{R8L}-@code{R15L}, @code{SPL}, @code{BPL}, @code{SIL},
@code{DIL}.

The Linux-AMD64 calling convention is to pass the first 6 integer
parameters in rdi, rsi, rdx, rcx, r8 and r9 and to return the result
in rax and rdx; to pass the first 8 FP parameters in xmm0--xmm7 and to
return FP results in xmm0--xmm1.  So @code{abi-code} words get the
data stack pointer in @code{di} and the address of the FP stack
pointer in @code{si}, and return the data stack pointer in @code{ax}.
The other caller-saved registers are: r10, r11, xmm8-xmm15.  This
calling convention reportedly is also used in other non-Microsoft OSs.
@c source: https://en.wikipedia.org/wiki/X86_calling_conventions#AMD64_ABI_convention

@c source: http://msdn.microsoft.com/en-us/library/9b372w95(v=VS.90).aspx
Windows x64 passes the first four integer parameters in rcx, rdx, r8
and r9 and return the integer result in rax.  The other caller-saved
registers are r10 and r11.

On the Linux platform, according to
@url{https://uclibc.org/docs/psABI-x86_64.pdf} page 21 the registers AX
CX DX SI DI R8 R9 R10 R11 are available for scratch.

The addressing modes for the AMD64 are:

@example
\ running word A produces a memory error as the registers are not initialised ;-)
ABI-CODE A  ( -- )
    500        #               AX  MOV     \ immediate
        DX              AX  MOV     \ register
        200             AX  MOV     \ direct addressing
        DX  )           AX  MOV     \ indirect addressing
    40  DX  D)          AX  MOV     \ base with displacement
        DX  CX      I)  AX  MOV     \ scaled index
        DX  CX  *4  I)  AX  MOV     \ scaled index
    40  DX  CX  *4  DI) AX  MOV     \ scaled index with displacement

        DI              AX  MOV     \ SP Out := SP in
                            RET
END-CODE
@end example

Here are a few examples of an AMD64 @code{abi-code} words:

@example
abi-code my+  ( n1 n2 -- n3 )
\ SP passed in di, returned in ax,  address of FP passed in si
8 di d) ax lea        \ compute new sp in result reg
di )    dx mov        \ get old tos
dx    ax ) add        \ add to new tos
ret
end-code
@end example

@example
\ Do nothing
ABI-CODE aNOP  ( -- )
       DI  )       AX      LEA          \ SP out := SP in  
                           RET
END-CODE
@end example


@example
\ Drop TOS
ABI-CODE aDROP  ( n -- )
   8   DI  D)      AX      LEA          \ SPout := SPin - 1
                           RET
END-CODE
@end example


@example
\ Push 5 on the data stack
ABI-CODE aFIVE   ( -- 5 )
   -8  DI  D)      AX      LEA          \ SPout := SPin + 1
   5   #           AX  )   MOV          \ TOS := 5
                           RET
END-CODE
@end example


@example
\ Push 10 and 20 into data stack
ABI-CODE aTOS2  ( -- n n )
   -16 DI  D)      AX      LEA          \ SPout := SPin + 2
   10  #       8   AX  D)  MOV          \ TOS - 1 := 10
   20  #           AX  )   MOV          \ TOS := 20
                           RET
END-CODE
@end example


@example
\ Get Time Stamp Counter as two 32 bit integers
\ The TSC is incremented every CPU clock pulse
ABI-CODE aRDTSC   ( -- TSCl TSCh )
                           RDTSC        \ DX:AX := TSC
   $FFFFFFFF #     AX      AND          \ Clear upper 32 bit AX
  0xFFFFFFFF #     DX      AND          \ Clear upper 32 bit DX
       AX          R8      MOV          \ Tempory save AX
   -16 DI  D)      AX      LEA          \ SPout := SPin + 2
       R8      8   AX  D)  MOV          \ TOS-1 := saved AX = TSC low
       DX          AX  )   MOV          \ TOS := Dx = TSC high
                           RET
END-CODE
@end example


@example
\ Get Time Stamp Counter as 64 bit integer
ABI-CODE RDTSC   ( -- TSC )
                           RDTSC        \ DX:AX := TSC
   $FFFFFFFF #     AX      AND          \ Clear upper 32 bit AX
   32  #           DX      SHL          \ Move lower 32 bit DX to upper 32 bit
       AX          DX      OR           \ Combine AX wit DX in DX
   -8  DI  D)      AX      LEA          \ SPout := SPin + 1
       DX          AX  )   MOV          \ TOS := DX
                           RET
END-CODE
@end example


@example
VARIABLE V

\ Assign 4 to variable V
ABI-CODE V=4 ( -- )
       BX                  PUSH         \ Save BX, used by gforth
   V   #           BX      MOV          \ BX := address of V
   4   #           BX )    MOV          \ Write 4 to V
       BX                  POP          \ Restore BX
       DI  )       AX      LEA          \ SPout := SPin
                           RET
END-CODE
@end example


@example
VARIABLE V

\ Assign 5 to variable V
ABI-CODE V=5 ( -- )
   V   #           CX      MOV          \ CX := address of V
   5   #           CX )    MOV          \ Write 5 to V
   DI )            AX      LEA          \ SPout := SPin
                           RET
END-CODE
@end example


@example
ABI-CODE TEST2  ( -- n n )
   -16 DI  D)  AX          LEA          \ SPout := SPin + 2
   5   #       CX          MOV          \ CX := 5
   5   #       CX          CMP
   0= IF
       1   #   8   AX  D)      MOV      \ If CX = 5 then TOS - 1 := 1  <--
   ELSE
       2   #   8   AX  D)      MOV      \ else TOS - 1 := 2
   THEN
   6   #       CX          CMP
   0= IF
       3   #       AX  )       MOV      \ If CX = 6 then TOS := 3
   ELSE
       4   #       AX  )       MOV      \ else TOS := 4  <--
   THEN
                           RET
END-CODE
@end example


@example
\ Do four loops. Expect : ( 4 3 2 1 -- )
ABI-CODE LOOP4  ( -- n n n n )
       DI          AX      MOV          \ SPout := SPin
   4   #           DX      MOV          \ DX := 4  loop counter
   BEGIN
       8   #           AX      SUB      \ SP := SP + 1
           DX          AX  )   MOV      \ TOS := DX
       1   #           DX      SUB      \ DX := DX - 1
   0= UNTIL
                           RET
END-CODE
@end example

Here's a AMD64 example that deals with FP values:

@example
abi-code my-f+  ( r1 r2 -- r )
\ SP passed in di, returned in ax,  address of FP passed in si
si )       dx mov         \ load fp
8 dx d)  xmm0 movsd       \ r2
dx )     xmm0 addsd       \ r1+r2
xmm0  8 dx d) movsd       \ store r
8 #      si ) add         \ update fp
di         ax mov         \ sp into return reg
ret
end-code
@end example

@node Alpha Assembler, MIPS assembler, AMD64 Assembler, Assembler and Code Words
@subsection Alpha Assembler

The Alpha assembler and disassembler were originally written by Bernd
Thallner.

The register names @code{a0}--@code{a5} are not available to avoid
shadowing hex numbers.

Immediate forms of arithmetic instructions are distinguished by a
@code{#} just before the @code{,}, e.g., @code{and#,} (note: @code{lda,}
does not count as arithmetic instruction).

You have to specify all operands to an instruction, even those that
other assemblers consider optional, e.g., the destination register for
@code{br,}, or the destination register and hint for @code{jmp,}.

You can specify conditions for @code{if,} by removing the first @code{b}
and the trailing @code{,} from a branch with a corresponding name; e.g.,

@example
11 fgt if, \ if F11>0e
  ...
endif,
@end example

@code{fbgt,} gives @code{fgt}.  

@node MIPS assembler, PowerPC assembler, Alpha Assembler, Assembler and Code Words
@subsection MIPS assembler

The MIPS assembler was originally written by Christian Pirker.

Currently the assembler and disassembler covers most of the MIPS32
architecture and doesn't support FP instructions.

The register names @code{$a0}--@code{$a3} are not available to avoid
shadowing hex numbers.  Use register numbers @code{$4}--@code{$7}
instead.

Nothing distinguishes registers from immediate values.  Use explicit
opcode names with the @code{i} suffix for instructions with immediate
argument.  E.g. @code{addiu,} in place of @code{addu,}.

Where the architecture manual specifies several formats for the
instruction (e.g., for @code{jalr,}),use the one with more arguments
(i.e. two for @code{jalr,}).  When in doubt, see
@code{arch/mips/testasm.fs} for an example of correct use.

Branches and jumps in the MIPS architecture have a delay slot.  You
have to fill it manually (the simplest way is to use @code{nop,}), the
assembler does not do it for you (unlike @command{as}).  Even
@code{if,}, @code{ahead,}, @code{until,}, @code{again,},
@code{while,}, @code{else,} and @code{repeat,} need a delay slot.
Since @code{begin,} and @code{then,} just specify branch targets, they
are not affected.  For branches the argument specifying the target is
a relative address.  Add the address of the delay slot to get the
absolute address.

Note that you must not put branches nor jumps (nor control-flow
instructions) into the delay slot.  Also it is a bad idea to put
pseudo-ops such as @code{li,} into a delay slot, as these may expand
to several instructions.  The MIPS I architecture also had load delay
slots, and newer MIPSes still have restrictions on using @code{mfhi,}
and @code{mflo,}.  Be careful to satisfy these restrictions, the
assembler does not do it for you.

Some example of instructions are:

@example
$ra  12 $sp  sw,         \ sw    ra,12(sp)
$4    8 $s0  lw,         \ lw    a0,8(s0)
$v0  $0  lui,            \ lui   v0,0x0
$s0  $s4  $12  addiu,    \ addiu s0,s4,0x12
$s0  $s4  $4  addu,      \ addu  s0,s4,$a0
$ra  $t9  jalr,          \ jalr  t9
@end example

You can specify the conditions for @code{if,} etc. by taking a
conditional branch and leaving away the @code{b} at the start and the
@code{,} at the end.  E.g.,

@example
4 5 eq if,
  ... \ do something if $4 equals $5
then,
@end example

The calling conventions for 32-bit MIPS machines is to pass the first
4 arguments in registers @code{$4}..@code{$7}, and to use
@code{$v0}-@code{$v1} for return values.  In addition to these
registers, it is ok to clobber registers @code{$t0}-@code{$t8} without
saving and restoring them.

If you use @code{jalr,} to call into dynamic library routines, you
must first load the called function's address into @code{$t9}, which
is used by position-indirect code to do relative memory accesses.

Here is an example of a MIPS32 @code{abi-code} word:

@example
abi-code my+  ( n1 n2 -- n3 )
  \ SP passed in $4, returned in $v0
  $t0  4 $4  lw,         \ load n1, n2 from stack
  $t1  0 $4  lw,    
  $t0  $t0  $t1  addu,   \ add n1+n2, result in $t0
  $t0  4 $4  sw,         \ store result (overwriting n1)
  $ra  jr,               \ return to caller
  $v0  $4  4  addiu,     \ (delay slot) return uptated SP in $v0
end-code
@end example

@node PowerPC assembler, ARM Assembler, MIPS assembler, Assembler and Code Words
@subsection PowerPC assembler

The PowerPC assembler and disassembler were contributed by Michal
Revucky.

This assembler does not follow the convention of ending mnemonic names
with a ``,'', so some mnemonic names shadow regular Forth words (in
particular: @code{and or xor fabs}); so if you want to use the Forth
words, you have to make them visible first, e.g., with @code{also
forth}.

Registers are referred to by their number, e.g., @code{9} means the
integer register 9 or the FP register 9 (depending on the
instruction).

Because there is no way to distinguish registers from immediate values,
you have to explicitly use the immediate forms of instructions, i.e.,
@code{addi,}, not just @code{add,}.

The assembler and disassembler usually support the most general form
of an instruction, but usually not the shorter forms (especially for
branches).


@node ARM Assembler, Other assemblers, PowerPC assembler, Assembler and Code Words
@subsection ARM Assembler

The ARM assembler includes all instruction of ARM architecture version
4, and the BLX instruction from architecture 5.  It does not (yet)
have support for Thumb instructions.  It also lacks support for any
co-processors.

The assembler uses a postfix syntax with the same operand order as
used in the ARM Architecture Reference Manual.  Mnemonics are suffixed
by a comma.

Registers are specified by their names @code{r0} through @code{r15},
with the aliases @code{pc}, @code{lr}, @code{sp}, @code{ip} and
@code{fp} provided for convenience.  Note that @code{ip} refers to
the``intra procedure call scratch register'' (@code{r12}) and does not
refer to an instruction pointer.  @code{sp} refers to the ARM ABI
stack pointer (@code{r13}) and not the Forth stack pointer.

Condition codes can be specified anywhere in the instruction, but will
be most readable if specified just in front of the mnemonic.  The 'S'
flag is not a separate word, but encoded into instruction mnemonics,
ie. just use @code{adds,} instead of @code{add,} if you want the
status register to be updated.

The following table lists the syntax of operands for general
instructions:

@example
Gforth          normal assembler      description
123 #           #123                  immediate
r12             r12                   register
r12 4 #LSL      r12, LSL #4           shift left by immediate
r12 r1 LSL      r12, LSL r1           shift left by register
r12 4 #LSR      r12, LSR #4           shift right by immediate
r12 r1 LSR      r12, LSR r1           shift right by register
r12 4 #ASR      r12, ASR #4           arithmetic shift right
r12 r1 ASR      r12, ASR r1           ... by register
r12 4 #ROR      r12, ROR #4           rotate right by immediate
r12 r1 ROR      r12, ROR r1           ... by register
r12 RRX         r12, RRX              rotate right with extend by 1
@end example

Memory operand syntax is listed in this table:

@example
Gforth            normal assembler      description
r4 ]              [r4]                  register
r4 4 #]           [r4, #+4]             register with immediate offset
r4 -4 #]          [r4, #-4]             with negative offset
r4 r1 +]          [r4, +r1]             register with register offset
r4 r1 -]          [r4, -r1]             with negated register offset
r4 r1 2 #LSL -]   [r4, -r1, LSL #2]     with negated and shifted offset
r4 4 #]!          [r4, #+4]!            immediate preincrement
r4 r1 +]!         [r4, +r1]!            register preincrement
r4 r1 -]!         [r4, +r1]!            register predecrement
r4 r1 2 #LSL +]!  [r4, +r1, LSL #2]!    shifted preincrement
r4 -4 ]#          [r4], #-4             immediate postdecrement
r4 r1 ]+          [r4], r1              register postincrement
r4 r1 ]-          [r4], -r1             register postdecrement
r4 r1 2 #LSL ]-   [r4], -r1, LSL #2     shifted postdecrement
' xyz >body [#]   xyz                   PC-relative addressing
@end example

Register lists for load/store multiple instructions are started and
terminated by using the words @code{@{} and @code{@}} respectively.
Between braces, register names can be listed one by one or register
ranges can be formed by using the postfix operator @code{r-r}.  The
@code{^} flag is not encoded in the register list operand, but instead
directly encoded into the instruction mnemonic, ie. use @code{^ldm,}
and @code{^stm,}.

Addressing modes for load/store multiple are not encoded as
instruction suffixes, but instead specified like an addressing mode,
Use one of @code{DA}, @code{IA}, @code{DB}, @code{IB}, @code{DA!},
@code{IA!}, @code{DB!} or @code{IB!}.

The following table gives some examples:

@example
Gforth                           normal assembler
r4 ia  @{ r0 r7 r8 @}  stm,        stmia    r4, @{r0,r7,r8@}
r4 db!  @{ r0 r7 r8 @}  ldm,       ldmdb    r4!, @{r0,r7,r8@}
sp ia!  @{ r0 r15 r-r @}  ^ldm,    ldmfd    sp!, @{r0-r15@}^
@end example

Control structure words typical for Forth assemblers are available:
@code{if,} @code{ahead,} @code{then,} @code{else,} @code{begin,}
@code{until,} @code{again,} @code{while,} @code{repeat,}
@code{repeat-until,}.  Conditions are specified in front of these words:

@example
r1 r2 cmp,    \ compare r1 and r2
eq if,        \ equal?
   ...          \ code executed if r1 == r2
then,
@end example

Example of a definition using the ARM assembler:

@example
abi-code my+ ( n1 n2 --  n3 )
   \ arm abi: r0=SP, r1=&FP, r2,r3,r12 saved by caller
   r0 IA!  @{ r2 r3 @}  ldm,     \ pop r2 = n2, r3 = n1
   r3  r2  r3         add,     \ r3 = n1+n1
   r3  r0 -4 #]!      str,     \ push r3
   pc  lr             mov,     \ return to caller, new SP in r0
end-code
@end example

@c !! ARM64
@c !! RISC-V

@node Other assemblers,  , ARM Assembler, Assembler and Code Words
@subsection Other assemblers

If you want to contribute another assembler/disassembler, please contact
us (@email{anton@@mips.complang.tuwien.ac.at}) to check if we have such
an assembler already.  If you are writing them from scratch, please use
a similar syntax style as the one we use (i.e., postfix, commas at the
end of the instruction names, @pxref{Common Assembler}); make the output
of the disassembler be valid input for the assembler, and keep the style
similar to the style we used.

Hints on implementation: The most important part is to have a good test
suite that contains all instructions.  Once you have that, the rest is
easy.  For actual coding you can take a look at
@file{arch/mips/disasm.fs} to get some ideas on how to use data for both
the assembler and disassembler, avoiding redundancy and some potential
bugs.  You can also look at that file (and @pxref{Advanced does> usage
example}) to get ideas how to factor a disassembler.

Start with the disassembler, because it's easier to reuse data from the
disassembler for the assembler than the other way round.

For the assembler, take a look at @file{arch/alpha/asm.fs}, which shows
how simple it can be.




@c -------------------------------------------------------------
@node Carnal words, Passing Commands to the OS, Assembler and Code Words, Words
@section Carnal words

These words deal with the mechanics of Gforth (in Forth circles called
``carnal knowledge'' of a Forth system), but we consider them stable
enough to document them.

@menu
* Header fields::
* Header methods::
* Threading Words::
@end menu

@node Header fields, Header methods, Carnal words, Carnal words
@subsection Header fields
@cindex header fields

In Gforth 1.0 we switched to a new word header layout.  For a detailed
description, read: Bernd Paysan and M. Anton Ertl.
@cite{@uref{http://www.euroforth.org/ef19/papers/paysan.pdf,The new
Gforth header}}. In 35th EuroForth Conference, pages 5-20, 2019.
Since this paper was published, xt and nt have been changed to point
to the parameter field, like the body, but otherwise it is still
up-to-date.

This section explains just the data structure and the words used to
access it.  A header has the following fields:

@example
name
>f+c
>link
>cfa
>namehm
>body
@end example

Currently Gforth has the names shown above for
getting from the xt/nt/body to the field, but apart from the standard
@code{>body} they are not stable Gforth words.  Instead, we provide
access words.  Note that the documented access words have survived the
reorganization of the header layout.

Some of the words expect an nt, some expect an xt.  Given that both nt
and xt point to the body of a word, what is the difference?  For most
words, the xt and nt use the same header, and with nt=xt, they point
to the same place.  However, for a synonym (@pxref{Aliases}) there is
a difference; consider the example

@example
create x
synonym y x
synonym z y
@end example

In this case the nt of @code{z} points to the body of @code{z}, while
the xt of @code{z} points to the body of @code{x}.  Words defined with
@code{alias} or @code{forward} (@pxref{Forward}) also have different
nts and xts.

The name field is variable-length and is accessed with
@code{name>string} (@pxref{Name token}).

The @code{>f+c} field contains flags and the name length (count).  You
read the count with @code{name>string}, and the flags with

doc-compile-only?

The @code{>link} field contains a link to the previous word in the
same word list.  You can read it with @code{name>link} (@pxref{Name
token}).

The name, @code{>f+c} and @code{>link} fields are not present for
@code{noname} words, but @code{name>string} and @code{name>link} work
nevertheless, producing 0 0 and 0, respectively.

The @code{>cfa} field (aka code field) contains the code address used
for @code{execute}ing the word; you can read it with
@code{>code-address} and write it with @code{code-address!}
(@pxref{Threading Words}).

The @code{>namehm} field contains the address of the header methods
table, described below.  You access it by performing or accessing
header methods (@pxref{Header methods}).

The @code{>body} (aka parameter) field contains data or threaded code
specific to the word type; its length depends on the word type.  E.g.,
for a @code{constant} it contains a cell with the value of the
constant.  You can access it through @code{>body}
(@pxref{CREATE..DOES> details}), but this is only standard for words
you defined with @code{create}.

@node Header methods, Threading Words, Header fields, Carnal words
@subsection Header methods
@cindex header methods

The new Gforth word header is object-oriented and supports the
following methods (method selectors):

@example
.hm label method          overrider        field
          execute         set-execute      >cfa
opt:      opt-compile,    set-optimizer    >hmcompile,
to:       defer!          set-to           >hmto
defer@@:   defer@@          set-defer@@       >hmdefer@@
extra:                                     >hmextra
>int:     name>interpret  set->int         >hm>int
>comp:    name>compile    set->comp        >hm>comp
>string:  name>string     set-name>string  >hm>string
>link:    name>link       set-name>link    >hm>link
@end example

Many of these words are not stable Gforth words, but Gforth has stable
higher-level words that we mention below.

You can look at the header methods of a word with

doc-.hm

Overrider (setter) words change the method implementation for the most
recent definition.  Quotations or closures restore the previous most
recent definition when they are completed, so they are not considered
most recent, and you can do things like:

@example
: my2dup over over ;
[: drop ]] over over [[ ;] set-optimizer
@end example

The @code{execute} method is actually stored in the @code{>cfa} field
in the header rather than in the header-methods table for performance
reasons; also it is implemented through a native-code address, while
the other methods are implemented by calling an xt.  The high-level
way to set this method is

doc-set-execute

To get a code address for use with @code{set-execute}, you can use
words like @code{docol:} or @code{>code-address}, @xref{Threading
Words}.

As an alternative to @code{set-execute}, there is also
@code{set-does>} (@pxref{User-defined Defining Words}), which takes an
xt.

Moreover, there are the low-level @code{code-address!} and
@code{definer!} (@pxref{Threading Words}).

The @code{opt-compile,} method is what @code{compile,} does on most
Gforth engines (@code{gforth-itc} uses @code{,} instead).  You can
define a more efficient implementation of @code{compile,} for the
current word with @code{set-optimizer} (@pxref{User-defined
compile-comma}).  Note that the end result must be equivalent to
@code{postpone literal postpone execute}.

As an example of the use of @code{set-optimizer}, consider the
following definition of @code{constant}:

@example
: constant ( n "name" -- ; name: -- n )
  create ,
  ['] @@ set-does>
;

5 constant five
: foo five ; see foo
@end example

The Forth system does not know that the value of a constant must not
be changed, and just sees a @code{create}d word (which can be changed
with @code{>body}), and @code{foo} first pushes the body address of
@code{five} and then fetches from there.  With @code{set-optimizer}
the definition of @code{constant} can be optimized as follows:

@example
: constant ( n "name" -- ; name: -- n )
  create ,
  ['] @@ set-does>
  [: >body @@ postpone literal ;] set-optimizer
;
@end example

Now @code{foo} contains the literal 5 rather than a call to
@code{five}.

Note that @code{set-execute} and @code{set-does>} perform
@code{set-optimizer} themselves in order to ensure that @code{execute}
and @code{compile,} agree, so if you want to add your own optimizer,
you should add it afterwards.

The @code{defer!} (aka @code{(to)} method (@pxref{User-defined TO and
DEFER@@}) implements @code{defer!}  for words defined with
@code{defer} and similar words, but it is also the core of @code{to}.
The general stack effect of the @code{defer!}/@code{(to)} method is
@code{( val xt -- )}, where @i{xt} identifies the word stored into,
and @i{val} is the value (of appropriate type) stored there.

E.g., one can implement @code{fvalue} as follows:

@example
: fvalue-to ( r xt -- ) >body f! ;

: fvalue ( r -- )
  create f,
  ['] f@@ set-does>
  ['] fvalue-to set-to ;

5e fvalue foo
: bar foo 1e f+ to foo ;
see bar
@end example

You can improve the generated code with @code{set-optimizer}:

@example
: compile-fvalue-to ( xt-value-to -- )
  drop ]] >body f! [[ ;
  
: fvalue-to ( r xt -- ) >body f! ;
' compile-fvalue-to set-optimizer

: fvalue ( r -- )
  create f,
  ['] f@@ set-does>
  [: >body ]] literal f@@ [[ ;] set-optimizer
  ['] fvalue-to set-to ;

5e fvalue foo
: bar foo 1e f+ to foo ;
see bar
@end example

In practice Gforth has a few additional twists to implement, e.g.,
@code{+TO}.

@code{Set-defer@@} (@pxref{User-defined TO and DEFER@@}) allows to
implement variants of the @code{defer@@} (@pxref{Deferred Words})
method for @code{defer}-like words.

The @code{>hmextra} field is used for cases where additional data
needs to be stored in the header methods table.  In particular, it
stores the xt passed to @code{set-does>} (and @code{does>} calls
@code{set-does>}) and the code address behind @code{;abi-code}.

The methods above all consume an xt, not an nt, but the override words
work on the most recent definition.  This means that if you use, e.g.,
@code{set-optimizer} on a synonym, the effect will probably not be
what you intended: When @code{compile,}ing the xt of the word, the
@code{opt-compile,} implementation of the original word will be used,
not the freshly-set one of the synonym.

The following methods consume an nt.

The @code{name>interpret} method is implemented as noop for most
words, except synonyms and similar words.

doc-set->int

The @code{name>compile} method produces the compilation semantics of
the nt.  By changing it with @code{set->comp}, you can change the
compilation semantics, but it's not as simple as just pushing the xt
of the desired compilation semantics, because of the stack effect of
@code{name>compile}.  Generally you should avoid changing the
compilation semantics, and if you do, use a higher-level word like
@code{immediate} or @code{interpret/compile:}, @xref{Combined words}.

doc-set->comp
doc-immediate?

@code{Name>string} and @code{Name>link} are methods in order to make
it possible to eliminate the name, @code{>f+c} and @code{link} fields
from noname headers, but still produce meaningful results when using
these words.  You will typically not change the implementations of
these methods except with @code{noname}, but we still have

doc-set-name>string
doc-set-name>link

@node Threading Words,  , Header methods, Carnal words
@subsection Threading Words
@cindex threading words

@cindex code field
@cindex CFA
@cindex code address
The terminology used here stems from indirect threaded Forth systems; in
such a system, the XT of a word is represented by the CFA (code field
address) of a word; the CFA points to a cell that contains the code
address.  The code address is the address of some machine code that
performs the run-time action of invoking the word (e.g., the
@code{dovar:} routine pushes the address of the body of the word (a
variable) on the stack).

These words provide access to code fields, code addresses and other
threading stuff in Gforth. It more or less abstracts away the
differences between direct and indirect threading.

Up to and including Gforth 0.7, the code address (plus, for
@code{does>}-defined words, the address returned by @code{>does-code})
was sufficient to know the type of the word.  However, since
Gforth-1.0 the behaviour or at least implementation of words like
@code{compile,} and @code{name>compile} can be determined
independently as described in @ref{Header methods}.

The following words for create a code field and at the same time
initialize the header methods:

doc-hmcopy,
doc-docol,
doc-docon,
doc-dovar,
doc-douser,
doc-dodefer,
doc-dofield,
doc-dovalue,
doc-doabicode,

For @code{does>}-defined words, use @code{hmcopy,}.

Or you use a higher-level word like @code{create-from}
(@pxref{Creating from a prototype}).

The following words were designed before the introduction of header
methods, and are therefore not the best (and recommended) way to deal
with different word types in Gforth.

In an indirect threaded Forth, you can get the code address of @i{name}
with @code{' @i{name} @@}; in Gforth you can get it with @code{' @i{name}
>code-address}, independent of the threading method.

doc-threading-method
doc->code-address
doc-code-address!

The code addresses produced by various defining words are produced by
the following words:

doc-docol:
doc-docon:
doc-dovar:
doc-douser:
doc-dodefer:
doc-dofield:
doc-dovalue:
doc-dodoes:
doc-doabicode:

@cindex @code{does>}-code
For a word @i{X} defined with @code{set-does>}, the code address
points to @code{dodoes:}, and the @code{>hmextra} field of the header
methods contains the xt of the word that is called after pushing the
body addres of @i{X}.

If you want to know whether a word is a @code{DOES>}-defined word, and
what Forth code it executes, @code{>does-code} tells you that:

doc->does-code

You can use the resulting @i{xt2} with @code{set-does>} (preferred) to
change the latest word or with

doc-does-code!

to change an arbitrary word.

@cindex definer
The following two words generalize @code{>code-address},
@code{>does-code}, @code{code-address!}, and @code{does-code!}:

doc->definer
doc-definer!

@code{Code-address!}, @code{does-code!}, and @code{definer!} update
the @code{opt-compile,} method to a somewhat generic compiler for that
word type (in particular, primitives get the slow
@code{general-compile,} method rather than the primitive-specific
@code{peephole-compile,}).

@c -------------------------------------------------------------
@node Passing Commands to the OS, Keeping track of Time, Carnal words, Words
@section Passing Commands to the Operating System
@cindex operating system - passing commands
@cindex shell commands

Gforth allows you to pass an arbitrary string to the host operating
system shell (if such a thing exists) for execution.

doc-sh
doc-system
doc-sh-get
doc-$?
doc-getenv

@c -------------------------------------------------------------
@node Keeping track of Time, Miscellaneous Words, Passing Commands to the OS, Words
@section Keeping track of Time
@cindex time-related words

doc-ms
doc-ns
doc-time&date
doc->time&date&tz
doc-utime
doc-ntime
doc-cputime


@c -------------------------------------------------------------
@node Miscellaneous Words,  , Keeping track of Time, Words
@section Miscellaneous Words
@cindex miscellaneous words

@comment TODO find homes for these

This section lists the Standard Forth words that are not documented
elsewhere in this manual. Ultimately, they all need proper homes.

doc-quit

The following Standard Forth words are not currently supported by Gforth 
(@pxref{Standard conformance}):

@code{EDITOR} 
@code{EMIT?} 
@code{FORGET} 

@c ******************************************************************
@node Error messages, Tools, Words, Top
@chapter Error messages
@cindex error messages
@cindex backtrace

A typical Gforth error message looks like this:

@example
in file included from \evaluated string/:-1
in file included from ./yyy.fs:1
./xxx.fs:4: Invalid memory address
>>>bar<<<
Backtrace:
$400E664C @@
$400E6664 foo
@end example

The message identifying the error is @code{Invalid memory address}.  The
error happened when text-interpreting line 4 of the file
@file{./xxx.fs}. This line is given (it contains @code{bar}), and the
word on the line where the error happened, is pointed out (with
@code{>>>} and @code{<<<}).

The file containing the error was included in line 1 of @file{./yyy.fs},
and @file{yyy.fs} was included from a non-file (in this case, by giving
@file{yyy.fs} as command-line parameter to Gforth).

At the end of the error message you find a return stack dump that can be
interpreted as a backtrace (possibly empty). On top you find the top of
the return stack when the @code{throw} happened, and at the bottom you
find the return stack entry just above the return stack of the topmost
text interpreter.

To the right of most return stack entries you see a guess for the word
that pushed that return stack entry as its return address. This gives a
backtrace. In our case we see that @code{bar} called @code{foo}, and
@code{foo} called @code{@@} (and @code{@@} had an @emph{Invalid memory
address} exception).

Note that the backtrace is not perfect: We don't know which return stack
entries are return addresses (so we may get false positives); and in
some cases (e.g., for @code{abort"}) we cannot determine from the return
address the word that pushed the return address, so for some return
addresses you see no names in the return stack dump.

@cindex @code{catch} and backtraces
The return stack dump represents the return stack at the time when a
specific @code{throw} was executed.  In programs that make use of
@code{catch}, it is not necessarily clear which @code{throw} should be
used for the return stack dump (e.g., consider one @code{throw} that
indicates an error, which is caught, and during recovery another error
happens; which @code{throw} should be used for the stack dump?).
Gforth presents the return stack dump for the first @code{throw} after
the last executed (not returned-to) @code{catch} or @code{nothrow};
this works well in the usual case. To get the right backtrace, you
usually want to insert @code{nothrow} or @code{['] false catch 2drop}
after a @code{catch} if the error is not rethrown.

@cindex @code{gforth-fast} and backtraces
@cindex @code{gforth-fast}, difference from @code{gforth}
@cindex backtraces with @code{gforth-fast}
@cindex return stack dump with @code{gforth-fast}
@code{Gforth} is able to do a return stack dump for throws generated
from primitives (e.g., invalid memory address, stack empty etc.);
@code{gforth-fast} is only able to do a return stack dump from a
directly called @code{throw} (including @code{abort} etc.).  Given an
exception caused by a primitive in @code{gforth-fast}, you will
typically see no return stack dump at all; however, if the exception is
caught by @code{catch} (e.g., for restoring some state), and then
@code{throw}n again, the return stack dump will be for the first such
@code{throw}.

@code{gforth-fast} also does not attempt to differentiate between
division by zero and division overflow, because that costs time in every
division.

@c ******************************************************************
@node Tools, Standard conformance, Error messages, Top
@chapter Tools

@menu
* Standard Report::             Report the words used, sorted by wordset.
* Stack depth changes::         Where does this stack item come from?
@end menu

See also @ref{Emacs and Gforth}.

@node Standard Report, Stack depth changes, Tools, Tools
@section @file{ans-report.fs}: Report the words used, sorted by wordset
@cindex @file{ans-report.fs}
@cindex report the words used in your program
@cindex words used in your program

If you want to label a Forth program as Standard Program, you must
document which wordsets the program uses.

The @file{ans-report.fs} tool makes it easy for you to determine which
words from which wordset and which non-standard words your application
uses. You simply have to include @file{ans-report.fs} before loading the
program you want to check. After loading your program, you can get the
report with @code{print-ans-report}. A typical use is to run this as
batch job like this:
@example
gforth ans-report.fs myprog.fs -e "print-ans-report bye"
@end example

The output looks like this (for @file{compat/control.fs}):
@example
The program uses the following words
from CORE :
: POSTPONE THEN ; immediate ?dup IF 0= 
from BLOCK-EXT :
\ 
from FILE :
( 
@end example

@file{ans-report.fs} reports both Forth-94 and Forth-2012 wordsets.
For words that are in both standards, it reports the wordset without
suffix (e.g., @code{CORE-EXT}).  For Forth-2012-only words, it reports
the wordset with a @code{-2012} suffix (e.g., @code{CORE-EXT-2012});
and likewise for the words that are Forth-94-only (i.e., that have
been removed in Forth-2012).

@subsection Caveats

Note that @file{ans-report.fs} just checks which words are used, not
whether they are used in a standard-conforming way!

Some words are defined in several wordsets in the
standard. @file{ans-report.fs} reports them for only one of the
wordsets, and not necessarily the one you expect. It depends on usage
which wordset is the right one to specify. E.g., if you only use the
compilation semantics of @code{S"}, it is a Core word; if you also use
its interpretation semantics, it is a File word.


@node Stack depth changes,  , Standard Report, Tools
@section Stack depth changes during interpretation
@cindex @file{depth-changes.fs}
@cindex depth changes during interpretation
@cindex stack depth changes during interpretation
@cindex items on the stack after interpretation

Sometimes you notice that, after loading a file, there are items left
on the stack.  The tool @file{depth-changes.fs} helps you find out
quickly where in the file these stack items are coming from.

The simplest way of using @file{depth-changes.fs} is to include it
before the file(s) you want to check, e.g.:

@example
gforth depth-changes.fs my-file.fs
@end example

This will compare the stack depths of the data and FP stack at every
empty line (in interpretation state) against these depths at the last
empty line (in interpretation state).  If the depths are not equal,
the position in the file and the stack contents are printed with
@code{~~} (@pxref{Debugging}).  This indicates that a stack depth
change has occured in the paragraph of non-empty lines before the
indicated line.  It is a good idea to leave an empty line at the end
of the file, so the last paragraph is checked, too.

Checking only at empty lines usually works well, but sometimes you
have big blocks of non-empty lines (e.g., when building a big table),
and you want to know where in this block the stack depth changed.  You
can check all interpreted lines with

@example
gforth depth-changes.fs -e "' all-lines is depth-changes-filter" my-file.fs
@end example

This checks the stack depth at every end-of-line.  So the depth change
occured in the line reported by the @code{~~} (not in the line
before).

Note that, while this offers better accuracy in indicating where the
stack depth changes, it will often report many intentional stack depth
changes (e.g., when an interpreted computation stretches across
several lines).  You can suppress the checking of some lines by
putting backslashes at the end of these lines (not followed by white
space), and using

@example
gforth depth-changes.fs -e "' most-lines is depth-changes-filter" my-file.fs
@end example

@c ******************************************************************
@node Standard conformance, Standard vs Extensions, Tools, Top
@chapter Standard conformance
@cindex Standard conformance of Gforth

To the best of our knowledge, Gforth is a

ANS Forth System and a Forth-2012 System
@itemize @bullet
@item providing the Core Extensions word set
@item providing the Block word set
@item providing the Block Extensions word set
@item providing the Double-Number word set
@item providing the Double-Number Extensions word set
@item providing the Exception word set
@item providing the Exception Extensions word set
@item providing the Facility word set
@item providing the Facility Extensions word set, except @code{EMIT?}
@item providing the File Access word set
@item providing the File Access Extensions word set
@item providing the Floating-Point word set
@item providing the Floating-Point Extensions word set
@item providing the Locals word set
@item providing the Locals Extensions word set
@item providing the Memory-Allocation word set
@item providing the Memory-Allocation Extensions word set
@item providing the Programming-Tools word set
@item providing the Programming-Tools Extensions word set, except @code{EDITOR} and @code{FORGET}
@item providing the Search-Order word set
@item providing the Search-Order Extensions word set
@item providing the String word set
@item providing the String Extensions word set
@item providing the Extended-Character wordset
@end itemize

Gforth has the following environmental restrictions:

@cindex environmental restrictions
@itemize @bullet
@item
While processing the OS command line, if an exception is not caught,
Gforth exits with a non-zero exit code instead of performing QUIT.

@item
When an @code{throw} is performed after a @code{query}, Gforth does not
always restore the input source specification in effect at the
corresponding catch.

@end itemize


@cindex system documentation
In addition, Standard Forth systems are required to document certain
implementation choices. This chapter tries to meet these requirements
for the Forth-94 standard.  For the Forth-2012 standard, we decided to
produce the additional documentation only if there is demand.  So if
you are really missing this documentation, please let us know.

In many cases, the following documentation gives a way to ask the
system for the information instead of providing the information
directly, in particular, if the information depends on the processor,
the operating system or the installation options chosen, or if they
are likely to change during the maintenance of Gforth.

@comment The framework for the rest has been taken from pfe.

@menu
* The Core Words::
* The optional Block word set::
* The optional Double Number word set::
* The optional Exception word set::
* The optional Facility word set::
* The optional File-Access word set::
* The optional Floating-Point word set::
* The optional Locals word set::
* The optional Memory-Allocation word set::
* The optional Programming-Tools word set::
* The optional Search-Order word set::
@end menu


@c =====================================================================
@node The Core Words, The optional Block word set, Standard conformance, Standard conformance
@comment  node-name,  next,  previous,  up
@section The Core Words
@c =====================================================================
@cindex core words, system documentation
@cindex system documentation, core words

@menu
* core-idef::                   Implementation Defined Options                   
* core-ambcond::                Ambiguous Conditions                
* core-other::                  Other System Documentation                  
@end menu

@c ---------------------------------------------------------------------
@node core-idef, core-ambcond, The Core Words, The Core Words
@subsection Implementation Defined Options
@c ---------------------------------------------------------------------
@cindex core words, implementation-defined options
@cindex implementation-defined options, core words


@table @i
@item (Cell) aligned addresses:
@cindex cell-aligned addresses
@cindex aligned addresses
processor-dependent. Gforth's alignment words perform natural alignment
(e.g., an address aligned for a datum of size 8 is divisible by
8). Unaligned accesses usually result in a @code{-23 THROW}.

@item @code{EMIT} and non-graphic characters:
@cindex @code{EMIT} and non-graphic characters
@cindex non-graphic characters and @code{EMIT}
The character is output using the C library function (actually, macro)
@code{putc}.

@item character editing of @code{ACCEPT} and @code{EXPECT}:
@cindex character editing of @code{ACCEPT} and @code{EXPECT}
@cindex editing in @code{ACCEPT} and @code{EXPECT}
@cindex @code{ACCEPT}, editing
@cindex @code{EXPECT}, editing
This is modeled on the GNU readline library (@pxref{Readline
Interaction, , Command Line Editing, readline, The GNU Readline
Library}) with Emacs-like key bindings. @kbd{Tab} deviates a little by
producing a full word completion every time you type it (instead of
producing the common prefix of all completions). @xref{Command-line editing}.

@item character set:
@cindex character set
The character set of your computer and display device. Gforth is
8-bit-clean (but some other component in your system may make trouble).

@item Character-aligned address requirements:
@cindex character-aligned address requirements
installation-dependent. Currently a character is represented by a C
@code{unsigned char}; in the future we might switch to @code{wchar_t}
(Comments on that requested).

@item character-set extensions and matching of names:
@cindex character-set extensions and matching of names
@cindex case-sensitivity for name lookup
@cindex name lookup, case-sensitivity
@cindex locale and case-sensitivity
Any character except the ASCII NUL character can be used in a
name. Matching is case-insensitive (except in @code{TABLE}s). The
matching is performed using the C library function @code{strncasecmp}, whose
function is probably influenced by the locale. E.g., the @code{C} locale
does not know about accents and umlauts, so they are matched
case-sensitively in that locale. For portability reasons it is best to
write programs such that they work in the @code{C} locale. Then one can
use libraries written by a Polish programmer (who might use words
containing ISO Latin-2 encoded characters) and by a French programmer
(ISO Latin-1) in the same program (of course, @code{WORDS} will produce
funny results for some of the words (which ones, depends on the font you
are using)). Also, the locale you prefer may not be available in other
operating systems. Hopefully, Unicode will solve these problems one day.

@item conditions under which control characters match a space delimiter:
@cindex space delimiters
@cindex control characters as delimiters
If @code{word} is called with the space character as a delimiter, all
white-space characters (as identified by the C macro @code{isspace()})
are delimiters. @code{Parse}, on the other hand, treats space like other
delimiters.  @code{Parse-name}, which is used by the outer
interpreter (aka text interpreter) by default, treats all white-space
characters as delimiters.

@item format of the control-flow stack:
@cindex control-flow stack, format
The data stack is used as control-flow stack. The size of a control-flow
stack item in cells is given by the constant @code{cs-item-size}. At the
time of this writing, an item consists of a (pointer to a) locals list
(third), an address in the code (second), and a tag for identifying the
item (TOS). The following tags are used: @code{defstart},
@code{live-orig}, @code{dead-orig}, @code{dest}, @code{do-dest},
@code{scopestart}.

@item conversion of digits > 35
@cindex digits > 35
The characters @code{[\]^_'} are the digits with the decimal value
36@minus{}41. There is no way to input many of the larger digits.

@item display after input terminates in @code{ACCEPT} and @code{EXPECT}:
@cindex @code{EXPECT}, display after end of input
@cindex @code{ACCEPT}, display after end of input
The cursor is moved to the end of the entered string. If the input is
terminated using the @kbd{Return} key, a space is typed.

@item exception abort sequence of @code{ABORT"}:
@cindex exception abort sequence of @code{ABORT"}
@cindex @code{ABORT"}, exception abort sequence
The error string is stored into the variable @code{"error} and a
@code{-2 throw} is performed.

@item input line terminator:
@cindex input line terminator
@cindex line terminator on input
@cindex newline character on input
For interactive input, @kbd{C-m} (CR) and @kbd{C-j} (LF) terminate
lines. One of these characters is typically produced when you type the
@kbd{Enter} or @kbd{Return} key.

@item maximum size of a counted string:
@cindex maximum size of a counted string
@cindex counted string, maximum size
@code{s" /counted-string" environment? drop .}. Currently 255 characters
on all platforms, but this may change.

@item maximum size of a parsed string:
@cindex maximum size of a parsed string
@cindex parsed string, maximum size
Given by the constant @code{/line}. Currently 255 characters.

@item maximum size of a definition name, in characters:
@cindex maximum size of a definition name, in characters
@cindex name, maximum length
MAXU/8

@item maximum string length for @code{ENVIRONMENT?}, in characters:
@cindex maximum string length for @code{ENVIRONMENT?}, in characters
@cindex @code{ENVIRONMENT?} string length, maximum
MAXU/8

@item method of selecting the user input device:
@cindex user input device, method of selecting
The user input device is the standard input. There is currently no way to
change it from within Gforth. However, the input can typically be
redirected in the command line that starts Gforth.

@item method of selecting the user output device:
@cindex user output device, method of selecting
@code{EMIT} and @code{TYPE} output to the file-id stored in the value
@code{outfile-id} (@code{stdout} by default). Gforth uses unbuffered
output when the user output device is a terminal, otherwise the output
is buffered.

@item methods of dictionary compilation:
What are we expected to document here?

@item number of bits in one address unit:
@cindex number of bits in one address unit
@cindex address unit, size in bits
@code{s" address-units-bits" environment? drop .}. 8 in all current
platforms.

@item number representation and arithmetic:
@cindex number representation and arithmetic
Processor-dependent. Binary two's complement on all current platforms.

@item ranges for integer types:
@cindex ranges for integer types
@cindex integer types, ranges
Installation-dependent. Make environmental queries for @code{MAX-N},
@code{MAX-U}, @code{MAX-D} and @code{MAX-UD}. The lower bounds for
unsigned (and positive) types is 0. The lower bound for signed types on
two's complement and one's complement machines machines can be computed
by adding 1 to the upper bound.

@item read-only data space regions:
@cindex read-only data space regions
@cindex data-space, read-only regions
The whole Forth data space is writable.

@item size of buffer at @code{WORD}:
@cindex size of buffer at @code{WORD}
@cindex @code{WORD} buffer size
@code{PAD HERE - .}. 104 characters on 32-bit machines. The buffer is
shared with the pictured numeric output string. If overwriting
@code{PAD} is acceptable, it is as large as the remaining dictionary
space, although only as much can be sensibly used as fits in a counted
string.

@item size of one cell in address units:
@cindex cell size
@code{1 cells .}.

@item size of one character in address units:
@cindex char size
@code{1 chars .}. 1 on all current platforms.

@item size of the keyboard terminal buffer:
@cindex size of the keyboard terminal buffer
@cindex terminal buffer, size
Varies. You can determine the size at a specific time using @code{lp@@
tib - .}. It is shared with the locals stack and TIBs of files that
include the current file. You can change the amount of space for TIBs
and locals stack at Gforth startup with the command line option
@code{-l}.

@item size of the pictured numeric output buffer:
@cindex size of the pictured numeric output buffer
@cindex pictured numeric output buffer, size
@code{PAD HERE - .}. 104 characters on 32-bit machines. The buffer is
shared with @code{WORD}.

@item size of the scratch area returned by @code{PAD}:
@cindex size of the scratch area returned by @code{PAD}
@cindex @code{PAD} size
The remainder of dictionary space. @code{unused pad here - - .}.

@item system case-sensitivity characteristics:
@cindex case-sensitivity characteristics
Dictionary searches are case-insensitive (except in
@code{TABLE}s). However, as explained above under @i{character-set
extensions}, the matching for non-ASCII characters is determined by the
locale you are using. In the default @code{C} locale all non-ASCII
characters are matched case-sensitively.

@item system prompt:
@cindex system prompt
@cindex prompt
@code{ ok} in interpret state, @code{ compiled} in compile state.

@item division rounding:
@cindex division rounding
The ordinary division words @code{/ mod /mod */ */mod} perform floored
division (with the default installation of Gforth).  You can check
this with @code{s" floored" environment? drop .}.  If you write
programs that need a specific division rounding, best use
@code{fm/mod} or @code{sm/rem} for portability.

@item values of @code{STATE} when true:
@cindex @code{STATE} values
-1.

@item values returned after arithmetic overflow:
On two's complement machines, arithmetic is performed modulo
2**bits-per-cell for single arithmetic and 4**bits-per-cell for double
arithmetic (with appropriate mapping for signed types). Division by
zero typically results in a @code{-55 throw} (Floating-point
unidentified fault) or @code{-10 throw} (divide by zero).  Integer
division overflow can result in these throws, or in @code{-11 throw};
in @code{gforth-fast} division overflow and divide by zero may also
result in returning bogus results without producing an exception.

@item whether the current definition can be found after @t{DOES>}:
@cindex @t{DOES>}, visibility of current definition
No.

@end table

@c ---------------------------------------------------------------------
@node core-ambcond, core-other, core-idef, The Core Words
@subsection Ambiguous conditions
@c ---------------------------------------------------------------------
@cindex core words, ambiguous conditions
@cindex ambiguous conditions, core words

@table @i

@item a name is neither a word nor a number:
@cindex name not found
@cindex undefined word
@code{-13 throw} (Undefined word).

@item a definition name exceeds the maximum length allowed:
@cindex word name too long
@code{-19 throw} (Word name too long)

@item addressing a region not inside the various data spaces of the forth system:
@cindex Invalid memory address
The stacks, code space and header space are accessible. Machine code space is
typically readable. Accessing other addresses gives results dependent on
the operating system. On decent systems: @code{-9 throw} (Invalid memory
address).

@item argument type incompatible with parameter:
@cindex argument type mismatch
This is usually not caught. Some words perform checks, e.g., the control
flow words, and issue a @code{ABORT"} or @code{-12 THROW} (Argument type
mismatch).

@item attempting to obtain the execution token of a word with undefined execution semantics:
@cindex compile-only warning, for @code{'} etc.
@cindex execution token of words with undefined execution semantics
The execution token represents the interpretation semantics of the
word.  Gforth defines interpretation semantics for all words; for
words where the standard does not define interpretation semantics, but
defines the execution semantics (except @code{LEAVE}), the
interpretation semantics are to perform the execution semantics.  For
words where the standard defines no interprtation semantics, but
defined compilation semantics (plus @code{LEAVE}), the interpretation
semantics are to perform the compilation semantics.  Some words are
marked as compile-only, and @code{'} gives a warning for these words.

@item dividing by zero:
@cindex dividing by zero
@cindex floating point unidentified fault, integer division
On some platforms, this produces a @code{-10 throw} (Division by
zero); on other systems, this typically results in a @code{-55 throw}
(Floating-point unidentified fault).

@item insufficient data stack or return stack space:
@cindex insufficient data stack or return stack space
@cindex stack overflow
@cindex address alignment exception, stack overflow
@cindex Invalid memory address, stack overflow
Depending on the operating system, the installation, and the invocation
of Gforth, this is either checked by the memory management hardware, or
it is not checked. If it is checked, you typically get a @code{-3 throw}
(Stack overflow), @code{-5 throw} (Return stack overflow), or @code{-9
throw} (Invalid memory address) (depending on the platform and how you
achieved the overflow) as soon as the overflow happens. If it is not
checked, overflows typically result in mysterious illegal memory
accesses, producing @code{-9 throw} (Invalid memory address) or
@code{-23 throw} (Address alignment exception); they might also destroy
the internal data structure of @code{ALLOCATE} and friends, resulting in
various errors in these words.

@item insufficient space for loop control parameters:
@cindex insufficient space for loop control parameters
Like other return stack overflows.

@item insufficient space in the dictionary:
@cindex insufficient space in the dictionary
@cindex dictionary overflow
If you try to allot (either directly with @code{allot}, or indirectly
with @code{,}, @code{create} etc.) more memory than available in the
dictionary, you get a @code{-8 throw} (Dictionary overflow). If you try
to access memory beyond the end of the dictionary, the results are
similar to stack overflows.

@item interpreting a word with undefined interpretation semantics:
@cindex interpreting a word with undefined interpretation semantics
@cindex Interpreting a compile-only word
Gforth defines interpretation semantics for all words; for words where
the standard defines execution semantics (except @code{LEAVE}), the
interpretation semantics are to perform the execution semantics.  For
words where the standard defines no interprtation semantics, but
defined compilation semantics (plus @code{LEAVE}), the interpretation
semantics are to perform the compilation semantics.  Some words are
marked as compile-only, and text-interpreting them gives a warning.

@item modifying the contents of the input buffer or a string literal:
@cindex modifying the contents of the input buffer or a string literal
These are located in writable memory and can be modified.

@item overflow of the pictured numeric output string:
@cindex overflow of the pictured numeric output string
@cindex pictured numeric output string, overflow
@code{-17 throw} (Pictured numeric ouput string overflow).

@item parsed string overflow:
@cindex parsed string overflow
@code{PARSE} cannot overflow. @code{WORD} does not check for overflow.

@item producing a result out of range:
@cindex result out of range
On two's complement machines, arithmetic is performed modulo
2**bits-per-cell for single arithmetic and 4**bits-per-cell for double
arithmetic (with appropriate mapping for signed types). Division by
zero typically results in a @code{-10 throw} (divide by zero) or
@code{-55 throw} (floating point unidentified fault). Overflow on
division may result in these errors or in @code{-11 throw} (result out
of range).  @code{Gforth-fast} may silently produce bogus results on
division overflow or division by zero.  @code{Convert} and
@code{>number} currently overflow silently.

@item reading from an empty data or return stack:
@cindex stack empty
@cindex stack underflow
@cindex return stack underflow
The data stack is checked by the outer (aka text) interpreter after
every word executed. If it has underflowed, a @code{-4 throw} (Stack
underflow) is performed. Apart from that, stacks may be checked or not,
depending on operating system, installation, and invocation. If they are
caught by a check, they typically result in @code{-4 throw} (Stack
underflow), @code{-6 throw} (Return stack underflow) or @code{-9 throw}
(Invalid memory address), depending on the platform and which stack
underflows and by how much. Note that even if the system uses checking
(through the MMU), your program may have to underflow by a significant
number of stack items to trigger the reaction (the reason for this is
that the MMU, and therefore the checking, works with a page-size
granularity).  If there is no checking, the symptoms resulting from an
underflow are similar to those from an overflow.  Unbalanced return
stack errors can result in a variety of symptoms, including @code{-9 throw}
(Invalid memory address) and Illegal Instruction (typically @code{-260
throw}).

@item unexpected end of the input buffer, resulting in an attempt to use a zero-length string as a name:
@cindex unexpected end of the input buffer
@cindex zero-length string as a name
@cindex Attempt to use zero-length string as a name
@code{Create} and its descendants perform a @code{-16 throw} (Attempt to
use zero-length string as a name). Words like @code{'} probably will not
find what they search. Note that it is possible to create zero-length
names with @code{nextname} (should it not?).

@item @code{>IN} greater than input buffer:
@cindex @code{>IN} greater than input buffer
The next invocation of a parsing word returns a string with length 0.

@item @code{RECURSE} appears after @code{DOES>}:
@cindex @code{RECURSE} appears after @code{DOES>}
Compiles a recursive call to the code after @code{DOES>}.

@item argument input source different than current input source for @code{RESTORE-INPUT}:
@cindex argument input source different than current input source for @code{RESTORE-INPUT}
@cindex argument type mismatch, @code{RESTORE-INPUT}
@cindex @code{RESTORE-INPUT}, Argument type mismatch
@code{-12 THROW}. Note that, once an input file is closed (e.g., because
the end of the file was reached), its source-id may be
reused. Therefore, restoring an input source specification referencing a
closed file may lead to unpredictable results instead of a @code{-12
THROW}.

In the future, Gforth may be able to restore input source specifications
from other than the current input source.

@item data space containing definitions gets de-allocated:
@cindex data space containing definitions gets de-allocated
Deallocation with @code{allot} is not checked. This typically results in
memory access faults or execution of illegal instructions.

@item data space read/write with incorrect alignment:
@cindex data space read/write with incorrect alignment
@cindex alignment faults
@cindex address alignment exception
Processor-dependent. Typically results in a @code{-23 throw} (Address
alignment exception). Under Linux-Intel on a 486 or later processor with
alignment turned on, incorrect alignment results in a @code{-9 throw}
(Invalid memory address). There are reportedly some processors with
alignment restrictions that do not report violations.

@item data space pointer not properly aligned, @code{,}, @code{C,}:
@cindex data space pointer not properly aligned, @code{,}, @code{C,}
Like other alignment errors.

@item less than u+2 stack items (@code{PICK} and @code{ROLL}):
Like other stack underflows.

@item loop control parameters not available:
@cindex loop control parameters not available
Not checked. The counted loop words simply assume that the top of return
stack items are loop control parameters and behave accordingly.

@item most recent definition does not have a name (@code{IMMEDIATE}):
@cindex most recent definition does not have a name (@code{IMMEDIATE})
@cindex last word was headerless
@code{abort" last word was headerless"}.

@item name not defined by @code{VALUE} used by @code{TO}:
@cindex name not defined by @code{VALUE} used by @code{TO}
@cindex @code{TO} on non-@code{VALUE}s
@cindex Invalid name argument, @code{TO}
@code{-32 throw} (Invalid name argument) (unless name is a local or was
defined by @code{CONSTANT}; in the latter case it just changes the constant).

@item name not found (@code{'}, @code{POSTPONE}, @code{[']}, @code{[COMPILE]}):
@cindex name not found (@code{'}, @code{POSTPONE}, @code{[']}, @code{[COMPILE]})
@cindex undefined word, @code{'}, @code{POSTPONE}, @code{[']}, @code{[COMPILE]}
@code{-13 throw} (Undefined word)

@item parameters are not of the same type (@code{DO}, @code{?DO}, @code{WITHIN}):
@cindex parameters are not of the same type (@code{DO}, @code{?DO}, @code{WITHIN})
Gforth behaves as if they were of the same type. I.e., you can predict
the behaviour by interpreting all parameters as, e.g., signed.

@item @code{POSTPONE} or @code{[COMPILE]} applied to @code{TO}:
@cindex @code{POSTPONE} or @code{[COMPILE]} applied to @code{TO}
Assume @code{: X POSTPONE TO ; IMMEDIATE}. @code{X} performs the
compilation semantics of @code{TO}.

@item String longer than a counted string returned by @code{WORD}:
@cindex string longer than a counted string returned by @code{WORD}
@cindex @code{WORD}, string overflow
Not checked. The string will be ok, but the count will, of course,
contain only the least significant bits of the length.

@item u greater than or equal to the number of bits in a cell (@code{LSHIFT}, @code{RSHIFT}):
@cindex @code{LSHIFT}, large shift counts
@cindex @code{RSHIFT}, large shift counts
Processor-dependent. Typical behaviours are returning 0 and using only
the low bits of the shift count.

@item word not defined via @code{CREATE}:
@cindex @code{>BODY} of non-@code{CREATE}d words
@code{>BODY} produces the PFA of the word no matter how it was defined.

@cindex @code{DOES>} of non-@code{CREATE}d words
@code{DOES>} changes the execution semantics of the last defined word no
matter how it was defined. E.g., @code{CONSTANT DOES>} is equivalent to
@code{CREATE , DOES>}.

@item words improperly used outside @code{<#} and @code{#>}:
Not checked. As usual, you can expect memory faults.

@end table


@c ---------------------------------------------------------------------
@node core-other,  , core-ambcond, The Core Words
@subsection Other system documentation
@c ---------------------------------------------------------------------
@cindex other system documentation, core words
@cindex core words, other system documentation

@table @i
@item nonstandard words using @code{PAD}:
@cindex @code{PAD} use by nonstandard words
None.

@item operator's terminal facilities available:
@cindex operator's terminal facilities available
After processing the OS's command line, Gforth goes into interactive mode,
and you can give commands to Gforth interactively. The actual facilities
available depend on how you invoke Gforth.

@item program data space available:
@cindex program data space available
@cindex data space available
@code{UNUSED .} gives the remaining dictionary space. The total
dictionary space can be specified with the @code{-m} switch
(@pxref{Invoking Gforth}) when Gforth starts up.

@item return stack space available:
@cindex return stack space available
You can compute the total return stack space in cells with
@code{s" RETURN-STACK-CELLS" environment? drop .}. You can specify it at
startup time with the @code{-r} switch (@pxref{Invoking Gforth}).

@item stack space available:
@cindex stack space available
You can compute the total data stack space in cells with
@code{s" STACK-CELLS" environment? drop .}. You can specify it at
startup time with the @code{-d} switch (@pxref{Invoking Gforth}).

@item system dictionary space required, in address units:
@cindex system dictionary space required, in address units
Type @code{here forthstart - .} after startup. At the time of this
writing, this gives 80080 (bytes) on a 32-bit system.
@end table


@c =====================================================================
@node The optional Block word set, The optional Double Number word set, The Core Words, Standard conformance
@section The optional Block word set
@c =====================================================================
@cindex system documentation, block words
@cindex block words, system documentation

@menu
* block-idef::                  Implementation Defined Options
* block-ambcond::               Ambiguous Conditions               
* block-other::                 Other System Documentation                 
@end menu


@c ---------------------------------------------------------------------
@node block-idef, block-ambcond, The optional Block word set, The optional Block word set
@subsection Implementation Defined Options
@c ---------------------------------------------------------------------
@cindex implementation-defined options, block words
@cindex block words, implementation-defined options

@table @i
@item the format for display by @code{LIST}:
@cindex @code{LIST} display format
First the screen number is displayed, then 16 lines of 64 characters,
each line preceded by the line number.

@item the length of a line affected by @code{\}:
@cindex length of a line affected by @code{\}
@cindex @code{\}, line length in blocks
64 characters.
@end table


@c ---------------------------------------------------------------------
@node block-ambcond, block-other, block-idef, The optional Block word set
@subsection Ambiguous conditions
@c ---------------------------------------------------------------------
@cindex block words, ambiguous conditions
@cindex ambiguous conditions, block words

@table @i
@item correct block read was not possible:
@cindex block read not possible
Typically results in a @code{throw} of some OS-derived value (between
-512 and -2048). If the blocks file was just not long enough, blanks are
supplied for the missing portion.

@item I/O exception in block transfer:
@cindex I/O exception in block transfer
@cindex block transfer, I/O exception
Typically results in a @code{throw} of some OS-derived value (between
-512 and -2048).

@item invalid block number:
@cindex invalid block number
@cindex block number invalid
@code{-35 throw} (Invalid block number)

@item a program directly alters the contents of @code{BLK}:
@cindex @code{BLK}, altering @code{BLK}
The input stream is switched to that other block, at the same
position. If the storing to @code{BLK} happens when interpreting
non-block input, the system will get quite confused when the block ends.

@item no current block buffer for @code{UPDATE}:
@cindex @code{UPDATE}, no current block buffer
@code{UPDATE} has no effect.

@end table

@c ---------------------------------------------------------------------
@node block-other,  , block-ambcond, The optional Block word set
@subsection Other system documentation
@c ---------------------------------------------------------------------
@cindex other system documentation, block words
@cindex block words, other system documentation

@table @i
@item any restrictions a multiprogramming system places on the use of buffer addresses:
No restrictions (yet).

@item the number of blocks available for source and data:
depends on your disk space.

@end table


@c =====================================================================
@node The optional Double Number word set, The optional Exception word set, The optional Block word set, Standard conformance
@section The optional Double Number word set
@c =====================================================================
@cindex system documentation, double words
@cindex double words, system documentation

@menu
* double-ambcond::              Ambiguous Conditions              
@end menu


@c ---------------------------------------------------------------------
@node double-ambcond,  , The optional Double Number word set, The optional Double Number word set
@subsection Ambiguous conditions
@c ---------------------------------------------------------------------
@cindex double words, ambiguous conditions
@cindex ambiguous conditions, double words

@table @i
@item @i{d} outside of range of @i{n} in @code{D>S}:
@cindex @code{D>S}, @i{d} out of range of @i{n} 
The least significant cell of @i{d} is produced.

@end table


@c =====================================================================
@node The optional Exception word set, The optional Facility word set, The optional Double Number word set, Standard conformance
@section The optional Exception word set
@c =====================================================================
@cindex system documentation, exception words
@cindex exception words, system documentation

@menu
* exception-idef::              Implementation Defined Options              
@end menu


@c ---------------------------------------------------------------------
@node exception-idef,  , The optional Exception word set, The optional Exception word set
@subsection Implementation Defined Options
@c ---------------------------------------------------------------------
@cindex implementation-defined options, exception words
@cindex exception words, implementation-defined options

@table @i
@item @code{THROW}-codes used in the system:
@cindex @code{THROW}-codes used in the system
The codes -256@minus{}-511 are used for reporting signals. The mapping
from OS signal numbers to throw codes is -256@minus{}@i{signal}. The
codes -512@minus{}-2047 are used for OS errors (for file and memory
allocation operations). The mapping from OS error numbers to throw codes
is -512@minus{}@code{errno}. One side effect of this mapping is that
undefined OS errors produce a message with a strange number; e.g.,
@code{-1000 THROW} results in @code{Unknown error 488} on my system.
@end table

@c =====================================================================
@node The optional Facility word set, The optional File-Access word set, The optional Exception word set, Standard conformance
@section The optional Facility word set
@c =====================================================================
@cindex system documentation, facility words
@cindex facility words, system documentation

@menu
* facility-idef::               Implementation Defined Options               
* facility-ambcond::            Ambiguous Conditions            
@end menu


@c ---------------------------------------------------------------------
@node facility-idef, facility-ambcond, The optional Facility word set, The optional Facility word set
@subsection Implementation Defined Options
@c ---------------------------------------------------------------------
@cindex implementation-defined options, facility words
@cindex facility words, implementation-defined options

@table @i
@item encoding of keyboard events (@code{EKEY}):
@cindex keyboard events, encoding in @code{EKEY}
@cindex @code{EKEY}, encoding of keyboard events
Keys corresponding to ASCII characters are encoded as ASCII
characters.  Other keys are encoded with the constants @code{k-left},
@code{k-right}, @code{k-up}, @code{k-down}, @code{k-home},
@code{k-end}, @code{k1}, @code{k2}, @code{k3}, @code{k4}, @code{k5},
@code{k6}, @code{k7}, @code{k8}, @code{k9}, @code{k10}, @code{k11},
@code{k12}, @code{k-winch}, @code{k-eof}.


@item duration of a system clock tick:
@cindex duration of a system clock tick
@cindex clock tick duration
System dependent. With respect to @code{MS}, the time is specified in
microseconds. How well the OS and the hardware implement this, is
another question.

@item repeatability to be expected from the execution of @code{MS}:
@cindex repeatability to be expected from the execution of @code{MS}
@cindex @code{MS}, repeatability to be expected
System dependent. On Unix, a lot depends on load. If the system is
lightly loaded, and the delay is short enough that Gforth does not get
swapped out, the performance should be acceptable. Under MS-DOS and
other single-tasking systems, it should be good.

@end table


@c ---------------------------------------------------------------------
@node facility-ambcond,  , facility-idef, The optional Facility word set
@subsection Ambiguous conditions
@c ---------------------------------------------------------------------
@cindex facility words, ambiguous conditions
@cindex ambiguous conditions, facility words

@table @i
@item @code{AT-XY} can't be performed on user output device:
@cindex @code{AT-XY} can't be performed on user output device
Largely terminal dependent. No range checks are done on the arguments.
No errors are reported. You may see some garbage appearing, you may see
simply nothing happen.

@end table


@c =====================================================================
@node The optional File-Access word set, The optional Floating-Point word set, The optional Facility word set, Standard conformance
@section The optional File-Access word set
@c =====================================================================
@cindex system documentation, file words
@cindex file words, system documentation

@menu
* file-idef::                   Implementation Defined Options
* file-ambcond::                Ambiguous Conditions                
@end menu

@c ---------------------------------------------------------------------
@node file-idef, file-ambcond, The optional File-Access word set, The optional File-Access word set
@subsection Implementation Defined Options
@c ---------------------------------------------------------------------
@cindex implementation-defined options, file words
@cindex file words, implementation-defined options

@table @i
@item file access methods used:
@cindex file access methods used
@code{R/O}, @code{R/W} and @code{BIN} work as you would
expect. @code{W/O} translates into the C file opening mode @code{w} (or
@code{wb}): The file is cleared, if it exists, and created, if it does
not (with both @code{open-file} and @code{create-file}).  Under Unix
@code{create-file} creates a file with 666 permissions modified by your
umask.

@item file exceptions:
@cindex file exceptions
The file words do not raise exceptions (except, perhaps, memory access
faults when you pass illegal addresses or file-ids).

@item file line terminator:
@cindex file line terminator
System-dependent. Gforth uses C's newline character as line
terminator. What the actual character code(s) of this are is
system-dependent.

@item file name format:
@cindex file name format
System dependent. Gforth just uses the file name format of your OS.

@item information returned by @code{FILE-STATUS}:
@cindex @code{FILE-STATUS}, returned information
@code{FILE-STATUS} returns the most powerful file access mode allowed
for the file: Either @code{R/O}, @code{W/O} or @code{R/W}. If the file
cannot be accessed, @code{R/O BIN} is returned. @code{BIN} is applicable
along with the returned mode.

@item input file state after an exception when including source:
@cindex exception when including source
All files that are left via the exception are closed.

@item @i{ior} values and meaning:
@cindex @i{ior} values and meaning
@cindex @i{wior} values and meaning
The @i{ior}s returned by the file and memory allocation words are
intended as throw codes. They typically are in the range
-512@minus{}-2047 of OS errors.  The mapping from OS error numbers to
@i{ior}s is -512@minus{}@i{errno}.

@item maximum depth of file input nesting:
@cindex maximum depth of file input nesting
@cindex file input nesting, maximum depth
limited by the amount of return stack, locals/TIB stack, and the number
of open files available. This should not give you troubles.

@item maximum size of input line:
@cindex maximum size of input line
@cindex input line size, maximum
@code{/line}. Currently 255.

@item methods of mapping block ranges to files:
@cindex mapping block ranges to files
@cindex files containing blocks
@cindex blocks in files
By default, blocks are accessed in the file @file{blocks.fb} in the
current working directory. The file can be switched with @code{USE}.

@item number of string buffers provided by @code{S"}:
@cindex @code{S"}, number of string buffers
As many as memory available; the strings are stored in memory blocks
allocated with ALLOCATE indefinitely.

@item size of string buffer used by @code{S"}:
@cindex @code{S"}, size of string buffer
@code{/line}. currently 255.

@end table

@c ---------------------------------------------------------------------
@node file-ambcond,  , file-idef, The optional File-Access word set
@subsection Ambiguous conditions
@c ---------------------------------------------------------------------
@cindex file words, ambiguous conditions
@cindex ambiguous conditions, file words

@table @i
@item attempting to position a file outside its boundaries:
@cindex @code{REPOSITION-FILE}, outside the file's boundaries
@code{REPOSITION-FILE} is performed as usual: Afterwards,
@code{FILE-POSITION} returns the value given to @code{REPOSITION-FILE}.

@item attempting to read from file positions not yet written:
@cindex reading from file positions not yet written
End-of-file, i.e., zero characters are read and no error is reported.

@item @i{file-id} is invalid (@code{INCLUDE-FILE}):
@cindex @code{INCLUDE-FILE}, @i{file-id} is invalid 
An appropriate exception may be thrown, but a memory fault or other
problem is more probable.

@item I/O exception reading or closing @i{file-id} (@code{INCLUDE-FILE}, @code{INCLUDED}):
@cindex @code{INCLUDE-FILE}, I/O exception reading or closing @i{file-id}
@cindex @code{INCLUDED}, I/O exception reading or closing @i{file-id}
The @i{ior} produced by the operation, that discovered the problem, is
thrown.

@item named file cannot be opened (@code{INCLUDED}):
@cindex @code{INCLUDED}, named file cannot be opened
The @i{ior} produced by @code{open-file} is thrown.

@item requesting an unmapped block number:
@cindex unmapped block numbers
There are no unmapped legal block numbers. On some operating systems,
writing a block with a large number may overflow the file system and
have an error message as consequence.

@item using @code{source-id} when @code{blk} is non-zero:
@cindex @code{SOURCE-ID}, behaviour when @code{BLK} is non-zero
@code{source-id} performs its function. Typically it will give the id of
the source which loaded the block. (Better ideas?)

@end table


@c =====================================================================
@node  The optional Floating-Point word set, The optional Locals word set, The optional File-Access word set, Standard conformance
@section The optional Floating-Point word set
@c =====================================================================
@cindex system documentation, floating-point words
@cindex floating-point words, system documentation

@menu
* floating-idef::               Implementation Defined Options
* floating-ambcond::            Ambiguous Conditions            
@end menu


@c ---------------------------------------------------------------------
@node floating-idef, floating-ambcond, The optional Floating-Point word set, The optional Floating-Point word set
@subsection Implementation Defined Options
@c ---------------------------------------------------------------------
@cindex implementation-defined options, floating-point words
@cindex floating-point words, implementation-defined options

@table @i
@item format and range of floating point numbers:
@cindex format and range of floating point numbers
@cindex floating point numbers, format and range
System-dependent; the @code{double} type of C.

@item results of @code{REPRESENT} when @i{float} is out of range:
@cindex  @code{REPRESENT}, results when @i{float} is out of range
System dependent; @code{REPRESENT} is implemented using the C library
function @code{ecvt()} and inherits its behaviour in this respect.

@item rounding or truncation of floating-point numbers:
@cindex rounding of floating-point numbers
@cindex truncation of floating-point numbers
@cindex floating-point numbers, rounding or truncation
System dependent; the rounding behaviour is inherited from the hosting C
compiler. IEEE-FP-based (i.e., most) systems by default round to
nearest, and break ties by rounding to even (i.e., such that the last
bit of the mantissa is 0).

@item size of floating-point stack:
@cindex floating-point stack size
@code{s" FLOATING-STACK" environment? drop .} gives the total size of
the floating-point stack (in floats). You can specify this on startup
with the command-line option @code{-f} (@pxref{Invoking Gforth}).

@item width of floating-point stack:
@cindex floating-point stack width 
@code{1 floats}.

@end table


@c ---------------------------------------------------------------------
@node floating-ambcond,  , floating-idef, The optional Floating-Point word set
@subsection Ambiguous conditions
@c ---------------------------------------------------------------------
@cindex floating-point words, ambiguous conditions
@cindex ambiguous conditions, floating-point words

@table @i
@item @code{df@@} or @code{df!} used with an address that is not double-float  aligned:
@cindex @code{df@@} or @code{df!} used with an address that is not double-float  aligned
System-dependent. Typically results in a @code{-23 THROW} like other
alignment violations.

@item @code{f@@} or @code{f!} used with an address that is not float  aligned:
@cindex @code{f@@} used with an address that is not float aligned
@cindex @code{f!} used with an address that is not float aligned
System-dependent. Typically results in a @code{-23 THROW} like other
alignment violations.

@item floating-point result out of range:
@cindex floating-point result out of range
System-dependent. Can result in a @code{-43 throw} (floating point
overflow), @code{-54 throw} (floating point underflow), @code{-41 throw}
(floating point inexact result), @code{-55 THROW} (Floating-point
unidentified fault), or can produce a special value representing, e.g.,
Infinity.

@item @code{sf@@} or @code{sf!} used with an address that is not single-float  aligned:
@cindex @code{sf@@} or @code{sf!} used with an address that is not single-float  aligned
System-dependent. Typically results in an alignment fault like other
alignment violations.

@item @code{base} is not decimal (@code{REPRESENT}, @code{F.}, @code{FE.}, @code{FS.}):
@cindex @code{base} is not decimal (@code{REPRESENT}, @code{F.}, @code{FE.}, @code{FS.})
The floating-point number is converted into decimal nonetheless.

@item Both arguments are equal to zero (@code{FATAN2}):
@cindex @code{FATAN2}, both arguments are equal to zero
System-dependent. @code{FATAN2} is implemented using the C library
function @code{atan2()}.

@item Using @code{FTAN} on an argument @i{r1} where cos(@i{r1}) is zero:
@cindex @code{FTAN} on an argument @i{r1} where cos(@i{r1}) is zero
System-dependent. Anyway, typically the cos of @i{r1} will not be zero
because of small errors and the tan will be a very large (or very small)
but finite number.

@item @i{d} cannot be presented precisely as a float in @code{D>F}:
@cindex @code{D>F}, @i{d} cannot be presented precisely as a float
The result is rounded to the nearest float.

@item dividing by zero:
@cindex dividing by zero, floating-point
@cindex floating-point dividing by zero
@cindex floating-point unidentified fault, FP divide-by-zero
Platform-dependent; can produce an Infinity, NaN, @code{-42 throw}
(floating point divide by zero) or @code{-55 throw} (Floating-point
unidentified fault).

@item exponent too big for conversion (@code{DF!}, @code{DF@@}, @code{SF!}, @code{SF@@}):
@cindex exponent too big for conversion (@code{DF!}, @code{DF@@}, @code{SF!}, @code{SF@@})
System dependent. On IEEE-FP based systems the number is converted into
an infinity.

@item @i{float}<1 (@code{FACOSH}):
@cindex @code{FACOSH}, @i{float}<1
@cindex floating-point unidentified fault, @code{FACOSH}
Platform-dependent; on IEEE-FP systems typically produces a NaN.

@item @i{float}<=-1 (@code{FLNP1}):
@cindex @code{FLNP1}, @i{float}<=-1
@cindex floating-point unidentified fault, @code{FLNP1}
Platform-dependent; on IEEE-FP systems typically produces a NaN (or a
negative infinity for @i{float}=-1).

@item @i{float}<=0 (@code{FLN}, @code{FLOG}):
@cindex @code{FLN}, @i{float}<=0
@cindex @code{FLOG}, @i{float}<=0
@cindex floating-point unidentified fault, @code{FLN} or @code{FLOG}
Platform-dependent; on IEEE-FP systems typically produces a NaN (or a
negative infinity for @i{float}=0).

@item @i{float}<0 (@code{FASINH}, @code{FSQRT}):
@cindex @code{FASINH}, @i{float}<0
@cindex @code{FSQRT}, @i{float}<0
@cindex floating-point unidentified fault, @code{FASINH} or @code{FSQRT}
Platform-dependent; for @code{fsqrt} this typically gives a NaN, for
@code{fasinh} some platforms produce a NaN, others a number (bug in the
C library?).

@item |@i{float}|>1 (@code{FACOS}, @code{FASIN}, @code{FATANH}):
@cindex @code{FACOS}, |@i{float}|>1
@cindex @code{FASIN}, |@i{float}|>1
@cindex @code{FATANH}, |@i{float}|>1
@cindex floating-point unidentified fault, @code{FACOS}, @code{FASIN} or @code{FATANH}
Platform-dependent; IEEE-FP systems typically produce a NaN.

@item integer part of float cannot be represented by @i{d} in @code{F>D}:
@cindex @code{F>D}, integer part of float cannot be represented by @i{d}
@cindex floating-point unidentified fault, @code{F>D}
Platform-dependent; typically, some double number is produced and no
error is reported.

@item string larger than pictured numeric output area (@code{f.}, @code{fe.}, @code{fs.}):
@cindex string larger than pictured numeric output area (@code{f.}, @code{fe.}, @code{fs.})
@code{Precision} characters of the numeric output area are used.  If
@code{precision} is too high, these words will smash the data or code
close to @code{here}.
@end table

@c =====================================================================
@node  The optional Locals word set, The optional Memory-Allocation word set, The optional Floating-Point word set, Standard conformance
@section The optional Locals word set
@c =====================================================================
@cindex system documentation, locals words
@cindex locals words, system documentation

@menu
* locals-idef::                 Implementation Defined Options                 
* locals-ambcond::              Ambiguous Conditions              
@end menu


@c ---------------------------------------------------------------------
@node locals-idef, locals-ambcond, The optional Locals word set, The optional Locals word set
@subsection Implementation Defined Options
@c ---------------------------------------------------------------------
@cindex implementation-defined options, locals words
@cindex locals words, implementation-defined options

@table @i
@item maximum number of locals in a definition:
@cindex maximum number of locals in a definition
@cindex locals, maximum number in a definition
@code{s" #locals" environment? drop .}. Currently 15. This is a lower
bound, e.g., on a 32-bit machine there can be 41 locals of up to 8
characters. The number of locals in a definition is bounded by the size
of locals-buffer, which contains the names of the locals.

@end table


@c ---------------------------------------------------------------------
@node locals-ambcond,  , locals-idef, The optional Locals word set
@subsection Ambiguous conditions
@c ---------------------------------------------------------------------
@cindex locals words, ambiguous conditions
@cindex ambiguous conditions, locals words

@table @i
@item executing a named local in interpretation state:
@cindex local in interpretation state
@cindex Interpreting a compile-only word, for a local
Compiles the local into the current definition (just as in compile
state); in addition text-interpreting a local in interpretation state
gives an ``is compile-only'' warning.

@item @i{name} not defined by @code{VALUE} or @code{(LOCAL)} (@code{TO}):
@cindex name not defined by @code{VALUE} or @code{(LOCAL)} used by @code{TO}
@cindex @code{TO} on non-@code{VALUE}s and non-locals
@cindex Invalid name argument, @code{TO}
@code{-32 throw} (Invalid name argument)

@end table


@c =====================================================================
@node  The optional Memory-Allocation word set, The optional Programming-Tools word set, The optional Locals word set, Standard conformance
@section The optional Memory-Allocation word set
@c =====================================================================
@cindex system documentation, memory-allocation words
@cindex memory-allocation words, system documentation

@menu
* memory-idef::                 Implementation Defined Options                 
@end menu


@c ---------------------------------------------------------------------
@node memory-idef,  , The optional Memory-Allocation word set, The optional Memory-Allocation word set
@subsection Implementation Defined Options
@c ---------------------------------------------------------------------
@cindex implementation-defined options, memory-allocation words
@cindex memory-allocation words, implementation-defined options

@table @i
@item values and meaning of @i{ior}:
@cindex  @i{ior} values and meaning
The @i{ior}s returned by the file and memory allocation words are
intended as throw codes. They typically are in the range
-512@minus{}-2047 of OS errors.  The mapping from OS error numbers to
@i{ior}s is -512@minus{}@i{errno}.

@end table

@c =====================================================================
@node  The optional Programming-Tools word set, The optional Search-Order word set, The optional Memory-Allocation word set, Standard conformance
@section The optional Programming-Tools word set
@c =====================================================================
@cindex system documentation, programming-tools words
@cindex programming-tools words, system documentation

@menu
* programming-idef::            Implementation Defined Options            
* programming-ambcond::         Ambiguous Conditions         
@end menu


@c ---------------------------------------------------------------------
@node programming-idef, programming-ambcond, The optional Programming-Tools word set, The optional Programming-Tools word set
@subsection Implementation Defined Options
@c ---------------------------------------------------------------------
@cindex implementation-defined options, programming-tools words
@cindex programming-tools words, implementation-defined options

@table @i
@item ending sequence for input following @code{;CODE} and @code{CODE}:
@cindex @code{;CODE} ending sequence
@cindex @code{CODE} ending sequence
@code{END-CODE}

@item manner of processing input following @code{;CODE} and @code{CODE}:
@cindex @code{;CODE}, processing input
@cindex @code{CODE}, processing input
The @code{ASSEMBLER} vocabulary is pushed on the search order stack, and
the input is processed by the text interpreter, (starting) in interpret
state.

@item search order capability for @code{EDITOR} and @code{ASSEMBLER}:
@cindex @code{ASSEMBLER}, search order capability
The Search-Order word set.

@item source and format of display by @code{SEE}:
@cindex @code{SEE}, source and format of output
The source for @code{see} is the executable code used by the inner
interpreter.  The current @code{see} tries to output Forth source code
(and on some platforms, assembly code for primitives) as well as
possible.

@end table

@c ---------------------------------------------------------------------
@node programming-ambcond,  , programming-idef, The optional Programming-Tools word set
@subsection Ambiguous conditions
@c ---------------------------------------------------------------------
@cindex programming-tools words, ambiguous conditions
@cindex ambiguous conditions, programming-tools words

@table @i

@item deleting the compilation word list (@code{FORGET}):
@cindex @code{FORGET}, deleting the compilation word list
Not implemented (yet).

@item fewer than @i{u}+1 items on the control-flow stack (@code{CS-PICK}, @code{CS-ROLL}):
@cindex @code{CS-PICK}, fewer than @i{u}+1 items on the control flow-stack
@cindex @code{CS-ROLL}, fewer than @i{u}+1 items on the control flow-stack
@cindex control-flow stack underflow
This typically results in an @code{abort"} with a descriptive error
message (may change into a @code{-22 throw} (Control structure mismatch)
in the future). You may also get a memory access error. If you are
unlucky, this ambiguous condition is not caught.

@item @i{name} can't be found (@code{FORGET}):
@cindex @code{FORGET}, @i{name} can't be found
Not implemented (yet).

@item @i{name} not defined via @code{CREATE}:
@cindex @code{;CODE}, @i{name} not defined via @code{CREATE}
@code{;CODE} behaves like @code{DOES>} in this respect, i.e., it changes
the execution semantics of the last defined word no matter how it was
defined.

@item @code{POSTPONE} applied to @code{[IF]}:
@cindex @code{POSTPONE} applied to @code{[IF]}
@cindex @code{[IF]} and @code{POSTPONE}
After defining @code{: X POSTPONE [IF] ; IMMEDIATE}. @code{X} is
equivalent to @code{[IF]}.

@item reaching the end of the input source before matching @code{[ELSE]} or @code{[THEN]}:
@cindex @code{[IF]}, end of the input source before matching @code{[ELSE]} or @code{[THEN]}
Continue in the same state of conditional compilation in the next outer
input source. Currently there is no warning to the user about this.

@item removing a needed definition (@code{FORGET}):
@cindex @code{FORGET}, removing a needed definition
Not implemented (yet).

@end table


@c =====================================================================
@node  The optional Search-Order word set,  , The optional Programming-Tools word set, Standard conformance
@section The optional Search-Order word set
@c =====================================================================
@cindex system documentation, search-order words
@cindex search-order words, system documentation

@menu
* search-idef::                 Implementation Defined Options                 
* search-ambcond::              Ambiguous Conditions              
@end menu


@c ---------------------------------------------------------------------
@node search-idef, search-ambcond, The optional Search-Order word set, The optional Search-Order word set
@subsection Implementation Defined Options
@c ---------------------------------------------------------------------
@cindex implementation-defined options, search-order words
@cindex search-order words, implementation-defined options

@table @i
@item maximum number of word lists in search order:
@cindex maximum number of word lists in search order
@cindex search order, maximum depth
@code{s" wordlists" environment? drop .}. Currently 16.

@item minimum search order:
@cindex minimum search order
@cindex search order, minimum
@code{root root}.

@end table

@c ---------------------------------------------------------------------
@node search-ambcond,  , search-idef, The optional Search-Order word set
@subsection Ambiguous conditions
@c ---------------------------------------------------------------------
@cindex search-order words, ambiguous conditions
@cindex ambiguous conditions, search-order words

@table @i
@item changing the compilation word list (during compilation):
@cindex changing the compilation word list (during compilation)
@cindex compilation word list, change before definition ends
The word is entered into the word list that was the compilation word list
at the start of the definition. Any changes to the name field (e.g.,
@code{immediate}) or the code field (e.g., when executing @code{DOES>})
are applied to the latest defined word (as reported by @code{latest} or
@code{latestxt}), if possible, irrespective of the compilation word list.

@item search order empty (@code{previous}):
@cindex @code{previous}, search order empty
@cindex vocstack empty, @code{previous}
@code{abort" Vocstack empty"}.

@item too many word lists in search order (@code{also}):
@cindex @code{also}, too many word lists in search order
@cindex vocstack full, @code{also}
@code{abort" Vocstack full"}.

@end table

@c ***************************************************************
@node Standard vs Extensions, Model, Standard conformance, Top
@chapter Should I use Gforth extensions?
@cindex Gforth extensions

As you read through the rest of this manual, you will see documentation
for @i{Standard} words, and documentation for some appealing Gforth
@i{extensions}. You might ask yourself the question: @i{``Should I
restrict myself to the standard, or should I use the extensions?''}

The answer depends on the goals you have for the program you are working
on:

@itemize @bullet

@item Is it just for yourself or do you want to share it with others?

@item
If you want to share it, do the others all use Gforth?

@item
If it is just for yourself, do you want to restrict yourself to Gforth?

@end itemize

If restricting the program to Gforth is ok, then there is no reason not
to use extensions.  It is still a good idea to keep to the standard
where it is easy, in case you want to reuse these parts in another
program that you want to be portable.

If you want to be able to port the program to other Forth systems, there
are the following points to consider:

@itemize @bullet

@item
Most Forth systems that are being maintained support Standard Forth.
So if your program complies with the standard, it will be portable
among many systems.

@item
A number of the Gforth extensions can be implemented in Standard Forth using
public-domain files provided in the @file{compat/} directory. These are
mentioned in the text in passing.  There is no reason not to use these
extensions, your program will still be Standard Forth compliant; just include
the appropriate compat files with your program.

@item
The tool @file{ans-report.fs} (@pxref{Standard Report}) makes it easy to
analyse your program and determine what non-Standard words it relies
upon.  However, it does not check whether you use standard words in a
non-standard way.

@item
Some techniques are not standardized by Standard Forth, and are hard or
impossible to implement in a standard way, but can be implemented in
most Forth systems easily, and usually in similar ways (e.g., accessing
word headers).  Forth has a rich historical precedent for programmers
taking advantage of implementation-dependent features of their tools
(for example, relying on a knowledge of the dictionary
structure). Sometimes these techniques are necessary to extract every
last bit of performance from the hardware, sometimes they are just a
programming shorthand.

@item
Does using a Gforth extension save more work than the porting this part
to other Forth systems (if any) will cost?

@item
Is the additional functionality worth the reduction in portability and
the additional porting problems?

@end itemize

In order to perform these considerations, you need to know what's
standard and what's not.  This manual generally states if something is
non-standard, but the authoritative source is the
@uref{https://forth-standard.org/standard/words,standard document}.
Appendix A of the Standard (@var{Rationale}) provides a valuable insight
into the thought processes of the technical committee.

Note also that portability between Forth systems is not the only
portability issue; there is also the issue of portability between
different platforms (processor/OS combinations).

@c ***************************************************************
@node Model, Integrating Gforth, Standard vs Extensions, Top
@chapter Model

This chapter has yet to be written. It will contain information, on
which internal structures you can rely.

@c ***************************************************************
@node Integrating Gforth, Emacs and Gforth, Model, Top
@chapter Integrating Gforth into C programs

Several people like to use Forth as scripting language for applications
that are otherwise written in C, C++, or some other language.

The Forth system ATLAST provides facilities for embedding it into
applications; unfortunately it has several disadvantages: most
importantly, it is not based on Standard Forth, and it is apparently dead
(i.e., not developed further and not supported). The facilities
provided by Gforth in this area are inspired by ATLAST's facilities, so
making the switch should not be hard.

We also tried to design the interface such that it can easily be
implemented by other Forth systems, so that we may one day arrive at a
standardized interface. Such a standard interface would allow you to
replace the Forth system without having to rewrite C code.

You embed the Gforth interpreter by linking with the library
@code{libgforth.a} or @code{libgforth.so} (give the compiler the
option @code{-lgforth}, or for one of the other engines
@code{-lgforth-fast}, @code{-lgforth-itc}, or @code{-lgforth-ditc}).
All global symbols in this library that belong to the interface, have
the prefix @code{gforth_}; if a common interface emerges, the
functions may also be available through @code{#define}s with the
prefix @code{forth_}.

You can include the declarations of Forth types, the functions and
variables of the interface with @code{#include <gforth.h>}.

You can now run a Gforth session by either calling @code{gforth_main}
or using the components:

@example
Cell gforth_main(int argc, char **argv, char **env)
@{
  Cell retvalue=gforth_start(argc, argv);

  if(retvalue == -56) @{ /* throw-code for quit */
    retvalue = gforth_bootmessage();     // show boot message
    if(retvalue == -56)
      retvalue = gforth_quit(); // run quit loop
  @}
  gforth_cleanup();
  gforth_printmetrics();
  // gforth_free_dict(); // if you want to restart, do this

  return retvalue;
@}
@end example

To interact with the Forth interpreter, there's @code{Xt
gforth_find(Char * name)} and @code{Cell gforth_execute(Xt xt)}.

More documentation needs to be put here.

@section Types

@code{Cell}, @code{UCell}: data stack elements.

@code{Float}: float stack element.

@code{Address}, @code{Xt}, @code{Label}: pointer typies to memory,
Forth words, and Forth instructions inside the VM.

@section Variables

Data and FP Stack pointer. Area sizes. Accessing the Stacks

@code{gforth_SP}, @code{gforth_FP}.

@section Functions

@example
void *gforth_engine(Xt *, stackpointers *);
Cell gforth_main(int argc, char **argv, char **env);
int gforth_args(int argc, char **argv, char **path, char **imagename);
ImageHeader* gforth_loader(char* imagename, char* path);
user_area* gforth_stacks(Cell dsize, Cell rsize, Cell fsize, Cell lsize);
void gforth_free_stacks(user_area* t);
void gforth_setstacks(user_area * t);
void gforth_free_dict();
Cell gforth_go(Xt* ip0);
Cell gforth_boot(int argc, char** argv, char* path);
void gforth_bootmessage();
Cell gforth_start(int argc, char ** argv);
Cell gforth_quit();
Xt gforth_find(Char * name);
Cell gforth_execute(Xt xt);
void gforth_cleanup();
void gforth_printmetrics();
void gforth_setwinch();
@end example

@section Signals

Gforth sets up signal handlers to catch exceptions and window size
changes.  This may interfere with your C program.

@c ******************************************************************
@node Emacs and Gforth, Image Files, Integrating Gforth, Top
@chapter Emacs and Gforth
@cindex Emacs and Gforth

@cindex @file{gforth.el}
@cindex @file{forth.el}
@cindex Rydqvist, Goran
@cindex Kuehling, David
@cindex comment editing commands
@cindex @code{\}, editing with Emacs
@cindex debug tracer editing commands
@cindex @code{~~}, removal with Emacs
@cindex Forth mode in Emacs

Gforth comes with @file{gforth.el}, an improved version of
@file{forth.el} by Goran Rydqvist (included in the TILE package). The
improvements are:

@itemize @bullet
@item
A better handling of indentation.
@item
A custom hilighting engine for Forth-code.
@item
Comment paragraph filling (@kbd{M-q})
@item
Commenting (@kbd{C-x \}) and uncommenting (@kbd{C-u C-x \}) of regions
@item
Removal of debugging tracers (@kbd{C-x ~}, @pxref{Debugging}).
@item
Support of the @code{info-lookup} feature for looking up the
documentation of a word.
@item
Support for reading and writing blocks files.
@end itemize

To get a basic description of these features, enter Forth mode and
type @kbd{C-h m}.

@cindex source location of error or debugging output in Emacs
@cindex error output, finding the source location in Emacs
@cindex debugging output, finding the source location in Emacs
In addition, Gforth supports Emacs quite well: The source code locations
given in error messages, debugging output (from @code{~~}) and failed
assertion messages are in the right format for Emacs' compilation mode
(@pxref{Compilation, , Running Compilations under Emacs, emacs, Emacs
Manual}) so the source location corresponding to an error or other
message is only a few keystrokes away (@kbd{C-x `} for the next error,
@kbd{C-c C-c} for the error under the cursor).

@cindex viewing the documentation of a word in Emacs
@cindex context-sensitive help
Moreover, for words documented in this manual, you can look up the
glossary entry quickly by using @kbd{C-h TAB}
(@code{info-lookup-symbol}, @pxref{Documentation, ,Documentation
Commands, emacs, Emacs Manual}).  This feature requires Emacs 20.3 or
later and does not work for words containing @code{:}.

@menu
* Installing gforth.el::        Making Emacs aware of Forth.
* Emacs Tags::                  Viewing the source of a word in Emacs.
* Hilighting::                  Making Forth code look prettier.
* Auto-Indentation::            Customizing auto-indentation.
* Blocks Files::                Reading and writing blocks files.
@end menu

@c ----------------------------------
@node Installing gforth.el, Emacs Tags, Emacs and Gforth, Emacs and Gforth
@section Installing gforth.el
@cindex @file{.emacs}
@cindex @file{gforth.el}, installation
To make the features from @file{gforth.el} available in Emacs, add
the following lines to your @file{.emacs} file:

@example
(autoload 'forth-mode "gforth.el")
(setq auto-mode-alist (cons '("\\.fs\\'" . forth-mode) 
			    auto-mode-alist))
(autoload 'forth-block-mode "gforth.el")
(setq auto-mode-alist (cons '("\\.fb\\'" . forth-block-mode) 
			    auto-mode-alist))
(add-hook 'forth-mode-hook (function (lambda ()
   ;; customize variables here:
   (setq forth-indent-level 4)
   (setq forth-minor-indent-level 2)
   (setq forth-hilight-level 3)
   ;;; ...
)))
@end example

@c ----------------------------------
@node Emacs Tags, Hilighting, Installing gforth.el, Emacs and Gforth
@section Emacs Tags
@cindex @file{TAGS} file
@cindex @file{etags.fs}
@cindex viewing the source of a word in Emacs
@cindex @code{require}, placement in files
@cindex @code{include}, placement in files
If you @code{require} @file{etags.fs}, a new @file{TAGS} file will be
produced (@pxref{Tags, , Tags Tables, emacs, Emacs Manual}) that
contains the definitions of all words defined afterwards. You can then
find the source for a word using @kbd{M-.}. Note that Emacs can use
several tags files at the same time (e.g., one for the Gforth sources
and one for your program, @pxref{Select Tags Table,,Selecting a Tags
Table,emacs, Emacs Manual}). The TAGS file for the preloaded words is
@file{$(datadir)/gforth/$(VERSION)/TAGS} (e.g.,
@file{/usr/local/share/gforth/0.2.0/TAGS}).  To get the best behaviour
with @file{etags.fs}, you should avoid putting definitions both before
and after @code{require} etc., otherwise you will see the same file
visited several times by commands like @code{tags-search}.

@c ----------------------------------
@node Hilighting, Auto-Indentation, Emacs Tags, Emacs and Gforth
@section Hilighting
@cindex hilighting Forth code in Emacs
@cindex highlighting Forth code in Emacs
@file{gforth.el} comes with a custom source hilighting engine.  When
you open a file in @code{forth-mode}, it will be completely parsed,
assigning faces to keywords, comments, strings etc.  While you edit
the file, modified regions get parsed and updated on-the-fly. 

Use the variable `forth-hilight-level' to change the level of
decoration from 0 (no hilighting at all) to 3 (the default).  Even if
you set the hilighting level to 0, the parser will still work in the
background, collecting information about whether regions of text are
``compiled'' or ``interpreted''.  Those information are required for
auto-indentation to work properly.  Set `forth-disable-parser' to
non-nil if your computer is too slow to handle parsing.  This will
have an impact on the smartness of the auto-indentation engine,
though.

Sometimes Forth sources define new features that should be hilighted,
new control structures, defining-words etc.  You can use the variable
`forth-custom-words' to make @code{forth-mode} hilight additional
words and constructs.  See the docstring of `forth-words' for details
(in Emacs, type @kbd{C-h v forth-words}).

`forth-custom-words' is meant to be customized in your
@file{.emacs} file.  To customize hilighing in a file-specific manner,
set `forth-local-words' in a local-variables section at the end of
your source file (@pxref{Local Variables in Files,, Variables, emacs, Emacs Manual}).

Example:
@example
0 [IF]
   Local Variables:
   forth-local-words:
      ((("t:") definition-starter (font-lock-keyword-face . 1)
        "[ \t\n]" t name (font-lock-function-name-face . 3))
       ((";t") definition-ender (font-lock-keyword-face . 1)))
   End:
[THEN]
@end example

@c ----------------------------------
@node Auto-Indentation, Blocks Files, Hilighting, Emacs and Gforth
@section Auto-Indentation
@cindex auto-indentation of Forth code in Emacs
@cindex indentation of Forth code in Emacs
@code{forth-mode} automatically tries to indent lines in a smart way,
whenever you type @key{TAB} or break a line with @kbd{C-m}.

Simple customization can be achieved by setting
`forth-indent-level' and `forth-minor-indent-level' in your
@file{.emacs} file. For historical reasons @file{gforth.el} indents
per default by multiples of 4 columns.  To use the more traditional
3-column indentation, add the following lines to your @file{.emacs}:

@example
(add-hook 'forth-mode-hook (function (lambda ()
   ;; customize variables here:
   (setq forth-indent-level 3)
   (setq forth-minor-indent-level 1)
)))
@end example

If you want indentation to recognize non-default words, customize it
by setting `forth-custom-indent-words' in your @file{.emacs}.  See the
docstring of `forth-indent-words' for details (in Emacs, type @kbd{C-h
v forth-indent-words}).

To customize indentation in a file-specific manner, set
`forth-local-indent-words' in a local-variables section at the end of
your source file (@pxref{Local Variables in Files, Variables,,emacs,
Emacs Manual}).

Example:
@example
0 [IF]
   Local Variables:
   forth-local-indent-words:
      ((("t:") (0 . 2) (0 . 2))
       ((";t") (-2 . 0) (0 . -2)))
   End:
[THEN]
@end example

@c ----------------------------------
@node Blocks Files,  , Auto-Indentation, Emacs and Gforth
@section Blocks Files
@cindex blocks files, use with Emacs
@code{forth-mode} Autodetects blocks files by checking whether the
length of the first line exceeds 1023 characters.  It then tries to
convert the file into normal text format.  When you save the file, it
will be written to disk as normal stream-source file.

If you want to write blocks files, use @code{forth-blocks-mode}.  It
inherits all the features from @code{forth-mode}, plus some additions:

@itemize @bullet
@item
Files are written to disk in blocks file format.
@item
Screen numbers are displayed in the mode line (enumerated beginning
with the value of `forth-block-base')
@item
Warnings are displayed when lines exceed 64 characters.
@item
The beginning of the currently edited block is marked with an
overlay-arrow. 
@end itemize

There are some restrictions you should be aware of.  When you open a
blocks file that contains tabulator or newline characters, these
characters will be translated into spaces when the file is written
back to disk.  If tabs or newlines are encountered during blocks file
reading, an error is output to the echo area. So have a look at the
`*Messages*' buffer, when Emacs' bell rings during reading.

Please consult the docstring of @code{forth-blocks-mode} for more
information by typing @kbd{C-h v forth-blocks-mode}).

@c ******************************************************************
@node Image Files, Engine, Emacs and Gforth, Top
@chapter Image Files
@cindex image file
@cindex @file{.fi} files
@cindex precompiled Forth code
@cindex dictionary in persistent form
@cindex persistent form of dictionary

An image file is a file containing an image of the Forth dictionary,
i.e., compiled Forth code and data residing in the dictionary.  By
convention, we use the extension @code{.fi} for image files.

@menu
* Image Licensing Issues::      Distribution terms for images.
* Image File Background::       Why have image files?
* Non-Relocatable Image Files::  don't always work.
* Data-Relocatable Image Files::  are better.
* Fully Relocatable Image Files::  better yet.
* Stack and Dictionary Sizes::  Setting the default sizes for an image.
* Running Image Files::         @code{gforth -i @i{file}} or @i{file}.
* Modifying the Startup Sequence::  and turnkey applications.
@end menu

@node Image Licensing Issues, Image File Background, Image Files, Image Files
@section Image Licensing Issues
@cindex license for images
@cindex image license

An image created with @code{gforthmi} (@pxref{gforthmi}) or
@code{savesystem} (@pxref{Non-Relocatable Image Files}) includes the
original image; i.e., according to copyright law it is a derived work of
the original image.

Since Gforth is distributed under the GNU GPL, the newly created image
falls under the GNU GPL, too. In particular, this means that if you
distribute the image, you have to make all of the sources for the image
available, including those you wrote.  For details see @ref{Copying, ,
GNU General Public License (Section 3)}.

If you create an image with @code{cross} (@pxref{cross.fs}), the image
contains only code compiled from the sources you gave it; if none of
these sources is under the GPL, the terms discussed above do not apply
to the image. However, if your image needs an engine (a gforth binary)
that is under the GPL, you should make sure that you distribute both in
a way that is at most a @emph{mere aggregation}, if you don't want the
terms of the GPL to apply to the image.

@node Image File Background, Non-Relocatable Image Files, Image Licensing Issues, Image Files
@section Image File Background
@cindex image file background

Gforth consists not only of primitives (in the engine), but also of
definitions written in Forth. Since the Forth compiler itself belongs to
those definitions, it is not possible to start the system with the
engine and the Forth source alone. Therefore we provide the Forth
code as an image file in nearly executable form. When Gforth starts up,
a C routine loads the image file into memory, optionally relocates the
addresses, then sets up the memory (stacks etc.) according to
information in the image file, and (finally) starts executing Forth
code.

The default image file is @file{gforth.fi} (in the @code{GFORTHPATH}).
You can use a different image by using the @code{-i},
@code{--image-file} or @code{--appl-image} options (@pxref{Invoking
Gforth}), e.g.:

@example
gforth-fast -i myimage.fi
@end example

There are different variants of image files, and they represent
different compromises between the goals of making it easy to generate
image files and making them portable.

@cindex relocation at run-time
Win32Forth 3.4 and Mitch Bradley's @code{cforth} use relocation at
run-time. This avoids many of the complications discussed below (image
files are data relocatable without further ado), but costs performance
(one addition per memory access) and makes it difficult to pass
addresses between Forth and library calls or other programs.

@cindex relocation at load-time
By contrast, the Gforth loader performs relocation at image load time. The
loader also has to replace tokens that represent primitive calls with the
appropriate code-field addresses (or code addresses in the case of
direct threading).

There are three kinds of image files, with different degrees of
relocatability: non-relocatable, data-relocatable, and fully relocatable
image files.

@cindex image file loader
@cindex relocating loader
@cindex loader for image files
These image file variants have several restrictions in common; they are
caused by the design of the image file loader:

@itemize @bullet
@item
There is only one segment; in particular, this means, that an image file
cannot represent @code{ALLOCATE}d memory chunks (and pointers to
them). The contents of the stacks are not represented, either.

@item
The only kinds of relocation supported are: adding the same offset to
all cells that represent data addresses; and replacing special tokens
with code addresses or with pieces of machine code.

If any complex computations involving addresses are performed, the
results cannot be represented in the image file. Several applications that
use such computations come to mind:

@itemize @minus
@item
Hashing addresses (or data structures which contain addresses) for table
lookup. If you use Gforth's @code{table}s or @code{wordlist}s for this
purpose, you will have no problem, because the hash tables are
recomputed automatically when the system is started. If you use your own
hash tables, you will have to do something similar.

@item
There's a cute implementation of doubly-linked lists that uses
@code{XOR}ed addresses. You could represent such lists as singly-linked
in the image file, and restore the doubly-linked representation on
startup.@footnote{In my opinion, though, you should think thrice before
using a doubly-linked list (whatever implementation).}

@item
The code addresses of run-time routines like @code{docol:} cannot be
represented in the image file (because their tokens would be replaced by
machine code in direct threaded implementations). As a workaround,
compute these addresses at run-time with @code{>code-address} from the
executions tokens of appropriate words (see the definitions of
@code{docol:} and friends in @file{kernel/getdoers.fs}).

@item
On many architectures addresses are represented in machine code in some
shifted or mangled form. You cannot put @code{CODE} words that contain
absolute addresses in this form in a relocatable image file. Workarounds
are representing the address in some relative form (e.g., relative to
the CFA, which is present in some register), or loading the address from
a place where it is stored in a non-mangled form.
@end itemize
@end itemize

@node  Non-Relocatable Image Files, Data-Relocatable Image Files, Image File Background, Image Files
@section Non-Relocatable Image Files
@cindex non-relocatable image files
@cindex image file, non-relocatable

These files are simple memory dumps of the dictionary. They are
specific to the executable (i.e., @file{gforth} file) they were
created with. What's worse, they are specific to the place on which
the dictionary resided when the image was created. Now, there is no
guarantee that the dictionary will reside at the same place the next
time you start Gforth, so there's no guarantee that a non-relocatable
image will work the next time (Gforth will complain instead of
crashing, though).  Indeed, on OSs with (enabled) address-space
randomization non-relocatable images are unlikely to work.

You can create a non-relocatable image file with @code{savesystem}, e.g.:

@example
gforth app.fs -e "savesystem app.fi bye"
@end example

doc-savesystem


@node Data-Relocatable Image Files, Fully Relocatable Image Files, Non-Relocatable Image Files, Image Files
@section Data-Relocatable Image Files
@cindex data-relocatable image files
@cindex image file, data-relocatable

These files contain relocatable data addresses, but fixed code
addresses (instead of tokens). They are specific to the executable
(i.e., @file{gforth} file) they were created with.  Also, they disable
dynamic native code generation (typically a factor of 2 in speed).
You get a data-relocatable image, if you pass the engine you want to
use through the @code{GFORTHD} environment variable to @file{gforthmi}
(@pxref{gforthmi}), e.g.

@example
GFORTHD="/usr/bin/gforth-fast --no-dynamic" gforthmi myimage.fi source.fs
@end example

Note that the @code{--no-dynamic} is required here for the image to
work (otherwise it will contain references to dynamically generated
code that is not saved in the image).


@node Fully Relocatable Image Files, Stack and Dictionary Sizes, Data-Relocatable Image Files, Image Files
@section Fully Relocatable Image Files
@cindex fully relocatable image files
@cindex image file, fully relocatable

@cindex @file{kern*.fi}, relocatability
@cindex @file{gforth.fi}, relocatability
These image files have relocatable data addresses, and tokens for code
addresses. They can be used with different binaries (e.g., with and
without debugging) on the same machine, and even across machines with
the same data formats (byte order, cell size, floating point format),
and they work with dynamic native code generation.  However, they are
usually specific to the version of Gforth they were created with. The
files @file{gforth.fi} and @file{kernl*.fi} are fully relocatable.

There are two ways to create a fully relocatable image file:

@menu
* gforthmi::                    The normal way
* cross.fs::                    The hard way
@end menu

@node gforthmi, cross.fs, Fully Relocatable Image Files, Fully Relocatable Image Files
@subsection @file{gforthmi}
@cindex @file{comp-i.fs}
@cindex @file{gforthmi}

You will usually use @file{gforthmi}. If you want to create an
image @i{file} that contains everything you would load by invoking
Gforth with @code{gforth @i{options}}, you simply say:
@example
gforthmi @i{file} @i{options}
@end example

E.g., if you want to create an image @file{asm.fi} that has the file
@file{asm.fs} loaded in addition to the usual stuff, you could do it
like this:

@example
gforthmi asm.fi asm.fs
@end example

@file{gforthmi} is implemented as a sh script and works like this: It
produces two non-relocatable images for different addresses and then
compares them. Its output reflects this: first you see the output (if
any) of the two Gforth invocations that produce the non-relocatable image
files, then you see the output of the comparing program: It displays the
offset used for data addresses and the offset used for code addresses;
moreover, for each cell that cannot be represented correctly in the
image files, it displays a line like this:

@example
     78DC         BFFFFA50         BFFFFA40
@end example

This means that at offset $78dc from @code{forthstart}, one input image
contains $bffffa50, and the other contains $bffffa40. Since these cells
cannot be represented correctly in the output image, you should examine
these places in the dictionary and verify that these cells are dead
(i.e., not read before they are written).

@cindex --application, @code{gforthmi} option
If you insert the option @code{--application} in front of the image file
name, you will get an image that uses the @code{--appl-image} option
instead of the @code{--image-file} option (@pxref{Invoking
Gforth}). When you execute such an image on Unix (by typing the image
name as command), the Gforth engine will pass all options to the image
instead of trying to interpret them as engine options.

If you type @file{gforthmi} with no arguments, it prints some usage
instructions.

@cindex @code{savesystem} during @file{gforthmi}
@cindex @code{bye} during @file{gforthmi}
@cindex doubly indirect threaded code
@cindex environment variables
@cindex @code{GFORTHD} -- environment variable
@cindex @code{GFORTH} -- environment variable
@cindex @code{gforth-ditc}
There are a few wrinkles: After processing the passed @i{options}, the
words @code{savesystem} and @code{bye} must be visible. A special
doubly indirect threaded version of the @file{gforth} executable is
used for creating the non-relocatable images; you can pass the exact
filename of this executable through the environment variable
@code{GFORTHD} (default: @file{gforth-ditc}); if you pass a version
that is not doubly indirect threaded, you will not get a fully
relocatable image, but a data-relocatable image
(@pxref{Data-Relocatable Image Files}), because there is no code
address offset). The normal @file{gforth} executable is used for
creating the relocatable image; you can pass the exact filename of
this executable through the environment variable @code{GFORTH}.

@node cross.fs,  , gforthmi, Fully Relocatable Image Files
@subsection @file{cross.fs}
@cindex @file{cross.fs}
@cindex cross-compiler
@cindex metacompiler
@cindex target compiler

You can also use @code{cross}, a batch compiler that accepts a Forth-like
programming language (@pxref{Cross Compiler}).

@code{cross} allows you to create image files for machines with
different data sizes and data formats than the one used for generating
the image file. You can also use it to create an application image that
does not contain a Forth compiler. These features are bought with
restrictions and inconveniences in programming. E.g., addresses have to
be stored in memory with special words (@code{A!}, @code{A,}, etc.) in
order to make the code relocatable.


@node Stack and Dictionary Sizes, Running Image Files, Fully Relocatable Image Files, Image Files
@section Stack and Dictionary Sizes
@cindex image file, stack and dictionary sizes
@cindex dictionary size default
@cindex stack size default

If you invoke Gforth with a command line flag for the size
(@pxref{Invoking Gforth}), the size you specify is stored in the
dictionary. If you save the dictionary with @code{savesystem} or create
an image with @file{gforthmi}, this size will become the default
for the resulting image file. E.g., the following will create a
fully relocatable version of @file{gforth.fi} with a 1MB dictionary:

@example
gforthmi gforth.fi -m 1M
@end example

In other words, if you want to set the default size for the dictionary
and the stacks of an image, just invoke @file{gforthmi} with the
appropriate options when creating the image.

@cindex stack size, cache-friendly
Note: For cache-friendly behaviour (i.e., good performance), you should
make the sizes of the stacks modulo, say, 2K, somewhat different. E.g.,
the default stack sizes are: data: 16k (mod 2k=0); fp: 15.5k (mod
2k=1.5k); return: 15k(mod 2k=1k); locals: 14.5k (mod 2k=0.5k).

@node Running Image Files, Modifying the Startup Sequence, Stack and Dictionary Sizes, Image Files
@section Running Image Files
@cindex running image files
@cindex invoking image files
@cindex image file invocation

@cindex -i, invoke image file
@cindex --image file, invoke image file
You can invoke Gforth with an image file @i{image} instead of the
default @file{gforth.fi} with the @code{-i} flag (@pxref{Invoking Gforth}):
@example
gforth -i @i{image}
@end example

@cindex executable image file
@cindex image file, executable
If your operating system supports starting scripts with a line of the
form @code{#! ...}, you just have to type the image file name to start
Gforth with this image file (note that the file extension @code{.fi} is
just a convention). I.e., to run Gforth with the image file @i{image},
you can just type @i{image} instead of @code{gforth -i @i{image}}.
This works because every @code{.fi} file starts with a line of this
format:

@example
#! /usr/local/bin/gforth-0.4.0 -i
@end example

The file and pathname for the Gforth engine specified on this line is
the specific Gforth executable that it was built against; i.e. the value
of the environment variable @code{GFORTH} at the time that
@file{gforthmi} was executed.

You can make use of the same shell capability to make a Forth source
file into an executable. For example, if you place this text in a file:

@example
#! /usr/local/bin/gforth

." Hello, world" CR
bye
@end example

@noindent
and then make the file executable (chmod +x in Unix), you can run it
directly from the command line. The sequence @code{#!} is used in two
ways; firstly, it is recognised as a ``magic sequence'' by the operating
system@footnote{The Unix kernel actually recognises two types of files:
executable files and files of data, where the data is processed by an
interpreter that is specified on the ``interpreter line'' -- the first
line of the file, starting with the sequence #!. There may be a small
limit (e.g., 32) on the number of characters that may be specified on
the interpreter line.} secondly it is treated as a comment character by
Gforth. Because of the second usage, a space is required between
@code{#!} and the path to the executable (moreover, some Unixes
require the sequence @code{#! /}).

Most Unix systems (including Linux) support exactly one option after
the binary name.  If that is not enough, you can use the following
trick:

@example
#! /bin/sh
: ## ; 0 [if]
exec gforth -m 10M -d 1M $0 "$@@"
[then]
." Hello, world" cr
bye \ caution: this prevents (further) processing of "$@@"
@end example

First this script is interpreted as shell script, which treats the
first two lines as (mostly) comments, then performs the third line,
which invokes gforth with this script (@code{$0}) as parameter and its
parameters as additional parameters (@code{"$@@"}).  Then this script
is interpreted as Forth script, which first defines a colon definition
@code{##}, then ignores everything up to @code{[then]} and finally
processes the following Forth code.  You can also use

@example
#0 [if]
@end example

in the second line, but this works only in Gforth-0.7.0 and later.

The @file{gforthmi} approach is the fastest one, the shell-based one
is slowest (needs to start an additional shell).  An additional
advantage of the shell approach is that it is unnecessary to know
where the Gforth binary resides, as long as it is in the @code{$PATH}.

doc-#!


@node Modifying the Startup Sequence,  , Running Image Files, Image Files
@section Modifying the Startup Sequence
@cindex startup sequence for image file
@cindex image file initialization sequence
@cindex initialization sequence of image file

You can add your own initialization to the startup sequence of an image
through the deferred word @code{'cold}. @code{'cold} is invoked just
before the image-specific command line processing (i.e., loading files
and evaluating (@code{-e}) strings) starts.

A sequence for adding your initialization usually looks like this:

@example
:noname
    Defers 'cold \ do other initialization stuff (e.g., rehashing wordlists)
    ... \ your stuff
; IS 'cold
@end example

After @code{'cold}, Gforth processes the image options
(@pxref{Invoking Gforth}), and then it performs @code{bootmessage},
another deferred word.  This normally prints Gforth's startup message
and does nothing else.

@cindex turnkey image files
@cindex image file, turnkey applications
So, if you want to make a turnkey image (i.e., an image for an
application instead of an extended Forth system), you can do this in
several ways:

@itemize @bullet

@item
If you want to do your interpretation of the OS command-line
arguments, hook into @code{'cold}.  In that case you probably also
want to build the image with @code{gforthmi --application}
(@pxref{gforthmi}) to keep the engine from processing OS command line
options.  You can then do your own command-line processing with
@code{next-arg} 

@item
If you want to have the normal Gforth processing of OS command-line
arguments, but specify your own command-line options, hook into
@code{process-option}.

@item
If you want to have more options in addition to the ones that come with
Gforth, define words into the @code{options} vocabulary.

@item
If you want to display your own boot message, hook into
@code{bootmessage}.

@end itemize

In either case, you probably do not want the word that you execute in
these hooks to exit normally, but use @code{bye} or @code{throw}.
Otherwise the Gforth startup process would continue and eventually
present the Forth command line to the user.

doc-'cold
doc-bootmessage
doc-'quit
doc-process-option

@c ******************************************************************
@node Engine, Cross Compiler, Image Files, Top
@chapter Engine
@cindex engine
@cindex virtual machine

Reading this chapter is not necessary for programming with Gforth. It
may be helpful for finding your way in the Gforth sources.

The ideas in this section have also been published in the following
papers: Bernd Paysan, @cite{ANS fig/GNU/??? Forth} (in German),
Forth-Tagung '93; M. Anton Ertl,
@cite{@uref{https://www.complang.tuwien.ac.at/papers/ertl93.ps.Z, A
Portable Forth Engine}}, EuroForth '93; M. Anton Ertl,
@cite{@uref{https://www.complang.tuwien.ac.at/papers/ertl02.ps.gz,
Threaded code variations and optimizations (extended version)}},
Forth-Tagung '02.

@menu
* Portability::
* Threading::
* Primitives::
* Performance::
@end menu

@node Portability, Threading, Engine, Engine
@section Portability
@cindex engine portability

An important goal of the Gforth Project is availability across a wide
range of personal machines. fig-Forth, and, to a lesser extent, F83,
achieved this goal by manually coding the engine in assembly language
for several then-popular processors. This approach is very
labor-intensive and the results are short-lived due to progress in
computer architecture.

@cindex C, using C for the engine
Others have avoided this problem by coding in C, e.g., Mitch Bradley
(cforth), Mikael Patel (TILE) and Dirk Zoller (pfe). This approach is
particularly popular for UNIX-based Forths due to the large variety of
architectures of UNIX machines. Unfortunately an implementation in C
does not mix well with the goals of efficiency and with using
traditional techniques: Indirect or direct threading cannot be expressed
in C, and switch threading, the fastest technique available in C, is
significantly slower. Another problem with C is that it is very
cumbersome to express double integer arithmetic.

@cindex GNU C for the engine
@cindex long long
Fortunately, there is a portable language that does not have these
limitations: GNU C, the version of C processed by the GNU C compiler
(@pxref{C Extensions, , Extensions to the C Language Family, gcc,
GNU C Manual}). Its labels as values feature (@pxref{Labels as Values, ,
Labels as Values, gcc, GNU C Manual}) makes direct and indirect
threading possible, its @code{long long} type (@pxref{Long Long, ,
Double-Word Integers, gcc, GNU C Manual}) corresponds to Forth's
double numbers on many systems.  GNU C is freely available on all
important (and many unimportant) UNIX machines, VMS, 80386s running
MS-DOS, the Amiga, and the Atari ST, so a Forth written in GNU C can run
on all these machines.

Writing in a portable language has the reputation of producing code that
is slower than assembly. For our Forth engine we repeatedly looked at
the code produced by the compiler and eliminated most compiler-induced
inefficiencies by appropriate changes in the source code.

@cindex explicit register declarations
@cindex --enable-force-reg, configuration flag
@cindex -DFORCE_REG
However, register allocation cannot be portably influenced by the
programmer, leading to some inefficiencies on register-starved
machines. We use explicit register declarations (@pxref{Explicit Reg
Vars, , Variables in Specified Registers, gcc, GNU C Manual}) to
improve the speed on some machines. They are turned on by using the
configuration flag @code{--enable-force-reg} (@code{gcc} switch
@code{-DFORCE_REG}). Unfortunately, this feature not only depends on the
machine, but also on the compiler version: On some machines some
compiler versions produce incorrect code when certain explicit register
declarations are used. So by default @code{-DFORCE_REG} is not used.

@node Threading, Primitives, Portability, Engine
@section Threading
@cindex inner interpreter implementation
@cindex threaded code implementation

@cindex labels as values
GNU C's labels as values extension (available since @code{gcc-2.0},
@pxref{Labels as Values, , Labels as Values, gcc, GNU C Manual})
makes it possible to take the address of @i{label} by writing
@code{&&@i{label}}.  This address can then be used in a statement like
@code{goto *@i{address}}. I.e., @code{goto *&&x} is the same as
@code{goto x}.

@cindex @code{NEXT}, indirect threaded
@cindex indirect threaded inner interpreter
@cindex inner interpreter, indirect threaded
With this feature an indirect threaded @code{NEXT} looks like:
@example
cfa = *ip++;
ca = *cfa;
goto *ca;
@end example
@cindex instruction pointer
For those unfamiliar with the names: @code{ip} is the Forth instruction
pointer; the @code{cfa} (code-field address) corresponds to Standard Forth's
execution token and points to the code field of the next word to be
executed; The @code{ca} (code address) fetched from there points to some
executable code, e.g., a primitive or the colon definition handler
@code{docol}.

@cindex @code{NEXT}, direct threaded
@cindex direct threaded inner interpreter
@cindex inner interpreter, direct threaded
Direct threading is even simpler:
@example
ca = *ip++;
goto *ca;
@end example

Of course we have packaged the whole thing neatly in macros called
@code{NEXT} and @code{NEXT1} (the part of @code{NEXT} after fetching the cfa).

@menu
* Scheduling::
* Direct or Indirect Threaded?::
* Dynamic Superinstructions::
* DOES>::
@end menu

@node Scheduling, Direct or Indirect Threaded?, Threading, Threading
@subsection Scheduling
@cindex inner interpreter optimization

There is a little complication: Pipelined and superscalar processors,
i.e., RISC and some modern CISC machines can process independent
instructions while waiting for the results of an instruction. The
compiler usually reorders (schedules) the instructions in a way that
achieves good usage of these delay slots. However, on our first tries
the compiler did not do well on scheduling primitives. E.g., for
@code{+} implemented as
@example
n=sp[0]+sp[1];
sp++;
sp[0]=n;
NEXT;
@end example
the @code{NEXT} comes strictly after the other code, i.e., there is
nearly no scheduling. After a little thought the problem becomes clear:
The compiler cannot know that @code{sp} and @code{ip} point to different
addresses (and the version of @code{gcc} we used would not know it even
if it was possible), so it could not move the load of the cfa above the
store to the TOS. Indeed the pointers could be the same, if code on or
very near the top of stack were executed. In the interest of speed we
chose to forbid this probably unused ``feature'' and helped the compiler
in scheduling: @code{NEXT} is divided into several parts:
@code{NEXT_P0}, @code{NEXT_P1} and @code{NEXT_P2}). @code{+} now looks
like:
@example
NEXT_P0;
n=sp[0]+sp[1];
sp++;
NEXT_P1;
sp[0]=n;
NEXT_P2;
@end example

There are various schemes that distribute the different operations of
NEXT between these parts in several ways; in general, different schemes
perform best on different processors.  We use a scheme for most
architectures that performs well for most processors of this
architecture; in the future we may switch to benchmarking and chosing
the scheme on installation time.


@node Direct or Indirect Threaded?, Dynamic Superinstructions, Scheduling, Threading
@subsection Direct or Indirect Threaded?
@cindex threading, direct or indirect?

Threaded forth code consists of references to primitives (simple machine
code routines like @code{+}) and to non-primitives (e.g., colon
definitions, variables, constants); for a specific class of
non-primitives (e.g., variables) there is one code routine (e.g.,
@code{dovar}), but each variable needs a separate reference to its data.

Traditionally Forth has been implemented as indirect threaded code,
because this allows to use only one cell to reference a non-primitive
(basically you point to the data, and find the code address there).

@cindex primitive-centric threaded code
However, threaded code in Gforth (since 0.6.0) uses two cells for
non-primitives, one for the code address, and one for the data address;
the data pointer is an immediate argument for the virtual machine
instruction represented by the code address.  We call this
@emph{primitive-centric} threaded code, because all code addresses point
to simple primitives.  E.g., for a variable, the code address is for
@code{lit} (also used for integer literals like @code{99}).

Primitive-centric threaded code allows us to use (faster) direct
threading as dispatch method, completely portably (direct threaded code
in Gforth before 0.6.0 required architecture-specific code).  It also
eliminates the performance problems related to I-cache consistency that
386 implementations have with direct threaded code, and allows
additional optimizations.

@cindex hybrid direct/indirect threaded code
There is a catch, however: the @var{xt} parameter of @code{execute} can
occupy only one cell, so how do we pass non-primitives with their code
@emph{and} data addresses to them?  Our answer is to use indirect
threaded dispatch for @code{execute} and other words that use a
single-cell xt.  So, normal threaded code in colon definitions uses
direct threading, and @code{execute} and similar words, which dispatch
to xts on the data stack, use indirect threaded code.  We call this
@emph{hybrid direct/indirect} threaded code.

@cindex engines, gforth vs.@: gforth-fast vs.@: gforth-itc
@cindex gforth engine
@cindex gforth-fast engine
The engines @command{gforth} and @command{gforth-fast} use hybrid
direct/indirect threaded code.  This means that with these engines you
cannot use @code{,} to compile an xt.  Instead, you have to use
@code{compile,}.

@cindex gforth-itc engine
If you want to compile xts with @code{,}, use @command{gforth-itc}.
This engine uses plain old indirect threaded code.  It still compiles in
a primitive-centric style, so you cannot use @code{compile,} instead of
@code{,} (e.g., for producing tables of xts with @code{] word1 word2
... [}).  If you want to do that, you have to use @command{gforth-itc}
and execute @code{' , is compile,}.  Your program can check if it is
running on a hybrid direct/indirect threaded engine or a pure indirect
threaded engine with @code{threading-method} (@pxref{Threading Words}).


@node Dynamic Superinstructions, DOES>, Direct or Indirect Threaded?, Threading
@subsection Dynamic Superinstructions
@cindex Dynamic superinstructions with replication
@cindex Superinstructions
@cindex Replication

The engines @command{gforth} and @command{gforth-fast} use another
optimization: Dynamic superinstructions with replication.  As an
example, consider the following colon definition:

@example
: squared ( n1 -- n2 )
  dup * ;
@end example

Gforth compiles this into the threaded code sequence

@example
dup
*
;s
@end example

Use @code{simple-see} (@pxref{Examining compiled code}) to see the
threaded code of a colon definition.

In normal direct threaded code there is a code address occupying one
cell for each of these primitives.  Each code address points to a
machine code routine, and the interpreter jumps to this machine code in
order to execute the primitive.  The routines for these three
primitives are (in @command{gforth-fast} on the 386):

@example
Code dup  
( $804B950 )  add     esi , # -4  \ $83 $C6 $FC 
( $804B953 )  add     ebx , # 4  \ $83 $C3 $4 
( $804B956 )  mov     dword ptr 4 [esi] , ecx  \ $89 $4E $4 
( $804B959 )  jmp     dword ptr FC [ebx]  \ $FF $63 $FC 
end-code
Code *  
( $804ACC4 )  mov     eax , dword ptr 4 [esi]  \ $8B $46 $4 
( $804ACC7 )  add     esi , # 4  \ $83 $C6 $4 
( $804ACCA )  add     ebx , # 4  \ $83 $C3 $4 
( $804ACCD )  imul    ecx , eax  \ $F $AF $C8 
( $804ACD0 )  jmp     dword ptr FC [ebx]  \ $FF $63 $FC 
end-code
Code ;s  
( $804A693 )  mov     eax , dword ptr [edi]  \ $8B $7 
( $804A695 )  add     edi , # 4  \ $83 $C7 $4 
( $804A698 )  lea     ebx , dword ptr 4 [eax]  \ $8D $58 $4 
( $804A69B )  jmp     dword ptr FC [ebx]  \ $FF $63 $FC 
end-code
@end example

With dynamic superinstructions and replication the compiler does not
just lay down the threaded code, but also copies the machine code
fragments, usually without the jump at the end.

@example
( $4057D27D )  add     esi , # -4  \ $83 $C6 $FC 
( $4057D280 )  add     ebx , # 4  \ $83 $C3 $4 
( $4057D283 )  mov     dword ptr 4 [esi] , ecx  \ $89 $4E $4 
( $4057D286 )  mov     eax , dword ptr 4 [esi]  \ $8B $46 $4 
( $4057D289 )  add     esi , # 4  \ $83 $C6 $4 
( $4057D28C )  add     ebx , # 4  \ $83 $C3 $4 
( $4057D28F )  imul    ecx , eax  \ $F $AF $C8 
( $4057D292 )  mov     eax , dword ptr [edi]  \ $8B $7 
( $4057D294 )  add     edi , # 4  \ $83 $C7 $4 
( $4057D297 )  lea     ebx , dword ptr 4 [eax]  \ $8D $58 $4 
( $4057D29A )  jmp     dword ptr FC [ebx]  \ $FF $63 $FC 
@end example

Only when a threaded-code control-flow change happens (e.g., in
@code{;s}), the jump is appended.  This optimization eliminates many of
these jumps and makes the rest much more predictable.  The speedup
depends on the processor and the application; on the Athlon and Pentium
III this optimization typically produces a speedup by a factor of 2.

The code addresses in the direct-threaded code are set to point to the
appropriate points in the copied machine code, in this example like
this:

@example
primitive  code address
   dup       $4057D27D
   *         $4057D286
   ;s        $4057D292
@end example

Thus there can be threaded-code jumps to any place in this piece of
code.  This also simplifies decompilation quite a bit.

@code{See-code} (@pxref{Examining compiled code}) shows the threaded
code intermingled with the native code of dynamic superinstructions.
These days some additional optimizations are applied for the
dynamically-generated native code, so the output of @code{see-code
squared} on @code{gforth-fast} on one particular AMD64 installation
looks like this:

@example
$7FB689C678C8 dup    1->2 
7FB68990C1B2:   mov     r15,r8
$7FB689C678D0 *    2->1 
7FB68990C1B5:   imul    r8,r15
$7FB689C678D8 ;s    1->1 
7FB68990C1B9:   mov     rbx,[r14]
7FB68990C1BC:   add     r14,$08
7FB68990C1C0:   mov     rax,[rbx]
7FB68990C1C3:   jmp     eax
@end example

@cindex --no-dynamic command-line option
@cindex --no-super command-line option
You can disable this optimization with @option{--no-dynamic}.  You can
use the copying without eliminating the jumps (i.e., dynamic
replication, but without superinstructions) with @option{--no-super};
this gives the branch prediction benefit alone; the effect on
performance depends on the CPU; on the Athlon and Pentium III the
speedup is a little less than for dynamic superinstructions with
replication.

@cindex patching threaded code
One use of these options is if you want to patch the threaded code.
With superinstructions, many of the dispatch jumps are eliminated, so
patching often has no effect.  These options preserve all the dispatch
jumps.

@cindex --dynamic command-line option
On some machines dynamic superinstructions are disabled by default,
because it is unsafe on these machines.  However, if you feel
adventurous, you can enable it with @option{--dynamic}.

@node DOES>,  , Dynamic Superinstructions, Threading
@subsection DOES>
@cindex @code{DOES>} implementation

@cindex @code{dodoes} routine
@cindex @code{DOES>}-code
One of the most complex parts of a Forth engine is @code{dodoes}, i.e.,
the chunk of code executed by every word defined by a
@code{CREATE}...@code{DOES>} pair; actually with primitive-centric code,
this is only needed if the xt of the word is @code{execute}d. The main
problem here is: How to find the Forth code to be executed, i.e. the
code after the @code{DOES>} (the @code{DOES>}-code)? There are two
solutions:

In fig-Forth the code field points directly to the @code{dodoes} and the
@code{DOES>}-code address is stored in the cell after the code address
(i.e. at @code{@i{CFA} cell+}). It may seem that this solution is
illegal in the Forth-79 and all later standards, because in fig-Forth
this address lies in the body (which is illegal in these
standards). However, by making the code field larger for all words this
solution becomes legal again.  We use this approach.  Leaving a cell
unused in most words is a bit wasteful, but on the machines we are
targeting this is hardly a problem.


@node Primitives, Performance, Threading, Engine
@section Primitives
@cindex primitives, implementation
@cindex virtual machine instructions, implementation

@menu
* Automatic Generation::
* TOS Optimization::
* Produced code::
@end menu

@node Automatic Generation, TOS Optimization, Primitives, Primitives
@subsection Automatic Generation
@cindex primitives, automatic generation

@cindex @file{prims2x.fs}

Since the primitives are implemented in a portable language, there is no
longer any need to minimize the number of primitives. On the contrary,
having many primitives has an advantage: speed. In order to reduce the
number of errors in primitives and to make programming them easier, we
provide a tool, the primitive generator (@file{prims2x.fs} aka Vmgen,
@pxref{Top, Vmgen, Introduction, vmgen, Vmgen}), that automatically
generates most (and sometimes all) of the C code for a primitive from
the stack effect notation.  The source for a primitive has the following
form:

@cindex primitive source format
@format
@i{Forth-name}  ( @i{stack-effect} )        @i{category}    [@i{pronounc.}]
[@code{""}@i{glossary entry}@code{""}]
@i{C code}
[@code{:}
@i{Forth code}]
@end format

The items in brackets are optional. The category and glossary fields
are there for generating the documentation, the Forth code is there
for manual implementations on machines without GNU C. E.g., the source
for the primitive @code{+} is:
@example
+    ( n1 n2 -- n )   core    plus
n = n1+n2;
@end example

This looks like a specification, but in fact @code{n = n1+n2} is C
code. Our primitive generation tool extracts a lot of information from
the stack effect notations@footnote{We use a one-stack notation, even
though we have separate data and floating-point stacks; The separate
notation can be generated easily from the unified notation.}: The number
of items popped from and pushed on the stack, their type, and by what
name they are referred to in the C code. It then generates a C code
prelude and postlude for each primitive. The final C code for @code{+}
looks like this:

@example
I_plus: /* + ( n1 n2 -- n ) */  /* label, stack effect */
/*  */                          /* documentation */
NAME("+")                       /* debugging output (with -DDEBUG) */
@{
DEF_CA                          /* definition of variable ca (indirect threading) */
Cell n1;                        /* definitions of variables */
Cell n2;
Cell n;
NEXT_P0;                        /* NEXT part 0 */
n1 = (Cell) sp[1];              /* input */
n2 = (Cell) TOS;
sp += 1;                        /* stack adjustment */
@{
n = n1+n2;                      /* C code taken from the source */
@}
NEXT_P1;                        /* NEXT part 1 */
TOS = (Cell)n;                  /* output */
NEXT_P2;                        /* NEXT part 2 */
@}
@end example

This looks long and inefficient, but the GNU C compiler optimizes quite
well and produces optimal code for @code{+} on, e.g., the R3000 and the
HP RISC machines: Defining the @code{n}s does not produce any code, and
using them as intermediate storage also adds no cost.

There are also other optimizations that are not illustrated by this
example: assignments between simple variables are usually for free (copy
propagation). If one of the stack items is not used by the primitive
(e.g.  in @code{drop}), the compiler eliminates the load from the stack
(dead code elimination). On the other hand, there are some things that
the compiler does not do, therefore they are performed by
@file{prims2x.fs}: The compiler does not optimize code away that stores
a stack item to the place where it just came from (e.g., @code{over}).

While programming a primitive is usually easy, there are a few cases
where the programmer has to take the actions of the generator into
account, most notably @code{?dup}, but also words that do not (always)
fall through to @code{NEXT}.

For more information

@node TOS Optimization, Produced code, Automatic Generation, Primitives
@subsection TOS Optimization
@cindex TOS optimization for primitives
@cindex primitives, keeping the TOS in a register

An important optimization for stack machine emulators, e.g., Forth
engines, is keeping  one or more of the top stack items in
registers.  If a word has the stack effect @i{in1}...@i{inx} @code{--}
@i{out1}...@i{outy}, keeping the top @i{n} items in registers
@itemize @bullet
@item
is better than keeping @i{n-1} items, if @i{x>=n} and @i{y>=n},
due to fewer loads from and stores to the stack.
@item is slower than keeping @i{n-1} items, if @i{x<>y} and @i{x<n} and
@i{y<n}, due to additional moves between registers.
@end itemize

@cindex -DUSE_TOS
@cindex -DUSE_NO_TOS
In particular, keeping one item in a register is never a disadvantage,
if there are enough registers. Keeping two items in registers is a
disadvantage for frequent words like @code{?branch}, constants,
variables, literals and @code{i}. Therefore our generator only produces
code that keeps zero or one items in registers. The generated C code
covers both cases; the selection between these alternatives is made at
C-compile time using the switch @code{-DUSE_TOS}. @code{TOS} in the C
code for @code{+} is just a simple variable name in the one-item case,
otherwise it is a macro that expands into @code{sp[0]}. Note that the
GNU C compiler tries to keep simple variables like @code{TOS} in
registers, and it usually succeeds, if there are enough registers.

@cindex -DUSE_FTOS
@cindex -DUSE_NO_FTOS
The primitive generator performs the TOS optimization for the
floating-point stack, too (@code{-DUSE_FTOS}). For floating-point
operations the benefit of this optimization is even larger:
floating-point operations take quite long on most processors, but can be
performed in parallel with other operations as long as their results are
not used. If the FP-TOS is kept in a register, this works. If
it is kept on the stack, i.e., in memory, the store into memory has to
wait for the result of the floating-point operation, lengthening the
execution time of the primitive considerably.

The TOS optimization makes the automatic generation of primitives a
bit more complicated. Just replacing all occurrences of @code{sp[0]} by
@code{TOS} is not sufficient. There are some special cases to
consider:
@itemize @bullet
@item In the case of @code{dup ( w -- w w )} the generator must not
eliminate the store to the original location of the item on the stack,
if the TOS optimization is turned on.
@item Primitives with stack effects of the form @code{--}
@i{out1}...@i{outy} must store the TOS to the stack at the start.
Likewise, primitives with the stack effect @i{in1}...@i{inx} @code{--}
must load the TOS from the stack at the end. But for the null stack
effect @code{--} no stores or loads should be generated.
@end itemize

@node Produced code,  , TOS Optimization, Primitives
@subsection Produced code
@cindex primitives, assembly code listing

@cindex @file{engine.s}
To see what assembly code is produced for the primitives on your machine
with your compiler and your flag settings, type @code{make engine.s} and
look at the resulting file @file{engine.s}.  Alternatively, you can also
disassemble the code of primitives with @code{see} on some architectures.

@node  Performance,  , Primitives, Engine
@section Performance
@cindex performance of some Forth interpreters
@cindex engine performance
@cindex benchmarking Forth systems
@cindex Gforth performance

On RISCs the Gforth engine is very close to optimal; i.e., it is usually
impossible to write a significantly faster threaded-code engine.

On register-starved machines like the 386 architecture processors
improvements are possible, because @code{gcc} does not utilize the
registers as well as a human, even with explicit register declarations;
e.g., Bernd Beuster wrote a Forth system fragment in assembly language
and hand-tuned it for the 486; this system is 1.19 times faster on the
Sieve benchmark on a 486DX2/66 than Gforth compiled with
@code{gcc-2.6.3} with @code{-DFORCE_REG}.  The situation has improved
with gcc-2.95 and gforth-0.4.9; now the most important virtual machine
registers fit in real registers (and we can even afford to use the TOS
optimization), resulting in a speedup of 1.14 on the sieve over the
earlier results.  And dynamic superinstructions provide another speedup
(but only around a factor 1.2 on the 486).

@cindex Win32Forth performance
@cindex NT Forth performance
@cindex eforth performance
@cindex ThisForth performance
@cindex PFE performance
@cindex TILE performance
The potential advantage of assembly language implementations is not
necessarily realized in complete Forth systems: We compared Gforth-0.5.9
(direct threaded, compiled with @code{gcc-2.95.1} and
@code{-DFORCE_REG}) with Win32Forth 1.2093 (newer versions are
reportedly much faster), LMI's NT Forth (Beta, May 1994) and Eforth
(with and without peephole (aka pinhole) optimization of the threaded
code); all these systems were written in assembly language. We also
compared Gforth with three systems written in C: PFE-0.9.14 (compiled
with @code{gcc-2.6.3} with the default configuration for Linux:
@code{-O2 -fomit-frame-pointer -DUSE_REGS -DUNROLL_NEXT}), ThisForth
Beta (compiled with @code{gcc-2.6.3 -O3 -fomit-frame-pointer}; ThisForth
employs peephole optimization of the threaded code) and TILE (compiled
with @code{make opt}). We benchmarked Gforth, PFE, ThisForth and TILE on
a 486DX2/66 under Linux. Kenneth O'Heskin kindly provided the results
for Win32Forth and NT Forth on a 486DX2/66 with similar memory
performance under Windows NT. Marcel Hendrix ported Eforth to Linux,
then extended it to run the benchmarks, added the peephole optimizer,
ran the benchmarks and reported the results.

We used four small benchmarks: the ubiquitous Sieve; bubble-sorting and
matrix multiplication come from the Stanford integer benchmarks and have
been translated into Forth by Martin Fraeman; we used the versions
included in the TILE Forth package, but with bigger data set sizes; and
a recursive Fibonacci number computation for benchmarking calling
performance. The following table shows the time taken for the benchmarks
scaled by the time taken by Gforth (in other words, it shows the speedup
factor that Gforth achieved over the other systems).

@example
relative       Win32-    NT       eforth       This-      
time     Gforth Forth Forth eforth  +opt   PFE Forth  TILE
sieve      1.00  2.16  1.78   2.16  1.32  2.46  4.96 13.37
bubble     1.00  1.93  2.07   2.18  1.29  2.21        5.70
matmul     1.00  1.92  1.76   1.90  0.96  2.06        5.32
fib        1.00  2.32  2.03   1.86  1.31  2.64  4.55  6.54
@end example

You may be quite surprised by the good performance of Gforth when
compared with systems written in assembly language. One important reason
for the disappointing performance of these other systems is probably
that they are not written optimally for the 486 (e.g., they use the
@code{lods} instruction). In addition, Win32Forth uses a comfortable,
but costly method for relocating the Forth image: like @code{cforth}, it
computes the actual addresses at run time, resulting in two address
computations per @code{NEXT} (@pxref{Image File Background}).

The speedup of Gforth over PFE, ThisForth and TILE can be easily
explained with the self-imposed restriction of the latter systems to
standard C, which makes efficient threading impossible (however, the
measured implementation of PFE uses a GNU C extension: @pxref{Global Reg
Vars, , Defining Global Register Variables, gcc, GNU C Manual}).
Moreover, current C compilers have a hard time optimizing other aspects
of the ThisForth and the TILE source.

The performance of Gforth on 386 architecture processors varies widely
with the version of @code{gcc} used. E.g., @code{gcc-2.5.8} failed to
allocate any of the virtual machine registers into real machine
registers by itself and would not work correctly with explicit register
declarations, giving a significantly slower engine (on a 486DX2/66
running the Sieve) than the one measured above.

Note that there have been several releases of Win32Forth since the
release presented here, so the results presented above may have little
predictive value for the performance of Win32Forth today (results for
the current release on an i486DX2/66 are welcome).

@cindex @file{Benchres}
In
@cite{@uref{https://www.complang.tuwien.ac.at/papers/ertl&maierhofer95.ps.gz,
Translating Forth to Efficient C}} by M. Anton Ertl and Martin
Maierhofer (presented at EuroForth '95), an indirect threaded version of
Gforth is compared with Win32Forth, NT Forth, PFE, ThisForth, and
several native code systems; that version of Gforth is slower on a 486
than the version used here. You can find a newer version of these
measurements at
@uref{https://www.complang.tuwien.ac.at/forth/performance.html}. You can
find numbers for Gforth on various machines in @file{Benchres}.

@c ******************************************************************
@c @node Binding to System Library, Cross Compiler, Engine, Top
@c @chapter Binding to System Library

@c ****************************************************************
@node Cross Compiler, MINOS2, Engine, Top
@chapter Cross Compiler
@cindex @file{cross.fs}
@cindex cross-compiler
@cindex metacompiler
@cindex target compiler

The cross compiler is used to bootstrap a Forth kernel. Since Gforth is
mostly written in Forth, including crucial parts like the outer
interpreter and compiler, it needs compiled Forth code to get
started. The cross compiler allows to create new images for other
architectures, even running under another Forth system.

@menu
* Using the Cross Compiler::
* How the Cross Compiler Works::
@end menu

@node Using the Cross Compiler, How the Cross Compiler Works, Cross Compiler, Cross Compiler
@section Using the Cross Compiler

The cross compiler uses a language that resembles Forth, but isn't. The
main difference is that you can execute Forth code after definition,
while you usually can't execute the code compiled by cross, because the
code you are compiling is typically for a different computer than the
one you are compiling on.

@c anton: This chapter is somewhat different from waht I would expect: I
@c would expect an explanation of the cross language and how to create an
@c application image with it.  The section explains some aspects of
@c creating a Gforth kernel.

The Makefile is already set up to allow you to create kernels for new
architectures with a simple make command. The generic kernels using the
GCC compiled virtual machine are created in the normal build process
with @code{make}. To create a embedded Gforth executable for e.g. the
8086 processor (running on a DOS machine), type

@example
make kernl-8086.fi
@end example

This will use the machine description from the @file{arch/8086}
directory to create a new kernel. A machine file may look like that:

@example
\ Parameter for target systems                         06oct92py

    4 Constant cell             \ cell size in bytes
    2 Constant cell<<           \ cell shift to bytes
    5 Constant cell>bit         \ cell shift to bits
    8 Constant bits/char        \ bits per character
    8 Constant bits/byte        \ bits per byte [default: 8]
    8 Constant float            \ bytes per float
    8 Constant /maxalign        \ maximum alignment in bytes
false Constant bigendian        \ byte order
( true=big, false=little )

include machpc.fs               \ feature list
@end example

This part is obligatory for the cross compiler itself, the feature list
is used by the kernel to conditionally compile some features in and out,
depending on whether the target supports these features.

There are some optional features, if you define your own primitives,
have an assembler, or need special, nonstandard preparation to make the
boot process work. @code{asm-include} includes an assembler,
@code{prims-include} includes primitives, and @code{>boot} prepares for
booting.

@example
: asm-include    ." Include assembler" cr
  s" arch/8086/asm.fs" included ;

: prims-include  ." Include primitives" cr
  s" arch/8086/prim.fs" included ;

: >boot          ." Prepare booting" cr
  s" ' boot >body into-forth 1+ !" evaluate ;
@end example

These words are used as sort of macro during the cross compilation in
the file @file{kernel/main.fs}. Instead of using these macros, it would
be possible --- but more complicated --- to write a new kernel project
file, too.

@file{kernel/main.fs} expects the machine description file name on the
stack; the cross compiler itself (@file{cross.fs}) assumes that either
@code{mach-file} leaves a counted string on the stack, or
@code{machine-file} leaves an address, count pair of the filename on the
stack.

The feature list is typically controlled using @code{SetValue}, generic
files that are used by several projects can use @code{DefaultValue}
instead. Both functions work like @code{Value}, when the value isn't
defined, but @code{SetValue} works like @code{to} if the value is
defined, and @code{DefaultValue} doesn't set anything, if the value is
defined.

@example
\ generic mach file for pc gforth                       03sep97jaw

true DefaultValue NIL  \ relocating

>ENVIRON

true DefaultValue file          \ controls the presence of the
                                \ file access wordset
true DefaultValue OS            \ flag to indicate a operating system

true DefaultValue prims         \ true: primitives are c-code

true DefaultValue floating      \ floating point wordset is present

true DefaultValue glocals       \ gforth locals are present
                                \ will be loaded
true DefaultValue dcomps        \ double number comparisons

true DefaultValue hash          \ hashing primitives are loaded/present

true DefaultValue xconds        \ used together with glocals,
                                \ special conditionals supporting gforths'
                                \ local variables
true DefaultValue header        \ save a header information

true DefaultValue backtrace     \ enables backtrace code

false DefaultValue ec
false DefaultValue crlf

cell 2 = [IF] &32 [ELSE] &256 [THEN] KB DefaultValue kernel-size

&16 KB          DefaultValue stack-size
&15 KB &512 +   DefaultValue fstack-size
&15 KB          DefaultValue rstack-size
&14 KB &512 +   DefaultValue lstack-size
@end example

@node How the Cross Compiler Works,  , Using the Cross Compiler, Cross Compiler
@section How the Cross Compiler Works

@node MINOS2, Bugs, Cross Compiler, Top
@chapter MINOS2, a GUI library

@menu
* MINOS2 object framework::
* MINOS2 tutorial::
@end menu

@node MINOS2 object framework, MINOS2 tutorial, MINOS2, MINOS2
@section MINOS2 object framework

@menu
* actor methods::
* widget methods::
@end menu

MINOS2 is a GUI library, written in @file{mini-oof2.fs}'s object model.
It has two main class hierarchies:

doc-actor
doc-widget

@node actor methods, widget methods, MINOS2 object framework, MINOS2 object framework
@subsection @code{actor} methods:

doc-caller-w
doc-active-w
doc-act-name$
doc-clicked
doc-scrolled
doc-touchdown
doc-touchup
doc-ukeyed
doc-ekeyed
doc-?inside
doc-focus
doc-defocus
doc-entered
doc-left
doc-show
doc-hide
doc-get
doc-set
doc-show-you

@node widget methods,  , actor methods, MINOS2 object framework
@subsection @code{widget} methods:

doc-parent-w
doc-act
doc-name$
doc-x
doc-y
doc-w
doc-h
doc-d
doc-gap
doc-baseline
doc-kerning
doc-raise
doc-border
doc-borderv
doc-bordert
doc-borderl
doc-w-color
doc-draw-init
doc-draw
doc-split
doc-lastfit
doc-hglue
doc-dglue
doc-vglue
doc-hglue@
doc-dglue@
doc-vglue@
doc-xywh
doc-xywhd
doc-!resize
doc-!size
doc-dispose-widget
doc-.widget
doc-par-split
doc-resized

Components are composed using a boxes&glue model similar to @LaTeX{},
including paragraph breaking.  For the sake of simplicity and
portability, MINOS2 only supports a single window, and uses OpenGL for
rendering.

MINOS2 furthermore supports animations with the @code{animation} class.
A color index texture is used for different color schemes, and
transition between neighboring schemes can also be animated.

doc->animate

You can create named color indexes and assign them color values for the
currently active color scheme.

doc-color:
doc-new-color:
doc-text-color:
doc-text-emoji-color:
doc-fade-color:
doc-text-emoji-fade-color:
doc-re-color
doc-re-text-color
doc-re-emoji-color
doc-re-fade-color
doc-re-text-emoji-fade-color

For a number of specific objects, there are early bound methods, that
only work on these objects

@itemize @bullet
@item Viewport

doc-vp-top
doc-vp-bottom
doc-vp-left
doc-vp-right
doc-vp-reslide
doc-vp-needed

@end itemize

@node MINOS2 tutorial,  , MINOS2 object framework, MINOS2
@section MINOS2 tutorial

Tutorials are small files, each showing a bit of MINOS2.  For the common
framework, the file @file{minos2/tutorial/tutorial.fs} needs to be
loaded first; all other tutorials in the command line argument are
included from within that file.  Scroll wheel or previous/next mouse
buttons as well as clicking on the left or right edge of the window
allow navigation between the different tutorials loaded.

I.e. to load the buttons tutorial, you start Gforth with

@example
gforth minos2/tutorial/tutorial.fs buttons.fs
@end example

Available tutorials:

@itemize @bullet
@item
@file{buttons.fs}: Clickable buttons

@item
@file{plots.fs}: Plot functions

@item
@file{markdown.fs}: Markdown document viewer

@item
@file{screenshot.fs}: Screenshot function
@end itemize

@node Bugs, Origin, MINOS2, Top
@appendix Bugs
@cindex bug reporting

Known bugs are described in the file @file{BUGS} in the Gforth distribution.

If you find a bug, please submit a bug report through
@uref{https://savannah.gnu.org/bugs/?func=addbug&group=gforth}.

@itemize @bullet
@item
A program (or a sequence of keyboard commands) that reproduces the bug.
@item
A description of what you think constitutes the buggy behaviour.
@item
The Gforth version used (it is announced at the start of an
interactive Gforth session).
@item
The machine and operating system (on Unix
systems @code{uname -a} will report this information).
@item
The installation options (you can find the configure options at the
start of @file{config.status}) and configuration (@code{configure}
output or @file{config.cache}).
@item
A complete list of changes (if any) you (or your installer) have made to the
Gforth sources.
@end itemize

For a thorough guide on reporting bugs read @ref{Bug Reporting, , How
to Report Bugs, gcc, GNU C Manual}.


@node Origin, Forth-related information, Bugs, Top
@appendix Authors and Ancestors of Gforth

@section Authors and Contributors
@cindex authors of Gforth
@cindex contributors to Gforth

The Gforth project was started in mid-1992 by Bernd Paysan and Anton
Ertl. The third major author was Jens Wilke.  Neal Crook contributed a
lot to the manual.  Assemblers and disassemblers were contributed by
Andrew McKewan, Christian Pirker, Bernd Thallner, and Michal Revucky.
Lennart Benschop (who was one of Gforth's first users, in mid-1993)
and Stuart Ramsden inspired us with their continuous feedback. Lennart
Benshop contributed @file{glosgen.fs}, while Stuart Ramsden has been
working on automatic support for calling C libraries. Helpful comments
also came from Paul Kleinrubatscher, Christian Pirker, Dirk Zoller,
Marcel Hendrix, John Wavrik, Barrie Stott, Marc de Groot, Jorge
Acerada, Bruce Hoyt, Robert Epprecht, Dennis Ruffer and David
N. Williams. Since the release of Gforth-0.2.1 there were also helpful
comments from many others; thank you all, sorry for not listing you
here (but digging through my mailbox to extract your names is on my
to-do list).

Gforth also owes a lot to the authors of the tools we used (GCC, CVS,
and autoconf, among others), and to the creators of the Internet: Gforth
was developed across the Internet, and its authors did not meet
physically for the first 4 years of development.

@section Pedigree
@cindex pedigree of Gforth

Gforth descends from bigFORTH (1993) and fig-Forth.  Of course, a
significant part of the design of Gforth was prescribed by Standard Forth.

Bernd Paysan wrote bigFORTH, a descendent from TurboForth, an unreleased
32 bit native code version of VolksForth for the Atari ST, written
mostly by Dietrich Weineck.

VolksForth was written by Klaus Schleisiek, Bernd Pennemann, Georg
Rehfeld and Dietrich Weineck for the C64 (called UltraForth there) in
the mid-80s and ported to the Atari ST in 1986.  It descends from fig-Forth.

@c Henry Laxen and Mike Perry wrote F83 as a model implementation of the
@c Forth-83 standard. !! Pedigree? When?

A team led by Bill Ragsdale implemented fig-Forth on many processors in
1979. Robert Selzer and Bill Ragsdale developed the original
implementation of fig-Forth for the 6502 based on microForth.

The principal architect of microForth was Dean Sanderson. microForth was
FORTH, Inc.'s first off-the-shelf product. It was developed in 1976 for
the 1802, and subsequently implemented on the 8080, the 6800 and the
Z80.

All earlier Forth systems were custom-made, usually by Charles Moore,
who discovered (as he puts it) Forth during the late 60s. The first full
Forth existed in 1971.

A part of the information in this section comes from
@cite{@uref{https://www.forth.com/resources/evolution/index.html,The
Evolution of Forth}} by Elizabeth D. Rather, Donald R. Colburn and
Charles H. Moore, presented at the HOPL-II conference and preprinted
in SIGPLAN Notices 28(3), 1993.  You can find more historical and
genealogical information about Forth there.  For a more general (and
graphical) Forth family tree look see
@cite{@uref{https://www.complang.tuwien.ac.at/forth/family-tree/},
Forth Family Tree and Timeline}.

@c ------------------------------------------------------------------
@node Forth-related information, Licenses, Origin, Top
@appendix Other Forth-related information
@cindex Forth-related information

@c anton: I threw most of this stuff out, because it can be found through
@c the FAQ and the FAQ is more likely to be up-to-date.

@cindex comp.lang.forth
@cindex frequently asked questions
There is an active news group (comp.lang.forth) discussing Forth
(including Gforth) and Forth-related issues. Its
@uref{https://www.complang.tuwien.ac.at/forth/faq/faq-general-2.html,FAQs}
(frequently asked questions and their answers) contains a lot of
information on Forth.  You should read it before posting to
comp.lang.forth.

The Forth standard is most usable in its
@uref{https://forth-standard.org/, HTML form}.

@c ---------------------------------------------------
@node  Licenses, Word Index, Forth-related information, Top
@appendix Licenses

@menu
* GNU Free Documentation License::  License for copying this manual.
* Copying::                     GPL (for copying this software).
@end menu

@node GNU Free Documentation License, Copying, Licenses, Licenses
@appendixsec GNU Free Documentation License
@include fdl.texi

@node Copying,  , GNU Free Documentation License, Licenses
@appendixsec GNU GENERAL PUBLIC LICENSE
@include gpl.texi



@c ------------------------------------------------------------------
@node Word Index, Concept Index, Licenses, Top
@unnumbered Word Index

This index is a list of Forth words that have ``glossary'' entries
within this manual. Each word is listed with its stack effect and
wordset.

@printindex fn

@c anton: the name index seems superfluous given the word and concept indices.

@c @node Name Index, Concept Index, Word Index, Top
@c @unnumbered Name Index

@c This index is a list of Forth words that have ``glossary'' entries
@c within this manual.

@c @printindex ky

@c -------------------------------------------------------
@node Concept Index,  , Word Index, Top
@unnumbered Concept and Word Index

Not all entries listed in this index are present verbatim in the
text. This index also duplicates, in abbreviated form, all of the words
listed in the Word Index (only the names are listed for the words here).

@printindex cp

@bye



