<!--
  - Authors:
  -   Denys Duchier <duchier@ps.uni-sb.de>
  -   Leif Kornstaedt <kornstae@ps.uni-sb.de>
  -   Christian Schulte <schulte@ps.uni-sb.de>
  -
  - Contributors:
  -   Daniel Simon <dansim@ps.uni-sb.de> (SGML Translation)
  -
  - Copyright:
  -   Denys Duchier, Leif Kornstaedt, and Christian Schulte, 1998
  -
  - Last change:
  -   $Date$ by $Author$
  -   $Revision$
  -
  - This file is part of Mozart, an implementation of Oz 3:
  -   http://www.mozart-oz.org
  -
  - See the file "LICENSE" or
  -   http://www.mozart-oz.org/LICENSE.html
  - for information on usage and redistribution
  - of this file, and for a DISCLAIMER OF ALL
  - WARRANTIES.
  -
  -->

<Chapter id="chapter.text">
  <Title/Text/
  <P>
    This chapter describes modules for handling data encoding textual
    information.  <Index/characters/<Def class=noindex/Characters/ are
    encoded as integers.  <Index/strings/<Def class=noindex/Strings/ are
    lists of characters.  <Index/virtual strings/<Def class=noindex/Virtual
    Strings/ are atoms, strings, byte strings, integers, and floats closed
    under virtual concatenation encoded by tuples with label&nbsp;<<'#'>>.
  <P>
    For example,
    <<<"Contains "#also#" numbers: "#(1#' '#2.045)>>>
    is a virtual string representing the string
    <<<"Contains also numbers: 1 2.045">>>

  <Section id="section.text.characters">
    <Title/Characters/
    <P>
      The module <Span class=index/<<Char>>/ contains procedures operating
      on characters.  Characters are integers between <<0>> and&nbsp;<<255>>,
      used for building strings.  For the encoding of characters by integers,
      we use the ISO&nbsp;8859-1 standard&nbsp;<Ptr to="ISO_8859_1">.
      The functionality provided by this module is similar to the
      <File/ctype.h/ module of ANSI&nbsp;C, see for instance&nbsp;<Ptr
      to="kernighan88">.
    <P>
      The procedures described herein can be used to compute with strings
      by using the generic procedures of the list module (see <Ptr
      to="section.records.lists">).  For example,
      <<<{Filter "r E!m O\nv7E" Char.isAlpha}>>>
      keeps only the letters of the given string and returns the
      string <<"rEmOvE">>.
      <List>
	<Entry><Span class=index/<<IsChar>>/
	  <Index class=tails/<<Char>><And><<is>>/
	<Synopsis>
	  <<{Char.is ?+{X} ??{B}}>>
	<Item>
	  tests whether ?_{X} is a character, &ie;, an integer between <<0>>
	  and&nbsp;<<255>> inclusively.

	<Entry><<isLower>>
	  <Index class=tails/<<Char>><And><<isLower>>/
	<Synopsis>
	  <<{Char.isLower ?+{Char} ??{B}}>>
	<Item>
	  tests whether ?_{Char} encodes a lower-case letter.

	<Entry><<isUpper>>
	  <Index class=tails/<<Char>><And><<isUpper>>/
	<Synopsis>
	  <<{Char.isUpper ?+{Char} ??{B}}>>
	<Item>
	  tests whether ?_{Char} encodes an upper-case letter.

	<Entry><<isDigit>>
	  <Index class=tails/<<Char>><And><<isDigit>>/
	<Synopsis>
	  <<{Char.isDigit ?+{Char} ??{B}}>>
	<Item>
	  tests whether ?_{Char} encodes a digit.

	<Entry><<isSpace>>
	  <Index class=tails/<<Char>><And><<isSpace>>/
	<Synopsis>
	  <<{Char.isSpace ?+{Char} ??{B}}>>
	<Item>
	  tests whether ?_{Char} encodes a white space character, &ie;, either
	  a space, a form feed (<<&\f>>), a newline (<<&\n>>), a carriage
	  return (<<&\r>>), a tab (<<&\t>>), a vertical tab (<<&\v>>) or
	  a non-breaking space (<<&\240>>).

	<Entry><<isPunct>>
	  <Index class=tails/<<Char>><And><<isPunct>>/
	<Synopsis>
	  <<{Char.isPunct ?+{Char} ??{B}}>>
	<Item>
	  tests whether ?_{Char} encodes a punctuation character, &ie;, a
	  visible character, which is not a space, a digit, or a letter.

	<Entry><<isCntrl>>
	  <Index class=tails/<<Char>><And><<isCntrl>>/
	<Synopsis>
	  <<{Char.isCntrl ?+{Char} ??{B}}>>
	<Item>
	  tests whether ?_{Char} encodes a control character.

	<Entry><<isAlpha>>
	  <Index class=tails/<<Char>><And><<isAlpha>>/
	<Synopsis>
	  <<{Char.isAlpha ?+{Char} ??{B}}>>
	<Item>
	  tests whether ?_{Char} encodes a letter.

	<Entry><<isAlNum>>
	  <Index class=tails/<<Char>><And><<isAlNum>>/
	<Synopsis>
	  <<{Char.isAlNum ?+{Char} ??{B}}>>
	<Item>
	  tests whether ?_{Char} encodes a letter or a digit.

	<Entry><<isGraph>>
	  <Index class=tails/<<Char>><And><<isGraph>>/
	<Synopsis>
	  <<{Char.isGraph ?+{Char} ??{B}}>>
	<Item>
	  tests whether ?_{Char} encodes a visible character.

	<Entry><<isPrint>>
	  <Index class=tails/<<Char>><And><<isPrint>>/
	<Synopsis>
	  <<{Char.isPrint ?+{Char} ??{B}}>>
	<Item>
	  tests whether ?_{Char} is a visible character or either the space
	  or non-breaking space character.

	<Entry><<isXDigit>>
	  <Index class=tails/<<Char>><And><<isXDigit>>/
	<Synopsis>
	  <<{Char.isXDigit ?+{Char} ??{B}}>>
	<Item>
	  tests whether ?_{Char} is a hexadecimal digit.

	<Entry><<type>>
	  <Index class=tails/<<Char>><And><<type>>/
	<Synopsis>
	  <<{Char.type ?+{Char} ??{A}}>>
	<Item>
	  maps ?_{Char} to its simple type&nbsp;?_{A}, &ie;, one of the
	  atoms <<lower>>, <<upper>>, <<digit>>, <<space>>, <<punct>>, or
	  <<other>>.

	<Entry><<toLower>>
	  <Index class=tails/<<Char>><And><<toLower>>/
	<Synopsis>
	  <<{Char.toLower ?+{Char1} ??{Char2}}>>
	<Item>
	  returns the corresponding lower-case letter if ?_{Char1} is an
	  upper-case letter, otherwise ?_{Char1} itself.

	<Entry><<toUpper>>
	  <Index class=tails/<<Char>><And><<toUpper>>/
	<Synopsis>
	  <<{Char.toUpper ?+{Char1} ??{Char2}}>>
	<Item>
	  returns the corresponding upper-case letter if ?_{Char1} is a
	  lower-case letter, otherwise ?_{Char1} itself.

	<Entry><<toAtom>>
	<Synopsis>
	  <<{Char.toAtom ?+{Char} ??{A}}>>
	<Item>
	  maps ?_{Char} to the corresponding atom&nbsp;?_{A}.  If ?_{Char}
	  is zero, ?_{A}&nbsp;will be the empty atom&nbsp;<<''>>.
      </List>

  <Section id="section.text.strings">
    <Title/Strings/
    <P>
      The module <Span class=index/<<String>>/ contains procedures operating
      on strings.  Strings are lists whose elements are characters (see
      <Ptr to="section.text.characters">).
      <List>
	<Entry><Span class=index/<<IsString>>/
	  <Index class=tails/<<String>><And><<is>>/
	<Synopsis>
	  <<{String.is ?+{X} ??{B}}>>
	<Item>
	  tests whether ?_{X} is string.

	<Entry><Span class=index/<<StringToAtom>>/
	  <Index class=tails/<<String>><And><<toAtom>>/
	<Synopsis>
	  <<{String.toAtom ?+{S} ??{A}}>>
	<Item>
	  converts a string&nbsp;?_{S} to an atom&nbsp;?_{A}.  ?_{S}&nbsp;must
	  not contain NUL characters.  This is the inverse of <<Atom.toString>>
	  (which see).

	<Entry><<isAtom>>
	  <Index class=tails/<<String>><And><<isAtom>>/
	<Synopsis>
	  <<{String.isAtom ?+{S} ??{B}}>>
	<Item>
	  tests whether the string&nbsp;?_{S} can be converted to an atom.

	<Entry><Span class=index/<<StringToInt>>/
	  <Index class=tails/<<String>><And><<toInt>>/
	<Synopsis>
	  <<{String.toInt ?+{S} ??{I}}>>
	<Item>
	  converts a string&nbsp;?_{S} to an integer&nbsp;?_{I}, according to
	  &Oz; concrete syntax.  See also <<IntToString>>.

	<Entry><<isInt>>
	  <Index class=tails/<<String>><And><<isInt>>/
	<Synopsis>
	  <<{String.isInt ?+{S} ??{B}}>>
	<Item>
	  tests whether the string&nbsp;?_{S} can be converted to an integer.

	<Entry><Span class=index/<<StringToFloat>>/
	  <Index class=tails/<<String>><And><<toFloat>>/
	<Synopsis>
	  <<{String.toFloat ?+{S} ??{F}}>>
	<Item>
	  converts a string&nbsp;?_{S} to a float&nbsp;?_{F}, according to
	  &Oz; concrete syntax.  See also <<FloatToString>>.

	<Entry><<isFloat>>
	  <Index class=tails/<<String>><And><<isFloat>>/
	<Synopsis>
	  <<{String.isFloat ?+{S} ??{B}}>>
	<Item>
	  tests whether the string&nbsp;?_{S} can be converted to a float.

	<Entry><<token>>
	  <Index class=tails/<<String>><And><<token>>/
	<Synopsis>
	  <<{String.token ?+{S1} ?_{X} ??{S2} ??{S3}}>>
	<Item>
	  splits the string&nbsp;?_{S1} into two substrings ?_{S2}
	  and&nbsp;?_{S3}.  ?_{S2} will contain all characters before the
	  first occurence of&nbsp;?_{X}, ?_{S3} all remaining characters
	  with&nbsp;?_{X} excluded.  If ?_{X} does not occur in&nbsp;?_{S1},
	  then ?_{S2} will be equal to&nbsp;?_{S1} and ?_{S3} will be the
	  empty string.
	<P>
	  For example,
	  <<<{String.token "a:b:c" &: S1 S2}>>>
	  binds <<S1>> to <<"a">> and <<S2>> to <<"b:c">>.

	<Entry><<tokens>>
	  <Index class=tails/<<String>><And><<tokens>>/
	<Synopsis>
	  <<{String.tokens ?+{S} ?_{X} ??{Ss}}>>
	<Item>
	  splits the string&nbsp;?_{S} into substrings&nbsp;?_{Ss} delimited
	  by occurrences of&nbsp;?_{X} in&nbsp;?_{S}.  Note that the final
	  empty string will be omitted if the last element of&nbsp;?_{S} is
	  an&nbsp;?_{X}.
	<P>
	  For example,
	  <<<{String.tokens "a:bb:cc:d:" &:}>>>
	  returns <<["a" "bb" "cc" "d"]>>.
      </List>

  <Section id="section.text.bytestrings">
    <Title/Byte Strings/
    <P>
      Module <Span class=index/<<ByteString>>/ provides an interface to a more
      economical representation for textual data: a simple array of bytes.
      In terms of memory, the economy is at least a factor of&nbsp;8, which
      may improve local processing, IO, pickle sizes, and remote
      communications (distributed processing).  However strings, &ie;,
      lists, are often more convenient for all forms of recursive processing
      (&eg;, <<Map>> or <<Filter>>).  Typically, you will be processing
      textual data in list form, but saving or communicating it in byte
      string form.
      <List>
	<Entry><Span class=index/<<IsByteString>>/
	  <Index class=tails/<<ByteString>><And><<is>>/
	<Synopsis>
	  <<{ByteString.is ?+{X} ??{B}}>>
	<Item>
	  tests whether ?_{X} is a byte string.

	<Entry><<make>>
	  <Index class=tails/<<ByteString>><And><<make>>/
	<Synopsis>
	  <<{ByteString.make ?+{V} ??{ByteString}}>>
	<Item>
	  returns a new byte string created from the virtual string&nbsp;?_{V}.

	<Entry><<get>>
	  <Index class=tails/<<ByteString>><And><<get>>/
	<Synopsis>
	  <<{ByteString.get ?+{ByteString} ?+{I} ??{C}}>>
	<Item>
	  retrieves the ?_{I}th character of byte string&nbsp;?_{ByteString}.
	  The first index is&nbsp;<<0>>.

	<Entry><<append>>
	  <Index class=tails/<<ByteString>><And><<append>>/
	<Synopsis>
	  <<{ByteString.append ?+{ByteString1} ?+{ByteString2} ??{ByteString3}}>>
	<Item>
	  returns the new byte string ?_{ByteString3} which is the
	  concatenation of ?_{ByteString1} and ?_{ByteString2}.

	<Entry><<slice>>
	  <Index class=tails/<<ByteString>><And><<slice>>/
	<Synopsis>
	  <<{ByteString.slice ?+{ByteString1} ?+{FromI} ?+{ToI} ??{ByteString2}}>>
	<Item>
	  returns a new byte string for the bytes in ?_{ByteString1} starting
	  at index ?_{FromI} and extending up to, but not including, index
	  ?_{ToI}.

	<Entry><<width>>
	  <Index class=tails/<<ByteString>><And><<width>>/
	<Entry><<length>>
	  <Index class=tails/<<ByteString>><And><<length>>/
	<Synopsis>
	  <<{ByteString.width ?+{ByteString} ??{I}}>>
	<Synopsis>
	  <<{ByteString.length ?+{ByteString} ??{I}}>>
	<Item>
	  returns the width&nbsp;?_{I} of ?_{ByteString}.

	<Entry><<toString>>
	  <Index class=tails/<<ByteString>><And><<toString>>/
	<Synopsis>
	  <<{ByteString.toString ?+{ByteString} ??{S}}>>
	<Item>
	  converts ?_{ByteString} to a string&nbsp;?_{S}.

	<Entry><<toStringWithTail>>
	  <Index class=tails/<<ByteString>><And><<toStringWithTail>>/
	<Synopsis>
	  <<{ByteString.toStringWithTail ?+{ByteString} ?_{X} ??{S}}>>
	<Item>
	  converts ?_{ByteString} to a string&nbsp;?_{S} ending
	  with&nbsp;?_{X}.  This is useful for subsequently instantiating
	  ?_{X}, &eg;, with another call to <<ByteString.toStringWithTail>>.

	<Entry><<strchr>>
	  <Index class=tails/<<ByteString>><And><<strchr>>/
	<Synopsis>
	  <<{ByteString.strchr ?+{ByteString} ?+{OffsetI} ?+{Char} ??{PosBI}}>>
	<Item>
	  returns the position ?_{PosI} of the first occurrence of ?_{Char}
	  in ?_{ByteString} starting at offset ?_{OffsetI}.  If none is found
	  <<false>> is returned instead.
      </List>

  <Section id="section.text.virtualstrings">
    <Title/Virtual Strings/
    <P>
      <Index/strings<And>virtual/
      The module <Span class=index/<<VirtualString>>/ contains procedures
      operating on virtual strings.  Virtual strings are designed as a
      convenient way to combine strings, byte strings, atoms, integers
      and floats to compound strings without explicit concatenation and
      conversion.
      <List>
	<Entry><Span class=index/<<IsVirtualString>>/
	  <Index class=tails/<<VirtualString>><And><<is>>/
	<Synopsis>
	  <<{VirtualString.is ?+{X} ??{B}}>>
	<Item>
	  tests whether ?_{X} is a virtual string.  Virtual strings are
	  defined recursively as the set of all integers, floats, atoms,
	  strings, byte strings, and tuples with label&nbsp;<<'#'>> whose
	  subtrees are virtual strings.

	<Entry><<toString>>
	  <Index class=tails/<<VirtualString>><And><<toString>>/
	<Synopsis>
	  <<{VirtualString.toString ?+{V} ??{S}}>>
	<Item>
	  converts a virtual string&nbsp;?_{V} to a string&nbsp;?_{S}.
	<P>
	  The transformation is straightforward:  Atoms (except <<nil>>
	  and&nbsp;<<'#'>>), integers, floats and byte strings are
	  transformed into strings using <<Atom.toString>>, <<Int.toString>>,
	  <<Float.toString>>, and <<ByteString.toString>> respectively, where
	  in numbers <Samp/-/ is used instead of <Samp/~/.  A tuple with
	  label&nbsp;<<'#'>> is transformed by concatenation of the
	  transformed subtrees.  Note that both <<nil>> and <<'#'>> denote
	  the empty string.
	<P>
	  The following relation holds for all virtual strings <<V1>>
	  and&nbsp;<<V2>>:
<<<{VirtualString.toString V1#V2}
= {Append
   {VirtualString.toString V1}
   {VirtualString.toString V2}}>>>
	  Thus, <<VirtualString.toString>> maps <<#>> homomorphically to
	  <<Append>>.

	<Entry><<toAtom>>
	  <Index class=tails/<<VirtualString>><And><<toAtom>>/
	<Synopsis>
	  <<{VirtualString.toAtom ?+{V} ??{A}}>>
	<Item>
	  converts a virtual string&nbsp;?_{V} to an atom&nbsp;?_{A}.
	<P>
	  This procedure can be defined as:
<<<fun {VirtualString.toAtom V}
   {String.toAtom {VirtualString.toString V}}
end>>>

	<Entry><<toByteString>>
	  <Index class=tails/<<VirtualString>><And><<toByteString>>/
	<Synopsis>
	  <<{VirtualString.toByteString ?+{V} ??{ByteString}}>>
	<Item>
	  converts a virtual string&nbsp;?_{V} to a byte string <<ByteString>>.
	<P>
	  This procedure is a synonym of <<ByteString.make>> (which see).

	<Entry><<length>>
	  <Index class=tails/<<VirtualString>><And><<length>>/
	<Synopsis>
	  <<{VirtualString.length ?+{V} ??{I}}>>
	<Item>
	  returns the length of a virtual string in characters.
	  Can be defined as:
	  <<<{Length {VirtualString.toString ?_{V}} ?_{I}}>>>

	<Entry><<changeSign>>
	  <Index class=tails/<<VirtualString>><And><<changeSign>>/
	<Synopsis>
	  <<{VirtualString.changeSign ?+{V1} ?_{X} ??{V2}}>>
	<Item>
	  returns a virtual string derived from&nbsp;?_{V1} where all
	  occurrences of the unary minus sign for integers and floats
	  are replaced by&nbsp;?_{X}.
      </List>
</Chapter>

<!--
Local Variables:
mode: sgml
mode: auto-fill
sgml-default-dtd-file: "chapter.ced"
sgml-auto-activate-dtd: t
adaptive-fill-mode: nil
End:
-->
