summaryrefslogtreecommitdiff
path: root/doc/ref
diff options
context:
space:
mode:
Diffstat (limited to 'doc/ref')
-rw-r--r--doc/ref/Makefile.am4
-rw-r--r--doc/ref/api-control.texi9
-rw-r--r--doc/ref/api-data.texi13
-rw-r--r--doc/ref/api-debug.texi23
-rw-r--r--doc/ref/api-evaluation.texi12
-rw-r--r--doc/ref/api-io.texi24
-rw-r--r--doc/ref/api-macros.texi8
-rw-r--r--doc/ref/api-peg.texi1036
-rw-r--r--doc/ref/compiler.texi92
-rw-r--r--doc/ref/guile-invoke.texi4
-rw-r--r--doc/ref/guile.texi2
-rw-r--r--doc/ref/libguile-concepts.texi26
-rw-r--r--doc/ref/posix.texi9
-rw-r--r--doc/ref/srfi-modules.texi23
-rw-r--r--doc/ref/vm.texi21
15 files changed, 1157 insertions, 149 deletions
diff --git a/doc/ref/Makefile.am b/doc/ref/Makefile.am
index d0ea94d51..75b1745fa 100644
--- a/doc/ref/Makefile.am
+++ b/doc/ref/Makefile.am
@@ -44,6 +44,7 @@ guile_TEXINFOS = preface.texi \
api-foreign.texi \
api-regex.texi \
api-lalr.texi \
+ api-peg.texi \
api-languages.texi \
api-evaluation.texi \
api-memory.texi \
@@ -118,8 +119,7 @@ EXTRA_DIST = ChangeLog-2008 $(PICTURES)
libguile-autoconf.texi: autoconf-macros.texi
autoconf-macros.texi: $(top_srcdir)/meta/guile.m4
- GUILE_INSTALL_LOCALE=1 GUILE_AUTO_COMPILE=0 \
- $(top_builddir)/meta/uninstalled-env guild \
+ GUILE_AUTO_COMPILE=0 $(top_builddir)/meta/uninstalled-env guild \
snarf-guile-m4-docs $(top_srcdir)/meta/guile.m4 \
> $(srcdir)/$@
diff --git a/doc/ref/api-control.texi b/doc/ref/api-control.texi
index 7ffb3f740..026308cac 100644
--- a/doc/ref/api-control.texi
+++ b/doc/ref/api-control.texi
@@ -494,14 +494,17 @@ those passed to @code{abort-to-prompt}.
@end deffn
@deffn {Scheme Procedure} make-prompt-tag [stem]
-Make a new prompt tag. Currently prompt tags are generated symbols.
-This may change in some future Guile version.
+Make a new prompt tag. A prompt tag is simply a unique object.
+Currently, a prompt tag is a fresh pair. This may change in some future
+Guile version.
@end deffn
@deffn {Scheme Procedure} default-prompt-tag
Return the default prompt tag. Having a distinguished default prompt
tag allows some useful prompt and abort idioms, discussed in the next
-section.
+section. Note that @code{default-prompt-tag} is actually a parameter,
+and so may be dynamically rebound using @code{parameterize}.
+@xref{Parameters}.
@end deffn
@deffn {Scheme Procedure} abort-to-prompt tag val1 val2 @dots{}
diff --git a/doc/ref/api-data.texi b/doc/ref/api-data.texi
index 096970cb5..760318028 100644
--- a/doc/ref/api-data.texi
+++ b/doc/ref/api-data.texi
@@ -4448,6 +4448,19 @@ returned is the number of bytes for @code{scm_to_latin1_stringn} and
for @code{scm_to_utf32_stringn}.
@end deftypefn
+It is not often the case, but sometimes when you are dealing with the
+implementation details of a port, you need to encode and decode strings
+according to the encoding and conversion strategy of the port. There
+are some convenience functions for that purpose as well.
+
+@deftypefn {C Function} SCM scm_from_port_string (const char *str, SCM port)
+@deftypefnx {C Function} SCM scm_from_port_stringn (const char *str, size_t len, SCM port)
+@deftypefnx {C Function} char* scm_to_port_string (SCM str, SCM port)
+@deftypefnx {C Function} char* scm_to_port_stringn (SCM str, size_t *lenp, SCM port)
+Like @code{scm_from_stringn} and friends, except they take their
+encoding and conversion strategy from a given port object.
+@end deftypefn
+
@node String Internals
@subsubsection String Internals
diff --git a/doc/ref/api-debug.texi b/doc/ref/api-debug.texi
index f6c706c78..4e1b82295 100644
--- a/doc/ref/api-debug.texi
+++ b/doc/ref/api-debug.texi
@@ -799,10 +799,11 @@ To digress, Guile's VM has 6 different hooks (@pxref{Hooks}) that can be
fired at different times, which may be accessed with the following
procedures.
-All hooks are called with one argument, the frame in
-question. @xref{Frames}. Since these hooks may be fired very
-frequently, Guile does a terrible thing: it allocates the frames on the
-C stack instead of the garbage-collected heap.
+The first argument of calls to these hooks is the frame in question.
+@xref{Frames}. Some hooks may call their procedures with more
+arguments. Since these hooks may be fired very frequently, Guile does a
+terrible thing: it allocates the frames on the C stack instead of the
+garbage-collected heap.
The upshot here is that the frames are only valid within the dynamic
extent of the call to the hook. If a hook procedure keeps a reference to
@@ -832,11 +833,8 @@ corresponding apply-hook.
@deffn {Scheme Procedure} vm-pop-continuation-hook vm
The hook that will be fired before returning from a frame.
-This hook is a bit trickier than the rest, in that there is a particular
-interpretation of the values on the stack. Specifically, the top value
-on the stack is the number of values being returned, and the next
-@var{n} values are the actual values being returned, with the last value
-highest on the stack.
+This hook fires with a variable number of arguments, corresponding to
+the values that the frame returns to its continuation.
@end deffn
@deffn {Scheme Procedure} vm-apply-hook vm
@@ -852,8 +850,11 @@ hook.
@deffn {Scheme Procedure} vm-abort-continuation-hook vm
The hook that will be called after aborting to a
-prompt. @xref{Prompts}. The stack will be in the same state as for
-@code{vm-pop-continuation-hook}.
+prompt. @xref{Prompts}.
+
+Like the pop-continuation hook, this hook fires with a variable number
+of arguments, corresponding to the values that returned to the
+continuation.
@end deffn
@deffn {Scheme Procedure} vm-restore-continuation-hook vm
diff --git a/doc/ref/api-evaluation.texi b/doc/ref/api-evaluation.texi
index 63b1d6059..f0edd1153 100644
--- a/doc/ref/api-evaluation.texi
+++ b/doc/ref/api-evaluation.texi
@@ -573,18 +573,6 @@ Call @var{proc} with the array of arguments @var{argv}, as a
@var{nargs}, as a @code{size_t}.
@end deffn
-@deffn {Scheme Procedure} apply:nconc2last lst
-@deffnx {C Function} scm_nconc2last (lst)
-@var{lst} should be a list (@var{arg1} @dots{} @var{argN}
-@var{arglst}), with @var{arglst} being a list. This function returns
-a list comprising @var{arg1} to @var{argN} plus the elements of
-@var{arglst}. @var{lst} is modified to form the return. @var{arglst}
-is not modified, though the return does share structure with it.
-
-This operation collects up the arguments from a list which is
-@code{apply} style parameters.
-@end deffn
-
@deffn {Scheme Procedure} primitive-eval exp
@deffnx {C Function} scm_primitive_eval (exp)
Evaluate @var{exp} in the top-level environment specified by
diff --git a/doc/ref/api-io.texi b/doc/ref/api-io.texi
index f1170eb2c..5ca3506a9 100644
--- a/doc/ref/api-io.texi
+++ b/doc/ref/api-io.texi
@@ -1066,28 +1066,6 @@ away from its default.
Calls the one-argument procedure @var{proc} with a newly created output
port. When the function returns, the string composed of the characters
written into the port is returned. @var{proc} should not close the port.
-
-Note that which characters can be written to a string port depend on the port's
-encoding. The default encoding of string ports is specified by the
-@code{%default-port-encoding} fluid (@pxref{Ports,
-@code{%default-port-encoding}}). For instance, it is an error to write Greek
-letter alpha to an ISO-8859-1-encoded string port since this character cannot be
-represented with ISO-8859-1:
-
-@example
-(define alpha (integer->char #x03b1)) ; GREEK SMALL LETTER ALPHA
-
-(with-fluids ((%default-port-encoding "ISO-8859-1"))
- (call-with-output-string
- (lambda (p)
- (display alpha p))))
-
-@result{}
-Throw to key `encoding-error'
-@end example
-
-Changing the string port's encoding to a Unicode-capable encoding such as UTF-8
-solves the problem.
@end deffn
@deffn {Scheme Procedure} call-with-input-string string proc
@@ -1101,8 +1079,6 @@ read. The value yielded by the @var{proc} is returned.
Calls the zero-argument procedure @var{thunk} with the current output
port set temporarily to a new string port. It returns a string
composed of the characters written to the current output.
-
-See @code{call-with-output-string} above for character encoding considerations.
@end deffn
@deffn {Scheme Procedure} with-input-from-string string thunk
diff --git a/doc/ref/api-macros.texi b/doc/ref/api-macros.texi
index a3fa83f5a..82a2c071e 100644
--- a/doc/ref/api-macros.texi
+++ b/doc/ref/api-macros.texi
@@ -726,7 +726,7 @@ Return the name of the module whose source contains the identifier
@var{id}.
@end deffn
-@deffn {Scheme Procedure} syntax-local-binding id
+@deffn {Scheme Procedure} syntax-local-binding id [#:resolve-syntax-parameters?=#t]
Resolve the identifer @var{id}, a syntax object, within the current
lexical environment, and return two values, the binding type and a
binding value. The binding type is a symbol, which may be one of the
@@ -739,6 +739,12 @@ of @code{eq?}) identifying this binding.
@item macro
A syntax transformer, either local or global. The value is the
transformer procedure.
+@item syntax-parameter
+A syntax parameter (@pxref{Syntax Parameters}). By default,
+@code{syntax-local-binding} will resolve syntax parameters, so that this
+value will not be returned. Pass @code{#:resolve-syntax-parameters? #f}
+to indicate that you are interested in syntax parameters. The value is
+the default transformer procedure, as in @code{macro}.
@item pattern-variable
A pattern variable, bound via syntax-case. The value is an opaque
object, internal to the expander.
diff --git a/doc/ref/api-peg.texi b/doc/ref/api-peg.texi
new file mode 100644
index 000000000..0e16aab7e
--- /dev/null
+++ b/doc/ref/api-peg.texi
@@ -0,0 +1,1036 @@
+@c -*-texinfo-*-
+@c This is part of the GNU Guile Reference Manual.
+@c Copyright (C) 2006, 2010, 2011
+@c Free Software Foundation, Inc.
+@c See the file guile.texi for copying conditions.
+
+@node PEG Parsing
+@section PEG Parsing
+
+Parsing Expression Grammars (PEGs) are a way of specifying formal
+languages for text processing. They can be used either for matching
+(like regular expressions) or for building recursive descent parsers
+(like lex/yacc). Guile uses a superset of PEG syntax that allows more
+control over what information is preserved during parsing.
+
+Wikipedia has a clear and concise introduction to PEGs if you want to
+familiarize yourself with the syntax:
+@url{http://en.wikipedia.org/wiki/Parsing_expression_grammar}.
+
+The module works by compiling PEGs down to lambda expressions. These
+can either be stored in variables at compile-time by the define macros
+(@code{define-peg-pattern} and @code{define-peg-string-patterns}) or calculated
+explicitly at runtime with the compile functions
+(@code{compile-peg-pattern} and @code{peg-string-compile}).
+
+They can then be used for either parsing (@code{match-pattern}) or searching
+(@code{search-for-pattern}). For convenience, @code{search-for-pattern}
+also takes pattern literals in case you want to inline a simple search
+(people often use regular expressions this way).
+
+The rest of this documentation consists of a syntax reference, an API
+reference, and a tutorial.
+
+@menu
+* PEG Syntax Reference::
+* PEG API Reference::
+* PEG Tutorial::
+* PEG Internals::
+@end menu
+
+@node PEG Syntax Reference
+@subsection PEG Syntax Reference
+
+@subsubheading Normal PEG Syntax:
+
+@deftp {PEG Pattern} sequence a b
+Parses @var{a}. If this succeeds, continues to parse @var{b} from the
+end of the text parsed as @var{a}. Succeeds if both @var{a} and
+@var{b} succeed.
+
+@code{"a b"}
+
+@code{(and a b)}
+@end deftp
+
+@deftp {PEG Pattern} {ordered choice} a b
+Parses @var{a}. If this fails, backtracks and parses @var{b}.
+Succeeds if either @var{a} or @var{b} succeeds.
+
+@code{"a/b"}
+
+@code{(or a b)}
+@end deftp
+
+@deftp {PEG Pattern} {zero or more} a
+Parses @var{a} as many times in a row as it can, starting each @var{a}
+at the end of the text parsed by the previous @var{a}. Always
+succeeds.
+
+@code{"a*"}
+
+@code{(* a)}
+@end deftp
+
+@deftp {PEG Pattern} {one or more} a
+Parses @var{a} as many times in a row as it can, starting each @var{a}
+at the end of the text parsed by the previous @var{a}. Succeeds if at
+least one @var{a} was parsed.
+
+@code{"a+"}
+
+@code{(+ a)}
+@end deftp
+
+@deftp {PEG Pattern} optional a
+Tries to parse @var{a}. Succeeds if @var{a} succeeds.
+
+@code{"a?"}
+
+@code{(? a)}
+@end deftp
+
+@deftp {PEG Pattern} {followed by} a
+Makes sure it is possible to parse @var{a}, but does not actually parse
+it. Succeeds if @var{a} would succeed.
+
+@code{"&a"}
+
+@code{(followed-by a)}
+@end deftp
+
+@deftp {PEG Pattern} {not followed by} a
+Makes sure it is impossible to parse @var{a}, but does not actually
+parse it. Succeeds if @var{a} would fail.
+
+@code{"!a"}
+
+@code{(not-followed-by a)}
+@end deftp
+
+@deftp {PEG Pattern} {string literal} ``abc''
+Parses the string @var{"abc"}. Succeeds if that parsing succeeds.
+
+@code{"'abc'"}
+
+@code{"abc"}
+@end deftp
+
+@deftp {PEG Pattern} {any character}
+Parses any single character. Succeeds unless there is no more text to
+be parsed.
+
+@code{"."}
+
+@code{peg-any}
+@end deftp
+
+@deftp {PEG Pattern} {character class} a b
+Alternative syntax for ``Ordered Choice @var{a} @var{b}'' if @var{a} and
+@var{b} are characters.
+
+@code{"[ab]"}
+
+@code{(or "a" "b")}
+@end deftp
+
+@deftp {PEG Pattern} {range of characters} a z
+Parses any character falling between @var{a} and @var{z}.
+
+@code{"[a-z]"}
+
+@code{(range #\a #\z)}
+@end deftp
+
+Example:
+
+@example
+"(a !b / c &d*) 'e'+"
+@end example
+
+Would be:
+
+@lisp
+(and
+ (or
+ (and a (not-followed-by b))
+ (and c (followed-by (* d))))
+ (+ "e"))
+@end lisp
+
+@subsubheading Extended Syntax
+
+There is some extra syntax for S-expressions.
+
+@deftp {PEG Pattern} ignore a
+Ignore the text matching @var{a}
+@end deftp
+
+@deftp {PEG Pattern} capture a
+Capture the text matching @var{a}.
+@end deftp
+
+@deftp {PEG Pattern} peg a
+Embed the PEG pattern @var{a} using string syntax.
+@end deftp
+
+Example:
+
+@example
+"!a / 'b'"
+@end example
+
+Is equivalent to
+
+@lisp
+(or (peg "!a") "b")
+@end lisp
+
+and
+
+@lisp
+(or (not-followed-by a) "b")
+@end lisp
+
+@node PEG API Reference
+@subsection PEG API Reference
+
+@subsubheading Define Macros
+
+The most straightforward way to define a PEG is by using one of the
+define macros (both of these macroexpand into @code{define}
+expressions). These macros bind parsing functions to variables. These
+parsing functions may be invoked by @code{match-pattern} or
+@code{search-for-pattern}, which return a PEG match record. Raw data can be
+retrieved from this record with the PEG match deconstructor functions.
+More complicated (and perhaps enlightening) examples can be found in the
+tutorial.
+
+@deffn {Scheme Macro} define-peg-string-patterns peg-string
+Defines all the nonterminals in the PEG @var{peg-string}. More
+precisely, @code{define-peg-string-patterns} takes a superset of PEGs. A normal PEG
+has a @code{<-} between the nonterminal and the pattern.
+@code{define-peg-string-patterns} uses this symbol to determine what information it
+should propagate up the parse tree. The normal @code{<-} propagates the
+matched text up the parse tree, @code{<--} propagates the matched text
+up the parse tree tagged with the name of the nonterminal, and @code{<}
+discards that matched text and propagates nothing up the parse tree.
+Also, nonterminals may consist of any alphanumeric character or a ``-''
+character (in normal PEGs nonterminals can only be alphabetic).
+
+For example, if we:
+@lisp
+(define-peg-string-patterns
+ "as <- 'a'+
+bs <- 'b'+
+as-or-bs <- as/bs")
+(define-peg-string-patterns
+ "as-tag <-- 'a'+
+bs-tag <-- 'b'+
+as-or-bs-tag <-- as-tag/bs-tag")
+@end lisp
+Then:
+@lisp
+(match-pattern as-or-bs "aabbcc") @result{}
+#<peg start: 0 end: 2 string: aabbcc tree: aa>
+(match-pattern as-or-bs-tag "aabbcc") @result{}
+#<peg start: 0 end: 2 string: aabbcc tree: (as-or-bs-tag (as-tag aa))>
+@end lisp
+
+Note that in doing this, we have bound 6 variables at the toplevel
+(@var{as}, @var{bs}, @var{as-or-bs}, @var{as-tag}, @var{bs-tag}, and
+@var{as-or-bs-tag}).
+@end deffn
+
+@deffn {Scheme Macro} define-peg-pattern name capture-type peg-sexp
+Defines a single nonterminal @var{name}. @var{capture-type} determines
+how much information is passed up the parse tree. @var{peg-sexp} is a
+PEG in S-expression form.
+
+Possible values for capture-type:
+
+@table @code
+@item all
+passes the matched text up the parse tree tagged with the name of the
+nonterminal.
+@item body
+passes the matched text up the parse tree.
+@item none
+passes nothing up the parse tree.
+@end table
+
+For Example, if we:
+@lisp
+(define-peg-pattern as body (+ "a"))
+(define-peg-pattern bs body (+ "b"))
+(define-peg-pattern as-or-bs body (or as bs))
+(define-peg-pattern as-tag all (+ "a"))
+(define-peg-pattern bs-tag all (+ "b"))
+(define-peg-pattern as-or-bs-tag all (or as-tag bs-tag))
+@end lisp
+Then:
+@lisp
+(match-pattern as-or-bs "aabbcc") @result{}
+#<peg start: 0 end: 2 string: aabbcc tree: aa>
+(match-pattern as-or-bs-tag "aabbcc") @result{}
+#<peg start: 0 end: 2 string: aabbcc tree: (as-or-bs-tag (as-tag aa))>
+@end lisp
+
+Note that in doing this, we have bound 6 variables at the toplevel
+(@var{as}, @var{bs}, @var{as-or-bs}, @var{as-tag}, @var{bs-tag}, and
+@var{as-or-bs-tag}).
+@end deffn
+
+@subsubheading Compile Functions
+It is sometimes useful to be able to compile anonymous PEG patterns at
+runtime. These functions let you do that using either syntax.
+
+@deffn {Scheme Procedure} peg-string-compile peg-string capture-type
+Compiles the PEG pattern in @var{peg-string} propagating according to
+@var{capture-type} (capture-type can be any of the values from
+@code{define-peg-pattern}).
+@end deffn
+
+
+@deffn {Scheme Procedure} compile-peg-pattern peg-sexp capture-type
+Compiles the PEG pattern in @var{peg-sexp} propagating according to
+@var{capture-type} (capture-type can be any of the values from
+@code{define-peg-pattern}).
+@end deffn
+
+The functions return syntax objects, which can be useful if you want to
+use them in macros. If all you want is to define a new nonterminal, you
+can do the following:
+
+@lisp
+(define exp '(+ "a"))
+(define as (compile (compile-peg-pattern exp 'body)))
+@end lisp
+
+You can use this nonterminal with all of the regular PEG functions:
+
+@lisp
+(match-pattern as "aaaaa") @result{}
+#<peg start: 0 end: 5 string: bbbbb tree: bbbbb>
+@end lisp
+
+@subsubheading Parsing & Matching Functions
+
+For our purposes, ``parsing'' means parsing a string into a tree
+starting from the first character, while ``matching'' means searching
+through the string for a substring. In practice, the only difference
+between the two functions is that @code{match-pattern} gives up if it can't
+find a valid substring starting at index 0 and @code{search-for-pattern} keeps
+looking. They are both equally capable of ``parsing'' and ``matching''
+given those constraints.
+
+@deffn {Scheme Procedure} match-pattern nonterm string
+Parses @var{string} using the PEG stored in @var{nonterm}. If no match
+was found, @code{match-pattern} returns false. If a match was found, a PEG
+match record is returned.
+
+The @code{capture-type} argument to @code{define-peg-pattern} allows you to
+choose what information to hold on to while parsing. The options are:
+
+@table @code
+@item all
+tag the matched text with the nonterminal
+@item body
+just the matched text
+@item none
+nothing
+@end table
+
+@lisp
+(define-peg-pattern as all (+ "a"))
+(match-pattern as "aabbcc") @result{}
+#<peg start: 0 end: 2 string: aabbcc tree: (as aa)>
+
+(define-peg-pattern as body (+ "a"))
+(match-pattern as "aabbcc") @result{}
+#<peg start: 0 end: 2 string: aabbcc tree: aa>
+
+(define-peg-pattern as none (+ "a"))
+(match-pattern as "aabbcc") @result{}
+#<peg start: 0 end: 2 string: aabbcc tree: ()>
+
+(define-peg-pattern bs body (+ "b"))
+(match-pattern bs "aabbcc") @result{}
+#f
+@end lisp
+@end deffn
+
+@deffn {Scheme Macro} search-for-pattern nonterm-or-peg string
+Searches through @var{string} looking for a matching subexpression.
+@var{nonterm-or-peg} can either be a nonterminal or a literal PEG
+pattern. When a literal PEG pattern is provided, @code{search-for-pattern} works
+very similarly to the regular expression searches many hackers are used
+to. If no match was found, @code{search-for-pattern} returns false. If a match
+was found, a PEG match record is returned.
+
+@lisp
+(define-peg-pattern as body (+ "a"))
+(search-for-pattern as "aabbcc") @result{}
+#<peg start: 0 end: 2 string: aabbcc tree: aa>
+(search-for-pattern (+ "a") "aabbcc") @result{}
+#<peg start: 0 end: 2 string: aabbcc tree: aa>
+(search-for-pattern "'a'+" "aabbcc") @result{}
+#<peg start: 0 end: 2 string: aabbcc tree: aa>
+
+(define-peg-pattern as all (+ "a"))
+(search-for-pattern as "aabbcc") @result{}
+#<peg start: 0 end: 2 string: aabbcc tree: (as aa)>
+
+(define-peg-pattern bs body (+ "b"))
+(search-for-pattern bs "aabbcc") @result{}
+#<peg start: 2 end: 4 string: aabbcc tree: bb>
+(search-for-pattern (+ "b") "aabbcc") @result{}
+#<peg start: 2 end: 4 string: aabbcc tree: bb>
+(search-for-pattern "'b'+" "aabbcc") @result{}
+#<peg start: 2 end: 4 string: aabbcc tree: bb>
+
+(define-peg-pattern zs body (+ "z"))
+(search-for-pattern zs "aabbcc") @result{}
+#f
+(search-for-pattern (+ "z") "aabbcc") @result{}
+#f
+(search-for-pattern "'z'+" "aabbcc") @result{}
+#f
+@end lisp
+@end deffn
+
+@subsubheading PEG Match Records
+The @code{match-pattern} and @code{search-for-pattern} functions both return PEG
+match records. Actual information can be extracted from these with the
+following functions.
+
+@deffn {Scheme Procedure} peg:string match-record
+Returns the original string that was parsed in the creation of
+@code{match-record}.
+@end deffn
+
+@deffn {Scheme Procedure} peg:start match-record
+Returns the index of the first parsed character in the original string
+(from @code{peg:string}). If this is the same as @code{peg:end},
+nothing was parsed.
+@end deffn
+
+@deffn {Scheme Procedure} peg:end match-record
+Returns one more than the index of the last parsed character in the
+original string (from @code{peg:string}). If this is the same as
+@code{peg:start}, nothing was parsed.
+@end deffn
+
+@deffn {Scheme Procedure} peg:substring match-record
+Returns the substring parsed by @code{match-record}. This is equivalent to
+@code{(substring (peg:string match-record) (peg:start match-record) (peg:end
+match-record))}.
+@end deffn
+
+@deffn {Scheme Procedure} peg:tree match-record
+Returns the tree parsed by @code{match-record}.
+@end deffn
+
+@deffn {Scheme Procedure} peg-record? match-record
+Returns true if @code{match-record} is a PEG match record, or false
+otherwise.
+@end deffn
+
+Example:
+@lisp
+(define-peg-pattern bs all (peg "'b'+"))
+
+(search-for-pattern bs "aabbcc") @result{}
+#<peg start: 2 end: 4 string: aabbcc tree: (bs bb)>
+
+(let ((pm (search-for-pattern bs "aabbcc")))
+ `((string ,(peg:string pm))
+ (start ,(peg:start pm))
+ (end ,(peg:end pm))
+ (substring ,(peg:substring pm))
+ (tree ,(peg:tree pm))
+ (record? ,(peg-record? pm)))) @result{}
+((string "aabbcc")
+ (start 2)
+ (end 4)
+ (substring "bb")
+ (tree (bs "bb"))
+ (record? #t))
+@end lisp
+
+@subsubheading Miscellaneous
+
+@deffn {Scheme Procedure} context-flatten tst lst
+Takes a predicate @var{tst} and a list @var{lst}. Flattens @var{lst}
+until all elements are either atoms or satisfy @var{tst}. If @var{lst}
+itself satisfies @var{tst}, @code{(list lst)} is returned (this is a
+flat list whose only element satisfies @var{tst}).
+
+@lisp
+(context-flatten (lambda (x) (and (number? (car x)) (= (car x) 1))) '(2 2 (1 1 (2 2)) (2 2 (1 1)))) @result{}
+(2 2 (1 1 (2 2)) 2 2 (1 1))
+(context-flatten (lambda (x) (and (number? (car x)) (= (car x) 1))) '(1 1 (1 1 (2 2)) (2 2 (1 1)))) @result{}
+((1 1 (1 1 (2 2)) (2 2 (1 1))))
+@end lisp
+
+If you're wondering why this is here, take a look at the tutorial.
+@end deffn
+
+@deffn {Scheme Procedure} keyword-flatten terms lst
+A less general form of @code{context-flatten}. Takes a list of terminal
+atoms @code{terms} and flattens @var{lst} until all elements are either
+atoms, or lists which have an atom from @code{terms} as their first
+element.
+@lisp
+(keyword-flatten '(a b) '(c a b (a c) (b c) (c (b a) (c a)))) @result{}
+(c a b (a c) (b c) c (b a) c a)
+@end lisp
+
+If you're wondering why this is here, take a look at the tutorial.
+@end deffn
+
+@node PEG Tutorial
+@subsection PEG Tutorial
+
+@subsubheading Parsing /etc/passwd
+This example will show how to parse /etc/passwd using PEGs.
+
+First we define an example /etc/passwd file:
+
+@lisp
+(define *etc-passwd*
+ "root:x:0:0:root:/root:/bin/bash
+daemon:x:1:1:daemon:/usr/sbin:/bin/sh
+bin:x:2:2:bin:/bin:/bin/sh
+sys:x:3:3:sys:/dev:/bin/sh
+nobody:x:65534:65534:nobody:/nonexistent:/bin/sh
+messagebus:x:103:107::/var/run/dbus:/bin/false
+")
+@end lisp
+
+As a first pass at this, we might want to have all the entries in
+/etc/passwd in a list.
+
+Doing this with string-based PEG syntax would look like this:
+@lisp
+(define-peg-string-patterns
+ "passwd <- entry* !.
+entry <-- (! NL .)* NL*
+NL < '\n'")
+@end lisp
+
+A @code{passwd} file is 0 or more entries (@code{entry*}) until the end
+of the file (@code{!.} (@code{.} is any character, so @code{!.} means
+``not anything'')). We want to capture the data in the nonterminal
+@code{passwd}, but not tag it with the name, so we use @code{<-}.
+
+An entry is a series of 0 or more characters that aren't newlines
+(@code{(! NL .)*}) followed by 0 or more newlines (@code{NL*}). We want
+to tag all the entries with @code{entry}, so we use @code{<--}.
+
+A newline is just a literal newline (@code{'\n'}). We don't want a
+bunch of newlines cluttering up the output, so we use @code{<} to throw
+away the captured data.
+
+Here is the same PEG defined using S-expressions:
+@lisp
+(define-peg-pattern passwd body (and (* entry) (not-followed-by peg-any)))
+(define-peg-pattern entry all (and (* (and (not-followed-by NL) peg-any))
+ (* NL)))
+(define-peg-pattern NL none "\n")
+@end lisp
+
+Obviously this is much more verbose. On the other hand, it's more
+explicit, and thus easier to build automatically. However, there are
+some tricks that make S-expressions easier to use in some cases. One is
+the @code{ignore} keyword; the string syntax has no way to say ``throw
+away this text'' except breaking it out into a separate nonterminal.
+For instance, to throw away the newlines we had to define @code{NL}. In
+the S-expression syntax, we could have simply written @code{(ignore
+"\n")}. Also, for the cases where string syntax is really much cleaner,
+the @code{peg} keyword can be used to embed string syntax in
+S-expression syntax. For instance, we could have written:
+
+@lisp
+(define-peg-pattern passwd body (peg "entry* !."))
+@end lisp
+
+However we define it, parsing @code{*etc-passwd*} with the @code{passwd}
+nonterminal yields the same results:
+
+@lisp
+(peg:tree (match-pattern passwd *etc-passwd*)) @result{}
+((entry "root:x:0:0:root:/root:/bin/bash")
+ (entry "daemon:x:1:1:daemon:/usr/sbin:/bin/sh")
+ (entry "bin:x:2:2:bin:/bin:/bin/sh")
+ (entry "sys:x:3:3:sys:/dev:/bin/sh")
+ (entry "nobody:x:65534:65534:nobody:/nonexistent:/bin/sh")
+ (entry "messagebus:x:103:107::/var/run/dbus:/bin/false"))
+@end lisp
+
+However, here is something to be wary of:
+
+@lisp
+(peg:tree (match-pattern passwd "one entry")) @result{}
+(entry "one entry")
+@end lisp
+
+By default, the parse trees generated by PEGs are compressed as much as
+possible without losing information. It may not look like this is what
+you want at first, but uncompressed parse trees are an enormous headache
+(there's no easy way to predict how deep particular lists will nest,
+there are empty lists littered everywhere, etc. etc.). One side-effect
+of this, however, is that sometimes the compressor is too aggressive.
+No information is discarded when @code{((entry "one entry"))} is
+compressed to @code{(entry "one entry")}, but in this particular case it
+probably isn't what we want.
+
+There are two functions for easily dealing with this:
+@code{keyword-flatten} and @code{context-flatten}. The
+@code{keyword-flatten} function takes a list of keywords and a list to
+flatten, then tries to coerce the list such that the first element of
+all sublists is one of the keywords. The @code{context-flatten}
+function is similar, but instead of a list of keywords it takes a
+predicate that should indicate whether a given sublist is good enough
+(refer to the API reference for more details).
+
+What we want here is @code{keyword-flatten}.
+@lisp
+(keyword-flatten '(entry) (peg:tree (match-pattern passwd *etc-passwd*))) @result{}
+((entry "root:x:0:0:root:/root:/bin/bash")
+ (entry "daemon:x:1:1:daemon:/usr/sbin:/bin/sh")
+ (entry "bin:x:2:2:bin:/bin:/bin/sh")
+ (entry "sys:x:3:3:sys:/dev:/bin/sh")
+ (entry "nobody:x:65534:65534:nobody:/nonexistent:/bin/sh")
+ (entry "messagebus:x:103:107::/var/run/dbus:/bin/false"))
+(keyword-flatten '(entry) (peg:tree (match-pattern passwd "one entry"))) @result{}
+((entry "one entry"))
+@end lisp
+
+Of course, this is a somewhat contrived example. In practice we would
+probably just tag the @code{passwd} nonterminal to remove the ambiguity
+(using either the @code{all} keyword for S-expressions or the @code{<--}
+symbol for strings)..
+
+@lisp
+(define-peg-pattern tag-passwd all (peg "entry* !."))
+(peg:tree (match-pattern tag-passwd *etc-passwd*)) @result{}
+(tag-passwd
+ (entry "root:x:0:0:root:/root:/bin/bash")
+ (entry "daemon:x:1:1:daemon:/usr/sbin:/bin/sh")
+ (entry "bin:x:2:2:bin:/bin:/bin/sh")
+ (entry "sys:x:3:3:sys:/dev:/bin/sh")
+ (entry "nobody:x:65534:65534:nobody:/nonexistent:/bin/sh")
+ (entry "messagebus:x:103:107::/var/run/dbus:/bin/false"))
+(peg:tree (match-pattern tag-passwd "one entry"))
+(tag-passwd
+ (entry "one entry"))
+@end lisp
+
+If you're ever uncertain about the potential results of parsing
+something, remember the two absolute rules:
+@enumerate
+@item
+No parsing information will ever be discarded.
+@item
+There will never be any lists with fewer than 2 elements.
+@end enumerate
+
+For the purposes of (1), "parsing information" means things tagged with
+the @code{any} keyword or the @code{<--} symbol. Plain strings will be
+concatenated.
+
+Let's extend this example a bit more and actually pull some useful
+information out of the passwd file:
+
+@lisp
+(define-peg-string-patterns
+ "passwd <-- entry* !.
+entry <-- login C pass C uid C gid C nameORcomment C homedir C shell NL*
+login <-- text
+pass <-- text
+uid <-- [0-9]*
+gid <-- [0-9]*
+nameORcomment <-- text
+homedir <-- path
+shell <-- path
+path <-- (SLASH pathELEMENT)*
+pathELEMENT <-- (!NL !C !'/' .)*
+text <- (!NL !C .)*
+C < ':'
+NL < '\n'
+SLASH < '/'")
+@end lisp
+
+This produces rather pretty parse trees:
+@lisp
+(passwd
+ (entry (login "root")
+ (pass "x")
+ (uid "0")
+ (gid "0")
+ (nameORcomment "root")
+ (homedir (path (pathELEMENT "root")))
+ (shell (path (pathELEMENT "bin") (pathELEMENT "bash"))))
+ (entry (login "daemon")
+ (pass "x")
+ (uid "1")
+ (gid "1")
+ (nameORcomment "daemon")
+ (homedir
+ (path (pathELEMENT "usr") (pathELEMENT "sbin")))
+ (shell (path (pathELEMENT "bin") (pathELEMENT "sh"))))
+ (entry (login "bin")
+ (pass "x")
+ (uid "2")
+ (gid "2")
+ (nameORcomment "bin")
+ (homedir (path (pathELEMENT "bin")))
+ (shell (path (pathELEMENT "bin") (pathELEMENT "sh"))))
+ (entry (login "sys")
+ (pass "x")
+ (uid "3")
+ (gid "3")
+ (nameORcomment "sys")
+ (homedir (path (pathELEMENT "dev")))
+ (shell (path (pathELEMENT "bin") (pathELEMENT "sh"))))
+ (entry (login "nobody")
+ (pass "x")
+ (uid "65534")
+ (gid "65534")
+ (nameORcomment "nobody")
+ (homedir (path (pathELEMENT "nonexistent")))
+ (shell (path (pathELEMENT "bin") (pathELEMENT "sh"))))
+ (entry (login "messagebus")
+ (pass "x")
+ (uid "103")
+ (gid "107")
+ nameORcomment
+ (homedir
+ (path (pathELEMENT "var")
+ (pathELEMENT "run")
+ (pathELEMENT "dbus")))
+ (shell (path (pathELEMENT "bin") (pathELEMENT "false")))))
+@end lisp
+
+Notice that when there's no entry in a field (e.g. @code{nameORcomment}
+for messagebus) the symbol is inserted. This is the ``don't throw away
+any information'' rule---we succesfully matched a @code{nameORcomment}
+of 0 characters (since we used @code{*} when defining it). This is
+usually what you want, because it allows you to e.g. use @code{list-ref}
+to pull out elements (since they all have known offsets).
+
+If you'd prefer not to have symbols for empty matches, you can replace
+the @code{*} with a @code{+} and add a @code{?} after the
+@code{nameORcomment} in @code{entry}. Then it will try to parse 1 or
+more characters, fail (inserting nothing into the parse tree), but
+continue because it didn't have to match the nameORcomment to continue.
+
+
+@subsubheading Embedding Arithmetic Expressions
+
+We can parse simple mathematical expressions with the following PEG:
+
+@lisp
+(define-peg-string-patterns
+ "expr <- sum
+sum <-- (product ('+' / '-') sum) / product
+product <-- (value ('*' / '/') product) / value
+value <-- number / '(' expr ')'
+number <-- [0-9]+")
+@end lisp
+
+Then:
+@lisp
+(peg:tree (match-pattern expr "1+1/2*3+(1+1)/2")) @result{}
+(sum (product (value (number "1")))
+ "+"
+ (sum (product
+ (value (number "1"))
+ "/"
+ (product
+ (value (number "2"))
+ "*"
+ (product (value (number "3")))))
+ "+"
+ (sum (product
+ (value "("
+ (sum (product (value (number "1")))
+ "+"
+ (sum (product (value (number "1")))))
+ ")")
+ "/"
+ (product (value (number "2")))))))
+@end lisp
+
+There is very little wasted effort in this PEG. The @code{number}
+nonterminal has to be tagged because otherwise the numbers might run
+together with the arithmetic expressions during the string concatenation
+stage of parse-tree compression (the parser will see ``1'' followed by
+``/'' and decide to call it ``1/''). When in doubt, tag.
+
+It is very easy to turn these parse trees into lisp expressions:
+
+@lisp
+(define (parse-sum sum left . rest)
+ (if (null? rest)
+ (apply parse-product left)
+ (list (string->symbol (car rest))
+ (apply parse-product left)
+ (apply parse-sum (cadr rest)))))
+
+(define (parse-product product left . rest)
+ (if (null? rest)
+ (apply parse-value left)
+ (list (string->symbol (car rest))
+ (apply parse-value left)
+ (apply parse-product (cadr rest)))))
+
+(define (parse-value value first . rest)
+ (if (null? rest)
+ (string->number (cadr first))
+ (apply parse-sum (car rest))))
+
+(define parse-expr parse-sum)
+@end lisp
+
+(Notice all these functions look very similar; for a more complicated
+PEG, it would be worth abstracting.)
+
+Then:
+@lisp
+(apply parse-expr (peg:tree (match-pattern expr "1+1/2*3+(1+1)/2"))) @result{}
+(+ 1 (+ (/ 1 (* 2 3)) (/ (+ 1 1) 2)))
+@end lisp
+
+But wait! The associativity is wrong! Where it says @code{(/ 1 (* 2
+3))}, it should say @code{(* (/ 1 2) 3)}.
+
+It's tempting to try replacing e.g. @code{"sum <-- (product ('+' / '-')
+sum) / product"} with @code{"sum <-- (sum ('+' / '-') product) /
+product"}, but this is a Bad Idea. PEGs don't support left recursion.
+To see why, imagine what the parser will do here. When it tries to
+parse @code{sum}, it first has to try and parse @code{sum}. But to do
+that, it first has to try and parse @code{sum}. This will continue
+until the stack gets blown off.
+
+So how does one parse left-associative binary operators with PEGs?
+Honestly, this is one of their major shortcomings. There's no
+general-purpose way of doing this, but here the repetition operators are
+a good choice:
+
+@lisp
+(use-modules (srfi srfi-1))
+
+(define-peg-string-patterns
+ "expr <- sum
+sum <-- (product ('+' / '-'))* product
+product <-- (value ('*' / '/'))* value
+value <-- number / '(' expr ')'
+number <-- [0-9]+")
+
+;; take a deep breath...
+(define (make-left-parser next-func)
+ (lambda (sum first . rest) ;; general form, comments below assume
+ ;; that we're dealing with a sum expression
+ (if (null? rest) ;; form (sum (product ...))
+ (apply next-func first)
+ (if (string? (cadr first));; form (sum ((product ...) "+") (product ...))
+ (list (string->symbol (cadr first))
+ (apply next-func (car first))
+ (apply next-func (car rest)))
+ ;; form (sum (((product ...) "+") ((product ...) "+")) (product ...))
+ (car
+ (reduce ;; walk through the list and build a left-associative tree
+ (lambda (l r)
+ (list (list (cadr r) (car r) (apply next-func (car l)))
+ (string->symbol (cadr l))))
+ 'ignore
+ (append ;; make a list of all the products
+ ;; the first one should be pre-parsed
+ (list (list (apply next-func (caar first))
+ (string->symbol (cadar first))))
+ (cdr first)
+ ;; the last one has to be added in
+ (list (append rest '("done"))))))))))
+
+(define (parse-value value first . rest)
+ (if (null? rest)
+ (string->number (cadr first))
+ (apply parse-sum (car rest))))
+(define parse-product (make-left-parser parse-value))
+(define parse-sum (make-left-parser parse-product))
+(define parse-expr parse-sum)
+@end lisp
+
+Then:
+@lisp
+(apply parse-expr (peg:tree (match-pattern expr "1+1/2*3+(1+1)/2"))) @result{}
+(+ (+ 1 (* (/ 1 2) 3)) (/ (+ 1 1) 2))
+@end lisp
+
+As you can see, this is much uglier (it could be made prettier by using
+@code{context-flatten}, but the way it's written above makes it clear
+how we deal with the three ways the zero-or-more @code{*} expression can
+parse). Fortunately, most of the time we can get away with only using
+right-associativity.
+
+@subsubheading Simplified Functions
+
+For a more tantalizing example, consider the following grammar that
+parses (highly) simplified C functions:
+
+@lisp
+(define-peg-string-patterns
+ "cfunc <-- cSP ctype cSP cname cSP cargs cLB cSP cbody cRB
+ctype <-- cidentifier
+cname <-- cidentifier
+cargs <-- cLP (! (cSP cRP) carg cSP (cCOMMA / cRP) cSP)* cSP
+carg <-- cSP ctype cSP cname
+cbody <-- cstatement *
+cidentifier <- [a-zA-z][a-zA-Z0-9_]*
+cstatement <-- (!';'.)*cSC cSP
+cSC < ';'
+cCOMMA < ','
+cLP < '('
+cRP < ')'
+cLB < '@{'
+cRB < '@}'
+cSP < [ \t\n]*")
+@end lisp
+
+Then:
+@lisp
+(match-pattern cfunc "int square(int a) @{ return a*a;@}") @result{}
+(32
+ (cfunc (ctype "int")
+ (cname "square")
+ (cargs (carg (ctype "int") (cname "a")))
+ (cbody (cstatement "return a*a"))))
+@end lisp
+
+And:
+@lisp
+(match-pattern cfunc "int mod(int a, int b) @{ int c = a/b;return a-b*c; @}") @result{}
+(52
+ (cfunc (ctype "int")
+ (cname "mod")
+ (cargs (carg (ctype "int") (cname "a"))
+ (carg (ctype "int") (cname "b")))
+ (cbody (cstatement "int c = a/b")
+ (cstatement "return a- b*c"))))
+@end lisp
+
+By wrapping all the @code{carg} nonterminals in a @code{cargs}
+nonterminal, we were able to remove any ambiguity in the parsing
+structure and avoid having to call @code{context-flatten} on the output
+of @code{match-pattern}. We used the same trick with the @code{cstatement}
+nonterminals, wrapping them in a @code{cbody} nonterminal.
+
+The whitespace nonterminal @code{cSP} used here is a (very) useful
+instantiation of a common pattern for matching syntactically irrelevant
+information. Since it's tagged with @code{<} and ends with @code{*} it
+won't clutter up the parse trees (all the empty lists will be discarded
+during the compression step) and it will never cause parsing to fail.
+
+@node PEG Internals
+@subsection PEG Internals
+
+A PEG parser takes a string as input and attempts to parse it as a given
+nonterminal. The key idea of the PEG implementation is that every
+nonterminal is just a function that takes a string as an argument and
+attempts to parse that string as its nonterminal. The functions always
+start from the beginning, but a parse is considered successful if there
+is material left over at the end.
+
+This makes it easy to model different PEG parsing operations. For
+instance, consider the PEG grammar @code{"ab"}, which could also be
+written @code{(and "a" "b")}. It matches the string ``ab''. Here's how
+that might be implemented in the PEG style:
+
+@lisp
+(define (match-and-a-b str)
+ (match-a str)
+ (match-b str))
+@end lisp
+
+As you can see, the use of functions provides an easy way to model
+sequencing. In a similar way, one could model @code{(or a b)} with
+something like the following:
+
+@lisp
+(define (match-or-a-b str)
+ (or (match-a str) (match-b str)))
+@end lisp
+
+Here the semantics of a PEG @code{or} expression map naturally onto
+Scheme's @code{or} operator. This function will attempt to run
+@code{(match-a str)}, and return its result if it succeeds. Otherwise it
+will run @code{(match-b str)}.
+
+Of course, the code above wouldn't quite work. We need some way for the
+parsing functions to communicate. The actual interface used is below.
+
+@subsubheading Parsing Function Interface
+
+A parsing function takes three arguments - a string, the length of that
+string, and the position in that string it should start parsing at. In
+effect, the parsing functions pass around substrings in pieces - the
+first argument is a buffer of characters, and the second two give a
+range within that buffer that the parsing function should look at.
+
+Parsing functions return either #f, if they failed to match their
+nonterminal, or a list whose first element must be an integer
+representing the final position in the string they matched and whose cdr
+can be any other data the function wishes to return, or '() if it
+doesn't have any more data.
+
+The one caveat is that if the extra data it returns is a list, any
+adjacent strings in that list will be appended by @code{match-pattern}. For
+instance, if a parsing function returns @code{(13 ("a" "b" "c"))},
+@code{match-pattern} will take @code{(13 ("abc"))} as its value.
+
+For example, here is a function to match ``ab'' using the actual
+interface.
+
+@lisp
+(define (match-a-b str len pos)
+ (and (<= (+ pos 2) len)
+ (string= str "ab" pos (+ pos 2))
+ (list (+ pos 2) '()))) ; we return no extra information
+@end lisp
+
+The above function can be used to match a string by running
+@code{(match-pattern match-a-b "ab")}.
+
+@subsubheading Code Generators and Extensible Syntax
+
+PEG expressions, such as those in a @code{define-peg-pattern} form, are
+interpreted internally in two steps.
+
+First, any string PEG is expanded into an s-expression PEG by the code
+in the @code{(ice-9 peg string-peg)} module.
+
+Then, then s-expression PEG that results is compiled into a parsing
+function by the @code{(ice-9 peg codegen)} module. In particular, the
+function @code{compile-peg-pattern} is called on the s-expression. It then
+decides what to do based on the form it is passed.
+
+The PEG syntax can be expanded by providing @code{compile-peg-pattern} more
+options for what to do with its forms. The extended syntax will be
+associated with a symbol, for instance @code{my-parsing-form}, and will
+be called on all PEG expressions of the form
+@lisp
+(my-parsing-form ...)
+@end lisp
+
+The parsing function should take two arguments. The first will be a
+syntax object containing a list with all of the arguments to the form
+(but not the form's name), and the second will be the
+@code{capture-type} argument that is passed to @code{define-peg-pattern}.
+
+New functions can be registered by calling @code{(add-peg-compiler!
+symbol function)}, where @code{symbol} is the symbol that will indicate
+a form of this type and @code{function} is the code generating function
+described above. The function @code{add-peg-compiler!} is exported from
+the @code{(ice-9 peg codegen)} module.
diff --git a/doc/ref/compiler.texi b/doc/ref/compiler.texi
index bfc633e57..553c270e7 100644
--- a/doc/ref/compiler.texi
+++ b/doc/ref/compiler.texi
@@ -1,6 +1,6 @@
@c -*-texinfo-*-
@c This is part of the GNU Guile Reference Manual.
-@c Copyright (C) 2008, 2009, 2010, 2013
+@c Copyright (C) 2008, 2009, 2010, 2011, 2012, 2013
@c Free Software Foundation, Inc.
@c See the file guile.texi for copying conditions.
@@ -161,12 +161,11 @@ different worlds indefinitely, as shown by the following quine:
@node The Scheme Compiler
@subsection The Scheme Compiler
-The job of the Scheme compiler is to expand all macros and all of
-Scheme to its most primitive expressions. The definition of
-``primitive'' is given by the inventory of constructs provided by
-Tree-IL, the target language of the Scheme compiler: procedure
-applications, conditionals, lexical references, etc. This is described
-more fully in the next section.
+The job of the Scheme compiler is to expand all macros and all of Scheme
+to its most primitive expressions. The definition of ``primitive'' is
+given by the inventory of constructs provided by Tree-IL, the target
+language of the Scheme compiler: procedure calls, conditionals, lexical
+references, etc. This is described more fully in the next section.
The tricky and amusing thing about the Scheme-to-Tree-IL compiler is
that it is completely implemented by the macro expander. Since the
@@ -184,10 +183,10 @@ The Scheme-to-Tree-IL expander may be invoked using the generic
@lisp
(compile '(+ 1 2) #:from 'scheme #:to 'tree-il)
@result{}
- #<<application> src: #f
- proc: #<<toplevel-ref> src: #f name: +>
- args: (#<<const> src: #f exp: 1>
- #<<const> src: #f exp: 2>)>
+ #<<call> src: #f
+ proc: #<<toplevel-ref> src: #f name: +>
+ args: (#<<const> src: #f exp: 1>
+ #<<const> src: #f exp: 2>)>
@end lisp
Or, since Tree-IL is so close to Scheme, it is often useful to expand
@@ -342,9 +341,9 @@ instruction.
Compilation of Tree-IL usually begins with a pass that resolves some
@code{<module-ref>} and @code{<toplevel-ref>} expressions to
-@code{<primitive-ref>} expressions. The actual compilation pass
-has special cases for applications of certain primitives, like
-@code{apply} or @code{cons}.
+@code{<primitive-ref>} expressions. The actual compilation pass has
+special cases for calls to certain primitives, like @code{apply} or
+@code{cons}.
@end deftp
@deftp {Scheme Variable} <lexical-ref> src name gensym
@deftpx {External Representation} (lexical @var{name} @var{gensym})
@@ -388,10 +387,19 @@ Defines a new top-level variable in the current procedure's module.
@deftpx {External Representation} (if @var{test} @var{then} @var{else})
A conditional. Note that @var{else} is not optional.
@end deftp
-@deftp {Scheme Variable} <application> src proc args
-@deftpx {External Representation} (apply @var{proc} . @var{args})
+@deftp {Scheme Variable} <call> src proc args
+@deftpx {External Representation} (call @var{proc} . @var{args})
A procedure call.
@end deftp
+@deftp {Scheme Variable} <primcall> src name args
+@deftpx {External Representation} (primcall @var{name} . @var{args})
+A call to a primitive. Equivalent to @code{(call (primitive @var{name})
+. @var{args})}. This construct is often more convenient to generate and
+analyze than @code{<call>}.
+
+As part of the compilation process, instances of @code{(call (primitive
+@var{name}) . @var{args})} are transformed into primcalls.
+@end deftp
@deftp {Scheme Variable} <sequence> src exps
@deftpx {External Representation} (begin . @var{exps})
Like Scheme's @code{begin}.
@@ -450,31 +458,6 @@ original binding names, @var{gensyms} are gensyms corresponding to the
A version of @code{<let>} that creates recursive bindings, like
Scheme's @code{letrec}, or @code{letrec*} if @var{in-order?} is true.
@end deftp
-@deftp {Scheme Variable} <dynlet> fluids vals body
-@deftpx {External Representation} (dynlet @var{fluids} @var{vals} @var{body})
-Dynamic binding; the equivalent of Scheme's @code{with-fluids}.
-@var{fluids} should be a list of Tree-IL expressions that will
-evaluate to fluids, and @var{vals} a corresponding list of expressions
-to bind to the fluids during the dynamic extent of the evaluation of
-@var{body}.
-@end deftp
-@deftp {Scheme Variable} <dynref> fluid
-@deftpx {External Representation} (dynref @var{fluid})
-A dynamic variable reference. @var{fluid} should be a Tree-IL
-expression evaluating to a fluid.
-@end deftp
-@deftp {Scheme Variable} <dynset> fluid exp
-@deftpx {External Representation} (dynset @var{fluid} @var{exp})
-A dynamic variable set. @var{fluid}, a Tree-IL expression evaluating
-to a fluid, will be set to the result of evaluating @var{exp}.
-@end deftp
-@deftp {Scheme Variable} <dynwind> winder body unwinder
-@deftpx {External Representation} (dynwind @var{winder} @var{body} @var{unwinder})
-A @code{dynamic-wind}. @var{winder} and @var{unwinder} should both
-evaluate to thunks. Ensure that the winder and the unwinder are called
-before entering and after leaving @var{body}. Note that @var{body} is
-an expression, without a thunk wrapper.
-@end deftp
@deftp {Scheme Variable} <prompt> tag body handler
@deftpx {External Representation} (prompt @var{tag} @var{body} @var{handler})
A dynamic prompt. Instates a prompt named @var{tag}, an expression,
@@ -509,7 +492,7 @@ Like Scheme's @code{receive} -- binds the values returned by
evaluating @code{exp} to the @code{lambda}-like bindings described by
@var{gensyms}. That is to say, @var{gensyms} may be an improper list.
-@code{<let-values>} is an optimization of @code{<application>} of the
+@code{<let-values>} is an optimization of a @code{<call>} to the
primitive, @code{call-with-values}.
@end deftp
@deftp {Scheme Variable} <fix> src names gensyms vals body
@@ -792,29 +775,36 @@ objcode)} module.
Returns @code{#f} if @var{obj} is object code, @code{#f} otherwise.
@end deffn
-@deffn {Scheme Procedure} bytecode->objcode bytecode
+@deffn {Scheme Procedure} bytecode->objcode bytecode [endianness]
@deffnx {C Function} scm_bytecode_to_objcode (bytecode)
Makes a bytecode object from @var{bytecode}, which should be a
-bytevector. @xref{Bytevectors}.
+bytevector. @xref{Bytevectors}. By default, the embedded length fields
+in the bytevector are interpreted in the native byte order.
@end deffn
-@deffn {Scheme Variable} load-objcode file
-@deffnx {C Function} scm_load_objcode (file)
+@deffn {Scheme Variable} load-thunk-from-file file
+@deffnx {C Function} scm_load_thunk_from_file (file)
Load object code from a file named @var{file}. The file will be mapped
into memory via @code{mmap}, so this is a very fast operation.
-On disk, object code has an sixteen-byte cookie prepended to it, to
-prevent accidental loading of arbitrary garbage.
+On disk, object code is embedded in ELF, a flexible container format
+created for use in UNIX systems. Guile has its own ELF linker and
+loader, so it uses the ELF format on all systems.
@end deffn
@deffn {Scheme Variable} write-objcode objcode file
@deffnx {C Function} scm_write_objcode (objcode)
-Write object code out to a file, prepending the sixteen-byte cookie.
+Embed object code into an ELF container, and write it out to a file.
+
+This procedure is part of a separate module, @code{(language objcode
+elf)}.
@end deffn
-@deffn {Scheme Variable} objcode->bytecode objcode
+@deffn {Scheme Variable} objcode->bytecode objcode [endianness]
@deffnx {C Function} scm_objcode_to_bytecode (objcode)
-Copy object code out to a bytevector for analysis by Scheme.
+Copy object code out to a bytevector for analysis by Scheme. By
+default, the length fields in the @code{struct scm_objcode} are
+interpreted in the native byte order.
@end deffn
The following procedure is actually in @code{(system vm program)}, but
diff --git a/doc/ref/guile-invoke.texi b/doc/ref/guile-invoke.texi
index 29f3c93c8..207c6f379 100644
--- a/doc/ref/guile-invoke.texi
+++ b/doc/ref/guile-invoke.texi
@@ -335,9 +335,7 @@ explicitly enable it by setting the variable to @code{1}.
Usually, installing the current locale is the right thing to do. It
allows Guile to correctly parse and print strings with non-ASCII
-characters. However, for compatibility with previous Guile 2.0
-releases, this option is off by default. The next stable release series
-of Guile (the 2.2 series) will install locales by default.
+characters. Therefore, this option is on by default.
@item GUILE_STACK_SIZE
@vindex GUILE_STACK_SIZE
diff --git a/doc/ref/guile.texi b/doc/ref/guile.texi
index 1e4a95a99..c3170ce46 100644
--- a/doc/ref/guile.texi
+++ b/doc/ref/guile.texi
@@ -308,6 +308,7 @@ available through both Scheme and C interfaces.
* Input and Output:: Ports, reading and writing.
* Regular Expressions:: Pattern matching and substitution.
* LALR(1) Parsing:: Generating LALR(1) parsers.
+* PEG Parsing:: Parsing Expression Grammars.
* Read/Load/Eval/Compile:: Reading and evaluating Scheme code.
* Memory Management:: Memory management and garbage collection.
* Modules:: Designing reusable code libraries.
@@ -336,6 +337,7 @@ available through both Scheme and C interfaces.
@include api-io.texi
@include api-regex.texi
@include api-lalr.texi
+@include api-peg.texi
@include api-evaluation.texi
@include api-memory.texi
@include api-modules.texi
diff --git a/doc/ref/libguile-concepts.texi b/doc/ref/libguile-concepts.texi
index 50c435584..0103ad173 100644
--- a/doc/ref/libguile-concepts.texi
+++ b/doc/ref/libguile-concepts.texi
@@ -1,7 +1,7 @@
@c -*-texinfo-*-
@c This is part of the GNU Guile Reference Manual.
-@c Copyright (C) 1996, 1997, 2000, 2001, 2002, 2003, 2004, 2005, 2010, 2013
-@c Free Software Foundation, Inc.
+@c Copyright (C) 1996, 1997, 2000, 2001, 2002, 2003, 2004, 2005, 2010,
+@c 2011, 2013 Free Software Foundation, Inc.
@c See the file guile.texi for copying conditions.
@node General Libguile Concepts
@@ -449,16 +449,18 @@ that are stored in local variables. When a thread puts itself into
guile mode for the first time, it gets a Scheme representation and is
listed by @code{all-threads}, for example.
-Threads in guile mode can block (e.g., do blocking I/O) without causing any
-problems@footnote{In Guile 1.8, a thread blocking in guile mode would prevent
-garbage collection to occur. Thus, threads had to leave guile mode whenever
-they could block. This is no longer needed with Guile 2.0.}; temporarily
-leaving guile mode with @code{scm_without_guile} before blocking slightly
-improves GC performance, though. For some common blocking operations, Guile
-provides convenience functions. For example, if you want to lock a pthread
-mutex while in guile mode, you might want to use @code{scm_pthread_mutex_lock}
-which is just like @code{pthread_mutex_lock} except that it leaves guile mode
-while blocking.
+Threads in guile mode can block (e.g., do blocking I/O) without causing
+any problems@footnote{In Guile 1.8, a thread blocking in guile mode
+would prevent garbage collection to occur. Thus, threads had to leave
+guile mode whenever they could block. This is no longer needed with
+Guile 2.@var{x}.}; temporarily leaving guile mode with
+@code{scm_without_guile} before blocking slightly improves GC
+performance, though. For some common blocking operations, Guile
+provides convenience functions. For example, if you want to lock a
+pthread mutex while in guile mode, you might want to use
+@code{scm_pthread_mutex_lock} which is just like
+@code{pthread_mutex_lock} except that it leaves guile mode while
+blocking.
All libguile functions are (intended to be) robust in the face of
diff --git a/doc/ref/posix.texi b/doc/ref/posix.texi
index b3a6a048f..40c20e7e9 100644
--- a/doc/ref/posix.texi
+++ b/doc/ref/posix.texi
@@ -1867,10 +1867,11 @@ the integer process ID of the child.
Note that it is unsafe to fork a process that has multiple threads
running, as only the thread that calls @code{primitive-fork} will
persist in the child. Any resources that other threads held, such as
-locked mutexes or open file descriptors, are lost. Indeed, @acronym{POSIX}
-specifies that only async-signal-safe procedures are safe to call after
-a multithreaded fork, which is a very limited set. Guile issues a
-warning if it detects a fork from a multi-threaded program.
+locked mutexes or open file descriptors, are lost. Indeed,
+@acronym{POSIX} specifies that only async-signal-safe procedures are
+safe to call after a multithreaded fork, which is a very limited set.
+Guile issues a warning if it detects a fork from a multi-threaded
+program.
If you are going to @code{exec} soon after forking, the procedures in
@code{(ice-9 popen)} may be useful to you, as they fork and exec within
diff --git a/doc/ref/srfi-modules.texi b/doc/ref/srfi-modules.texi
index 31407e828..d97f49820 100644
--- a/doc/ref/srfi-modules.texi
+++ b/doc/ref/srfi-modules.texi
@@ -146,6 +146,7 @@ guile-2 ;; starting from Guile 2.x
r5rs
srfi-0
srfi-4
+srfi-6
srfi-13
srfi-14
srfi-23
@@ -177,8 +178,8 @@ how to load it with the Guile mechanism.
@cindex @code{guile-2} SRFI-0 feature
@cindex portability between 2.0 and older versions
Likewise, testing the @code{guile-2} feature allows code to be portable
-between Guile 2.0 and previous versions of Guile. For instance, it
-makes it possible to write code that accounts for Guile 2.0's compiler,
+between Guile 2.@var{x} and previous versions of Guile. For instance, it
+makes it possible to write code that accounts for Guile 2.@var{x}'s compiler,
yet be correctly interpreted on 1.8 and earlier versions:
@example
@@ -1851,19 +1852,11 @@ uniform numeric vector, it is returned unchanged.
@cindex SRFI-6
SRFI-6 defines the procedures @code{open-input-string},
-@code{open-output-string} and @code{get-output-string}.
-
-Note that although versions of these procedures are included in the
-Guile core, the core versions are not fully conformant with SRFI-6:
-attempts to read or write characters that are not supported by the
-current @code{%default-port-encoding} will fail.
-
-We therefore recommend that you import this module, which supports all
-characters:
-
-@example
-(use-modules (srfi srfi-6))
-@end example
+@code{open-output-string} and @code{get-output-string}. These
+procedures are included in the Guile core, so using this module does not
+make any difference at the moment. But it is possible that support for
+SRFI-6 will be factored out of the core library in the future, so using
+this module does not hurt, after all.
@node SRFI-8
@subsection SRFI-8 - receive
diff --git a/doc/ref/vm.texi b/doc/ref/vm.texi
index 9936ad97d..1e10eb02e 100644
--- a/doc/ref/vm.texi
+++ b/doc/ref/vm.texi
@@ -1,6 +1,6 @@
@c -*-texinfo-*-
@c This is part of the GNU Guile Reference Manual.
-@c Copyright (C) 2008,2009,2010,2013
+@c Copyright (C) 2008,2009,2010,2011,2013
@c Free Software Foundation, Inc.
@c See the file guile.texi for copying conditions.
@@ -79,9 +79,9 @@ but it is not normally used at runtime.)
The upside of implementing the interpreter in Scheme is that we preserve
tail calls and multiple-value handling between interpreted and compiled
-code. The downside is that the interpreter in Guile 2.0 is slower than
-the interpreter in 1.8. We hope the that the compiler's speed makes up
-for the loss!
+code. The downside is that the interpreter in Guile 2.@var{x} is slower
+than the interpreter in 1.8. We hope the that the compiler's speed makes
+up for the loss!
Also note that this decision to implement a bytecode compiler does not
preclude native compilation. We can compile from bytecode to native
@@ -1120,18 +1120,17 @@ wind/unwind thunk pair. @code{unwind} instructions should be properly
paired with their winding instructions, like @code{wind}.
@end deffn
-@deffn Instruction wind-fluids n
-Pop off @var{n} values and @var{n} fluids from the stack, in that order.
-Set the fluids to the values by creating a with-fluids object and
-pushing that object on the dynamic stack. @xref{Fluids and Dynamic
-States}.
+@deffn Instruction push-fluid
+Pop a value and a fluid from the stack, in that order. Set the fluid
+to the value by creating a with-fluids object and pushing that object
+on the dynamic stack. @xref{Fluids and Dynamic States}.
@end deffn
-@deffn Instruction unwind-fluids
+@deffn Instruction pop-fluid
Pop a with-fluids object from the dynamic stack, and swap the current
values of its fluids with the saved values of its fluids. In this way,
the dynamic environment is left as it was before the corresponding
-@code{wind-fluids} instruction was processed.
+@code{wind-fluid} instruction was processed.
@end deffn
@deffn Instruction fluid-ref