summaryrefslogtreecommitdiff
path: root/doc/ref/api-io.texi
diff options
context:
space:
mode:
authorAndy Wingo <wingo@pobox.com>2016-05-16 10:44:21 +0200
committerAndy Wingo <wingo@pobox.com>2016-05-16 10:48:35 +0200
commit1e058add7b9568fb3a37e4fa82360d183d0a26ee (patch)
tree2482575eab3cd6b9e47ce98c29f03ed317af7194 /doc/ref/api-io.texi
parentda456d23beb1b13063a9817d63797aa08949ba20 (diff)
downloadguile-1e058add7b9568fb3a37e4fa82360d183d0a26ee.tar.gz
U+FFFD is the input substitution character
* libguile/ports.c (UNICODE_REPLACEMENT_CHARACTER): * libguile/ports.c (peek_utf8_codepoint) (scm_port_decode_char, peek_iconv_codepoint): * module/ice-9/sports.scm (peek-char-and-len/utf8): (peek-char-and-len/iconv): Return U+FFFD when we get a decoding error when reading, instead of '?', in accordance with Unicode recommendations. * test-suite/tests/iconv.test: * test-suite/tests/ports.test: * test-suite/tests/rdelim.test: Update tests. * NEWS: Update.
Diffstat (limited to 'doc/ref/api-io.texi')
-rw-r--r--doc/ref/api-io.texi76
1 files changed, 42 insertions, 34 deletions
diff --git a/doc/ref/api-io.texi b/doc/ref/api-io.texi
index 23d3b50cd..5b200977b 100644
--- a/doc/ref/api-io.texi
+++ b/doc/ref/api-io.texi
@@ -78,11 +78,17 @@ string doesn't depend on its context: the same byte sequence will always
return the same string. A couple of modal encodings are in common use,
like ISO-2022-JP and ISO-2022-KR, and they are not yet supported.
-Each port also has an associated conversion strategy: what to do when
-a Guile character can't be converted to the port's encoded character
-representation for output. There are three possible strategies: to
-raise an error, to replace the character with a hex escape, or to
-replace the character with a substitute character.
+@cindex port conversion strategy
+@cindex conversion strategy, port
+@cindex decoding error
+@cindex encoding error
+Each port also has an associated conversion strategy, which determines
+what to do when a Guile character can't be converted to the port's
+encoded character representation for output. There are three possible
+strategies: to raise an error, to replace the character with a hex
+escape, or to replace the character with a substitute character. Port
+conversion strategies are also used when decoding characters from an
+input port.
Finally, all ports have associated input and output buffers, as
appropriate. Buffering is a common strategy to limit the overhead of
@@ -142,14 +148,10 @@ its input and output. The value @code{#f} is equivalent to @code{"ISO-8859-1"}.
@deffn {Scheme Procedure} set-port-conversion-strategy! port sym
@deffnx {C Function} scm_set_port_conversion_strategy_x (port, sym)
-Sets the behavior of the interpreter when outputting a character that
-is not representable in the port's current encoding. @var{sym} can be
-either @code{'error}, @code{'substitute}, or @code{'escape}. If it is
-@code{'error}, an error will be thrown when an nonconvertible character
-is encountered. If it is @code{'substitute}, then nonconvertible
-characters will be replaced with approximate characters, or with
-question marks if no approximately correct character is available. If
-it is @code{'escape}, it will appear as a hex escape when output.
+Sets the behavior of Guile when outputting a character that is not
+representable in the port's current encoding, or when Guile encounters a
+decoding error when trying to read a character. @var{sym} can be either
+@code{error}, @code{substitute}, or @code{escape}.
If @var{port} is an open port, the conversion error behavior
is set for that port. If it is @code{#f}, it is set as the
@@ -157,15 +159,27 @@ default behavior for any future ports that get created in
this thread.
@end deffn
+For an output port, a there are three possible port conversion
+strategies. The @code{error} strategy will throw an error when a
+nonconvertible character is encountered. The @code{substitute} strategy
+will replace nonconvertible characters with a question mark (@samp{?}).
+Finally the @code{escape} strategy will print nonconvertible characters
+as a hex escape, using the escaping that is recognized by Guile's string
+syntax. Note that if the port's encoding is a Unicode encoding, like
+@code{UTF-8}, then encoding errors are impossible.
+
+For an input port, the @code{error} strategy will cause Guile to throw
+an error if it encounters an invalid encoding, such as might happen if
+you tried to read @code{ISO-8859-1} as @code{UTF-8}. The error is
+thrown before advancing the read position. The @code{substitute}
+strategy will replace the bad bytes with a U+FFFD replacement character,
+in accordance with Unicode recommendations. When reading from an input
+port, the @code{escape} strategy is treated as if it were @code{error}.
+
@deffn {Scheme Procedure} port-conversion-strategy port
@deffnx {C Function} scm_port_conversion_strategy (port)
-Returns the behavior of the port when outputting a character that is
-not representable in the port's current encoding. It returns the
-symbol @code{error} if unrepresentable characters should cause
-exceptions, @code{substitute} if the port should try to replace
-unrepresentable characters with question marks or approximate
-characters, or @code{escape} if unrepresentable characters should be
-converted to string escapes.
+Returns the behavior of the port when outputting a character that is not
+representable in the port's current encoding.
If @var{port} is @code{#f}, then the current default behavior will be
returned. New ports will have this default behavior when they are
@@ -179,9 +193,9 @@ and for other conversion routines such as @code{scm_to_stringn},
@code{pointer->string}.
Its value must be one of the symbols described above, with the same
-semantics: @code{'error}, @code{'substitute}, or @code{'escape}.
+semantics: @code{error}, @code{substitute}, or @code{escape}.
-When Guile starts, its value is @code{'substitute}.
+When Guile starts, its value is @code{substitute}.
Note that @code{(set-port-conversion-strategy! #f @var{sym})} is
equivalent to @code{(fluid-set! %default-port-conversion-strategy
@@ -226,13 +240,10 @@ interactive port that has no ready characters.
@rnindex read-char
@deffn {Scheme Procedure} read-char [port]
@deffnx {C Function} scm_read_char (port)
-Return the next character available from @var{port}, updating
-@var{port} to point to the following character. If no more
-characters are available, the end-of-file object is returned.
-
-When @var{port}'s data cannot be decoded according to its character
-encoding, a @code{decoding-error} is raised and @var{port} is not
-advanced past the erroneous byte sequence.
+Return the next character available from @var{port}, updating @var{port}
+to point to the following character. If no more characters are
+available, the end-of-file object is returned. A decoding error, if
+any, is handled in accordance with the port's conversion strategy.
@end deffn
@deftypefn {C Function} size_t scm_c_read (SCM port, void *buffer, size_t size)
@@ -262,8 +273,8 @@ return the value returned by the preceding call to
an interactive port will hang waiting for input whenever a call
to @code{read-char} would have hung.
-As for @code{read-char}, a @code{decoding-error} may be raised
-if such a situation occurs.
+As for @code{read-char}, decoding errors are handled in accordance with
+the port's conversion strategy.
@end deffn
@deffn {Scheme Procedure} unread-char cobj [port]
@@ -627,9 +638,6 @@ Push the terminating delimiter (if any) back on to the port.
Return a pair containing the string read from the port and the
terminating delimiter or end-of-file object.
@end table
-
-Like @code{read-char}, this procedure can throw to @code{decoding-error}
-(@pxref{Reading, @code{read-char}}).
@end deffn
@c begin (scm-doc-string "rdelim.scm" "read-line!")