summaryrefslogtreecommitdiff
path: root/libguile/strings.c
AgeCommit message (Collapse)AuthorFilesLines
2011-06-18Merge remote-tracking branch 'local-2.0/stable-2.0'Andy Wingo1-17/+24
Conflicts: module/ice-9/psyntax-pp.scm module/language/tree-il/compile-glil.scm
2011-06-16Fix several POSIX functions to use the locale encodingMark H Weaver1-17/+24
* libguile/strings.c (scm_i_allocate_string_pointers): Encode strings using the current locale. Previously, Latin-1 was used. Indirectly, this affects the encoding of strings in `system*', `execl', `execlp', `execle', `environ', and `dynamic-args-call'. (scm_makfromstrs): In header comment, clarify that the C strings are interpreted according to the current locale encoding. * NEWS: Add NEWS entry.
2011-05-26Merge remote-tracking branch 'local-2.0/stable-2.0'Andy Wingo1-2/+2
2011-05-26deprecate scm_immutable_{double_,}cellAndy Wingo1-2/+2
* libguile/inline.h: * libguile/deprecated.h: * libguile/deprecated.c (scm_immutable_cell, scm_immutable_double_cell): Deprecate these, as the GC_STUBBORN API doesn't do anything any more. * libguile/strings.c (scm_i_c_make_symbol): Change the one use of scm_immutable_double_cell to scm_double_cell.
2011-05-12remove all deprecated codeAndy Wingo1-60/+0
* libguile/async.c: * libguile/async.h: * libguile/debug.h: * libguile/deprecated.c: * libguile/deprecated.h: * libguile/evalext.h: * libguile/gc-malloc.c: * libguile/gc.h: * libguile/gen-scmconfig.c: * libguile/numbers.c: * libguile/ports.c: * libguile/ports.h: * libguile/procprop.c: * libguile/procprop.h: * libguile/read.c: * libguile/socket.c: * libguile/srfi-4.h: * libguile/strings.c: * libguile/strings.h: * libguile/tags.h: * module/ice-9/boot-9.scm: * module/ice-9/deprecated.scm: Remove all deprecated code. CPP defines that were not previously issuing warnings were changed so that their expansions would indicate the replacement forms to use, e.g. scm_sizet__GONE__REPLACE_WITH__size_t. The two exceptions were SCM_LISTN, which did not produce warnings before, and the string-filter argument order stuff. Drops the initial dirty memory usage of Guile down to 2.8 MB on my machine, from 4.4 MB.
2011-05-07bytevectors have internal parent fieldAndy Wingo1-1/+1
* libguile/bytevectors.h (SCM_BYTEVECTOR_HEADER_SIZE): Bump, giving bytevectors another word: a parent pointer. Will allow for sub-bytevectors and efficient mmap bindings. * libguile/bytevectors.c (make_bytevector): (make_bytevector_from_buffer): Init parent to #f. (scm_c_take_bytevector, scm_c_take_typed_bytevector): Another argument, the parent, which gets set in the bytevector. * libguile/foreign.c (scm_pointer_to_bytevector): Use the parent field instead of registering a weak reference from bytevector to foreign pointer. * libguile/objcodes.c (scm_objcode_to_bytecode): Use the parent field to avoid copying the objcode. * libguile/srfi-4.c (DEFINE_SRFI_4_C_FUNCS): * libguile/strings.c (scm_from_stringn): * libguile/vm.c (really_make_boot_program): * libguile/r6rs-ports.c (scm_get_bytevector_some) (scm_get_bytevector_all, bytevector_output_port_procedure): Set the parent to #f.
2011-04-01latin1 subr and message in internal scm_{encoding,decoding}_errorAndy Wingo1-4/+4
* libguile/strings.c (scm_encoding_error, scm_decoding_error): Use scm_from_latin1_string for the subr and message args, as these are internal functions, and we know their callers.
2011-03-20Make VM string literals immutable.Ludovic Courtès1-23/+27
* libguile/strings.c (scm_i_make_string, scm_i_make_wide_string): Add `read_only_p' parameter. All callers updated. * libguile/vm-i-loader.c (load_string, load_wide_string): Push read-only strings. * test-suite/tests/strings.test ("literals"): New test prefix.
2011-03-17scm_{to,from}_locale_string use current locale, not current portsAndy Wingo1-33/+3
* libguile/strings.c (scm_to_locale_stringn, scm_from_locale_stringn): Use the encoding of the current locale, not of the current i/o ports. Also use the current conversion strategy. * doc/ref/api-data.texi (Conversion to/from C): Update docs.
2011-02-02Change `scm_encoding_error' to pass the port and faulty character.Ludovic Courtès1-12/+12
* libguile/strings.c (scm_encoding_error): Remove the `from', `to', and `string_or_bv' parameters; add `port' and `chr'. (scm_to_stringn): Update accordingly. * libguile/strings.h (scm_encoding_error): Update accordingly. * libguile/ports.c (scm_ungetc): Update accordingly. * libguile/print.c (iprin1, scm_write_char): Update accordingly. * test-suite/tests/encoding-escapes.test ("display output errors")["ultima", "Rashomon"]: Check the arguments of `encoding-error'. ["tekniko"]: New test. * test-suite/tests/ports.test ("string ports")["wrong encoding"]: Adjust to new `encoding-error' arguments.
2011-02-02Have `read-char' & co. throw to `decoding-error'.Ludovic Courtès1-4/+21
* libguile/ports.c (scm_read_char): Mention `decoding-error' in the docstring. (get_codepoint): Change to return an error code; add `codepoint' output parameter. Don't raise an error from here. (scm_getc): Raise an error with `scm_decoding_error' if `get_codepoint' returns an error. (scm_peek_char): Likewise. Update docstring. * libguile/strings.c (scm_decoding_error_key): New variable. (scm_decoding_error): New function. (scm_from_stringn): Use `scm_decoding_error' instead of `scm_encoding_error'. * libguile/strings.h (scm_decoding_error): New declaration. * test-suite/tests/ports.test ("string ports")["read-char, wrong encoding, error"]: Change to expect `decoding-error'. Make sure PORT points past the error. ["read-char, wrong encoding, escape"]: Likewise. ["peek-char, wrong encoding, error"]: New test. * test-suite/tests/r6rs-ports.test ("7.2.11 Binary Output")["put-bytevector with wrong-encoding string port"]: Change to expect `decoding-error'. ("8.2.6 Input and output ports")["transcoded-port [error handling mode = raise]"]: Likewise. * test-suite/tests/rdelim.test ("read-line")["decoding error", "decoding error, substitute"]: New tests. * doc/ref/api-io.texi (Reading): Update documentation of `read-char' and `peek-char'. (Line/Delimited): Update documentation of `read-line'.
2011-01-26Add `scm_{to,from}_utf32_string'.Ludovic Courtès1-0/+58
* libguile/strings.c (scm_from_utf32_string, scm_from_utf32_stringn, scm_to_utf32_string, scm_to_utf32_stringn): New functions. * libguile/strings.h (scm_from_utf32_string, scm_from_utf32_stringn, scm_to_utf32_string, scm_to_utf32_stringn): New declarations. * doc/ref/api-data.texi (Conversion to/from C): Document `scm_{to,from}_{utf8,utf32}_stringn'.
2011-01-26Optimize `scm_{to,from}_latin1_string'.Ludovic Courtès1-2/+29
* libguile/strings.c (scm_from_latin1_stringn): Directly return a narrow string instead of going through `scm_from_stringn'. (scm_to_latin1_stringn): Directly return a copy of STR's raw bytes when it's narrow.
2011-01-23Hide the string escaping hacks.Ludovic Courtès1-7/+11
* libguile/strings.c (scm_i_unistring_escapes_to_guile_escapes): Rename to... (unistring_escapes_to_guile_escapes): ... this. Make `static'. (scm_i_unistring_escapes_to_r6rs_escapes): Rename to... (unistring_escapes_to_r6rs_escapes): ... this. Make `static'. * libguile/strings.h (scm_i_unistring_escapes_to_guile_escapes, scm_i_unistring_escapes_to_r6rs_escapes): Remove declarations.
2011-01-07fix scm_from_stringn empty string caseAndy Wingo1-3/+3
* libguile/strings.c (scm_from_stringn): Fix empty string case (oops...).
2011-01-07use scm_from_latin1_symboln for string literals and load-symbolAndy Wingo1-24/+24
* libguile/bytevectors.c: * libguile/eval.c: * libguile/goops.c: * libguile/i18n.c: * libguile/load.c: * libguile/memoize.c: * libguile/modules.c: * libguile/ports.c: * libguile/print.c: * libguile/procs.c: * libguile/programs.c: * libguile/read.c: * libguile/script.c: * libguile/srfi-14.c: * libguile/stacks.c: * libguile/strings.c: * libguile/throw.c: * libguile/vm.c: Use scm_from_latin1_symboln to make symbols from string literals, because they aren't in the user's locale -- they are in ASCII, and we can optimize this case. * libguile/vm-i-loader.c: Also use scm_from_latin1_symboln when loading narrow symbols.
2011-01-07add scm_{to,from}_{utf8,latin1}_string{n,}Andy Wingo1-32/+49
* libguile/strings.h: * libguile/strings.c (scm_from_latin1_string, scm_to_latin1_string): New functions, in terms of the latin1_stringn variants. (scm_from_utf8_string, scm_from_utf8_stringn) (scm_to_utf8_string, scm_to_utf8_stringn): New functions. (scm_i_from_utf8_string, scm_i_to_utf8_string): Removed these internal functions. (scm_from_stringn): Handle -1 as a length. Unlike the previous behavior of scm_from_locale_string (NULL), which returned the empty string, we now raise an error. The null pointer is not the same as the empty string. * libguile/stime.c (scm_strftime, scm_strptime): Adapt to publishing of utf8 functions.
2010-12-16Inline `scm_is_string'.Ludovic Courtès1-5/+0
* libguile/strings.c (scm_is_string): Move to... * libguile/inline.h (scm_is_string): ... here. Inline.
2010-12-04make-string et al nulls memory if not given an initializerAndy Wingo1-3/+7
* libguile/gc-malloc.c: Add a note that the gc-malloc does not clear the memory block, so users need to make sure it is initialized. * libguile/bitvectors.c (scm_c_make_bitvector): * libguile/bytevectors.c (scm_make_bytevector): * libguile/strings.c (scm_c_make_string): If no initializer is given, initialize the bytes to 0. Prevents information leakage if an app uses make-string et al without initializers. * libguile/foreign.c (make_cif): Initialize this too, to prevent leakage in the struct holes. Paranoia...
2010-11-19Include <alloca.h> wherever `alloca' is used.Ludovic Courtès1-0/+1
Patch provided by <carlo.bramix@libero.it> (tiny change). * libguile/control.c, libguile/fluids.c, libguile/foreign.c, libguile/hashtab.c, libguile/strings.c: Include <alloca.h>.
2010-09-15Fix write-beyond-end-of-string error in the conversion to R6RS string escapes.Ludovic Courtès1-4/+16
Reported by Mike Gran <spk121@yahoo.com>. * libguile/strings.c (scm_i_unistring_escapes_to_guile_escapes, scm_i_unistring_escapes_to_r6rs_escapes): Augment comments. (scm_to_stringn): When `handler == SCM_FAILED_CONVERSION_ESCAPE_SEQUENCE && SCM_R6RS_ESCAPES_P', realloc BUF so that it's large enough for the worst case. * libguile/print.c (display_character): When `result != NULL && strategy == SCM_FAILED_CONVERSION_ESCAPE_SEQUENCE && SCM_R6RS_ESCAPES_P', make LOCALE_ENCODED large enough to hold an R6RS escape.
2010-09-14Internally expose `scm_i_unistring_escapes_to_{guile,r6rs}_escapes'.Ludovic Courtès1-11/+11
* libguile/strings.c (unistring_escapes_to_guile_escapes): Rename to... (scm_i_unistring_escapes_to_guile_escapes): ... this. Change `char **bufp' to `char *buf'; leave realloc responsibility to the caller. Update caller. (unistring_escapes_to_r6rs_escapes): Rename to... (scm_i_unistring_escapes_to_r6rs_escapes): ... this. Likewise.
2010-09-12Provide non-locale C/Scheme string conversion functionsMichael Gran1-0/+11
* doc/ref/api-data.texi: document scm_to_stringn, scm_from_stringn, scm_to_latin1_stringn, and scm_from_latin1_stringn * libguile/strings.h (scm_to_stringn): make public (scm_to_latin1_stringn): new declaration (scm_from_latin1_stringn): new declaration * libguile/strings.c (scm_to_latin1_stringn): new function (scm_from_latin1_stringn): new function
2010-07-15Expose `scm_encoding_error'.Ludovic Courtès1-1/+1
* libguile/strings.c (scm_encoding_error): Make public. * libguile/strings.h (scm_encoding_error): New internal declaration.
2010-07-04Add `scm_i_string_data'.Ludovic Courtès1-3/+21
* libguile/strings.c (STRINGBUF_CONTENTS): New macro. (STRINGBUF_CHARS, STRINGBUF_WIDE_CHARS): Use it. (scm_i_string_data): New function. * libguile/strings.h (scm_i_string_data): New declaration.
2010-03-18Improve encoding error reporting.Ludovic Courtès1-23/+34
* libguile/strings.c (scm_encoding_error): Change arguments to convey more information. Raise the error with `scm_throw ()', passing all the information to the handler. (scm_from_stringn, scm_to_stringn): Update accordingly. * test-suite/tests/ports.test ("string ports")["wrong encoding"]: Check the arguments passed to the `throw' handler. * test-suite/tests/r6rs-ports.test ("7.2.11 Binary Output")["put-bytevector with wrong-encoding string port"]: Likewise.
2010-01-23R6RS string escapes broken on string outputMichael Gran1-11/+87
scm_to_stringn failed to do the necessary escape conversion for R6RS hex escapes * libguile/strings.c (unistring_escapes_to_r6rs_escapes): new function (scm_to_stringn): use new function when r6rs hex escapes are enabled * test-suite/tests/reader.test: new test for string display
2010-01-07Use `encoding-error' instead of `misc-error' for string encoding errors.Ludovic Courtès1-13/+23
* libguile/strings.c (scm_encoding_error): New function. (scm_from_stringn, scm_to_stringn): Use it instead of `scm_misc_error ()'. * test-suite/lib.scm (exception:encoding-error): Adjust accordingly. * test-suite/tests/encoding-escapes.test (exception:conversion): Remove. Use `exception:encoding-error' instead. * test-suite/tests/encoding-iso88591.test: Likewise. * test-suite/tests/encoding-iso88597.test: Likewise. * test-suite/tests/encoding-utf8.test: Likewise.
2010-01-07Add in-source documentation of `scm_to_stringn ()'.Ludovic Courtès1-2/+5
* libguile/strings.c (scm_to_stringn): Add documentation comment.
2010-01-06Attempt to narrow normalized strings.Julian Graham1-1/+6
* libguile/strings.c (normalize_str): Clean up indentation. Add call to scm_i_try_narrow_string.
2010-01-03Support for Unicode string normalization functionsJulian Graham1-0/+73
* libguile/strings.c, libguile/strings.h (normalize_str, scm_string_normalize_nfc, scm_string_normalize_nfd, scm_normalize_nfkc, scm_string_normalize_nfkd): New functions. * test-suite/tests/strings.test: Unit tests for `string-normalize-nfc', `string-normalize-nfd', `string-normalize-nfkc', and `string-normalize-nfkd'. * doc/ref/api-data.texi (String Comparison): Documentation for normalization functions.
2009-12-29fix bug in string array implementation type maskAndy Wingo1-1/+1
* libguile/strings.c (SCM_ARRAY_IMPLEMENTATION): The mask for the string array implementation should be 0x7f, without masking out 0x2. Otherwise numbers were being thought to be vectors! * test-suite/tests/unif.test: Add test. * libguile/vectors.c (SCM_ARRAY_IMPLEMENTATION): Only register one implementation, because weak vectors can be checked with the mask & ~2, and the functions are the same.
2009-12-15Remove extraneous semicolon after `SCM_ARRAY_IMPLEMENTATION' & co.Ludovic Courtès1-2/+2
* libguile/bitvectors.c, libguile/srfi-4.c, libguile/strings.c, libguile/vectors.c: Remove extraneous semicolon after `SCM_ARRAY_IMPLEMENTATION' and `SCM_VECTOR_IMPLEMENTATION' invocations.
2009-12-05decruftify scm_sys_protectsAndy Wingo1-0/+2
* libguile/root.h * libguile/root.c (scm_sys_protects): It used to be that for some reason we'd define a special array of "protected" values. This was a little silly, always, but with the BDW GC it's completely unnecessary. Also many of these variables were unused, and none of them were good API. So remove this array, and either eliminate, make static, or make internal the various values. * libguile/snarf.h: No need to generate calls to scm_permanent_object. * guile-readline/readline.c (scm_init_readline): No need to call scm_permanent_object. * libguile/array-map.c (ramap, rafe): Remove the dubious nullvect optimizations. * libguile/async.c (scm_init_async): No need to init scm_asyncs, it is no more. * libguile/eval.c (scm_init_eval): No need to init scm_listofnull, it is no more. * libguile/gc.c: Make scm_protects a static var. (scm_storage_prehistory): Change the sanity check to use the address of protects. (scm_init_gc_protect_object): No need to clear the scm_sys_protects, as it is no more. * libguile/keywords.c: Make the keyword obarray a static var. * libguile/numbers.c: Make flo0 a static var. * libguile/objprop.c: Make object_whash a static var. * libguile/properties.c: Make properties_whash a static var. * libguile/srcprop.h: * libguile/srcprop.c: Make scm_source_whash a global with internal linkage. * libguile/strings.h: * libguile/strings.c: Make scm_nullstr a global with internal linkage. * libguile/vectors.c (scm_init_vectors): No need to init scm_nullvect, it's unused.
2009-11-17Fix stylistic issues revealed by "make syntax-check".Ludovic Courtès1-5/+5
* libguile/gc-malloc.c (scm_must_free): Remove unnecessary `if' before `free ()'. * libguile/stime.c (scm_localtime, scm_mktime): Likewise. * libguile/eval.i.c (ceval): Don't cast the result of alloca(3). * libguile/i18n.c (SCM_STRING_TO_U32_BUF): Likewise. * test-suite/standalone/test-unwind.c: Likewise. * libguile/strings.c (scm_i_deprecated_string_chars): Don't end error message in period.
2009-11-17Remove references to undefined macros.Ludovic Courtès1-6/+10
The intent is to allow compilation with `-Wundef', which in turn should make it easier to catch erroneous uses of nonexistent macros. * libguile/__scm.h: Don't assume `BUILDING_LIBGUILE' is defined. * libguile/conv-uinteger.i.c (SCM_TO_TYPE_PROTO): Remove unneeded CPP conditional on `TYPE_MIN == 0'. * libguile/fports.c: Check for the definition of `HAVE_CHSIZE' and `HAVE_FTRUNCATE', not for their value. * libguile/ports.c: Likewise. * libguile/numbers.c (guile_ieee_init): Likewise with `HAVE_DINFINITY' and `HAVE_DQNAN'. * test-suite/standalone/test-conversion.c (ieee_init): Likewise. * libguile/strings.c: Likewise with `SCM_STRING_LENGTH_HISTOGRAM'. * libguile/strings.h: Likewise. * libguile/tags.h: Likewise with `HAVE_INTTYPES_H' and `HAVE_STDINT_H'. * libguile/threads.c: Likewise with `HAVE_PTHREAD_GET_STACKADDR_NP'. * libguile/vm-engine.c (VM_NAME): Likewise with `VM_CHECK_IP'. * libguile/gen-scmconfig.c (main): Use "#ifdef HAVE_", not "#if HAVE_". * libguile/socket.c (scm_setsockopt): Likewise.
2009-11-01Merge branch 'bdw-gc-static-alloc'Ludovic Courtès1-7/+12
Conflicts: acinclude.m4 libguile/__scm.h libguile/bdw-gc.h libguile/eval.c
2009-09-09Merge branch 'master' into boehm-demers-weiser-gcLudovic Courtès1-2/+7
Conflicts: libguile/gc_os_dep.c
2009-09-09Make scm_i_from_stringn into API for use with libguilereadlineMichael Gran1-4/+4
* libguile/strings.c (scm_i_from_stringn): renamed to scm_from_stringn. All callers changed. * libguile/strings.h: change declaration of scm_i_from_stringn to scm_from_stringn * libguile/strports.c (scm_strport_to_string): scm_i_from_stringn -> scm_from_stringn * guile-readline/readline.c (internal_readline): scm_i_from_stringn -> scm_from_stringn
2009-09-02Merge branch 'boehm-demers-weiser-gc' into bdw-gc-static-allocLudovic Courtès1-203/+1023
Conflicts: acinclude.m4 libguile/strings.c
2009-09-01Remove the distinction between inline/outline storage for stringbufs.Ludovic Courtès1-129/+90
* libguile/strings.c (STRINGBUF_HEADER_SIZE, STRINGBUF_HEADER_BYTES): New macros. (STRINGBUF_F_INLINE, STRINGBUF_INLINE, STRINGBUF_OUTLINE_CHARS, STRINGBUF_OUTLINE_LENGTH, STRINGBUF_INLINE_CHARS, STRINGBUF_INLINE_LENGTH, STRINGBUF_MAX_INLINE_LEN): Remove. (STRINGBUF_CHARS, STRINGBUF_WIDE_CHARS): Adjust to return a fixed location. (STRINGBUF_LENGTH): Get the length from word 1. (make_stringbuf, make_wide_stringbuf): Adjust to use a contiguous memory region. (wide_stringbuf): Renamed from `widen_stringbuf'. Adjust similarly. Return the new stringbuf. Callers updated. (narrow_stringbuf): Likewise. (scm_sys_string_dump, scm_sys_symbol_dump): Remove `stringbuf-inline' pair. * test-suite/tests/strings.test ("string internals")["null strings are inlined", "short Latin-1 encoded strings are inlined", "long Latin-1 encoded strings are not inlined", "short UCS-4 encoded strings are not inlined", "long UCS-4 encoded strings are not inlined"]: Remove. * test-suite/tests/symbols.test ("symbol internals")["null symbols are inlined", "short Latin-1 encoded symbols are inlined", "long Latin-1 encoded symbols are not inlined", "short UCS-4 encoded symbols are not inlined", "long UCS-4 encoded symbols are not inlined"]: Remove.
2009-09-01Fix leaky handling of `scm_take_locale_{symbol,string} ()'.Ludovic Courtès1-37/+6
* libguile/strings.c (scm_i_take_stringbufn, scm_i_c_take_symbol): Remove. (scm_take_locale_stringn): Rewrite in terms of `scm_from_locale_stringn ()'. * libguile/strings.h (scm_i_c_take_symbol, scm_i_take_stringbufn): Remove declarations.
2009-08-30Fix escape sequence normalization for wide stringsMichael Gran1-0/+2
* libguile/strings.c (scm_to_stringn): convert unistring escapes to guile escapes for both wide and narrow strings
2009-08-30Fix encoding errors with strings returned by string portsMichael Gran1-6/+9
String ports, being 8-bit, store strings using the character encoding of the port. This fixes a bug where the default character encoding, and not the port's encoding, was being used to convert the string port data back to a string. * libguile/strports.c: extra comments (scm_strport_to_string): use port's encoding when converting port data to a string * libguile/strings.c (scm_i_from_stringn): renamed from scm_from_stringn and made internal. All callers changed. (scm_from_stringn): renamed to scm_i_from_stringn. * libguile/strings.h: declaration for scm_i_from_stringn
2009-08-28Merge branch 'master' into boehm-demers-weiser-gcLudovic Courtès1-50/+338
Conflicts: libguile/Makefile.am libguile/bytevectors.c libguile/gc-card.c libguile/gc-mark.c libguile/programs.c libguile/srcprop.c libguile/srfi-14.c libguile/symbols.c libguile/threads.c libguile/unif.c libguile/vm.c
2009-08-25Merge commit 'origin/master'Andy Wingo1-26/+238
Conflicts: libguile/unif.c
2009-08-25Merge wip-array refactor, up to cd43fdc5b7a7cAndy Wingo1-0/+31
Conflicts: NEWS libguile/print.c
2009-08-25Add full Unicode capability to ports and the default readerMichael Gran1-43/+132
Ports are given two additional properties: a character encoding and a conversion failure strategy. These properties have getters and setters. The new properties are used to convert any locale text to/from the internal representation of strings. If unspecified, ports use a default value. The default value of these properties is held in a fluid. The default character encoding can be modified by calling setlocale. ISO-8859-1 is treated specially. Since it is a native encoding of strings, it can be processed more quickly. Source code is assumed to be ISO-8859-1 unless otherwise specified. The encoding of a source code file can be given as 'coding: XXXXX' in a magic comment at the top of a file. The C functions that deal with encoding often use a null pointer as shorthand for the native Latin-1 encoding, for efficiency's sake. * test-suite/tests/encoding-iso88591.test: new tests * test-suite/tests/encoding-iso88597.test: new tests * test-suite/tests/encoding-utf8.test: new tests * test-suite/tests/encoding-escapes.test: new tests * test-suite/tests/numbers.test: declare 'binary' encoding * test-suite/tests/ports.test: declare 'binary' encoding * test-suite/tests/r6rs-ports.test: declare 'binary' encoding * module/system/base/compile.scm (compile-file): use source-code file's self-declared encoding when compiling files * libguile/strports.c: store string ports in locale encoding (scm_strport_to_locale_u8vector, scm_call_with_output_locale_u8vector) (scm_open_input_locale_u8vector, scm_get_output_locale_u8vector): new functions * libguile/strings.h: new declaration for scm_i_string_contains_char * libguile/strings.c (scm_i_string_contains_char): new function (scm_from_stringn, scm_to_stringn): use NULL for Latin-1 (scm_from_locale_stringn, scm_to_locale_stringn): respect character encoding of input and output ports * libguile/read.h: declaration for scm_scan_for_encoding * libguile/read.c: (read_token): now takes scheme string instead of C string/length (read_complete_token): new function (scm_read_sexp, scm_read_number, scm_read_mixed_case_symbol) (scm_read_number_and_radix, scm_read_quote, scm_read_semicolon_comment) (scm_read_srfi4_vector, scm_read_bytevector, scm_read_guile_bit_vector) (scm_read_scsh_block_comment, scm_read_commented_expression) (scm_read_extended_symbol, scm_read_sharp_extension, scm_read_shart) (scm_read_expression): use scm_t_wchar for char type, use read_complete_token (scm_scan_for_encoding): new function to find a file's character encoding (scm_file_encoding): new function to find a port's character encoding * libguile/rdelim.c: don't unpack strings * libguile/print.h: declaration for modified function scm_i_charprint * libguile/print.c: use locale when printing characters and strings (scm_i_charprint): input parameter is now scm_t_wchar (scm_simple_format): don't unpack strings * libguile/posix.h: new declaration for scm_setbinary. * libguile/posix.c (scm_setlocale): set default and stdio port encodings based on the locale's character encoding (scm_setbinary): new function * libguile/ports.h (scm_t_port): add encoding and failed conversion handler to port type. Declarations for new or modified functions scm_getc, scm_unget_byte, scm_ungetc, scm_i_get_port_encoding, scm_i_set_port_encoding_x, scm_port_encoding, scm_set_port_encoding_x, scm_i_get_conversion_strategy, scm_i_set_conversion_strategy_x, scm_port_conversion_strategy, scm_set_port_conversion_strategy_x. * libguile/ports.c: assign the current ports to zero on startup so we can see if they've been set. (scm_current_input_port, scm_current_output_port, scm_current_error_port): return #f if the port is not yet initialized (scm_new_port_table_entry): set up a new port's encoding and illegal sequence handler based on the thread's current defaults (scm_i_remove_port): free port encoding name when port is removed (scm_i_mode_bits_n): now takes a scheme string instead of a c string and length. All callers changed. (SCM_MBCHAR_BUF_SIZE): new const (scm_getc): new function, since the scm_getc in inline.h is now scm_get_byte_or_eof. This pulls one codepoint from a port. (scm_lfwrite_substr, scm_lfwrite_str): now uses port's encoding (scm_unget_byte): new function, incorportaing the low-level functionality of scm_ungetc (scm_ungetc): uses scm_unget_byte * libguile/numbers.h (scm_t_wchar): compilation order problem with scm_t_wchar being use in functions in multiple headers. Forward declare scm_t_wchar. * libguile/load.c (scm_primitive_load): scan for file encoding at top of file and use it to set the load port's encoding * libguile/inline.h (scm_get_byte_or_eof): new function incorporating most of the functionality of scm_getc. * libguile/fports.c (fport_fill_input): now returns scm_t_wchar * libguile/chars.h (scm_t_wchar): avoid compilation order problem with declaration of scm_t_wchar
2009-08-23Modify socket and time functions for wide stringsMichael Gran1-10/+133
* libguile/socket.c (scm_recv): receive the message without holding the stringbuf writing lock (scm_send): try to narrow a string before using it * libguile/stime.c (strftime): convert string to UTF-8 so that it can be safely passed to strftime (strptime): convert input string to UTF-8 so that it can be safely passed through strptime * libguile/strings.c (narrow_stringbuf): new function (scm_i_try_narrow_string): new function * libguile/strings.h: new declaration for scm_i_try_narrow_string
2009-08-21Use string accessors for string->number conversionMichael Gran1-0/+23
* libguile/numbers.c (scm_i_print_fraction): use string accessors (XDIGIT2UINT): use libunistring function (mem2uinteger, mem2integer, mem2decimal_from_point, mem2ureal) (mem2complex): take scheme string instead of c string; use accessors (scm_i_string_to_number): new function (scm_c_locale_string_to_number): use scm_i_string_to_number * libguile/numbers.h: declaration for scm_i_string_to_number * libguile/strings.c (scm_i_string_strcmp): new function * libguile/strings.h: declaration for scm_i_string_strcmp