summaryrefslogtreecommitdiff
path: root/libguile/strings.h
AgeCommit message (Collapse)AuthorFilesLines
2012-02-23Revert "wrap iconv_open / iconv_close with a lock to help in thread/fork issues"Andy Wingo1-4/+1
This reverts commit 8dfb7bbfd908ca883d0fdd0d868e13e6b20803ae.
2012-02-17wrap iconv_open / iconv_close with a lock to help in thread/fork issuesAndy Wingo1-1/+4
* libguile/bytevectors.c (STRING_TO_UTF, scm_string_to_utf8) (UTF_TO_STRING): * libguile/ports.c (open_iconv_descriptors, close_iconv_descriptors): * libguile/strings.c (scm_from_stringn, scm_to_stringn): Wrap operations that acquire and destroy iconv contexts with a mutex. While iconv is threadsafe, internally it uses a lock, and we need to make sure when we fork() that no one has that lock -- so we surround it with another one. Gross.
2012-01-10Merge remote-tracking branch 'origin/stable-2.0'Andy Wingo1-1/+1
Conflicts: libguile/__scm.h libguile/array-map.c libguile/procprop.c libguile/tags.h module/ice-9/deprecated.scm module/ice-9/psyntax-pp.scm module/ice-9/psyntax.scm test-suite/standalone/test-num2integral.c test-suite/tests/regexp.test
2012-01-05Placate a number of `syntax-check' verifications.Ludovic Courtès1-1/+1
- "filesystem" -> "file system" - remove doubled words - use EXIT_* macros instead of literal numbers - update `syntax-check' exclusion files
2011-05-12remove all deprecated codeAndy Wingo1-15/+0
* libguile/async.c: * libguile/async.h: * libguile/debug.h: * libguile/deprecated.c: * libguile/deprecated.h: * libguile/evalext.h: * libguile/gc-malloc.c: * libguile/gc.h: * libguile/gen-scmconfig.c: * libguile/numbers.c: * libguile/ports.c: * libguile/ports.h: * libguile/procprop.c: * libguile/procprop.h: * libguile/read.c: * libguile/socket.c: * libguile/srfi-4.h: * libguile/strings.c: * libguile/strings.h: * libguile/tags.h: * module/ice-9/boot-9.scm: * module/ice-9/deprecated.scm: Remove all deprecated code. CPP defines that were not previously issuing warnings were changed so that their expansions would indicate the replacement forms to use, e.g. scm_sizet__GONE__REPLACE_WITH__size_t. The two exceptions were SCM_LISTN, which did not produce warnings before, and the string-filter argument order stuff. Drops the initial dirty memory usage of Guile down to 2.8 MB on my machine, from 4.4 MB.
2011-03-20Make VM string literals immutable.Ludovic Courtès1-2/+5
* libguile/strings.c (scm_i_make_string, scm_i_make_wide_string): Add `read_only_p' parameter. All callers updated. * libguile/vm-i-loader.c (load_string, load_wide_string): Push read-only strings. * test-suite/tests/strings.test ("literals"): New test prefix.
2011-02-02Change `scm_encoding_error' to pass the port and faulty character.Ludovic Courtès1-3/+1
* libguile/strings.c (scm_encoding_error): Remove the `from', `to', and `string_or_bv' parameters; add `port' and `chr'. (scm_to_stringn): Update accordingly. * libguile/strings.h (scm_encoding_error): Update accordingly. * libguile/ports.c (scm_ungetc): Update accordingly. * libguile/print.c (iprin1, scm_write_char): Update accordingly. * test-suite/tests/encoding-escapes.test ("display output errors")["ultima", "Rashomon"]: Check the arguments of `encoding-error'. ["tekniko"]: New test. * test-suite/tests/ports.test ("string ports")["wrong encoding"]: Adjust to new `encoding-error' arguments.
2011-02-02Have `read-char' & co. throw to `decoding-error'.Ludovic Courtès1-0/+2
* libguile/ports.c (scm_read_char): Mention `decoding-error' in the docstring. (get_codepoint): Change to return an error code; add `codepoint' output parameter. Don't raise an error from here. (scm_getc): Raise an error with `scm_decoding_error' if `get_codepoint' returns an error. (scm_peek_char): Likewise. Update docstring. * libguile/strings.c (scm_decoding_error_key): New variable. (scm_decoding_error): New function. (scm_from_stringn): Use `scm_decoding_error' instead of `scm_encoding_error'. * libguile/strings.h (scm_decoding_error): New declaration. * test-suite/tests/ports.test ("string ports")["read-char, wrong encoding, error"]: Change to expect `decoding-error'. Make sure PORT points past the error. ["read-char, wrong encoding, escape"]: Likewise. ["peek-char, wrong encoding, error"]: New test. * test-suite/tests/r6rs-ports.test ("7.2.11 Binary Output")["put-bytevector with wrong-encoding string port"]: Change to expect `decoding-error'. ("8.2.6 Input and output ports")["transcoded-port [error handling mode = raise]"]: Likewise. * test-suite/tests/rdelim.test ("read-line")["decoding error", "decoding error, substitute"]: New tests. * doc/ref/api-io.texi (Reading): Update documentation of `read-char' and `peek-char'. (Line/Delimited): Update documentation of `read-line'.
2011-01-26Add `scm_{to,from}_utf32_string'.Ludovic Courtès1-0/+5
* libguile/strings.c (scm_from_utf32_string, scm_from_utf32_stringn, scm_to_utf32_string, scm_to_utf32_stringn): New functions. * libguile/strings.h (scm_from_utf32_string, scm_from_utf32_stringn, scm_to_utf32_string, scm_to_utf32_stringn): New declarations. * doc/ref/api-data.texi (Conversion to/from C): Document `scm_{to,from}_{utf8,utf32}_stringn'.
2011-01-23Hide the string escaping hacks.Ludovic Courtès1-4/+0
* libguile/strings.c (scm_i_unistring_escapes_to_guile_escapes): Rename to... (unistring_escapes_to_guile_escapes): ... this. Make `static'. (scm_i_unistring_escapes_to_r6rs_escapes): Rename to... (unistring_escapes_to_r6rs_escapes): ... this. Make `static'. * libguile/strings.h (scm_i_unistring_escapes_to_guile_escapes, scm_i_unistring_escapes_to_r6rs_escapes): Remove declarations.
2011-01-07add scm_{to,from}_{utf8,latin1}_string{n,}Andy Wingo1-5/+18
* libguile/strings.h: * libguile/strings.c (scm_from_latin1_string, scm_to_latin1_string): New functions, in terms of the latin1_stringn variants. (scm_from_utf8_string, scm_from_utf8_stringn) (scm_to_utf8_string, scm_to_utf8_stringn): New functions. (scm_i_from_utf8_string, scm_i_to_utf8_string): Removed these internal functions. (scm_from_stringn): Handle -1 as a length. Unlike the previous behavior of scm_from_locale_string (NULL), which returned the empty string, we now raise an error. The null pointer is not the same as the empty string. * libguile/stime.c (scm_strftime, scm_strptime): Adapt to publishing of utf8 functions.
2010-12-17Remove conflicting `scm_is_string' declaration.Ludovic Courtès1-1/+0
* libguile/strings.h: Move `scm_is_string' declaration... * libguile/inline.h: ... here. Reported by Noah Lavine <noah.b.lavine@gmail.com>.
2010-09-14Internally expose `scm_i_unistring_escapes_to_{guile,r6rs}_escapes'.Ludovic Courtès1-0/+4
* libguile/strings.c (unistring_escapes_to_guile_escapes): Rename to... (scm_i_unistring_escapes_to_guile_escapes): ... this. Change `char **bufp' to `char *buf'; leave realloc responsibility to the caller. Update caller. (unistring_escapes_to_r6rs_escapes): Rename to... (scm_i_unistring_escapes_to_r6rs_escapes): ... this. Likewise.
2010-09-12Provide non-locale C/Scheme string conversion functionsMichael Gran1-8/+8
* doc/ref/api-data.texi: document scm_to_stringn, scm_from_stringn, scm_to_latin1_stringn, and scm_from_latin1_stringn * libguile/strings.h (scm_to_stringn): make public (scm_to_latin1_stringn): new declaration (scm_from_latin1_stringn): new declaration * libguile/strings.c (scm_to_latin1_stringn): new function (scm_from_latin1_stringn): new function
2010-07-15Expose `scm_encoding_error'.Ludovic Courtès1-0/+4
* libguile/strings.c (scm_encoding_error): Make public. * libguile/strings.h (scm_encoding_error): New internal declaration.
2010-07-04Add `scm_i_string_data'.Ludovic Courtès1-1/+3
* libguile/strings.c (STRINGBUF_CONTENTS): New macro. (STRINGBUF_CHARS, STRINGBUF_WIDE_CHARS): Use it. (scm_i_string_data): New function. * libguile/strings.h (scm_i_string_data): New declaration.
2010-01-03Support for Unicode string normalization functionsJulian Graham1-0/+5
* libguile/strings.c, libguile/strings.h (normalize_str, scm_string_normalize_nfc, scm_string_normalize_nfd, scm_normalize_nfkc, scm_string_normalize_nfkd): New functions. * test-suite/tests/strings.test: Unit tests for `string-normalize-nfc', `string-normalize-nfd', `string-normalize-nfkc', and `string-normalize-nfkd'. * doc/ref/api-data.texi (String Comparison): Documentation for normalization functions.
2009-12-05decruftify scm_sys_protectsAndy Wingo1-0/+2
* libguile/root.h * libguile/root.c (scm_sys_protects): It used to be that for some reason we'd define a special array of "protected" values. This was a little silly, always, but with the BDW GC it's completely unnecessary. Also many of these variables were unused, and none of them were good API. So remove this array, and either eliminate, make static, or make internal the various values. * libguile/snarf.h: No need to generate calls to scm_permanent_object. * guile-readline/readline.c (scm_init_readline): No need to call scm_permanent_object. * libguile/array-map.c (ramap, rafe): Remove the dubious nullvect optimizations. * libguile/async.c (scm_init_async): No need to init scm_asyncs, it is no more. * libguile/eval.c (scm_init_eval): No need to init scm_listofnull, it is no more. * libguile/gc.c: Make scm_protects a static var. (scm_storage_prehistory): Change the sanity check to use the address of protects. (scm_init_gc_protect_object): No need to clear the scm_sys_protects, as it is no more. * libguile/keywords.c: Make the keyword obarray a static var. * libguile/numbers.c: Make flo0 a static var. * libguile/objprop.c: Make object_whash a static var. * libguile/properties.c: Make properties_whash a static var. * libguile/srcprop.h: * libguile/srcprop.c: Make scm_source_whash a global with internal linkage. * libguile/strings.h: * libguile/strings.c: Make scm_nullstr a global with internal linkage. * libguile/vectors.c (scm_init_vectors): No need to init scm_nullvect, it's unused.
2009-11-17Remove references to undefined macros.Ludovic Courtès1-1/+1
The intent is to allow compilation with `-Wundef', which in turn should make it easier to catch erroneous uses of nonexistent macros. * libguile/__scm.h: Don't assume `BUILDING_LIBGUILE' is defined. * libguile/conv-uinteger.i.c (SCM_TO_TYPE_PROTO): Remove unneeded CPP conditional on `TYPE_MIN == 0'. * libguile/fports.c: Check for the definition of `HAVE_CHSIZE' and `HAVE_FTRUNCATE', not for their value. * libguile/ports.c: Likewise. * libguile/numbers.c (guile_ieee_init): Likewise with `HAVE_DINFINITY' and `HAVE_DQNAN'. * test-suite/standalone/test-conversion.c (ieee_init): Likewise. * libguile/strings.c: Likewise with `SCM_STRING_LENGTH_HISTOGRAM'. * libguile/strings.h: Likewise. * libguile/tags.h: Likewise with `HAVE_INTTYPES_H' and `HAVE_STDINT_H'. * libguile/threads.c: Likewise with `HAVE_PTHREAD_GET_STACKADDR_NP'. * libguile/vm-engine.c (VM_NAME): Likewise with `VM_CHECK_IP'. * libguile/gen-scmconfig.c (main): Use "#ifdef HAVE_", not "#if HAVE_". * libguile/socket.c (scm_setsockopt): Likewise.
2009-11-01Merge branch 'bdw-gc-static-alloc'Ludovic Courtès1-0/+11
Conflicts: acinclude.m4 libguile/__scm.h libguile/bdw-gc.h libguile/eval.c
2009-10-02Use `SCM_DEPRECATED' in declarations of deprecated functions/variables.Ludovic Courtès1-3/+3
* libguile/deprecated.c (SCM_BUILDING_DEPRECATED_CODE): New macro. * libguile/async.c (SCM_BUILDING_DEPRECATED_CODE): Likewise. * libguile/macros.c (SCM_BUILDING_DEPRECATED_CODE): Likewise. * libguile/async.h, libguile/deprecated.h, libguile/eval.h, libguile/gc.h, libguile/gc.h, libguile/macros.h, libguile/ports.h, libguile/srfi-4.h, libguile/strings.h: Change declarations of deprecated functions and variables to use `SCM_DEPRECATED' instead of `SCM_API'.
2009-09-09Merge branch 'master' into boehm-demers-weiser-gcLudovic Courtès1-0/+4
Conflicts: libguile/gc_os_dep.c
2009-09-09Make scm_i_from_stringn into API for use with libguilereadlineMichael Gran1-2/+2
* libguile/strings.c (scm_i_from_stringn): renamed to scm_from_stringn. All callers changed. * libguile/strings.h: change declaration of scm_i_from_stringn to scm_from_stringn * libguile/strports.c (scm_strport_to_string): scm_i_from_stringn -> scm_from_stringn * guile-readline/readline.c (internal_readline): scm_i_from_stringn -> scm_from_stringn
2009-09-07Fix broken interaction between readline and UnicodeMichael Gran1-2/+2
This requires separate small fixes. Readline has internal logic to deal with multi-byte characters, so it wants bytes, not characters. scm_c_read gets called by the vm when readline is activated, and it was truncating multi-byte characters because soft ports didn't have the UCS-4 capability. Soft ports need the capability to read UCS-4 characters. Since soft ports may have a single byte buffer, full characters need to be stored into the pushback buffer. This broke the optimizations in scm_c_read for using an alternate buffer for single-byte-buffered ports, because the opimization wasn't expecting anything in the pushback buffer. * libguile/vports.c (sf_fill_input): store complete chars, not single bytes * libguile/ports.c (scm_c_read): don't use optimized path for non Latin-1. Add debug prints. * libguile/string.h: make scm_i_from_stringn and scm_i_string_ref public so that readline can use them * guile-readline/readline.c: read bytes, not complete chars, from the input port. Convert output to the output port's locale
2009-09-02Merge branch 'boehm-demers-weiser-gc' into bdw-gc-static-allocLudovic Courtès1-29/+73
Conflicts: acinclude.m4 libguile/strings.c
2009-09-01Fix leaky handling of `scm_take_locale_{symbol,string} ()'.Ludovic Courtès1-4/+0
* libguile/strings.c (scm_i_take_stringbufn, scm_i_c_take_symbol): Remove. (scm_take_locale_stringn): Rewrite in terms of `scm_from_locale_stringn ()'. * libguile/strings.h (scm_i_c_take_symbol, scm_i_take_stringbufn): Remove declarations.
2009-08-30Fix encoding errors with strings returned by string portsMichael Gran1-0/+4
String ports, being 8-bit, store strings using the character encoding of the port. This fixes a bug where the default character encoding, and not the port's encoding, was being used to convert the string port data back to a string. * libguile/strports.c: extra comments (scm_strport_to_string): use port's encoding when converting port data to a string * libguile/strings.c (scm_i_from_stringn): renamed from scm_from_stringn and made internal. All callers changed. (scm_from_stringn): renamed to scm_i_from_stringn. * libguile/strings.h: declaration for scm_i_from_stringn
2009-08-28Merge branch 'master' into boehm-demers-weiser-gcLudovic Courtès1-1/+6
Conflicts: libguile/Makefile.am libguile/bytevectors.c libguile/gc-card.c libguile/gc-mark.c libguile/programs.c libguile/srcprop.c libguile/srfi-14.c libguile/symbols.c libguile/threads.c libguile/unif.c libguile/vm.c
2009-08-25Add full Unicode capability to ports and the default readerMichael Gran1-0/+1
Ports are given two additional properties: a character encoding and a conversion failure strategy. These properties have getters and setters. The new properties are used to convert any locale text to/from the internal representation of strings. If unspecified, ports use a default value. The default value of these properties is held in a fluid. The default character encoding can be modified by calling setlocale. ISO-8859-1 is treated specially. Since it is a native encoding of strings, it can be processed more quickly. Source code is assumed to be ISO-8859-1 unless otherwise specified. The encoding of a source code file can be given as 'coding: XXXXX' in a magic comment at the top of a file. The C functions that deal with encoding often use a null pointer as shorthand for the native Latin-1 encoding, for efficiency's sake. * test-suite/tests/encoding-iso88591.test: new tests * test-suite/tests/encoding-iso88597.test: new tests * test-suite/tests/encoding-utf8.test: new tests * test-suite/tests/encoding-escapes.test: new tests * test-suite/tests/numbers.test: declare 'binary' encoding * test-suite/tests/ports.test: declare 'binary' encoding * test-suite/tests/r6rs-ports.test: declare 'binary' encoding * module/system/base/compile.scm (compile-file): use source-code file's self-declared encoding when compiling files * libguile/strports.c: store string ports in locale encoding (scm_strport_to_locale_u8vector, scm_call_with_output_locale_u8vector) (scm_open_input_locale_u8vector, scm_get_output_locale_u8vector): new functions * libguile/strings.h: new declaration for scm_i_string_contains_char * libguile/strings.c (scm_i_string_contains_char): new function (scm_from_stringn, scm_to_stringn): use NULL for Latin-1 (scm_from_locale_stringn, scm_to_locale_stringn): respect character encoding of input and output ports * libguile/read.h: declaration for scm_scan_for_encoding * libguile/read.c: (read_token): now takes scheme string instead of C string/length (read_complete_token): new function (scm_read_sexp, scm_read_number, scm_read_mixed_case_symbol) (scm_read_number_and_radix, scm_read_quote, scm_read_semicolon_comment) (scm_read_srfi4_vector, scm_read_bytevector, scm_read_guile_bit_vector) (scm_read_scsh_block_comment, scm_read_commented_expression) (scm_read_extended_symbol, scm_read_sharp_extension, scm_read_shart) (scm_read_expression): use scm_t_wchar for char type, use read_complete_token (scm_scan_for_encoding): new function to find a file's character encoding (scm_file_encoding): new function to find a port's character encoding * libguile/rdelim.c: don't unpack strings * libguile/print.h: declaration for modified function scm_i_charprint * libguile/print.c: use locale when printing characters and strings (scm_i_charprint): input parameter is now scm_t_wchar (scm_simple_format): don't unpack strings * libguile/posix.h: new declaration for scm_setbinary. * libguile/posix.c (scm_setlocale): set default and stdio port encodings based on the locale's character encoding (scm_setbinary): new function * libguile/ports.h (scm_t_port): add encoding and failed conversion handler to port type. Declarations for new or modified functions scm_getc, scm_unget_byte, scm_ungetc, scm_i_get_port_encoding, scm_i_set_port_encoding_x, scm_port_encoding, scm_set_port_encoding_x, scm_i_get_conversion_strategy, scm_i_set_conversion_strategy_x, scm_port_conversion_strategy, scm_set_port_conversion_strategy_x. * libguile/ports.c: assign the current ports to zero on startup so we can see if they've been set. (scm_current_input_port, scm_current_output_port, scm_current_error_port): return #f if the port is not yet initialized (scm_new_port_table_entry): set up a new port's encoding and illegal sequence handler based on the thread's current defaults (scm_i_remove_port): free port encoding name when port is removed (scm_i_mode_bits_n): now takes a scheme string instead of a c string and length. All callers changed. (SCM_MBCHAR_BUF_SIZE): new const (scm_getc): new function, since the scm_getc in inline.h is now scm_get_byte_or_eof. This pulls one codepoint from a port. (scm_lfwrite_substr, scm_lfwrite_str): now uses port's encoding (scm_unget_byte): new function, incorportaing the low-level functionality of scm_ungetc (scm_ungetc): uses scm_unget_byte * libguile/numbers.h (scm_t_wchar): compilation order problem with scm_t_wchar being use in functions in multiple headers. Forward declare scm_t_wchar. * libguile/load.c (scm_primitive_load): scan for file encoding at top of file and use it to set the load port's encoding * libguile/inline.h (scm_get_byte_or_eof): new function incorporating most of the functionality of scm_getc. * libguile/fports.c (fport_fill_input): now returns scm_t_wchar * libguile/chars.h (scm_t_wchar): avoid compilation order problem with declaration of scm_t_wchar
2009-08-23Modify socket and time functions for wide stringsMichael Gran1-0/+3
* libguile/socket.c (scm_recv): receive the message without holding the stringbuf writing lock (scm_send): try to narrow a string before using it * libguile/stime.c (strftime): convert string to UTF-8 so that it can be safely passed to strftime (strptime): convert input string to UTF-8 so that it can be safely passed through strptime * libguile/strings.c (narrow_stringbuf): new function (scm_i_try_narrow_string): new function * libguile/strings.h: new declaration for scm_i_try_narrow_string
2009-08-21Use string accessors for string->number conversionMichael Gran1-0/+1
* libguile/numbers.c (scm_i_print_fraction): use string accessors (XDIGIT2UINT): use libunistring function (mem2uinteger, mem2integer, mem2decimal_from_point, mem2ureal) (mem2complex): take scheme string instead of c string; use accessors (scm_i_string_to_number): new function (scm_c_locale_string_to_number): use scm_i_string_to_number * libguile/numbers.h: declaration for scm_i_string_to_number * libguile/strings.c (scm_i_string_strcmp): new function * libguile/strings.h: declaration for scm_i_string_strcmp
2009-08-19Rename string-width to string-bytes-per-charMichael Gran1-1/+1
* libguile/strings.h: rename scm_string_width to scm_string_bytes_per_char * libguile/strings.c (scm_string_width): renamed to scm_string_bytes_per_char (scm_string_bytes_per_char): renamed from scm_string_width * module/language/assembly/compile-bytecode.scm (write-bytecode): string-width -> string-bytes-per-char * module/language/glil/compile-assembly.scm (dump-object): string-width -> string-bytes-per-char
2009-08-18Merge branch 'master' into boehm-demers-weiser-gcLudovic Courtès1-24/+67
Conflicts: lib/Makefile.am libguile/Makefile.am libguile/frames.c libguile/gc-card.c libguile/gc-freelist.c libguile/gc-mark.c libguile/gc-segment.c libguile/gc_os_dep.c libguile/load.c libguile/macros.c libguile/objcodes.c libguile/programs.c libguile/strings.c libguile/vm.c m4/gnulib-cache.m4 m4/gnulib-comp.m4 m4/inline.m4
2009-08-12Don't include libunistring headers in Guile public headersMichael Gran1-3/+12
This requres the creation of a new type scm_t_string_failed_conversion_handler to replace libunistring's enum iconveh_ilseq_handler. * libguile/strings.h: don't include <uniconv.h> (scm_t_string_failed_conversion_handler): new enum type (SCM_FAILED_CONVERSION_ERROR, SCM_FAILED_CONVERSION_QUESTION_MARK): (SCM_FAILED_CONVERSION_ESCAPE_SEQUENCE): new enum type values * libguile/strings.c (scm_to_stringn): now takes type scm_t_string_failed_conversion_handler. All callers changed. * libguile/print.c: include <uniconv.h> * libguile/ports.c (scm_lfwrite_substr): use scm_t_string_conversion_handler's constants * libguile/gen-scmconfig.c (SCM_ICONVEH_ERROR): (SCM_ICONVEH_QUESTION_MARK, SCM_ICONVEH_ESCAPE_SEQUENCE): store iconveh_ilseq_hander constants as #define's
2009-08-10Make scm_charprint and scm_i_string_wide_chars SCM_INTERNAL.Michael Gran1-1/+1
Also, scm_charprint is renamed to scm_i_charprint. * libguile/strings.h: make scm_i_string_wide_chars internal. * libguile/print.h: rename scm_charprint to scm_i_charprint. Make internal. * libguile/print.c (scm_i_charprint): renamed from scm_charprint (scm_charprint): renamed to scm_i_charprint. All callers changed.
2009-08-10Improve %string-dump and %symbol-dumpMichael Gran1-0/+8
%string-dump and %symbol-dump are modified to return assocation lists of string and symbol attributes instead of printing to stderr. They are no longer conditional on SCM_DEBUG. * libguile/strings.c (scm_sys_string_dump) (scm_sys_symbol_dump): now returns alist of properties. No longer require that SCM_DEBUG be defined. (scm_sys_stringbuf_hist): now conditional on SCM_STRING_LENGTH_HISTOGRAM * libguile/strings.h: scm_sys_string_dump and scm_sys_symbol dump are now declared as API
2009-08-08Add Unicode strings and symbolsMichael Gran1-17/+42
This adds full Unicode strings as a datatype, and it adds some minimal functionality. The terminal and port encoding is assumed to be ISO-8859-1. Non-ISO-8859-1 characters are written or input as string character escapes. The string character escapes now have 3 forms: \xXX \uXXXX and \UXXXXXX, for unprintable characters that have 2, 4 or 6 hex digits. The process for writing to strings has been modified. There is now a function scm_i_string_start_writing that does the copy-on-write conversion if necessary. To compile strings that may be wide, the VM storage of strings and string-likes has changed. Most string-using functions have not yet been updated and may break when used with wide strings. * module/language/assembly/compile-bytecode.scm (write-bytecode): use variable width string bytecode format * module/language/assembly.scm (byte-length): use variable width bytecode format * libguile/vm-i-loader.c (load-string, load-symbol): (load-keyword, define): use variable-width bytecode format * libguile/vm-engine.h (FETCH_WIDTH): new macro * libguile/strings.h: new declarations * libguile/strings.c (make_wide_stringbuf): new function (widen_stringbuf): new function (scm_i_make_wide_string): new function (scm_i_is_narrow_string): new function (scm_i_string_wide_chars): new function (scm_i_string_start_writing): new function (scm_i_string_ref): new function (scm_i_string_set_x): new function (scm_i_is_narrow_symbol): new function (scm_i_symbol_wide_chars, scm_i_symbol_ref): new function (scm_string_width): new function (unistring_escapes_to_guile_escapes): new function (scm_to_stringn): new function (scm_i_stringbuf_free): modify for wide strings (scm_i_substring_copy): modify for wide strings (scm_i_string_chars, scm_string_append): modify for wide strings (scm_i_make_symbol, scm_to_locale_stringn): modify for wide strings (scm_string_dump, scm_symbol_dump, scm_to_locale_stringbuf): (scm_string, scm_i_deprecated_string_chars): modify for wide strings (scm_from_locale_string, scm_from_locale_stringn): add null test * libguile/srfi-13.c: add calls for scm_i_string_start_writing for each call of scm_i_string_stop_writing (scm_string_for_each): modify for wide strings * libguile/socket.c: add calls for scm_i_string_start_writing for each call of scm_i_string_stop_writing * libguile/rw.c: add calls for scm_i_string_start_writing for each call of scm_i_string_stop_writing * libguile/read.c (scm_read_string): allow reading of wide strings * libguile/print.h: add declaration for scm_charprint * libguile/print.c (iprin1): print wide strings and add new string escapes (scm_charprint): new function * libguile/ports.h: new declarations for scm_lfwrite_substr and scm_lfwrite_str * libguile/ports.c (update_port_lf): new function (scm_lfwrite): use update_port_lf (scm_lfwrite_substr): new function (scm_lfwrite_str): new function * test-suite/tests/asm-to-bytecode.test ("compiler"): add string width byte to sting-like asm tests
2009-06-17Change Guile license to LGPLv3+Neil Jerram1-6/+7
(Not quite finished, the following will be done tomorrow. module/srfi/*.scm module/rnrs/*.scm module/scripts/*.scm testsuite/*.scm guile-readline/* )
2009-01-19Merge branch 'boehm-demers-weiser-gc' into bdw-gc-static-allocLudovic Courtès1-1/+0
2009-01-18Use `scm_gc malloc_pointerless ()' in `scm_i allocate_string_pointers ()'.Ludovic Courtès1-1/+0
* libguile/dynl.c (free_string_pointers): Remove. (scm_dynamic_args_call): Remove reference to `free_string_pointers ()' and remove dynwind. * libguile/posix.c (free_string_pointers): Remove. (scm_execl, scm_execlp, scm_execle, scm_environ): Remove references to `free_string_pointers ()'. * libguile/simpos.c (free_string_pointers): Remove. (scm_system_star): Remove reference to `free_string_pointers ()', remove enclosing dynwind. * libguile/strings.c (scm_i_allocate_string_pointers): Use `scm_gc_malloc_pointerless ()' and `scm_gc_malloc ()' instead of `scm_malloc ()' and `scm_to_locale_string ()', so that the result is automatically GC'd when no longer referenced. Remove unneeded dynwind. (scm_i_free_string_pointers): Remove. * libguile/strings.h (scm_i_free_string_pointers): Remove declaration.
2009-01-13Expose some of the string/stringbuf internal flags and tags.Ludovic Courtès1-1/+12
* libguile/strings.h (scm_tc7_ro_string, SCM_I_STRINGBUF_F_SHARED, SCM_I_STRINGBUF_F_INLINE): New macros. * libguile/strings.c (STRINGBUF_F_SHARED): Alias for `SCM_I_STRINGBUF_F_SHARED'. (STRINGBUF_F_INLINE): Alias for `SCM_I_STRINGBUF_F_INLINE'. (RO_STRING_TAG): Alias for `scm_tc7_ro_string'.
2008-10-11Merge branch 'master' into boehm-demers-weiser-gcLudovic Courtès1-1/+0
Conflicts: libguile/Makefile.am libguile/threads.c
2008-10-09Revert "Make literal strings (i.e., returned by `read') read-only."Ludovic Courtès1-1/+0
This reverts commit fb2f8886c4d537b0c7d3e9e78a8d4e5e272a36f4. The rationale is that `read' must return mutable strings, as reported by szgyg <szgyg@ludens.elte.hu>.
2008-09-23Merge branch 'master' into boehm-demers-weiser-gcLudovic Courtès1-0/+1
2008-09-23Make literal strings (i.e., returned by `read') read-only.Ludovic Courtès1-0/+1
* libguile/read.c (scm_read_string): Use `scm_i_make_read_only_string ()' to return a read-only string, as mandated by R5RS. Reported by Bill Schottstaedt <bil@ccrma.Stanford.EDU>. * libguile/strings.c (scm_i_make_read_only_string): New function. (scm_i_shared_substring_read_only): Special-case the empty string so that the read-only and read-write empty strings are `eq?'. This optimization is relied on by the `substring/shared' `empty string' test case in `srfi-13.test'. * libguile/strings.h (scm_i_make_read_only_string): New declaration. * test-suite/tests/strings.test ("string-set!")["literal string"]: New test. * NEWS: Update.
2008-09-15Remove unused GC string/symbol functions.Ludovic Courtès1-9/+0
* libguile/strings.c (scm_i_stringbuf_mark, scm_i_stringbuf_free, scm_i_string_mark, scm_i_string_free, scm_i_symbol_mark, scm_i_symbol_free): Remove. * libguile/strings.h: Remove corresponding declarations.
2008-07-05Add `scm_c_symbol_length ()'.Ludovic Courtès1-0/+1
2008-05-31Add `SCM_INTERNAL' macro, use it.Ludovic Courtès1-30/+30
2006-04-17merge from 1.8 branchKevin Ryde1-1/+1
2006-01-24patches by Ludovic Courtès for symbol generation.Han-Wen Nienhuys1-0/+7