diff options
author | Julian Graham <julian.graham@aya.yale.edu> | 2010-01-03 01:06:05 -0500 |
---|---|---|
committer | Julian Graham <julian.graham@aya.yale.edu> | 2010-01-03 01:08:37 -0500 |
commit | edb7bb4766773cffa8262b4cd8bb980888913d65 (patch) | |
tree | f3eef8fb0e0b5941bd4630688cd4d7798cfb2abc /doc/ref/api-data.texi | |
parent | 441891f376221d7bcd1f6fc8927595fe25417255 (diff) | |
download | guile-edb7bb4766773cffa8262b4cd8bb980888913d65.tar.gz |
Support for Unicode string normalization functions
* libguile/strings.c, libguile/strings.h (normalize_str,
scm_string_normalize_nfc, scm_string_normalize_nfd, scm_normalize_nfkc,
scm_string_normalize_nfkd): New functions.
* test-suite/tests/strings.test: Unit tests for `string-normalize-nfc',
`string-normalize-nfd', `string-normalize-nfkc', and
`string-normalize-nfkd'.
* doc/ref/api-data.texi (String Comparison): Documentation for normalization
functions.
Diffstat (limited to 'doc/ref/api-data.texi')
-rwxr-xr-x | doc/ref/api-data.texi | 64 |
1 files changed, 64 insertions, 0 deletions
diff --git a/doc/ref/api-data.texi b/doc/ref/api-data.texi index e847c9caa..8e797acc3 100755 --- a/doc/ref/api-data.texi +++ b/doc/ref/api-data.texi @@ -3273,6 +3273,70 @@ Compute a hash value for @var{S}. the optional argument @var{bound} is a non-ne Compute a hash value for @var{S}. the optional argument @var{bound} is a non-negative exact integer specifying the range of the hash function. A positive value restricts the return value to the range [0,bound). @end deffn +Because the same visual appearance of an abstract Unicode character can +be obtained via multiple sequences of Unicode characters, even the +case-insensitive string comparison functions described above may return +@code{#f} when presented with strings containing different +representations of the same character. For example, the Unicode +character ``LATIN SMALL LETTER S WITH DOT BELOW AND DOT ABOVE'' can be +represented with a single character (U+1E69) or by the character ``LATIN +SMALL LETTER S'' (U+0073) followed by the combining marks ``COMBINING +DOT BELOW'' (U+0323) and ``COMBINING DOT ABOVE'' (U+0307). + +For this reason, it is often desirable to ensure that the strings +to be compared are using a mutually consistent representation for every +character. The Unicode standard defines two methods of normalizing the +contents of strings: Decomposition, which breaks composite characters +into a set of constituent characters with an ordering defined by the +Unicode Standard; and composition, which performs the converse. + +There are two decomposition operations. ``Canonical decomposition'' +produces character sequences that share the same visual appearance as +the original characters, while ``compatiblity decomposition'' produces +ones whose visual appearances may differ from the originals but which +represent the same abstract character. + +These operations are encapsulated in the following set of normalization +forms: + +@table @dfn +@item NFD +Characters are decomposed to their canonical forms. + +@item NFKD +Characters are decomposed to their compatibility forms. + +@item NFC +Characters are decomposed to their canonical forms, then composed. + +@item NFKC +Characters are decomposed to their compatibility forms, then composed. + +@end table + +The functions below put their arguments into one of the forms described +above. + +@deffn {Scheme Procedure} string-normalize-nfd s +@deffnx {C Function} scm_string_normalize_nfd (s) +Return the @code{NFD} normalized form of @var{s}. +@end deffn + +@deffn {Scheme Procedure} string-normalize-nfkd s +@deffnx {C Function} scm_string_normalize_nfkd (s) +Return the @code{NFKD} normalized form of @var{s}. +@end deffn + +@deffn {Scheme Procedure} string-normalize-nfc s +@deffnx {C Function} scm_string_normalize_nfc (s) +Return the @code{NFC} normalized form of @var{s}. +@end deffn + +@deffn {Scheme Procedure} string-normalize-nfkc s +@deffnx {C Function} scm_string_normalize_nfkc (s) +Return the @code{NFKC} normalized form of @var{s}. +@end deffn + @node String Searching @subsubsection String Searching |