summaryrefslogtreecommitdiff
path: root/doc/ref/libguile-foreign-objects.texi
diff options
context:
space:
mode:
Diffstat (limited to 'doc/ref/libguile-foreign-objects.texi')
-rw-r--r--doc/ref/libguile-foreign-objects.texi491
1 files changed, 491 insertions, 0 deletions
diff --git a/doc/ref/libguile-foreign-objects.texi b/doc/ref/libguile-foreign-objects.texi
new file mode 100644
index 000000000..f668eedca
--- /dev/null
+++ b/doc/ref/libguile-foreign-objects.texi
@@ -0,0 +1,491 @@
+@c -*-texinfo-*-
+@c This is part of the GNU Guile Reference Manual.
+@c Copyright (C) 1996, 1997, 2000, 2001, 2002, 2003, 2004, 2005, 2010, 2011, 2013, 2014, 2018
+@c Free Software Foundation, Inc.
+@c See the file guile.texi for copying conditions.
+
+@node Defining New Foreign Object Types
+@section Defining New Foreign Object Types
+
+The @dfn{foreign object type} facility is Guile's mechanism for
+importing object and types from C or other languages into Guile's
+system. If you have a C @code{struct foo} type, for example, you can
+define a corresponding Guile foreign object type that allows Scheme code
+to handle @code{struct foo *} objects.
+
+To define a new foreign object type, the programmer provides Guile with
+some essential information about the type --- what its name is, how many
+fields it has, and its finalizer (if any) --- and Guile allocates a
+fresh type for it. Foreign objects can be accessed from Scheme or from
+C.
+
+@menu
+* Defining Foreign Object Types::
+* Creating Foreign Objects::
+* Type Checking of Foreign Objects::
+* Foreign Object Memory Management::
+* Foreign Objects and Scheme::
+@end menu
+
+@node Defining Foreign Object Types
+@subsection Defining Foreign Object Types
+
+To create a new foreign object type from C, call
+@code{scm_make_foreign_object_type}. It returns a value of type
+@code{SCM} which identifies the new type.
+
+Here is how one might declare a new type representing eight-bit
+gray-scale images:
+
+@example
+#include <libguile.h>
+
+struct image @{
+ int width, height;
+ char *pixels;
+
+ /* The name of this image */
+ SCM name;
+
+ /* A function to call when this image is
+ modified, e.g., to update the screen,
+ or SCM_BOOL_F if no action necessary */
+ SCM update_func;
+@};
+
+static SCM image_type;
+
+void
+init_image_type (void)
+@{
+ SCM name, slots;
+ scm_t_struct_finalize finalizer;
+
+ name = scm_from_utf8_symbol ("image");
+ slots = scm_list_1 (scm_from_utf8_symbol ("data"));
+ finalizer = NULL;
+
+ image_type =
+ scm_make_foreign_object_type (name, slots, finalizer);
+@}
+@end example
+
+The result is an initialized @code{image_type} value that identifies the
+new foreign object type. The next section describes how to create
+foreign objects and how to access their slots.
+
+
+@node Creating Foreign Objects
+@subsection Creating Foreign Objects
+
+Foreign objects contain zero or more ``slots'' of data. A slot can hold
+a pointer, an integer that fits into a @code{size_t} or @code{ssize_t},
+or a @code{SCM} value.
+
+All objects of a given foreign type have the same number of slots. In
+the example from the previous section, the @code{image} type has one
+slot, because the slots list passed to
+@code{scm_make_foreign_object_type} is of length one. (The actual names
+given to slots are unimportant for most users of the C interface, but
+can be used on the Scheme side to introspect on the foreign object.)
+
+To construct a foreign object and initialize its first slot, call
+@code{scm_make_foreign_object_1 (@var{type}, @var{first_slot_value})}.
+There are similarly named constructors for initializing 0, 1, 2, or 3
+slots, or initializing @var{n} slots via an array. @xref{Foreign
+Objects}, for full details. Any fields that are not explicitly
+initialized are set to 0.
+
+To get or set the value of a slot by index, you can use the
+@code{scm_foreign_object_ref} and @code{scm_foreign_object_set_x}
+functions. These functions take and return values as @code{void *}
+pointers; there are corresponding convenience procedures like
+@code{_signed_ref}, @code{_unsigned_set_x} and so on for dealing with
+slots as signed or unsigned integers.
+
+Foreign objects fields that are pointers can be tricky to manage. If
+possible, it is best that all memory that is referenced by a foreign
+object be managed by the garbage collector. That way, the GC can
+automatically ensure that memory is accessible when it is needed, and
+freed when it becomes inaccessible. If this is not the case for your
+program -- for example, if you are exposing an object to Scheme that was
+allocated by some other, Guile-unaware part of your program -- then you
+will probably need to implement a finalizer. @xref{Foreign Object
+Memory Management}, for more.
+
+Continuing the example from the previous section, if the global variable
+@code{image_type} contains the type returned by
+@code{scm_make_foreign_object_type}, here is how we could construct a
+foreign object whose ``data'' field contains a pointer to a freshly
+allocated @code{struct image}:
+
+@example
+SCM
+make_image (SCM name, SCM s_width, SCM s_height)
+@{
+ struct image *image;
+ int width = scm_to_int (s_width);
+ int height = scm_to_int (s_height);
+
+ /* Allocate the `struct image'. Because we
+ use scm_gc_malloc, this memory block will
+ be automatically reclaimed when it becomes
+ inaccessible, and its members will be traced
+ by the garbage collector. */
+ image = (struct image *)
+ scm_gc_malloc (sizeof (struct image), "image");
+
+ image->width = width;
+ image->height = height;
+
+ /* Allocating the pixels with
+ scm_gc_malloc_pointerless means that the
+ pixels data is collectable by GC, but
+ that GC shouldn't spend time tracing its
+ contents for nested pointers because there
+ aren't any. */
+ image->pixels =
+ scm_gc_malloc_pointerless (width * height, "image pixels");
+
+ image->name = name;
+ image->update_func = SCM_BOOL_F;
+
+ /* Now wrap the struct image* in a new foreign
+ object, and return that object. */
+ return scm_make_foreign_object_1 (image_type, image);
+@}
+@end example
+
+We use @code{scm_gc_malloc_pointerless} for the pixel buffer to tell the
+garbage collector not to scan it for pointers. Calls to
+@code{scm_gc_malloc}, @code{scm_make_foreign_object_1}, and
+@code{scm_gc_malloc_pointerless} raise an exception in out-of-memory
+conditions; the garbage collector is able to reclaim previously
+allocated memory if that happens.
+
+
+@node Type Checking of Foreign Objects
+@subsection Type Checking of Foreign Objects
+
+Functions that operate on foreign objects should check that the passed
+@code{SCM} value indeed is of the correct type before accessing its
+data. They can do this with @code{scm_assert_foreign_object_type}.
+
+For example, here is a simple function that operates on an image object,
+and checks the type of its argument.
+
+@example
+SCM
+clear_image (SCM image_obj)
+@{
+ int area;
+ struct image *image;
+
+ scm_assert_foreign_object_type (image_type, image_obj);
+
+ image = scm_foreign_object_ref (image_obj, 0);
+ area = image->width * image->height;
+ memset (image->pixels, 0, area);
+
+ /* Invoke the image's update function. */
+ if (scm_is_true (image->update_func))
+ scm_call_0 (image->update_func);
+
+ return SCM_UNSPECIFIED;
+@}
+@end example
+
+
+@node Foreign Object Memory Management
+@subsection Foreign Object Memory Management
+
+Once a foreign object has been released to the tender mercies of the
+Scheme system, it must be prepared to survive garbage collection. In
+the example above, all the memory associated with the foreign object is
+managed by the garbage collector because we used the @code{scm_gc_}
+allocation functions. Thus, no special care must be taken: the garbage
+collector automatically scans them and reclaims any unused memory.
+
+However, when data associated with a foreign object is managed in some
+other way---e.g., @code{malloc}'d memory or file descriptors---it is
+possible to specify a @dfn{finalizer} function to release those
+resources when the foreign object is reclaimed.
+
+As discussed in @pxref{Garbage Collection}, Guile's garbage collector
+will reclaim inaccessible memory as needed. This reclamation process
+runs concurrently with the main program. When Guile analyzes the heap
+and determines that an object's memory can be reclaimed, that memory is
+put on a ``free list'' of objects that can be reclaimed. Usually that's
+the end of it---the object is available for immediate re-use. However
+some objects can have ``finalizers'' associated with them---functions
+that are called on reclaimable objects to effect any external cleanup
+actions.
+
+Finalizers are tricky business and it is best to avoid them. They can
+be invoked at unexpected times, or not at all---for example, they are
+not invoked on process exit. They don't help the garbage collector do
+its job; in fact, they are a hindrance. Furthermore, they perturb the
+garbage collector's internal accounting. The GC decides to scan the
+heap when it thinks that it is necessary, after some amount of
+allocation. Finalizable objects almost always represent an amount of
+allocation that is invisible to the garbage collector. The effect can
+be that the actual resource usage of a system with finalizable objects
+is higher than what the GC thinks it should be.
+
+All those caveats aside, some foreign object types will need finalizers.
+For example, if we had a foreign object type that wrapped file
+descriptors---and we aren't suggesting this, as Guile already has ports
+---then you might define the type like this:
+
+@example
+static SCM file_type;
+
+static void
+finalize_file (SCM file)
+@{
+ int fd = scm_foreign_object_signed_ref (file, 0);
+ if (fd >= 0)
+ @{
+ scm_foreign_object_signed_set_x (file, 0, -1);
+ close (fd);
+ @}
+@}
+
+static void
+init_file_type (void)
+@{
+ SCM name, slots;
+ scm_t_struct_finalize finalizer;
+
+ name = scm_from_utf8_symbol ("file");
+ slots = scm_list_1 (scm_from_utf8_symbol ("fd"));
+ finalizer = finalize_file;
+
+ image_type =
+ scm_make_foreign_object_type (name, slots, finalizer);
+@}
+
+static SCM
+make_file (int fd)
+@{
+ return scm_make_foreign_object_1 (file_type, (void *) fd);
+@}
+@end example
+
+@cindex finalizer
+@cindex finalization
+
+Note that the finalizer may be invoked in ways and at times you might
+not expect. In a Guile built without threading support, finalizers are
+invoked via ``asyncs'', which interleaves them with running Scheme code;
+@pxref{Asyncs}. If the user's Guile is built with support for threads,
+the finalizer will probably be called by a dedicated finalization
+thread, unless the user invokes @code{scm_run_finalizers ()} explicitly.
+
+In either case, finalizers run concurrently with the main program, and
+so they need to be async-safe and thread-safe. If for some reason this
+is impossible, perhaps because you are embedding Guile in some
+application that is not itself thread-safe, you have a few options. One
+is to use guardians instead of finalizers, and arrange to pump the
+guardians for finalizable objects. @xref{Guardians}, for more
+information. The other option is to disable automatic finalization
+entirely, and arrange to call @code{scm_run_finalizers ()} at
+appropriate points. @xref{Foreign Objects}, for more on these
+interfaces.
+
+Finalizers are allowed to allocate memory, access GC-managed memory, and
+in general can do anything any Guile user code can do. This was not the
+case in Guile 1.8, where finalizers were much more restricted. In
+particular, in Guile 2.0, finalizers can resuscitate objects. We do not
+recommend that users avail themselves of this possibility, however, as a
+resuscitated object can re-expose other finalizable objects that have
+been already finalized back to Scheme. These objects will not be
+finalized again, but they could cause use-after-free problems to code
+that handles objects of that particular foreign object type. To guard
+against this possibility, robust finalization routines should clear
+state from the foreign object, as in the above @code{free_file} example.
+
+One final caveat. Foreign object finalizers are associated with the
+lifetime of a foreign object, not of its fields. If you access a field
+of a finalizable foreign object, and do not arrange to keep a reference
+on the foreign object itself, it could be that the outer foreign object
+gets finalized while you are working with its field.
+
+For example, consider a procedure to read some data from a file, from
+our example above.
+
+@example
+SCM
+read_bytes (SCM file, SCM n)
+@{
+ int fd;
+ SCM buf;
+ size_t len, pos;
+
+ scm_assert_foreign_object_type (file_type, file);
+
+ fd = scm_foreign_object_signed_ref (file, 0);
+ if (fd < 0)
+ scm_wrong_type_arg_msg ("read-bytes", SCM_ARG1,
+ file, "open file");
+
+ len = scm_to_size_t (n);
+ SCM buf = scm_c_make_bytevector (scm_to_size_t (n));
+
+ pos = 0;
+ while (pos < len)
+ @{
+ char *bytes = SCM_BYTEVECTOR_CONTENTS (buf);
+ ssize_t count = read (fd, bytes + pos, len - pos);
+ if (count < 0)
+ scm_syserror ("read-bytes");
+ if (count == 0)
+ break;
+ pos += count;
+ @}
+
+ scm_remember_upto_here_1 (file);
+
+ return scm_values (scm_list_2 (buf, scm_from_size_t (pos)));
+@}
+@end example
+
+After the prelude, only the @code{fd} value is used and the C compiler
+has no reason to keep the @code{file} object around. If
+@code{scm_c_make_bytevector} results in a garbage collection,
+@code{file} might not be on the stack or anywhere else and could be
+finalized, leaving @code{read} to read a closed (or, in a multi-threaded
+program, possibly re-used) file descriptor. The use of
+@code{scm_remember_upto_here_1} prevents this, by creating a reference
+to @code{file} after all data accesses. @xref{Garbage Collection
+Functions}.
+
+@code{scm_remember_upto_here_1} is only needed on finalizable objects,
+because garbage collection of other values is invisible to the program
+-- it happens when needed, and is not observable. But if you can, save
+yourself the headache and build your program in such a way that it
+doesn't need finalization.
+
+
+@node Foreign Objects and Scheme
+@subsection Foreign Objects and Scheme
+
+It is also possible to create foreign objects and object types from
+Scheme, and to access fields of foreign objects from Scheme. For
+example, the file example from the last section could be equivalently
+expressed as:
+
+@example
+(define-module (my-file)
+ #:use-module (system foreign-object)
+ #:use-module ((oop goops) #:select (make))
+ #:export (make-file))
+
+(define (finalize-file file)
+ (let ((fd (struct-ref file 0)))
+ (unless (< fd 0)
+ (struct-set! file 0 -1)
+ (close-fdes fd))))
+
+(define <file>
+ (make-foreign-object-type '<file> '(fd)
+ #:finalizer finalize-file))
+
+(define (make-file fd)
+ (make <file> #:fd fd))
+@end example
+
+Here we see that the result of @code{make-foreign-object-type}, which is
+the equivalent of @code{scm_make_foreign_object_type}, is a struct
+vtable. @xref{Vtables}, for more information. To instantiate the
+foreign object, which is really a Guile struct, we use @code{make}. (We
+could have used @code{make-struct/no-tail}, but as an implementation
+detail, finalizers are attached in the @code{initialize} method called
+by @code{make}). To access the fields, we use @code{struct-ref} and
+@code{struct-set!}. @xref{Structure Basics}.
+
+There is a convenience syntax, @code{define-foreign-object-type}, that
+defines a type along with a constructor, and getters for the fields. An
+appropriate invocation of @code{define-foreign-object-type} for the
+file object type could look like this:
+
+@example
+(use-modules (system foreign-object))
+
+(define-foreign-object-type <file>
+ make-file
+ (fd)
+ #:finalizer finalize-file)
+@end example
+
+This defines the @code{<file>} type with one field, a @code{make-file}
+constructor, and a getter for the @code{fd} field, bound to @code{fd}.
+
+Foreign object types are not only vtables but are actually GOOPS
+classes, as hinted at above. @xref{GOOPS}, for more on Guile's
+object-oriented programming system. Thus one can define print and
+equality methods using GOOPS:
+
+@example
+(use-modules (oop goops))
+
+(define-method (write (file <file>) port)
+ ;; Assuming existence of the `fd' getter
+ (format port "#<<file> ~a>" (fd file)))
+
+(define-method (equal? (a <file>) (b <file>))
+ (eqv? (fd a) (fd b)))
+@end example
+
+One can even sub-class foreign types.
+
+@example
+(define-class <named-file> (<file>)
+ (name #:init-keyword #:name #:init-value #f #:accessor name))
+@end example
+
+The question arises of how to construct these values, given that
+@code{make-file} returns a plain old @code{<file>} object. It turns out
+that you can use the GOOPS construction interface, where every field of
+the foreign object has an associated initialization keyword argument.
+
+@example
+(define* (my-open-file name #:optional (flags O_RDONLY))
+ (make <named-file> #:fd (open-fdes name flags) #:name name))
+
+(define-method (write (file <named-file>) port)
+ (format port "#<<file> ~s ~a>" (name file) (fd file)))
+@end example
+
+@xref{Foreign Objects}, for full documentation on the Scheme interface
+to foreign objects. @xref{GOOPS}, for more on GOOPS.
+
+As a final note, you might wonder how this system supports encapsulation
+of sensitive values. First, we have to recognize that some facilities
+are essentially unsafe and have global scope. For example, in C, the
+integrity and confidentiality of a part of a program is at the mercy of
+every other part of that program -- because any part of the program can
+read and write anything in its address space. At the same time,
+principled access to structured data is organized in C on lexical
+boundaries; if you don't expose accessors for your object, you trust
+other parts of the program not to work around that barrier.
+
+The situation is not dissimilar in Scheme. Although Scheme's unsafe
+constructs are fewer in number than in C, they do exist. The
+@code{(system foreign)} module can be used to violate confidentiality
+and integrity, and shouldn't be exposed to untrusted code. Although
+@code{struct-ref} and @code{struct-set!} are less unsafe, they still
+have a cross-cutting capability of drilling through abstractions.
+Performing a @code{struct-set!} on a foreign object slot could cause
+unsafe foreign code to crash. Ultimately, structures in Scheme are
+capabilities for abstraction, and not abstractions themselves.
+
+That leaves us with the lexical capabilities, like constructors and
+accessors. Here is where encapsulation lies: the practical degree to
+which the innards of your foreign objects are exposed is the degree to
+which their accessors are lexically available in user code. If you want
+to allow users to reference fields of your foreign object, provide them
+with a getter. Otherwise you should assume that the only access to your
+object may come from your code, which has the relevant authority, or via
+code with access to cross-cutting @code{struct-ref} and such, which also
+has the cross-cutting authority.