Image buffer protocol Isis software documentation

Image buffer protocol

This document explains the various routines for creating, accessing, and manipulating visual images in Isis. In this protocol, images are always stored as raw data. The individual samples of an image may be any C data type, but certain operations may only work on certain types of images. For example, most of the image compositing routines in the image processing library will only work on byte-sampled images.

Images must have exactly 2 dimensions, and they may have one or more channels, such as red, green, blue, and alpha. Images in this protocol also have horizontal and vertical step factors that indicate the the offset of the next horizontal and vertical picture element from the current one, thereby allowing one to subsample or isolate parts of images for an operation without incurring any extra overhead.

An image may be either planar or interleaved when it is first allocated. In planar images, all of the samples in each channel are stored consecutively, and the channels themselves are consecutive in memory as well (RRR...GGG...BBB...). In interleaved images, the samples from the individual channels are interleaved with each other in memory (RGBRGBRGB...). You may explicitly allocate either type of image, but in most cases there is no reason to choose one or the other format. However, certain libraries will function more efficiently when you use certain formats. For example, OpenGL works better with interleaved images, whereas the DAT library requires the use of planar images. See the documentation of each library for more details.

Isis lists are used to store information about images. You can access the fields of these lists to obtain information about an image, including the addresses of its memory buffers. Normally, you will not need to look inside these lists at all.

Creating and destroying image buffers

(new-standard-image chans dims) # allocate a planar byte image (new-image c-type chans dims) # allocate a planar image (new-image c-type chans dims steps addrs) # refer to an image in memory (new-planar-image c-type chans dims) # allocate a planar image (new-interleaved-image c-type chans dims) # allocate an interleaved image (free-image image) # free the memory used by an image

new-standard-image allocates a planar byte-sampled image buffer suitable for visual data. chans should be an integer number of channels, and dims should be a list of 2 integers [xsize ysize] indicating the size of the image. All dimensions and step factors should be specified this way.

new-image allows you to create images of any sample type (not just bytes). There are two ways to use it. The first way (3 arguments) allocates a new planar image buffer with the specified sample type, channels, and dimensions. The sample type should be one of the C type constants from the memory manipulation library.

The second way of using new-image (5 arguments) allows you to refer directly to an image that already exists in memory. steps should be a list of 2 integers [xstep ystep] indicating the offset in samples (not bytes) of the next horizontal and vertical pixels from the current pixel. addrs should be list of addresses where the image data for each channel starts. The number of items in this list should match the number of channels.

new-planar-image and new-interleaved-image allow you to explicitly allocate planar or interleaved images. See the previous section for a discussion on the difference between planar and interleaved images. Note that new-planar-image works the same as the 3-argument form of new-image.

All of the image creation routines return an image description list that should be used in subsequent operations on the image.

free-image should be called on any image you do not intend to use any more. If the image memory was allocated directly by one of the functions above, it will be released. On the other hand, if the image refers to memory that was previously allocated or if it is actually a sub-image or sub-sampled version of another image, the memory will not be freed.

Examples

The following creates a new planar byte-sampled image with 3 channels and a size of 320 by 240 pixels. This image would be suitable for storing RGB visual image data.

The following creates a new planar one-channel, integer-sampled, 512 by 512 pixel image. It might be suitable for use as a depth buffer, or as an accumulation buffer for summing several byte-sampled images.

In the example below, assume that addr is an address that points to the start of a memory buffer containing a byte-sampled, 640 by 480 pixel image that is stored in an interleaved fashion (RGBRGBRGB rather than RRRGGGBBB). The following creates an Isis image that refers to that memory. Recall that the addresses must point to the first element in each channel, hence the need to add the offsets.

The following frees all of the images created above. Recall that image2 refers to previously allocated memory, so calling free-image on it will have no effect.

Obtaining information about images

(image buf-type) # sample type (image buf-alloc) # allocation status (image buf-chans) # number of channels (image buf-dims) # dimensions (image buf-steps) # step factors (image buf-addrs) # channel addresses As mentioned before, information about images is stored in lists with a particular format. You can access the elements of this list using the index constants shown above to get the information you need about an image. The items in the list are mostly self-explanatory. The allocation status is a boolean value indicating whether the memory buffers associated with the image should be freed when free-image is called on the image. If the image refers to previously allocated memory, or if it is a sub-image or sub-sampled version of another image, this field will be set to False.

Example

The following shows how to extract some information about image1 from the previous example.

Accessing individual pixels

(get-pixel pos image) # get the value of a pixel (set-pixel val pos image) # set the value of a pixel These functions retrieve and set the value of an individual pixel in an image. pos should be a list of two integers [xpos ypos]. get-pixel converts the value of the pixel to the most appropriate Isis type, and conversly, set-pixel converts the value to the sample type of the image. Pixel values are returned in lists with one element per channel, and they should be passed to set-pixel in the same format. These functions should not be used to process every pixel in an image. Specialized C code should be written to perform full-size image processing.

Isolating and combining parts of images

(isolate-channel chan image) # isolate one channel (isolate-channels firstchan lastchan image) # isolate range of channels (isolate-sub-image rectpos dims image) # isolate a section of an image (combine-channels image image ...) # combine channels in images (refer-image image) # isolate "entire" image These procedures allow you to isolate certain parts of an image for an operation. None of these procedures actually allocates any new memory or performs any transfer of data. Instead, references to the appropriate parts of data in the original image buffers are created. Therefore, these procedures may be used liberally, without fear of extensive system slow-down.

isolate-channel will isolate a particular channel from an image, and isolate-channels will isolate a range of channels, between lowchan and highchan inclusive.

Use isolate-sub-image to isolate a particular rectangle inside of an image. You must supply the location of the upper left corner of the rectangle, and the dimensions of the rectangle (both as lists of 2 integers of course).

To combine the channels of several images into one image, call combine-channels with all of the images in the desired channel order. The images must have the same type, dimensions, and step factors.

To simply create a new reference to an entire image as it stands, use refer-image.

The images returned from all of these procedures will always have an allocation status of False so that a subsequent call to free-image will not deallocate their memory buffers.

Examples

The following creates a 3-channel image and initializes its channels to values of 230, 128, and 42 respectively (red, green, and blue). The initialization is accomplished by using the image-fill-constant primitive from the image processing library. The image is then displayed in an X window.

The following isolates a sub-image of the first and second channels (red and green) and fills that sub-image with a value of 255, thereby creating a yellow rectangle.

Intersecting images

(intersect-images offset image1 image2) (intersect-rectangles offset rect1size rect2size) intersect-images intersects two image buffers that are offset by a certain amount that you specify. The procedure returns two sub-image buffers corresponding to where the two original buffers overlap. This routine is very important because most of the image processing procedures described later will only work on buffers that are the same size. Therefore, intersect-images must be called (if necessary) in order to isolate identically-sized sub-images of the buffers involved in the operation you want to perform.

intersect-images expects 3 arguments: the position of origin of first buffer with respect to the origin the second, and then the first and second image buffers, respectively. A list of 2 corresponding sub-image buffers is returned. You can then apply the image processing operation on these sub-images.

intersect-rectangles does exactly the same thing with rectangles instead of actual images. You pass the relative position of R1 with respect to R2, and the sizes of the two rectangles being intersected. The return is a list of 3 items: the origin for the sub-rectangle in R1, the origin for the sub-rectangle in R2, and the size of the sub-rectangle. This procedure is called by intersect-images but can also be used independently.

Example

The following creates an x-window, retrieves an image from a URL using retreive-url-image from the Internet utilities library, creates a second large image and makes it all red, then transfers the first image into the second at position [100 100], and finally displays the image. Try it now:

Simple remappings

(horizontal-mirror-image image) (vertical-mirror-image image) (transpose-image image) (subsample-image subfactor image) Again, these functions do not allocate any memory or perform any data transfers. transpose-image effectively swaps the x and y axes of an image. The subfactor passed to subsample-image can be just an integer, or a list of two integers. For example, (subsample-image 2 image) makes the image half of its original size. (subsample-image [3 5] image) scales the image by one-third in the horizontal direction and one-fifth in the vertical direction.

Reading and writing images

(write-image file image) (read-image file image) These functions read and write image buffers to the specified file handle, which may be a real file, a tcp connection, a pipe, etc. Images are written to the file in separate channels (not interleaved), i.e. all of the first channel is written, then all of the second channel, and so on. When reading, the image being received from the file must have the same number of channels and dimensions as the image buffer it is being read into, otherwise strange behavior may result. Therefore, when handling images with varying sizes, information about image dimensions should be stored or transmitted separately so that properly sized image buffers can be created prior to calling read-image.

For efficiency, if the step factor of the image buffer is not [ 1 xsize ], the image is first transferred to a temporary buffer so that only one read or write system call is necessary. This strategy speeds up transfer times greatly, especially when using an optimized version of Isis.

Miscellaneous utilities

(bounding-rectangle point1 point2 point3 ...) (rectangle-size rect) Rectangles in the image buffer package should all be specified as [x1 y1 x2 y2], where x1 < x2 and

y1 <
y2

. bounding-rectangle returns the coordinates of a rectangle that bounds all of the points passed as arguments. rectangle-size returns the size of a rectangle. (image-buf-error opname image image ...) (image-type-error opname image image ...) These procedures print error messages. Each expects a string operation name and any number of images as arguments. image-buf-error prints a message stating that the image buffers are incompatible for the operation. image-type-error prints a message stating that the operation is unsupported for the given image types. These functions are used in several places internally, especially in the image processing interface routines.

Requirements:

Scripts: (load "image-buffer.isis")