@need 3200

@node Arrays
@chapter Arrays

Dynamically allocated arrays are a common feature of C programs.
The Hackerlab C library provides two array-like data structures:
variable size arrays, and power-of-two size sparse arrays.


Variable size arrays are contiguous regions of memory, similar
to memory allocated by malloc.  The size of a variable size array,
measured as the number of elements it contains, can be retrieved
at run-time using the function @code{ar_size}.


Power-of-two size sparse arrays are a tree structure, used to 
represent an array holding a number of elements which is a power
of two.  Access to these arrays is very fast, and they are memory
efficient for very large arrays which are sparsely populated.

@menu
* Variable Size Arrays::
* Power-of-Two Sparse Arrays::
@end menu

@need 3200

@node Variable Size Arrays
@section Variable Size Arrays

@geindex array

@geindex variable size array

@geindex power of two size array

@geindex aparse array



A @geindex variable size array
@dfn{variable size array} is a dynamically allocated block of memory,
similar to a block returned by @code{lim_malloc}, except that a variable sized
array is tagged with its size, measured in the number of array elements.


A null pointer counts as an array of @code{0} elements.  For example, if
@code{ar_size}, which returns the size of a variable sized arrary, is
passed @code{0}, it returns @code{0}.  That means there is no special function
to allocate a new variable sized array -- instead, array pointers
should be initialized to @code{0}.  This example creates an array with ten
integers by using @code{ar_ref}:

@example
@group
     @{
       int * the_array;
       int * tenth_element;
@end group
@group
@end group
@end example
@example
@group
       the_array = 0;
       tenth_element = (int *)ar_ref (&the_array,
                                      lim_use_must_malloc,
                                      9,
                                      sizeof (int));
     @}
@end group
@group
@end group
@end example

A variable size array can be used as a stack. (See @code{ar_push} and
@code{ar_pop}.)


Array functions use the @code{lim_malloc} family of functions to allcoate
memory.  (See @ref{Allocation With Limitations}.)

@menu
* Basic Variable Size Array Functions::
* Variable Sized Arrays as Stacks::
@end menu

@need 3200

@node Basic Variable Size Array Functions
@subsection Basic Variable Size Array Functions
@sp 1
@need 1750
@noindent
@b{* Function} @code{ar_size}@fuindex ar_size
@*
@example
@group
int ar_size (void * base,
           alloc_limits limits,
           size_t elt_size);
@end group
@group
@end group
@end example

Return the number of elements in the array.  If @code{base == 0}, return
@code{0}.


@code{limits} is the allocation limits associated with this array.

@sp 1
@need 1750
@noindent
@b{* Function} @code{ar_ref}@fuindex ar_ref
@*
@example
@group
void * ar_ref (void ** base,
               alloc_limits limits,
               int n,
               int szof);
@end group
@group
@end group
@end example

Return the address of element @code{n} of an array, expanding the array
to @code{n+1} elements, if necessary.


@code{base} is a pointer to a pointer to the array.


@code{limits} is the allocation limits associated with this array.


@code{szof} is the size, in bytes, of one element of the array.


If this function adds new elements to an array, those elements are
filled with @code{0} bytes.


This function may resize and relocate the array.  If it does,
@code{*base} is updated to point to the new location of the array.

@sp 1
@need 1750
@noindent
@b{* Function} @code{ar_setsize}@fuindex ar_setsize
@*
@example
@group
void ar_setsize (void ** base,
               alloc_limits limits,
               int n,
               size_t szof);
@end group
@group
@end group
@end example

Resize the array so that it contains exactly @code{n} elements.


@code{base} is a pointer to a pointer to the array.


@code{limits} is the allocation limits associated with this array.


@code{szof} is the size, in bytes, of one element.


If this function adds new elements to an array, those elements are
filled with @code{0} bytes.


This function can be used to make an array smaller, but doing so
does not reclaim any storage. (See @code{ar_compact}.)

@sp 1
@need 1750
@noindent
@b{* Function} @code{ar_compact}@fuindex ar_compact
@*
@example
@group
void ar_compact (void ** base,
               alloc_limits limits,
               size_t szof);
@end group
@group
@end group
@end example

Resize an array so that it is only as large as it needs to be.


@code{base} is a pointer to a pointer to the array.


@code{limits} is the allocation limits associated with this array.


@code{szof} is the size, in bytes, of one element.


This function may resize and relocate the array.  If it does,
@code{*base} is updated to point to the new location of the array.


Functions like @code{ar_setsize} can be used to make an array smaller, but
doing so does not reclaim any storage used by the array and does
not move the array in memory.


This function does attempt to reclaim storage (by using
@code{lim_realloc}).  If the array occupies significantly more memory
than needed, this function will move it to a smaller block.
If @code{lim_realloc} returns @code{0}, this function has no effect.

@sp 1
@need 1750
@noindent
@b{* Function} @code{ar_free}@fuindex ar_free
@*
@example
@group
void ar_free (void ** base, alloc_limits limits);
@end group
@group
@end group
@end example

Release storage associated with the array pointed to by @code{*base}.
Set @code{*base} to @code{0}.


@code{limits} is the allocation limits associated with this array.




@need 3200

@node Variable Sized Arrays as Stacks
@subsection Variable Sized Arrays as Stacks
@sp 1
@need 1750
@noindent
@b{* Function} @code{ar_push}@fuindex ar_push
@*
@example
@group
void * ar_push (void ** base,
              alloc_limits limits,
              size_t szof);
@end group
@group
@end group
@end example

Return the address of element @code{n} in an array previously containing
only @code{n-1} elements.


@code{base} is a pointer to a pointer to the array.


@code{limits} is the allocation limits associated with this array.


@code{szof} is the size, in bytes, of one element.


The new array element is filled with @code{0} bytes.


This function may resize and relocate the array.  If it does,
@code{*base} is updated to point to the new location of the array.

@sp 1
@need 1750
@noindent
@b{* Function} @code{ar_pop}@fuindex ar_pop
@*
@example
@group
void * ar_pop (void ** base,
             alloc_limits limits,
             size_t szof);
@end group
@group
@end group
@end example

Return the address of the @code{n}th element in an array previously
containing @code{n} elements.  Resize the array so that it contains
exactly @code{n-1} elements.


@code{base} is a pointer to a pointer to the array.


@code{limits} is the allocation limits associated with this array.


@code{szof} is the size, in bytes, of one element.


This function may resize and relocate the array.  If it does,
@code{*base} is updated to point to the new location of the array.

@sp 1
@need 1750
@noindent
@b{* Function} @code{ar_copy}@fuindex ar_copy
@*
@example
@group
void * ar_copy (void * base,
              alloc_limits limits,
              size_t szof);
@end group
@group
@end group
@end example

Create a new array which is a copy of the array pointed to by
@code{base}.


@code{limits} is the allocation limits associated with this array.







@need 3200

@node Power-of-Two Sparse Arrays
@section Power-of-Two Sparse Arrays
@noindent
@strong{#include <hackerlab/arrays/pow2-array.h>}@*
@sp 1

@geindex power of two size array

@geindex sparse array



A @code{pow2_array} (@geindex power-of-two sparse array
@dfn{power-of-two sparse array}) is an array-like
data structure.  It always holds a number of elements which is
some power of two.  It provides reasonably fast access to elements
(but slower than ordinary arrays).  It provides good memory efficiency
for sparsely populated arrays.


@strong{NOTE:} this interface is net yet complete.  Some details of the
existing interface may change in future releases.

@menu
* The pow2_array Data Structure::
* Allocating Power-of-Two Sparse Arrays::
* Accessing Elements in Sparse Arrays::
@end menu

@need 3200

@node The pow2_array Data Structure
@subsection The pow2_array Data Structure

A @code{pow2_array} is represented by a tree structure of uniform depth.
Leaf elements are ordinary (dynamically allocated) arrays, each
leaf having the same number of elements.


So that sparsely populated arrays can be stored efficiently in
memory, subtrees which are populated entirely with default values
are represented in one of two ways: the root of such subtrees may
be represented as a @code{NULL} pointer; or the root of such as subtree
may be represented by a @geindex default node
@dfn{default node}.  In the latter case, one
default node exists for each level of the tree (a default root, a
default second-level node, a default leaf node, and so on).
Representation by null pointer saves memory by not allocating
default nodes.  Representation by default nodes speeds up access,
in some cases.


For each level of the tree, two values are defined: a @code{shift} and a
@code{mask}.  For a given internal node of the tree, the subtree
containing the @code{N}th element below that node is stored in the
subtree:

@example
@group
             (N >> shift) & mask
@end group
@group
@end group
@end example

The index of the same element within that subtree is:

@example
@group
             N & ((1 << shift) - 1)
@end group
@group
@end group
@end example

For leaf nodes, @code{shift} is @code{0}.


The opaque type @code{pow2_array_rules} holds the set of @code{shift} and @code{mask}
values which define a tree structure for arrays of some size.
The opaque type @code{pow2_array} holds a particular array.


Here is a simple example: a sparse array containing @code{8} elements.
(In ordinary use, we would presumably choose a much larger power
of two.)


We will define oen possible tree structures for this array: a two
level tree with four elements in each leaf.  Other structures are
possible: we might have defined a two-level structure with two
elements in each leaf or a three-level structure with @code{2} in each
leaf, and two sub-trees below each internal node.


For the two level tree with four elements per leaf node, we have:

@example
@group
     root:                   shift == 2
                             mask == 1
@end group
@group
@end group
@end example
@example
@group
     leaf nodes:             shift == 0
                             mask == 3
@end group
@group
@end group
@end example

The default leaf node, at address @code{Ld} is a four element array:

@example
@group
     Ld:
     ---------------------------- 
     | dflt | dflt | dflt | dflt| 
     ---------------------------- 
@end group
@group
@end group
@end example

The default root node, at address @code{Rd} is a two element array:

@example
@group
     Rd:
     -----------
     | Ld | Ld |
     -----------
@end group
@group
@end group
@end example

An array with a non-default value (@code{V}) in element @code{2}, but default
values everywhere else might look like:

@example
@group
                     root:
                     ----------------
                     |  leaf  |  Ld |
                     --/----------|--
                      /           |
@end group
@group
                     /            |
@end group
@group
          /---------/             |
     leaf:                       Ld:
     --------------------------  -----------------------------
     | dflt | V | dflt | dflt |  | dflt | dflt | dflt | dflt |
     --------------------------  -----------------------------
@end group
@group
@end group
@end example

Suppose that elements in this array are of type @code{T}.  Then, using
the @code{shift} and @code{mask} values given above, the address of element
@code{N} in that tree is:

@example
@group
     (T *)((char *)root[(N >> 2) & 1] + ((N & ((1 << 2) - 1)) & 3))
@end group
@group
@end group
@end example

That is the address returned by the function @code{pow2_array_rref}.
Note that this address might be in @code{leaf}, or it might be in the
default leaf @code{Ld}.


When modifying a particular element, it is important to not modify
the default leaf.  A copy-on-write strategy is used.  For example,
before modifying element @code{7}, the tree is rewritten:

@example
@group
                     root:
                     --------------------
                     |  leaf0  |  Leaf1 |
                     --/------------/----
                      /            /
@end group
@group
                     /            /
@end group
@group
          /---------/            /
    leaf0:                      leaf1:
    --------------------------  -----------------------------
    | dflt | V | dflt | dflt |  | dflt | dflt | dflt | dflt |
    --------------------------  -----------------------------
@end group
@group
@end group
@end example

The function @code{pow2_array_ref} performs that copy-on-write operation
and then returns an element address similarly to @code{pow2_array_rref}.


If the default value for elements is a region of memory filled with
@code{0} bytes, a tree can be represented without using default nodes.
For example, the array containing an element only in element @code{2}
would be represented:

@example
@group
                     root:
                     ----------------
                     |  leaf  |  0  |
                     --/-------------
                      /           
@end group
@group
                     /            
@end group
@group
          /---------/             
     leaf:                       
     -----------------  
     | 0 | V | 0 | 0 |  
     -----------------  
@end group
@group
@end group
@end example

A tree of this variety is created by not specifying a default leaf
node when calling @code{make_pow2_array_rules}.  @code{pow2_array_ref} returns
a NULL pointer if an element is accessed which is not currently in 
such a tree.


The function @code{pow2_array_compact} compresses the representation of a
sparse array by eliminating identical subtrees.  For example, after
calling @code{pow2_array_compact} on an array with default values everywhere
except elements @code{1} and @code{5}, the tree would look like:

@example
@group
                     root:
                     -------------------
                     |  leaf  |  Leaf |
                     -------\-------/---
                             \     /
@end group
@group
                              \   /
@end group
@group
                               \ /
                                leaf:
                                --------------------------
                                | dflt | V | dflt | dflt |
                                --------------------------
@end group
@group
@end group
@end example

After calling @code{pow2_array_compact}, it is no longer safe to call
@code{pow2_array_ref} for the same array.  @code{pow2_array_ref} is safe.
@code{pow2_array_compact} is useful in combination with
@code{pow2_array_print}.




@need 3200

@node Allocating Power-of-Two Sparse Arrays
@subsection Allocating Power-of-Two Sparse Arrays
@sp 1
@need 1750
@noindent
@b{* Function} @code{make_pow2_array_rules}@fuindex make_pow2_array_rules
@*
@example
@group
pow2_array_rules make_pow2_array_rules (alloc_limits limits,
                                        size_t elt_size,
                                        void * default_page,
                                        int shift,
                                        size_t mask,
                                        ...);
@end group
@group
@end group
@end example

Return the @code{pow2_array_rules} which defines the tree structure
for a particular type of sparse array.


@code{limits} is used when allocating the @code{pow2_array_rules} and default
nodes.  See @ref{Allocation With Limitations}.


@code{elt_size} is the size, in bytes, of individual elements.


@code{default_page} is either @code{0} or a default leaf node. 


The remaining arguments are a series of @code{shift} and @code{mask} pairs,
ending with a pair in which @code{shift} is @code{0}.


See @ref{The pow2_array Data Structure} for more information about
default leaf nodes, shifts, and masks.


If allocation fails, this funtion returns @code{0}.

@sp 1
@need 1750
@noindent
@b{* Function} @code{pow2_array_alloc}@fuindex pow2_array_alloc
@*
@example
@group
pow2_array pow2_array_alloc (alloc_limits limits,
                             pow2_array_rules rules);
@end group
@group
@end group
@end example

Allocate a sparse array.


@code{limits} is used when allocating the array.  It is also used by
@code{pow2_array_ref} when allocating nodes within the array.
See @ref{Allocation With Limitations}.


@code{rules} defines the tree structure for the array and should be
an object returned by @code{make_pow2_array_rules}.


If allocation fails, this funtion returns @code{0}.




@need 3200

@node Accessing Elements in Sparse Arrays
@subsection Accessing Elements in Sparse Arrays
@sp 1
@need 1750
@noindent
@b{* Function} @code{pow2_array_rref}@fuindex pow2_array_rref
@*
@example
@group
void * pow2_array_rref (pow2_array array, size_t addr);
@end group
@group
@end group
@end example

Return the address if the @code{addr} element within @code{array}.


The value pointed to by this address should not be modified.


If the element has never been modified, and no default leaf node
was passed to @code{make_pow2_array_rules}, this function returns @code{0}.

@sp 1
@need 1750
@noindent
@b{* Function} @code{pow2_array_ref}@fuindex pow2_array_ref
@*
@example
@group
void * pow2_array_ref (pow2_array array, size_t addr);
@end group
@group
@end group
@end example

Return the address if the @code{addr} element within @code{array}.


The value pointed to by this address may be modified.


This function might allocate memory if the element has not
previously been modified.  If allocation fails, this function
returns @code{0}.










