<?xml version="1.0" encoding="iso-8859-1"?>
<!DOCTYPE book PUBLIC "-//Torsten Bronger//DTD tbook 1.5.2//EN"
                      "/usr/local/share/xml/tbook/tbook.dtd">
<book>
  <frontmatter>
    <title><visual markup="tt">PyTables</visual> User's Guide</title>
    <author>Francesc Altet</author>
    <author>Ivan Vilata</author>
    <author>Scott Prater</author>
    <author>Vicent Mas</author>
    <author>Tom Hedley</author>
    <author>Antonio Valentino</author>
    <author>Jeffrey Whitaker</author>

    <subtitle>Hierarchical datasets in Python - Release 1.3.3</subtitle>

    <date>$LastChangedDate: 2006-03-30 14:05:30 +0200 (Thu, 30 Mar 2006) $</date>
    <year>2002, 2003, 2004, 2005, 2006</year>
    <graphics kind="bitmap" file="logo3-ombra"/>
<!--     <graphics kind="bitmap" file="logo4-ombra"/> -->
<!--     <graphics kind="vector" scale="0.3" file="enjoylogo-1"/> -->
<!--     <graphics kind="bitmap" file="pytables-logo-menut"/> -->
<!--     <city>Castelló de la Plana, Spain</city> -->
    <typeset>Francesc Altet, Scott Prater, Ivan Vilata, Vicent Mas, Tom Hedley, Antonio Valentino and Jeffrey Whitaker</typeset>

    <legalnotice>

<!--	   Copyright (c) 2002, 2003, 2004, 2005 Cárabos Coop. V. -->

      <p><visual markup="bf">Copyright Notice and Statement for
	  <verb>PyTables</verb> Software Library and
	  Utilities</visual> <newline vspace="0.25cm"/>
      </p>

      <p>Redistribution and use in source and binary forms, with or
	without modification, are permitted provided that the
	following conditions are met:

	<newline vspace="0.1cm"/>
      </p>

      <p>1. Redistributions of source code must retain the above copyright
	notice, this list of conditions and the following disclaimer.
      </p>
      <p>2. Redistributions in binary form must reproduce the above
	copyright notice, this list of conditions and the following
	disclaimer in the documentation and/or other materials
	provided with the distribution.

	<newline vspace="0.25cm"/>
      </p>

      <p>THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY
	EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
	THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
	PARTICULAR PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL THE
	AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
	SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
	NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
	LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
	HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
	CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR
	OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE,
	EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

	<newline vspace="1cm"/>
      </p>

<!-- NCSA license -->

      <p><visual markup="bf">Copyright Notice and Statement for NCSA
	  Hierarchical Data Format (HDF) Software Library and
	  Utilities</visual>
	<newline vspace="0.1cm"/>
      </p>


      <p>NCSA HDF5 (Hierarchical Data Format 5) Software Library and
	Utilities Copyright 1998, 1999, 2000, 2001, 2002, 2003, 2004,
	2005 by the Board of Trustees of the University of Illinois.
	All rights reserved.  <newline vspace="0.25cm"/>
      </p>

      <p>See more information about the terms of this license at:
	<newline/> <url
	name="http://hdf.ncsa.uiuc.edu/HDF5/doc/Copyright.html"><verb>http://hdf.ncsa.uiuc.edu/HDF5/doc/Copyright.html</verb></url>
	<newline vspace="1cm"/>
      </p>

      <!-- lrucache license -->

      <p><visual markup="bf">Copyright Notice and Statement
	  for the lrucache.py module
	</visual>
	<newline vspace="0.1cm"/>
      </p>

      <p>Copyright 2004 Evan Prodromou.
	Licensed under the Academic Free License 2.1.
      </p>

      <p>See more information about the terms of this license at:
	<newline/> <url
	name="http://opensource.org/licenses/afl-2.1.php"><verb>http://opensource.org/licenses/afl-2.1.php</verb></url>
	<newline vspace="1cm"/>
      </p>

      <!-- numarray license -->

      <!-- This is not needed anymore, as PyTables does not include a
      private copy of the numarray.records module -->

<!--       <p><visual markup="bf">Copyright Notice and Statement for AURA -->
<!-- 	  <em>numarray</em> software library</visual> -->
<!-- 	<newline vspace="0.25cm"/> -->
<!--       </p> -->


<!--       <p>Copyright (C) 2001 Association of Universities for Research -->
<!-- 	in Astronomy (AURA) <newline vspace="0.25cm"/> -->
<!--       </p> -->


<!--       <p> -->
<!-- 	THIS SOFTWARE IS PROVIDED BY AURA ``AS IS'' AND ANY EXPRESS OR -->
<!-- 	IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED -->
<!-- 	WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR -->
<!-- 	PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL AURA BE LIABLE FOR -->
<!-- 	ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR -->
<!-- 	CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, -->
<!-- 	PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, -->
<!-- 	DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND -->
<!-- 	ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT -->
<!-- 	LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING -->
<!-- 	IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF -->
<!-- 	THE POSSIBILITY OF SUCH DAMAGE. -->
<!--       </p> -->

    </legalnotice>
  </frontmatter>

  <!--  This is only for the article
  <abstract>

  <p><visual markup="tt">PyTables</visual> is a Python package whose
    goal is to provide a powerful and easy-to-use interface for
    managing data tables and NumPy/Numeric/numarray Python objects
    organized in a hierarchical structure. Such a tables are defined
    as a collection of records whose values are stored in fixed-length
    fields. The excellent HDF5 library (http://hdf.ncsa.uiuc.edu/HDF5)
    is the foundation for the underlying hierarchical data
    organization.
  </p>

  <p><visual markup="tt">PyTables</visual> is designed to be an
    easy-to-use as well as high-performance interface to HDF5.  The
    latest improvements introduced in Python 2.2 (such as generators,
    iterators and metaclasses in new-brand classes) have been
    used. The HDF5 library is accessed through the Pyrex creation
    extension tool.
  </p>

  </abstract> -->

  <mainmatter>

<!--    <chapter kind='preface'> -->
<!--       <heading></heading> -->

    <part>
      <heading>The <visual markup="tt">PyTables</visual> Core
	Library</heading>

    <chapter>
      <heading>Introduction</heading>

      <!-- SGP:  I translated this aphorism, but not any of the
      others.  I'd translate all the aphorisms, as it's a little
      pedantic to assume your readers can read French, Spanish,
      Valencian, Catalan, Portuguese, etc. -->

      <aphorism>La sabiduría no vale la pena si no es posible servirse
	de ella para inventar una nueva manera de preparar los
	garbanzos.<newline/>(Wisdom isn't worth anything if you can't
	use it to come up with a new way to cook garbanzos).
	<caption>A wise Catalan<newline/>in "Cien años de
	soledad"<newline/> Gabriel García Márquez</caption>
      </aphorism>

      <p>The goal of <verb>PyTables</verb> is to enable the end user
	to manipulate easily data <visual markup="bf">tables</visual>
	and <visual markup="bf">array</visual> objects in a
	hierarchical structure. The foundation of the underlying
	hierarchical data organization is the excellent
	<verb>HDF5</verb> library (see <cite
	refid="HDFWhatIs"></cite>).<!-- Right now,
	<verb>PyTables</verb> --> <!-- provides limited support for
	importing generic HDF5 files --> <!-- (i.e., those which were
	created with other tools than --> <!-- <verb>PyTables</verb>),
	but I hope to add the more interesting --> <!-- tools as time
	goes on. Still, I think <verb>PyTables</verb> can --> <!--
	read a wide range of such files. -->
      </p>
      <p>
	It should be noted that this package is not intended to
	serve as a complete wrapper for the entire HDF5 API, but only to
	provide a flexible, <em>very pythonic</em> tool to deal with
	(arbitrarily) large amounts of data (typically bigger than
	available memory) in tables and arrays organized in a
	hierarchical and persistent disk storage structure.
      </p>

      <p>A table is defined as a collection of records whose values
	are stored in <em>fixed-length</em> fields. All records have
	the same structure and all values in each field have the same
	<em>data type</em>. The terms <em>fixed-length</em> and strict
	<em>data types</em> may seem to be a strange requirement for
	an interpreted language like Python, but they serve a useful
	function if the goal is to save very large quantities of data
	(such as is generated by many data acquisition systems,
	Internet services or scientific applications, for example) in
	an efficient manner that reduces demand on CPU time and I/O.
      </p>

      <p>In order to emulate in Python records mapped to HDF5 C
	structs <verb>PyTables</verb> implements a special
	class so as to easily define all its
	fields and other properties.  <verb>PyTables</verb> also
	provides a powerful interface to mine data in tables. Records
	in tables are also known in the <verb>HDF5</verb> naming
	scheme as <em>compound</em> data types.
      </p>

      <p>For example, you can define arbitrary tables in Python
	simply by declaring a class with name field and types
	information, such as in the following example:
      </p>

<verbatim>
class Particle(IsDescription):
    name      = StringCol(16)   # 16-character String
    idnumber  = Int64Col()      # Signed 64-bit integer
    ADCcount  = UInt16Col()     # Unsigned short integer
    TDCcount  = UInt8Col()      # unsigned byte
    grid_i    = Int32Col()      # integer
    grid_j    = IntCol()        # integer (equivalent to Int32Col)
    class Properties(IsDescription):  # A sub-structure (nested data-type)
        pressure = Float32Col(shape=(2,3)) # 2-D float array (single-precision)
        energy   = FloatCol(shape=(2,3,4)) # 3-D float array (double-precision)
</verbatim>

      <p>You then pass this class to the table constructor,
	fill its rows with your values, and save (arbitrarily large)
	collections of them to a file for persistent storage. After
	that, the data can be retrieved and post-processed quite
	easily with <visual markup="tt">PyTables</visual> or even with
	another <verb>HDF5</verb> application (in C, Fortran, Java or
	whatever language that provides a library to interface with HDF5).
      </p>

      <p>Other important entities in <visual
        markup="tt">PyTables</visual> are the <visual
        markup="bf">array</visual> objects that are analogous to
        tables with the difference that all of their components are
        homogeneous.  They come in different flavors, like
        <em>generic</em> (they provide a quick and fast way to deal
        with for numerical arrays), <em>enlargeable</em> (arrays can
        be extended in any single dimension) and <em>variable
        length</em> (each row in the array can have a different number
        of elements).
      </p>

      <p>The next section describes the most interesting capabilities of
	<verb>PyTables</verb>.
      </p>

      <section>
	<heading>Main Features</heading>
	<p>
	  <verb>PyTables</verb> takes advantage of the object
	  orientation and introspection capabilities offered by
	  <verb>Python</verb>, the <verb>HDF5</verb> powerful data
	  management features and <verb>numarray</verb> flexibility
	  and high-performance manipulation of large sets of objects
	  organized in grid-like fashion to provide these features:
	</p>

	<itemize>
	  <item><em>Support for table entities:</em> You can tailor
	    your data adding or deleting records in your tables. A
	    large number of rows (up to 2**62), i.e. much more than
	    will fit into memory is supported as well.
	  </item>
	  <item><em>Multidimensional and nested table cells:</em> You
	    can declare a column to consist of general array cells as
	    well as scalars, which is the only dimensionality allowed
	    the majority of relational databases. You can even declare
	    columns that are made of other columns (of different
	    types), which is known as struct types.
	  </item>
	  <item><em>Indexing support for columns of tables:</em> Very
	    useful if you have large tables and you want to quickly
	    look up for values in columns satisfying some criteria.
	  </item>
	  <item><em>Support for numerical arrays:</em>
	    <verb>NumPy</verb> (see <cite refid="NumPy"></cite>),
	    <verb>Numeric</verb> (see <cite refid="Numeric"></cite>)
	    and <verb>numarray</verb> (see <cite
	    refid="Numarray"></cite>) arrays can be used as a useful
	    complement of tables to store homogeneous data.
	  </item>
          <item><em>Enlargeable arrays:</em> You can add new elements
            to existing arrays on disk in any dimension you want (but
            only one). Besides, you can access to only a slice of your
            datasets by using the powerful extended slicing mechanism,
            without need to load all your complete dataset in-memory.
          </item>
          <item><em>Variable length arrays:</em> The number of
            elements in these arrays can be variable from row to
            row. This provides a lot of flexibility when dealing with
            complex data.
          </item>
	  <item><em>Supports a hierarchical data model:</em> Allows
	    the user to clearly structure all the data.
	    <verb>PyTables</verb> builds up an <em>object
	    tree</em> in memory that replicates the underlying file
	    data structure. Access to the file objects is achieved by
	    walking through and manipulating this object tree.
	  </item>
	  <item><em>User defined metadata:</em> Besides supporting
	    system metadata (like the number of rows of a table,
	    shape, flavor, etc.) the user may specify its own metadata
	    (as for example, room temperature, or protocol for IP
	    traffic that was collected) that complement the meaning of
	    his actual data.
	  </item>
	  <item><em>Ability to read/modify generic HDF5 files:</em>
	    <verb>PyTables</verb> can access a wide range of objects
	    in generic HDF5 files, like compound type datasets (that
	    can be mapped to <verb>Table</verb> objects), homogeneous
	    datasets (that can be mapped to <verb>Array</verb>
	    objects) or variable length record datasets (that can be
	    mapped to <verb>VLArray</verb> objects).  Besides, if a
	    dataset is not supported, it will be mapped into a special
	    <verb>UnImplemented</verb> class (see <ref
	    refid="UnImplementedClassDescr"></ref>), that will let the
	    user see that the data is there, although it would be
	    unreachable (still, you will be able to access the
	    attributes and some metadata in the dataset).  With that,
	    <verb>PyTables</verb> probably can access and
	    <em>modify</em> most of the HDF5 files out there.
	  </item>
	  <item><em>Data compression:</em> Supports data compression
	    (using the <visual markup="bf">Zlib</visual>, <visual
	    markup="bf">LZO</visual> and <visual
	    markup="bf">bzip2</visual> compression libraries) out of
	    the box. This is important when you have repetitive data
	    patterns and don't want to spend time searching for an
	    optimized way to store them (saving you time spent
	    analyzing your data organization).
	  </item>
	  <item><em>High performance I/O:</em> On modern systems
	    storing large amounts of data, tables and array objects can be
	    read and written at a speed only limited by the
	    performance of the underlying I/O subsystem. Moreover, if
	    your data is compressible, even that limit is surmountable!
	  </item>
	  <item><em>Support of files bigger than 2 GB:</em>
            <verb>PyTables</verb> automatically inherits
	    this capability from the underlying HDF5 library (assuming
	    your platform supports the C long long integer, or, on
	    Windows, __int64).
	  </item>
	  <item><em>Architecture-independent:</em> <visual
	    markup="tt">PyTables</visual> has been carefully coded (as
	    has HDF5 itself) with little-endian/big-endian byte
	    orderings issues in mind.  So, you can write a file on a
	    big-endian machine (like a Sparc or MIPS) and read it on
	    other little-endian machine (like an Intel or Alpha)
	    without problems. In addition, it has been tested
	    successfully with 64 bit platforms (Intel-64, AMD-64,
	    PowerPC-G5, MIPS, UltraSparc) using code generated with 64
	    bit aware compilers.
	  </item>

	</itemize>

      </section>

      <section id="ObjectTreeSection">
	<heading>The Object Tree</heading>

	<p>The hierarchical model of the underlying HDF5 library
	  allows <verb>PyTables</verb> to manage tables and arrays in
	  a tree-like structure. In order to achieve this, an
	  <em>object tree</em> entity is <em>dynamically</em> created
	  imitating the HDF5 structure on disk.  The HDF5 objects are
	  read by walking through this object tree.  You can get a
	  good picture of what kind of data is kept in the object by
	  examining the <em>metadata</em> nodes.
	</p>

	<p>The different nodes in the object tree are instances of
	  <verb>PyTables</verb> classes. There are several types of
	  classes, but the most important ones are the
	  <verb>Node</verb>, <verb>Group</verb> and <verb>Leaf</verb>
	  classes.  All nodes in a <verb>PyTables</verb> tree are
	  instances of the <verb>Node</verb> class.
	  <verb>Group</verb> and <verb>Leaf</verb> classes are
	  descendants of <verb>Node</verb>.  <verb>Group</verb>
	  instances (referred to as <em>groups</em> from now on) are a
	  grouping structure containing instances of zero or more
	  groups or leaves, together with supplementary metadata.
	  <verb>Leaf</verb> instances (referred to as <em>leaves</em>)
	  are containers for actual data and can not contain further
	  groups or leaves. The <verb>Table</verb>,
	  <verb>Array</verb>, <verb>CArray</verb>,
	  <verb>EArray</verb>, <verb>VLArray</verb> and
	  <verb>UnImplemented</verb> classes are descendants of
	  <verb>Leaf</verb>, and inherit all its properties.
	</p>

	<p>Working with groups and leaves is similar in many ways to
	  working with directories and files on a Unix filesystem,
	  i.e. a node (file or directory) is always a <em>child</em>
	  of one and only one group (directory), its <em>parent
	  group</em><footnote><verb>PyTables</verb> does not support
	  hard links &ndash; for the moment.</footnote>.  Inside of
	  that group, the node is accessed by its <em>name</em>.  As
	  is the case with Unix directories and files, objects in
	  the object tree are often referenced by giving their full
	  (absolute) path names. In <verb>PyTables</verb> this full
	  path can be specified either as string (such as
	  <verb>'/subgroup2/table3'</verb>, using <verb>/</verb>
	  as a parent/child separator) or as a complete object
	  path written in a format known as the <em>natural name</em>
	  schema (such as <verb>file.root.subgroup2.table3</verb>).
	</p>

	<p>Support for <em>natural naming</em> is a key aspect of
	  <verb>PyTables</verb>.  It means that the names of instance
	  variables of the node objects are the same as the names of
	  the element's children<footnote>I got this simple but
	  powerful idea from the excellent <visual
	  markup="tt">Objectify</visual> module by David Mertz (see
	  <cite refid="Objectify"></cite>)</footnote>. This is very
	  <em>Pythonic</em> and intuitive in many cases. Check the
	  tutorial <ref refid="readingAndSelectingUsage">section</ref>
	  for usage examples.
	</p>

	<p>You should also be aware that not all the data present in a
	  file is loaded into the object tree.  Only the
	  <em>metadata</em> (i.e. special data that describes the
	  structure of the actual data) is loaded. The actual data is
	  not read until you request it (by calling a method on a
	  particular node). Using the object tree (the metadata) you
	  can retrieve information about the objects on disk such as
	  table names, titles, name columns, data types in columns,
	  numbers of rows, or, in the case of arrays, the shapes,
	  typecodes, etc. of the array.  You can also search through
	  the tree for specific kinds of data then read it and process
	  it. In a certain sense, you can think of
	  <verb>PyTables</verb> as a tool that applies the same
	  introspection capabilities of Python objects to large
	  amounts of data in persistent storage.
	</p>

	<p>It is worth to note that, from version 1.2 on, PyTables
	  sports a <em>node cache system</em> that loads nodes on
	  demand, and unloads nodes that have not been used for some
	  time (i.e. following a <em>Least Recent Used</em> schema).
	  This feature allows opening HDF5 files with large
	  hierarchies very quickly and with a low memory consumption,
	  while retaining all the powerful browsing capabilities of the
	  previous implementation of the object tree. See <cite
	  refid="NewObjectTreeCacheRef"></cite> for more facts about
	  the advantages introduced by this new node cache system.
	</p>

	<p>To better understand the dynamic nature of this object tree
	  entity, let's start with a sample <verb>PyTables</verb>
	  script (you can find it in
	  <verb>examples/objecttree.py</verb>) to create a HDF5 file:
	</p>

	<!-- IVB: Thus the sample values of the string array
	do not look like keywords. -->
	<verbatim>
from tables import *

class Particle(IsDescription):
    identity = StringCol(length=22, dflt=" ", pos = 0)  # character String
    idnumber = Int16Col(1, pos = 1)  # short integer
    speed    = Float32Col(1, pos = 2)  # single-precision

# Open a file in "w"rite mode
fileh = openFile("objecttree.h5", mode = "w")
# Get the HDF5 root group
root = fileh.root

# Create the groups:
group1 = fileh.createGroup(root, "group1")
group2 = fileh.createGroup(root, "group2")

# Now, create an array in the root group
array1 = fileh.createArray(root, "array1",
                           ["this is", "a string array"], "String array")
# Create 2 new tables in group1 and group2
table1 = fileh.createTable(group1, "table1", Particle)
table2 = fileh.createTable("/group2", "table2", Particle)
# Create one more Array in group1
array2 = fileh.createArray("/group1", "array2", [1,2,3,4])

# Now, fill the tables:
for table in (table1, table2):
    # Get the record object associated with the table:
    row = table.row
    # Fill the table with 10 records
    for i in xrange(10):
        # First, assign the values to the Particle record
        row['identity']  = 'This is particle: %2d' % (i)
        row['idnumber'] = i
        row['speed']  = i * 2.
        # This injects the Record values
        row.append()

    # Flush the table buffers
    table.flush()

# Finally, close the file (this also will flush all the remaining buffers!)
fileh.close()
	</verbatim>

	<p>This small program creates a simple HDF5 file called
	  <verb>objecttree.h5</verb> with the structure that appears
	  in <ref refid="objecttree-h5">figure</ref><footnote>We have
	  used ViTables (see <cite refid="ViTablesRef"></cite>) in
	  order to create this snapshot.</footnote>. When the file is
	  created, the metadata in the object tree is updated in
	  memory while the actual data is saved to disk. When you
	  close the file the object tree is no longer
	  available. However, when you reopen this file the object
	  tree will be reconstructed in memory from the metadata on
	  disk, allowing you to work with it in exactly the same way
	  as when you originally created it.
	</p>

	<figure id="objecttree-h5">
	  <graphics file="objecttree-h5" scale="0.5" kind="bitmap">
	  </graphics>
	  <caption>An HDF5 example with 2 subgroups, 2 tables and 1
	    array.</caption>
	</figure>

	<p>In <ref refid="objecttree">figure</ref> you can see an
	  example of the object tree created when the above
	  <verb>objecttree.h5</verb> file is read (in fact, such an object is
	  always created when reading any supported generic HDF5
	  file).  It's worthwhile to take your time to understand
	  it<footnote>Bear in mind, however, that this
	  diagram is <visual markup="bf">not</visual> a standard UML
	  class diagram; it is rather meant to show the connections
	  between the <verb>PyTables</verb> objects and some of its
	  most important attributes and methods.</footnote>. It will
	  help you to avoid programming mistakes.
	</p>

	<figure id="objecttree">
	  <graphics file="objecttree" scale="0.4" kind="vector">

	  <!-- If you want to convert the eps to jpeg use the command:
pstoimg -scale 0.75 -aaliastext -type png -crop a -interlace objecttree.eps
	  and then, convert the png file again to jpg with:
	  convert objecttree.png objecttree.jpg
	  Use this tag to include the jpeg file:
	  <graphics file="objecttree" scale="0.5" kind="bitmap">
	  -->
	  </graphics>
	  <caption>A <visual markup="tt">PyTables</visual> object tree
	      example.
	  </caption>
	</figure>

      </section>

    </chapter>

    <chapter>
      <heading>Installation</heading>

      <aphorism>Make things as simple as possible, but not any
	simpler.  <caption>Albert Einstein</caption> </aphorism>

      <p>The Python <verb>Distutils</verb> are used to build and
	install <verb>PyTables</verb>, so it is fairly simple to get
	the application up and running. If you want to install the
	package from sources go to the next section. But if you are
	running Windows and want to install precompiled binaries jump
	to <ref refid="binaryInstallationDescr">section</ref>). In
	addition, packages are available for many different Linux
	distributions, for instance <url
	name="http://www.t2-project.org">T2 Project</url>, <url
	name="http://www.rocklinux.org/"><verb>RockLinux</verb></url>,
	<url name="http://www.debian.org/"><verb>Debian</verb></url>,
	or <url
	name="http://www.gentoo.org/"><verb>Gentoo</verb></url>, among
	others.  There also packages for other Unices like <url
	name="http://www.freshports.org/"><verb>FreeBSD</verb></url>
	or <url
	name="http://www.opendarwin.org/"><verb>MacOSX</verb></url>
      </p>

      <section id="sourceInstallationDescr">
	<heading>Installation from source</heading>

	<p>
	  These instructions are for both Unix/Linux and Windows
	  systems. If you are using Windows, it is assumed that you
	  have a recent version of <verb>MS Visual C++</verb> (>= 6.0)
	  compiler installed. A <verb>GCC</verb> compiler is assumed
	  for Unix, but other compilers should work as well.
	</p>

	<p>Extensions in <visual markup="tt">PyTables</visual> have
	  been developed in Pyrex (see <cite refid="Pyrex"></cite>)
	  and C language. You can rebuild everything from scratch if
	  you have Pyrex installed, but this is not necessary, as the
	  Pyrex compiled source is included in the distribution.
	</p>

	<p>To compile <verb>PyTables</verb> you will need a recent
	  version of <verb>Python</verb>, the <verb>HDF5</verb> (C
	  flavor) library, and the <verb>numarray</verb> (see <cite
	  refid="Numarray"></cite>) package. Although you won't need
	  <verb>NumPy</verb> (see <cite refid="NumPy"></cite>) or
	  <verb>Numeric</verb> (see <cite refid="Numeric"></cite>) in
	  order to compile PyTables, they are supported; you only need
	  a reasonably recent version of them (>= 0.9.8 for NumPy and
	  >= 24.2 for Numeric) if you plan on using them in your
	  applications. If you already have <verb>NumPy</verb> and/or
	  <verb>Numeric</verb> installed, the test driver module will
	  detect them and will run the tests for <verb>NumPy</verb>
	  and/or <verb>Numeric</verb> automatically.
	</p>

        <subsection id="PrerequisitesSourceDescr">
	  <heading>Prerequisites</heading>

	  <p>First, make sure that you have at least Python 2.3 or 2.4
	    (Python 2.2 is unsupported), HDF5 1.6.4 and numarray 1.5.0
	    or higher installed (I'm using HDF5 1.6.5 and numarray
	    1.5.1 currently). If you don't, fetch and install them
	    before proceeding.
	  </p>
	  <p>Compile and install these packages (but see <ref
	    refid="prerequisitesBinInst">section</ref> for
	    instructions on how to install precompiled binaries if you
	    are not willing to compile the prerequisites on Windows
	    systems).
	  </p>
	  <p>For compression (and possibly improved performance), you
	    will need to install the <verb>Zlib</verb> (see <cite
	    refid="zlibRef"></cite>), which is also required by
	    <verb>HDF5</verb> as well. You may also optionally install
	    the excellent <verb>LZO</verb> compression library (see
	    <cite refid="lzoRef"></cite> and section <ref
	    refid="compressionIssues"></ref>).  The high-performance
	    bzip2 compression library can also be used with PyTables
	    (see <cite refid="bzip2Ref"></cite>).  The use of the UCL
	    compression library is in process of being
	    <em>deprecated</em><footnote>This is because of recurrent
	    memory problems in some platforms (perhaps some bad
	    interaction between UCL and <em>something</em>
	    else). Eventually, UCL support will be dropped in the
	    future, so, please, refrain to create datasets compressed
	    with it.</footnote>, so it is recommended to not use it
	    unless you have to (you still have data files compressed
	    with UCL). Meanwhile, you can force its support in
	    PyTables by passing the <verb>--force-ucl</verb> flag to
	    <verb>setup.py</verb> (see later).
	  </p>
	  <description>
	    <term>Unix</term>
	    <item>
	      <p><verb>setup.py</verb> will detect <verb>HDF5</verb>,
		<verb>LZO</verb>, <verb>UCL</verb> or
		<verb>bzip2</verb> libraries and include files under
		<verb>/usr</verb> or <verb>/usr/local</verb>; this
		will cover most manual installations as well as
		installations from packages.  If <verb>setup.py</verb>
		can not find <verb>libhdf5</verb> (or
		<verb>liblzo</verb>, <verb>libucl</verb> or
		<verb>libbz2</verb> that you may wish to use) or if
		you have several versions of a library installed and
		want to use a particular one, then you can set the
		path to the resource in the environment, setting the
		values of the <verb>HDF5_DIR</verb>,
		<verb>LZO_DIR</verb>, <verb>UCL_DIR</verb> or
		<verb>BZIP2_DIR</verb> environment variables to the
		path to the particular resource.  You may also specify
		the locations of the resource root directories on the
		<verb>setup.py</verb> command line. For example:
	      </p>

	      <verbatim>
		--hdf5=/stuff/hdf5-1.6.5
		--lzo=/stuff/lzo-1.08
		--bzip2=/stuff/bzip2-1.0.3
	      </verbatim>

	      <p>You can force the compilation of the deprecated UCL
		  compressor by passing the --force-ucl flag:
	      </p>

	      <verbatim>
		--ucl=/stuff/ucl-1.03 --force-ucl
	      </verbatim>

	      <p>If your <verb>HDF5</verb> library was built as
		a shared library not in the runtime load path, then
		you can specify the additional linker flags needed to
		find the shared library on the command line as well.
		For example:

		<verbatim>
		  --lflags="-Xlinker -rpath -Xlinker /stuff/hdf5-1.6.5/lib"
		</verbatim>

		You may also want to try setting the LD_LIBRARY_PATH
		environment variable to point to the directory where
		the shared libraries can be found.  Check your
		compiler and linker documentation as well as the
		Python <verb>Distutils</verb> documentation for the
		correct syntax or environment variable names.
	      </p>

	      <p>It is also possible to link with specific libraries by
		setting the <verb>LIBS</verb> environment variable:
		<verbatim>
		  LIBS="hdf5-1.6.5"
		  LIBS="hdf5-1.6.5 nsl"
		</verbatim>
	      </p>
	    </item>

	    <term>Windows</term>
	    <item>
	      <p>Once you have installed the prerequisites,
		<verb>setup.py</verb> needs to know where the
		necessary library <em>stub</em> (<verb>.lib</verb>)
		and <em>header</em> (<verb>.h</verb>) files are
		installed.  Set the following environment variables:
	      </p>

	      <description>
		<term>HDF5_DIR</term><item>Points to the root HDF5
		  directory (where the include/ and dll/ directories
		  can be found). <em>Mandatory</em>.</item>

<!-- 		<term>ZLIB_DIR</term> <item>Points to the root ZLIB -->
<!-- 		  directory (where the include/ and lib/ directories -->
<!-- 		  can be found). <em>Mandatory</em>.</item> -->

		<term>LZO_DIR</term> <item>Points to the root LZO
		  directory (where the include/ and lib/ directories
		  can be found). <em>Optional</em>.</item>

		<term>BZIP2_DIR</term> <item>Points to the root bzip2
		  directory (where the include/ and lib/ directories
		  can be found). <em>Optional</em>.</item>

		<term>UCL_DIR</term> <item>Points to the root UCL
		  directory (where the include/ and lib/ directories
		  can be found). <em>Optional, but
		  discouraged</em>.</item>

	      </description>

	      <p>For example:
		<verbatim>
		  set HDF5_DIR=c:\stuff\5-165-win
		  set LZO_DIR=c:\stuff\lzo-1-08
		  set BZIP2_DIR=c:\stuff\bzip2-1-0-3
		</verbatim>

		Or, you can pass this information to
		<verb>setup.py</verb> by setting the appropriate
		arguments on the command line. For example:
	      </p>

	      <verbatim>
		--hdf5=c:\stuff\5-165-win
		--lzo=c:\stuff\lzo-1-08
		--bzip2=c:\stuff\bzip2-1-0-3
	      </verbatim>

	      <p>You can force the compilation of the deprecated UCL
		  compressor by passing the --force-ucl flag:
	      </p>

	      <verbatim>
		--ucl=c:\stuff\ucl-1-02 --force-ucl
	      </verbatim>

	      <p>You can get ready-to-use Windows binaries and other
	      development files for most of those libraries from the
	      GnuWin32 project (see <cite refid="GnuWin32"></cite>).
	      </p>
	    </item>

	  </description>

	</subsection>

        <subsection id="PyTablesSourceInstallationDescr">
	  <heading><visual markup="tt">PyTables</visual> package
	    installation
	  </heading>

          <p>Once you have installed the HDF5 library and the numarray
            package, you can proceed with the <verb>PyTables</verb>
            package itself:
          </p>

	  <enumerate>
	    <item>
	      <p>Run this command from the main <verb>PyTables</verb>
		distribution directory, including any extra command line
		arguments as discussed above:
		<verbatim>
		  python setup.py build_ext --inplace
		</verbatim>
		Depending on the compiler flags used when compiling your
		Python executable, there may appear many warnings. Don't
		worry, almost all of them are caused by variables declared
		but never used. That's normal in Pyrex extensions.
		</p>
	      </item>
	      <item>
		<p>To run the test suite, change into the
		  <verb>tables/tests</verb> directory and execute this
		  command:
		</p>
		<description>
		  <term>Unix</term> <item>In the shell <verb>sh</verb>
		    and its variants:
		    <verbatim>
		      PYTHONPATH=../..  python test_all.py
		    </verbatim>
		  </item>
		  <term>Windows</term>
		  <item>Open a DOS terminal and type:
		    <verbatim>
		      set PYTHONPATH=..\..
		      python test_all.py
		    </verbatim>
		  </item>
		</description>

	      <p>If you would like to see verbose output from the
		tests simply add the flag <verb>-v</verb> and/or the
		word <verb>verbose</verb> to the command line. You can
		also run only the tests in a particular test
		module. For example, to execute just the
		<verb>types</verb> test:
		<verbatim>
		  python test_types.py -v
		</verbatim>
		If a test fails, please enable verbose output (the
		<verb>-v</verb> flag <visual markup="bf">and</visual>
		<verb>verbose</verb> option), run the failing test
		module again, and, very important, get your
		<verb>PyTables</verb> version information by running
		the command:
		<verbatim>
		  python test_all.py --show-versions
		</verbatim>
		and send back the output to developers so that we may
		continue improving <verb>PyTables</verb>.
	      </p>

	      <p> If you run into problems because Python can not load the
		HDF5 library or other shared libraries:
	      </p>
	      <description>
		<term>Unix</term>
		<item>
		  Try setting the LD_LIBRARY_PATH environment variable to
		  point to the directory where the missing libraries can
		  be found.
		</item>
		<term>Windows</term> <item>Put the DLL libraries
		  (<verb>hdf5dll.dll</verb> and, optionally,
		  <verb>lzo1.dll</verb> and <verb>bzip2.dll</verb>) in
		  a directory listed in your <verb>PATH</verb>
		  environment variable or in
		  <verb>python_installation_path\Lib\site-packages\tables</verb>
		  (the last directory may have not exist yet, so if
		  you want to install the DLLs there, you should do so
		  <em>after</em> installing the PyTables package).
		  The <verb>setup.py</verb> installation program will
		  print out a warning to that effect if the libraries
		  can not be found.
		</item>
	      </description>
	    </item>
	    <item>
		  <!-- SGP: What if they're not root, or if they're on
		  a Windows machine?  Can you set the install prefix
		  on the command line?  Where does PyTables install
		  itself by default?  Amplify this part.  Done! -->

	      <p>To install the entire <visual
		  markup="tt">PyTables</visual> Python package, change
		back to the root distribution directory and run the
		following command (make sure you have sufficient
		permissions to write to the directories where the
		<verb>PyTables</verb> files will be installed):

		<verbatim>
		  python setup.py install
		</verbatim>

		Of course, you will need super-user privileges if you
		want to install <verb>PyTables</verb> on a
		system-protected area. You can select, though, a
		different place to install the package using the
		<verb>--prefix</verb> flag:

		<verbatim>
		  python setup.py install --prefix="/home/myuser/mystuff"
		</verbatim>

		Have in mind, however, that if you use the
		<verb>--prefix</verb> flag to install in a non-standard
		place, you should properly setup your
		<verb>PYTHONPATH</verb> environment variable, so that
		the Python interpreter would be able to find your new
		<verb>PyTables</verb> installation.
	      </p>

	      <p>You have more installation options available in
		the Distutils package. Issue a:
		<verbatim>
		  python setup.py install --help
		</verbatim>
		for more information on that subject.
	      </p>

	    </item>
	  </enumerate>

	  <p>That's it! Now you can skip to the next chapter to learn how to use
	    <verb>PyTables</verb>. <!-- <newline vspace="4.0cm"/> -->
	  </p>

	</subsection>

      </section>

      <section id="binaryInstallationDescr">
	<heading>Binary installation (Windows)</heading>

	<p>This section is intended for installing precompiled
	  binaries on Windows platforms. You may also find it useful
	  for instructions on how to install <em>binary
	  prerequisites</em> even if you want to compile
	  <verb>PyTables</verb> itself on Windows.
	</p>

	<subsection id="prerequisitesBinInst">
	  <heading>Windows prerequisites</heading>

	  <p>First, make sure that you have Python 2.3, 2.4 or higher
 	    (Python 2.2 is unsupported), HDF5 1.6.5 or higher and
 	    numarray 1.5.0 or higher installed (I have built the
 	    <verb>PyTables</verb> binaries using HDF5 1.6.5 and
 	    numarray 1.5.1).
          </p>
          <p>For the HDF5 library it should be enough to manually copy
	    the <verb>hdf5dll.dll</verb>, <verb>zlib1.dll</verb>
	    <visual markup="bf">and</visual> <verb>szipdll.dll</verb>
	    files to a directory in your <verb>PATH</verb> environment
	    variable (for example <verb>C:\WINDOWS\SYSTEM32</verb>) or
<!-- 	    in <verb>python_installation_path\DLLs</verb> or in -->
	    <verb>python_installation_path\Lib\site-packages\tables</verb>
	    (the last directory may have not exist yet, so if you want
	    to install the DLLs there, you should do so <em>after</em>
	    installing the PyTables package).
	  </p>
	  <p><visual markup="bf">Caveat:</visual> When downloading the
	    binary distribution for HDF5 libraries, select one
	    compiled with MSVC 6.0 if you are using Python 2.3.x, such
	    as the package <verb>5-165-win.zip</verb>. The file
	    <verb>5-165-win-net.zip</verb> was compiled with the MSVC
	    7.1 (aka "<verb>.NET 2003</verb>") and you <visual
	    markup="bf">must</visual> choose if you want to run
	    PyTables with Python 2.4.x series. You have been warned!
	  </p>

	  <p>To enable compression with optional LZO or bzip2
	    libraries (see the <ref
	    refid="compressionIssues">section</ref> for hints about
	    how they may be used to improve performance), fetch and
	    install the <verb>LZO</verb> (choose v1.x,
	    <verb>LZO</verb> v2.x is not supported in precompiled
	    Windows builds) and <verb>bzip2</verb> binaries from <cite
	    refid="GnuWin32"></cite><footnote>Note that support for
	    the UCL compressor has been declared deprecated and has
	    not been added in the binary build of PyTables for
	    Windows.</footnote>. Normally, you will only need to fetch
	    and install the<newline/>
	    <verb>&lt;package>-&lt;version>-bin.zip</verb> file and
	    copy the <verb>lzo1.dll</verb> or <verb>bzip2.dll</verb>
	    files in a directory in the <verb>PATH</verb> environment
	    variable, or in
	    <verb>python_installation_path\Lib\site-packages\tables</verb>
	    (the last directory may have not exist yet, so if you want
	    to install the DLLs there, you should do so <em>after</em>
	    installing the PyTables package), so that they can be
	    found by the <visual markup="tt">PyTables</visual>
	    extensions.
	  </p>
	  <p>Please, note that PyTables has internal machinery for
	    dealing with uninstalled optional compression libraries,
	    so, you don't need to install any of LZO or bzip2 dynamic
	    libraries if you don't want to.
          </p>

        </subsection>

        <subsection id="PyTablesBinInstallDescr">
	  <heading><visual markup="tt">PyTables</visual> package
	    installation
	  </heading>

	  <p>Download the
	    <verb>tables-&lt;version>.win32-py&lt;version>.exe</verb>
	    file and execute it.
	  </p>

	  <p>You can (<em>you should</em>) test your installation by
	    unpacking the source tar-ball, changing to the
	    <verb>tables/tests/</verb> subdirectory and executing the
	    <verb>test_all.py</verb> script. If all the tests pass
	    (possibly with a few warnings, related to the potential
	    unavailability of LZO or bzip2 libs) you already have a
	    working, well-tested copy of <verb>PyTables</verb>
	    installed! If any test fails, please try to locate which
	    test module is failing and execute:

	    <verbatim>
	      python test_&lt;module>.py -v verbose
	    </verbatim>

	    and also:

	    <verbatim>
	      python test_all.py --show-versions
	    </verbatim>

	    and mail the output to the developers so that the problem
	    can be fixed in future releases.
	  </p>

	  <newline vspace="0.25cm"/>

	  <p>You can proceed now to the next chapter to see how to use
	    <verb>PyTables</verb>.
	  </p>

	</subsection>

      </section>


    </chapter>

    <chapter id="usage">
      <heading>Tutorials</heading>

<!--       <aphorism>Tout le malheur des hommes vient d'une seule chose, -->
<!-- 	qui est de ne savoir pas demeurer en repos, dans une chambre. -->
<!-- 	<caption>Blaise Pascal</caption> -->
<!--       </aphorism> -->

      <aphorism>Seràs la clau que obre tots els panys, <newline/>
	seràs la llum, la llum il.limitada, <newline/> seràs confí on
	l'aurora comença, <newline/>seràs forment, escala il.luminada!
	<newline/><caption>M'aclame a tu <newline/>Lyrics: Vicent
	Andrés i Estellés <newline/>Music: Ovidi Montllor</caption>
      </aphorism>


      <p>This chapter consists of a series of simple yet comprehensive
	tutorials that will enable you to understand
	<verb>PyTables</verb>' main features. If you would like more
	information about some particular instance variable, global
	function, or method, look at the doc strings or go to the
	library reference in <ref
	refid="libraryReference">chapter</ref>. If you are reading
	this in PDF or HTML formats, follow the corresponding
	hyperlink near each newly introduced entity.
      </p>

      <p>Please, note that throughout this document the terms
	<em>column</em> and <em>field</em> will be used
	interchangeably, as will the terms <em>row</em> and
	<em>record</em>.
      </p>

      <section>
	<heading>Getting started</heading>

	<p>In this section, we will see how to define our own records
	  in Python and save collections of them (i.e. a <visual
	  markup="bf">table</visual>) into a file. Then we will select
	  some of the data in the table using Python cuts and create
	  <verb>numarray</verb> arrays to store this selection as
	  separate objects in a tree.
	</p>
	<p>
	  In <em>examples/tutorial1-1.py</em> you will find the
	  working version of all the code in this
	  section. Nonetheless, this tutorial series has been written
	  to allow you reproduce it in a Python interactive console. I
	  encourage you to do parallel testing and inspect the created
	  objects (variables, docs, children objects, etc.) during the
	  course of the tutorial!
	</p>

	<subsection>
	  <heading>Importing <visual markup="tt">tables</visual>
	    objects</heading>

          <p>Before starting you need to import the
	    public objects in the <verb>tables</verb> package. You
	    normally do that by executing:
	  </p>
	  <verbatim>
>>> import tables
	  </verbatim>
	  <p>This is the recommended way to import <verb>tables</verb>
	    if you don't want to pollute your namespace. However,
	    <verb>PyTables</verb> has a very reduced set of
	    first-level primitives, so you may consider using the
	    alternative:
	  </p>
	  <verbatim>
>>> from tables import *
	  </verbatim>
	  <p>which will export in your caller application namespace the
	    following functions: <verb>openFile()</verb>, <verb>copyFile()</verb>,
	    <verb>isHDF5File()</verb>, <verb>isPyTablesFile()</verb> and
	    <verb>whichLibVersion()</verb>. This is a rather reduced set
	    of functions, and for convenience, we will use this
	    technique to access them.
	  </p>
	  <p>If you are going to work with <verb>numarray</verb> (or
	    <verb>NumPy</verb> or <verb>Numeric</verb>) arrays (and
	    normally, you will) you will also need to import functions
	    from them.  So most <verb>PyTables</verb> programs begin
	    with:
	  </p>
	  <verbatim>
>>> import tables        # but in this tutorial we use "from tables import *"
>>> import numarray      # or "import numpy" or "import Numeric"
	  </verbatim>
	</subsection>

	<subsection>
	  <heading>Declaring a Column Descriptor</heading>

	  <p>Now, imagine that we have a particle detector and we want
	    to create a table object in order to save data
	    retrieved from it. You need first to define the table, the
	    number of columns it has, what kind of object is contained
	    in each column, and so on.
	  </p>
	  <p>Our particle detector has a TDC (Time to Digital
	    Converter) counter with a dynamic range of 8 bits and an
	    ADC (Analogical to Digital Converter) with a range of 16
	    bits. For these values, we will define 2 fields in our
	    record object called <verb>TDCcount</verb> and
	    <verb>ADCcount</verb>. We also want to save the grid
	    position in which the particle has been detected, so we
	    will add two new fields called <verb>grid_i</verb> and
	    <verb>grid_j</verb>. Our instrumentation also can obtain
	    the pressure and energy of the particle. The resolution of
	    the pressure-gauge allows us to use a simple-precision
	    float to store <verb>pressure</verb> readings, while the
	    <verb>energy</verb> value will need a double-precision
	    float. Finally, to track the particle we want to assign it
	    a name to identify the kind of the particle it is and a
	    unique numeric identifier. So we will add two more fields:
	    <verb>name</verb> will be a string of up to 16 characters,
	    and <verb>idnumber</verb> will be an integer of 64 bits
	    (to allow us to store records for extremely large numbers
	    of particles).  <!-- SGP: Whew.  You didn't exactly choose
	    the most intuitive example.  If you want this to be
	    understandable to people outside the physics community,
	    you may want to come up with a different example (say, a
	    database of dirty pictures). -->
	  </p>
	  <p>Having determined our columns and their types, we can now
	    declare a new <verb>Particle</verb> class that will
	    contain all this information:
	  </p>

	  <verbatim>
>>> class Particle(IsDescription):
...     name      = StringCol(16)   # 16-character String
...     idnumber  = Int64Col()      # Signed 64-bit integer
...     ADCcount  = UInt16Col()     # Unsigned short integer
...     TDCcount  = UInt8Col()      # unsigned byte
...     grid_i    = Int32Col()      # integer
...     grid_j    = IntCol()        # integer (equivalent to Int32Col)
...     pressure  = Float32Col()    # float  (single-precision)
...     energy    = FloatCol()      # double (double-precision)
...
>>>
	  </verbatim>
	  <p>This definition class is self-explanatory.  Basically,
	    you declare a class variable for each field you need.  As
	    its value you assign an instance of the appropriate
	    <verb>Col</verb> subclass, according to the kind of column
	    defined (the data type, the length, the shape, etc).  See
	    the <ref refid="ColClassDescr">section</ref> for a
	    complete description of these subclasses. See also <ref
	    refid="datatypesSupported">appendix</ref> for a list of
	    data types supported by the <verb>Col</verb> constructor.
	  </p>
	  <p>From now on, we can use <verb>Particle</verb> instances
	    as a descriptor for our detector data table. We will see
	    later on how to pass this object to construct the table.
	    But first, we must create a file where all the actual data
	    pushed into our table will be saved.
	  </p>

	</subsection>

	<subsection>
	  <heading>Creating a <visual markup="tt">PyTables</visual> file from scratch</heading>

	  <p>Use the first-level <verb>openFile</verb> (see <ref
	    refid="openFileDescr"></ref>) function to create a 
	    <verb>PyTables</verb> file:
	  </p>

	  <verbatim>
>>> h5file = openFile("tutorial1.h5", mode = "w", title = "Test file")
	  </verbatim>
	  <p><verb>openFile</verb> (<ref
	    refid="openFileDescr">see</ref>) is one of the objects
	    imported by the "<verb>from tables import *</verb>"
	    statement. Here, we are saying that we want to create a
	    new file in the current working directory called
	    "<verb>tutorial1.h5</verb>" in "<verb>w</verb>"rite mode
	    and with an descriptive title string ("<verb>Test
	    file</verb>").  This function attempts to open the file,
	    and if successful, returns the <verb>File</verb> (<ref
	    refid="FileClassDescr">see</ref>) object instance
	    <verb>h5file</verb>.  The root of the object tree is
	    specified in the instance's <verb>root</verb> attribute.
	  </p>
	</subsection>

	<subsection>
	  <heading>Creating a new group</heading>

	  <p>Now, to better organize our data, we will create a group
	    called <em>detector</em> that branches from the root
	    node. We will save our particle data table in this group.
	  </p>
	  <verbatim>
>>> group = h5file.createGroup("/", 'detector', 'Detector information')
	  </verbatim>

	  <p>Here, we have taken the <verb>File</verb> instance
	    <verb>h5file</verb> and invoked its
	    <verb>createGroup</verb> method (<ref
	    refid="createGroupDescr">see</ref>) to create a new group
	    called <em>detector</em> branching from "<em>/</em>"
	    (another way to refer to the <verb>h5file.root</verb>
	    object we mentioned above). This will create a new
	    <verb>Group</verb> (see<ref
	    refid="GroupClassDescr"></ref>) object instance that will
	    be assigned to the variable <verb>group</verb>.
	  </p>

	</subsection>
	<subsection>
	  <heading>Creating a new table</heading>

	  <p>Let's now create a <verb>Table</verb> (see <ref
	    refid="TableClassDescr"></ref>) object as a branch off the
	    newly-created group. We do that by calling the
	    <verb>createTable</verb> (see <ref
	    refid="createTableDescr"></ref>) method of the
	    <verb>h5file</verb> object:
	  </p>
	  <verbatim>
>>> table = h5file.createTable(group, 'readout', Particle, "Readout example")
	  </verbatim>

	  <p>We create the <verb>Table</verb> instance under
	    <verb>group</verb>.  We assign this table the node name
	    "<em>readout</em>".  The <verb>Particle</verb> class
	    declared before is the <em>description</em> parameter (to
	    define the columns of the table) and finally we set
	    "<em>Readout example</em>" as the <verb>Table</verb>
	    title. With all this information, a new <verb>Table</verb>
	    instance is created and assigned to the variable
	    <em>table</em>.
	  </p>

	  <p>If you are curious about how the object tree looks right
	    now, simply <verb>print</verb> the <verb>File</verb>
	    instance variable <em>h5file</em>, and examine the output:
	  </p>

	  <verbatim>
>>> print h5file
Filename: 'tutorial1.h5' Title: 'Test file' Last modif.: 'Sun Jul 27 14:00:13 2003'
/ (Group) 'Test file'
/detector (Group) 'Detector information'
/detector/readout (Table(0,)) 'Readout example'

	  </verbatim>

	  <p>As you can see, a dump of the object tree is displayed.
	    It's easy to see the <verb>Group</verb> and
	    <verb>Table</verb> objects we have just created. If you
	    want more information, just type the variable containing the
	    <verb>File</verb> instance:
	  </p>

	  <verbatim>
>>> h5file
File(filename='tutorial1.h5', title='Test file', mode='w', trMap={}, rootUEP='/')
/ (Group) 'Test file'
/detector (Group) 'Detector information'
/detector/readout (Table(0,)) 'Readout example'
  description := {
    "ADCcount": Col('UInt16', shape=1, itemsize=2, dflt=0),
    "TDCcount": Col('UInt8', shape=1, itemsize= 1, dflt=0),
    "energy": Col('Float64', shape=1, itemsize=8, dflt=0.0),
    "grid_i": Col('Int32', shape=1, itemsize=4, dflt=0),
    "grid_j": Col('Int32', shape=1, itemsize=4, dflt=0),
    "idnumber": Col('Int64', shape=1, itemsize=8, dflt=0),
    "name": Col('CharType', shape=1, itemsize=16, dflt=None),
    "pressure": Col('Float32', shape=1, itemsize=4, dflt=0.0) }
  byteorder := little

	  </verbatim>

	  <p>More detailed information is displayed about each object
	    in the tree. Note how <verb>Particle</verb>, our table
	    descriptor class, is printed as part of the
	    <em>readout</em> table description information. In
	    general, you can obtain much more information about the
	    objects and their children by just printing them. That
	    introspection capability is very useful, and I recommend
	    that you use it extensively.
	  </p>

	  <p>The time has come to fill this table with some
	    values. First we will get a pointer to the
	    <verb>Row</verb> (see <ref refid="RowClassDescr"></ref>)
	    instance of this <verb>table</verb> instance:
	  </p>
	  <verbatim>
>>> particle = table.row
	  </verbatim>

	  <p>The <verb>row</verb> attribute of <verb>table</verb>
	    points to the <verb>Row</verb> instance that will be used
	    to write data rows into the table. We write data simply by
	    assigning the <verb>Row</verb> instance the values for
	    each row as if it were a dictionary (although it is
	    actually an <em>extension class</em>), using the column
	    names as keys.
	  </p>

	  <p>Below is an example of how to write rows:
	  </p>

	  <verbatim>
>>> for i in xrange(10):
...     particle['name']  = 'Particle: %6d' % (i)
...     particle['TDCcount'] = i % 256
...     particle['ADCcount'] = (i * 256) % (1 &lt;&lt; 16)
...     particle['grid_i'] = i
...     particle['grid_j'] = 10 - i
...     particle['pressure'] = float(i*i)
...     particle['energy'] = float(particle['pressure'] ** 4)
...     particle['idnumber'] = i * (2 ** 34)
...     particle.append()
...
>>>
	  </verbatim>

	  <p>This code should be easy to understand. The lines inside
	    the loop just assign values to the different columns in
	    the Row instance <verb>particle</verb> (<ref
	    refid="RowClassDescr">see</ref>). A call to its
	    <verb>append()</verb> method writes this
	    information to the <verb>table</verb> I/O buffer.
	  </p>

	  <p>After we have processed all our data, we should flush the
	    table's I/O buffer if we want to write all
	    this data to disk. We achieve that by calling the
	    <verb>table.flush()</verb> method.
	  </p>
	  <verbatim>
>>> table.flush()
	  </verbatim>

	</subsection>

	<subsection id="readingAndSelectingUsage">
	  <heading>Reading (and selecting) data in a table</heading>

	  <p>Ok. We have our data on disk, and now we need to access
	    it and select from specific columns the values we are
	    interested in. See the example below:
	  </p>

	  <verbatim>
>>> table = h5file.root.detector.readout
>>> pressure = [ x['pressure'] for x in table.iterrows()
...              if x['TDCcount']>3 and 20&lt;=x['pressure']&lt;50 ]
>>> pressure
[25.0, 36.0, 49.0]
	  </verbatim>

	  <p>The first line creates a "shortcut"
	    to the <em>readout</em> table deeper on the
	    object tree. As you can see, we use the <visual
	    markup="bf">natural naming</visual> schema to access
	    it. We also could have used the
	    <verb>h5file.getNode()</verb> method, as we will do
	    later on.
	  </p>

	  <p>You will recognize the last two lines as a Python list
	    comprehension. It loops over the rows in <em>table</em> as
	    they are provided by the <verb>table.iterrows()</verb>
	    iterator (see <ref refid="Table.iterrows"></ref>). The
	    iterator returns values until all the data in table is
	    exhausted. These rows are filtered using the expression:
	    <verbatim>
	      x['TDCcount'] > 3 and x['pressure'] &lt;50
	    </verbatim>
	    We select the value of the <verb>pressure</verb> column from
	    filtered records to create the final list and assign it to
	    <verb>pressure</verb> variable.
	  </p>

	  <p>We could have used a normal <verb>for</verb> loop to
	    accomplish the same purpose, but I find comprehension
	    syntax to be more compact and elegant.
	  </p>

	  <p>Let's select the <verb>name</verb> column for the same
	    set of cuts:
	  </p>

	  <verbatim>
>>> names=[ x['name'] for x in table if x['TDCcount']>3 and 20&lt;=x['pressure']&lt;50 ]
>>> names
['Particle:      5', 'Particle:      6', 'Particle:      7']
	  </verbatim>

	  <p>Note how we have omitted the <verb>iterrows()</verb> call
	    in the list comprehension. The <verb>Table</verb> class
	    has an implementation of the special method
	    <verb>__iter__()</verb> that iterates over all the rows in
	    the table. In fact, <verb>iterrows()</verb> internally
	    calls this special <verb>__iter__()</verb> method.
	    Accessing all the rows in a table using this method is
	    very convenient, especially when working with the data
	    interactively.
	  </p>

	  <p>That's enough about selections. The next section will show
	    you how to save these selected results to a file.
	  </p>

	</subsection>

	<subsection>
	  <heading>Creating new array objects</heading>

	  <p>In order to separate the selected data from the mass of
	    detector data, we will create a new group
	    <verb>columns</verb> branching off the root
	    group. Afterwards, under this group, we will create two
	    arrays that will contain the selected data. First, we
	    create the group:
	  </p>

	  <verbatim>
>>> gcolumns = h5file.createGroup(h5file.root, "columns", "Pressure and Name")
	  </verbatim>

	  <p>Note that this time we have specified the first parameter
	    using <em>natural naming</em>
	    (<verb>h5file.root</verb>) instead of with an absolute
	    path string ("/").
	  </p>

	  <p>Now, create the first of the two <verb>Array</verb>
	    objects we've just mentioned:
	  </p>

	  <verbatim>
>>> h5file.createArray(gcolumns, 'pressure', array(pressure),
...                     "Pressure column selection")
/columns/pressure (Array(3,)) 'Pressure column selection'
  type = Float64
  itemsize = 8
  flavor = 'numarray'
  byteorder = 'little'
	  </verbatim>

	  <p>We already know the first two parameters of the
	    <verb>createArray</verb> (see <ref
	    refid="createArrayDescr"></ref>) methods (these are the
	    same as the first two in <verb>createTable</verb>): they
	    are the parent group <em>where</em> <verb>Array</verb>
	    will be created and the <verb>Array</verb> instance
	    <em>name</em>.  The third parameter is the <em>object</em>
	    we want to save to disk. In this case, it is a
	    <verb>numarray</verb> array that is built from the
	    selection list we created before.  The fourth parameter is
	    the <em>title</em>.
	  </p>

	  <p>Now, we will save the second array. It contains the list
	    of strings we selected before: we save this object as-is,
	    with no further conversion.
	  </p>

	  <verbatim>
>>> h5file.createArray(gcolumns, 'name', names, "Name column selection")
/columns/name Array(4,) 'Name column selection'
  type = 'CharType'
  itemsize = 16
  flavor = 'List'
  byteorder = 'little'
	  </verbatim>

	  <p>As you can see, <verb>createArray()</verb> accepts
	    <em>names</em> (which is a regular Python list) as an
	    <em>object</em> parameter. Actually, it accepts a variety
	    of different regular objects (see <ref
	    refid="createArrayDescr"></ref>) as parameters. The
	    <verb>flavor</verb> attribute (see the output above) saves
	    the original kind of object that was saved. Based on this
	    <em>flavor</em>, <verb>PyTables</verb> will be able to
	    retrieve exactly the same object from disk later on.
	  </p>
	  <p>Note that in these examples, the <verb>createArray</verb>
	    method returns an <verb>Array</verb> instance that is not
	    assigned to any variable. Don't worry, this is intentional
	    to show the kind of object we have created by displaying
	    its representation.  The <verb>Array</verb> objects have
	    been attached to the object tree and saved to disk, as you
	    can see if you print the complete object tree:
	  </p>
	  <verbatim>
>>> print h5file
Filename: 'tutorial1.h5' Title: 'Test file' Last modif.: 'Sun Jul 27 14:00:13 2003'
/ (Group) 'Test file'
/columns (Group) 'Pressure and Name'
/columns/name (Array(3,)) 'Name column selection'
/columns/pressure (Array(3,)) 'Pressure column selection'
/detector (Group) 'Detector information'
/detector/readout (Table(10,)) 'Readout example'

	  </verbatim>
	</subsection>

	<subsection>
	  <heading>Closing the file and looking at its content</heading>

	  <p>To finish this first tutorial, we use the
	    <verb>close</verb> method of the h5file <verb>File</verb>
	    object to close the file before exiting Python:
	  </p>
	  <verbatim>
>>> h5file.close()
>>> ^D
	  </verbatim>

	  <p>You have now created your first <verb>PyTables</verb>
	    file with a table and two arrays. You can examine it with
	    any generic HDF5 tool, such as <verb>h5dump</verb> or
	    <verb>h5ls</verb>. Here is what the
	    <verb>tutorial1.h5</verb> looks like when read with the
	    <verb>h5ls</verb> program:
	  </p>
	  <verbatim>
$ h5ls -rd tutorial1.h5
/columns                 Group
/columns/name            Dataset {3}
    Data:
        (0) "Particle:      5", "Particle:      6", "Particle:      7"
/columns/pressure        Dataset {3}
    Data:
        (0) 25, 36, 49
/detector                Group
/detector/readout        Dataset {10/Inf}
    Data:
        (0) {0, 0, 0, 0, 10, 0, "Particle:      0", 0},
        (1) {256, 1, 1, 1, 9, 17179869184, "Particle:      1", 1},
        (2) {512, 2, 256, 2, 8, 34359738368, "Particle:      2", 4},
        (3) {768, 3, 6561, 3, 7, 51539607552, "Particle:      3", 9},
        (4) {1024, 4, 65536, 4, 6, 68719476736, "Particle:      4", 16},
        (5) {1280, 5, 390625, 5, 5, 85899345920, "Particle:      5", 25},
        (6) {1536, 6, 1679616, 6, 4, 103079215104, "Particle:      6", 36},
        (7) {1792, 7, 5764801, 7, 3, 120259084288, "Particle:      7", 49},
        (8) {2048, 8, 16777216, 8, 2, 137438953472, "Particle:      8", 64},
        (9) {2304, 9, 43046721, 9, 1, 154618822656, "Particle:      9", 81}
	  </verbatim>

	  <p>Here's the outputs as displayed by the "ptdump"
	   <verb>PyTables</verb> utility (located in
	   <verb>utils/</verb> directory):
	  </p>

	  <verbatim>
$ ptdump tutorial1.h5
Filename: 'tutorial1.h5' Title: 'Test file' Last modif.: 'Sun Jul 27 14:40:51 2003'
/ (Group) 'Test file'
/columns (Group) 'Pressure and Name'
/columns/name (Array(3,)) 'Name column selection'
/columns/pressure (Array(3,)) 'Pressure column selection'
/detector (Group) 'Detector information'
/detector/readout (Table(10,)) 'Readout example'

	  </verbatim>

	  <p>You can pass the <verb>-v</verb> or <verb>-d</verb>
	    options to <verb>ptdump</verb> if you want more
	    verbosity. Try them out!
	  </p>

	  <p>Also, in <ref refid="tutorial1-1-tableview">figure</ref>,
	    you can admire how the <verb>tutorial1.h5</verb> looks
	    like using the <url
	    name="http://www.carabos.com/products/vitables.html">ViTables
	    </url> graphical interface .
	  </p>

	  <figure id="tutorial1-1-tableview">
	    <graphics file="tutorial1-1-tableview" scale="0.5" kind="bitmap">
	    </graphics>
	    <caption>The initial version of the data file for tutorial 1,
	      with a view of the data objects.
	    </caption>
	  </figure>

	</subsection>
      </section>

      <section>
	<heading>Browsing the <visual markup="it">object tree</visual>
	</heading>

	<p>In this section, we will learn how to browse the tree and
	  retrieve data and also meta-information about the actual
	  data.
	</p>
	<p>
	  In <em>examples/tutorial1-2.py</em> you will find the
	  working version of all the code in this section. As before,
	  you are encouraged to use a python shell and inspect the
	  object tree during the course of the tutorial.
	</p>

	<subsection>
	  <heading>Traversing the object tree</heading>

	  <p>Let's start by opening the file we created in last
	    tutorial section.
	  </p>

	  <verbatim>
>>> h5file = openFile("tutorial1.h5", "a")
	  </verbatim>

	  <p>This time, we have opened the file in "a"ppend mode. We
	    use this mode to add more information to the file.
	  </p>
	  <p><verb>PyTables</verb>, following the Python tradition,
	    offers powerful introspection capabilities, i.e. you can
	    easily ask information about any component of the object
	    tree as well as search the tree.
	  </p>
	  <p>To start with, you can get a preliminary overview of the
	    object tree by simply printing the existing
	    <verb>File</verb> instance:
	  </p>

	  <verbatim>
>>> print h5file
Filename: 'tutorial1.h5' Title: 'Test file' Last modif.: 'Sun Jul 27 14:40:51 2003'
/ (Group) 'Test file'
/columns (Group) 'Pressure and Name'
/columns/name (Array(3,)) 'Name column selection'
/columns/pressure (Array(3,)) 'Pressure column selection'
/detector (Group) 'Detector information'
/detector/readout (Table(10,)) 'Readout example'

	  </verbatim>

	  <p>It looks like all of our objects are there.  Now let's
	    make use of the <verb>File</verb> iterator to see to list
	    all the nodes in the object tree:
	  </p>

	  <verbatim>
>>> for node in h5file:
...   print node
...
/ (Group) 'Test file'
/columns (Group) 'Pressure and Name'
/detector (Group) 'Detector information'
/columns/name (Array(3,)) 'Name column selection'
/columns/pressure (Array(3,)) 'Pressure column selection'
/detector/readout (Table(10,)) 'Readout example'
	  </verbatim>
	  <p>We can use the <verb>walkGroups</verb> method (see <ref
	    refid="walkGroupsDescr"></ref>) of the <verb>File</verb> class
	    to list only the <em>groups</em> on tree:
	  </p>

	  <verbatim>
>>> for group in h5file.walkGroups("/"):
...   print group
...
/ (Group) 'Test file'
/columns (Group) 'Pressure and Name'
/detector (Group) 'Detector information'
	  </verbatim>

	  <p>Note that <verb>walkGroups()</verb> actually returns an
	    <em>iterator</em>, not a list of objects. Using this
	    iterator with the <verb>listNodes()</verb> method is a
	    powerful combination. Let's see an example listing of all
	    the arrays in the tree:
	  </p>

	  <verbatim>
>>> for group in h5file.walkGroups("/"):
...     for array in h5file.listNodes(group, classname = 'Array'):
...         print array
...
/columns/name Array(3,) 'Name column selection'
/columns/pressure Array(3,) 'Pressure column selection'
	  </verbatim>

	  <p><verb>listNodes()</verb> (see <ref
	    refid="File.listNodes"></ref>) returns a list containing
	    all the nodes hanging off a specific <verb>Group</verb>.
	    If the <em>classname</em> keyword is specified, the method
	    will filter out all instances which are not descendants of
	    the class. We have asked for only <verb>Array</verb>
	    instances. There exist also an iterator counterpart called
	    <verb>iterNodes()</verb> (see <ref
	    refid="File.iterNodes"></ref>) that might be handy is some
	    situations, like for example when dealing with groups with
	    a large number of nodes behind it.
	  </p>

	  <p>We can combine both calls by using the
	      <verb>walkNodes(where, classname)</verb> special method
	      of the <verb>File</verb> object (<ref
	      refid="File.walkNodes">see</ref>). For example:
	  </p>

	  <verbatim>
>>> for array in h5file.walkNodes("/", "Array"):
...   print array
...
/columns/name (Array(3,)) 'Name column selection'
/columns/pressure (Array(3,)) 'Pressure column selection'
	  </verbatim>

	  <p>This is a nice shortcut when working interactively.</p>

	  <p>Finally, we will list all the <verb>Leaf</verb>,
	    i.e. <verb>Table</verb> and <verb>Array</verb> instances
	    (see <ref refid="LeafClassDescr"></ref> for detailed
	    information on <verb>Leaf</verb> class), in the
	    <verb>/detector</verb> group. Note that only one instance
	    of the <verb>Table</verb> class
	    (i.e. <verb>readout</verb>) will be selected in this group
	    (as should be the case):
	  </p>

	  <verbatim>
>>> for leaf in h5file.root.detector._f_walkNodes('Leaf'):
...   print leaf
...
/detector/readout (Table(10,)) 'Readout example'
	  </verbatim>

	  <p>We have used a call to the
	    <verb>Group._f_walkNodes(classname, recursive)</verb>
	    method (<ref refid="Group._f_walkNodes"></ref>), using the
	    <em>natural naming</em> path specification.</p>

	  <p>Of course you can do more sophisticated node selections
	    using these powerful methods. But first, let's take a look
	    at some important <verb>PyTables</verb> object instance
	    variables.
	  </p>

	</subsection>

	<subsection>
	  <heading>Setting and getting user attributes</heading>

	  <p>PyTables provides an easy and concise way to complement
	    the meaning of your node objects on the tree by using the
	    <verb>AttributeSet</verb> class (see <ref
	    refid="AttributeSetClassDescr">section</ref>). You can
	    access this object through the standard attribute
	    <verb>attrs</verb> in <verb>Leaf</verb> nodes and
	    <verb>_v_attrs</verb> in <verb>Group</verb> nodes.
	  </p>

	  <p>For example, let's imagine that we want to save the date
	    indicating when the data in <verb>/detector/readout</verb>
	    table has been acquired, as well as the temperature during
	    the gathering process:
	  </p>

	  <verbatim>
>>> table = h5file.root.detector.readout
>>> table.attrs.gath_date = "Wed, 06/12/2003 18:33"
>>> table.attrs.temperature = 18.4
>>> table.attrs.temp_scale = "Celsius"
	  </verbatim>

	  <p>Now, let's set a somewhat more complex attribute in the
	    <verb>/detector</verb> group:
	  </p>
          <!-- SGP:  What is this attribute? -->
	  <verbatim>
>>> detector = h5file.root.detector
>>> detector._v_attrs.stuff = [5, (2.3, 4.5), "Integer and tuple"]
	  </verbatim>

	  <p>Note how the AttributeSet instance is accessed with
	    the <verb>_v_attrs</verb> attribute because detector is a
	    <verb>Group</verb> node. In general, you can save any
	    standard Python data structure as an attribute node.
	    See <ref refid="AttributeSetClassDescr">section</ref> for
	    a more detailed explanation of how they are serialized for
	    export to disk.
	  </p>

	  <p>Retrieving the attributes is equally simple:
	  </p>

	  <verbatim>
>>> table.attrs.gath_date
'Wed, 06/12/2003 18:33'
>>> table.attrs.temperature
18.399999999999999
>>> table.attrs.temp_scale
'Celsius'
>>> detector._v_attrs.stuff
[5, (2.2999999999999998, 4.5), 'Integer and tuple']
	  </verbatim>

	  <p>You can probably guess how to delete attributes:
	  </p>

	  <verbatim>
>>> del table.attrs.gath_date
	  </verbatim>

	  <p>If you want to examine the current user attribute set
	  of <verb>/detector/table</verb>, you can print its
	  representation (try hitting the <verb>TAB</verb> key
	  twice if you are on a Unix Python console with the
	  <verb>rlcompleter</verb> module active):
	  </p>

	  <verbatim>
>>> table.attrs
/detector/readout (AttributeSet), 2 attributes:
   [temp_scale := 'Celsius',
    temperature := 18.399999999999999]
	  </verbatim>

	  <p>You can get a list of all attributes or only the user
	     or system attributes with the <verb>_f_list()</verb> method.
	  </p>

	  <verbatim>
>>> print table.attrs._f_list("all")
['CLASS', 'FIELD_0_NAME', 'FIELD_1_NAME', 'FIELD_2_NAME', 'FIELD_3_NAME',
 'FIELD_4_NAME', 'FIELD_5_NAME', 'FIELD_6_NAME', 'FIELD_7_NAME', 'NROWS',
 'TITLE', 'VERSION', 'temp_scale', 'temperature']
>>> print table.attrs._f_list("user")
['temp_scale', 'temperature']
>>> print table.attrs._f_list("sys")
['CLASS', 'FIELD_0_NAME', 'FIELD_1_NAME', 'FIELD_2_NAME', 'FIELD_3_NAME',
 'FIELD_4_NAME', 'FIELD_5_NAME', 'FIELD_6_NAME', 'FIELD_7_NAME', 'NROWS',
 'TITLE', 'VERSION']
	  </verbatim>

	  <p>You can also rename attributes:
	  </p>

	  <verbatim>
>>> table.attrs._f_rename("temp_scale","tempScale")
>>> print table.attrs._f_list()
['tempScale', 'temperature']
	  </verbatim>

	  <p>However, you can not set, delete or rename read-only
	    attributes:
	  </p>

	  <verbatim>
>>> table.attrs._f_rename("VERSION", "version")
Traceback (most recent call last):
  File "&gt;stdin>", line 1, in ?
  File "/home/falted/PyTables/pytables-0.7/tables/AttributeSet.py",
  line 249, in _f_rename
    raise AttributeError, \
AttributeError: Read-only attribute ('VERSION') cannot be renamed
	  </verbatim>

	  <p>If you would terminate your session now, you would be
	    able to use the <verb>h5ls</verb> command to read the
	    <verb>/detector/readout</verb> attributes from the file
	    written to disk:
	  </p>

	  <verbatim>
$ h5ls -vr tutorial1.h5/detector/readout
Opened "tutorial1.h5" with sec2 driver.
/detector/readout        Dataset {10/Inf}
    Attribute: CLASS     scalar
        Type:      6-byte null-terminated ASCII string
        Data:  "TABLE"
    Attribute: VERSION   scalar
        Type:      4-byte null-terminated ASCII string
        Data:  "2.0"
    Attribute: TITLE     scalar
        Type:      16-byte null-terminated ASCII string
        Data:  "Readout example"
    Attribute: FIELD_0_NAME scalar
        Type:      9-byte null-terminated ASCII string
        Data:  "ADCcount"
    Attribute: FIELD_1_NAME scalar
        Type:      9-byte null-terminated ASCII string
        Data:  "TDCcount"
    Attribute: FIELD_2_NAME scalar
        Type:      7-byte null-terminated ASCII string
        Data:  "energy"
    Attribute: FIELD_3_NAME scalar
        Type:      7-byte null-terminated ASCII string
        Data:  "grid_i"
    Attribute: FIELD_4_NAME scalar
        Type:      7-byte null-terminated ASCII string
        Data:  "grid_j"
    Attribute: FIELD_5_NAME scalar
        Type:      9-byte null-terminated ASCII string
        Data:  "idnumber"
    Attribute: FIELD_6_NAME scalar
        Type:      5-byte null-terminated ASCII string
        Data:  "name"
    Attribute: FIELD_7_NAME scalar
        Type:      9-byte null-terminated ASCII string
        Data:  "pressure"
    Attribute: tempScale scalar
        Type:      8-byte null-terminated ASCII string
        Data:  "Celsius"
    Attribute: temperature {1}
        Type:      native double
        Data:  18.4
    Attribute: NROWS     {1}
        Type:      native int
        Data:  10
    Location:  0:1:0:1952
    Links:     1
    Modified:  2003-07-24 13:59:19 CEST
    Chunks:    {2048} 96256 bytes
    Storage:   470 logical bytes, 96256 allocated bytes, 0.49% utilization
    Type:      struct {
                   "ADCcount"         +0    native unsigned short
                   "TDCcount"         +2    native unsigned char
                   "energy"           +3    native double
                   "grid_i"           +11   native int
                   "grid_j"           +15   native int
                   "idnumber"         +19   native long long
                   "name"             +27   16-byte null-terminated ASCII string
                   "pressure"         +43   native float
               } 47 bytes


	  </verbatim>


	  <p>Attributes are a useful mechanism to add persistent
	    (meta) information to your data.
	  </p>

	</subsection>

	<subsection>
	  <heading>Getting object metadata</heading>

	  <p>Each object in <verb>PyTables</verb> has
	    <em>metadata</em> information about the data in the
	    file. Normally this <em>meta-information</em> is accessible
	    through the node instance variables. Let's take a look at
	    some examples:
	  </p>

	  <verbatim>
>>> print "Object:", table
Object: /detector/readout Table(10,) 'Readout example'
>>> print "Table name:", table.name
Table name: readout
>>> print "Table title:", table.title
Table title: Readout example
>>> print "Number of rows in table:", table.nrows
Number of rows in table: 10
>>> print "Table variable names with their type and shape:"
Table variable names with their type and shape:
>>> for name in table.colnames:
...   print name, ':= %s, %s' % (table.coltypes[name], table.colshapes[name])
...
ADCcount := UInt16, 1
TDCcount := UInt8, 1
energy := Float64, 1
grid_i := Int32, 1
grid_j := Int32, 1
idnumber := Int64, 1
name := CharType, 1
pressure := Float32, 1
	  </verbatim>

	  <p>
	    Here, the <verb>name</verb>, <verb>title</verb>,
	    <verb>nrows</verb>, <verb>colnames</verb>,
	    <verb>coltypes</verb> and <verb>colshapes</verb>
	    attributes (see <ref
	    refid="TableInstanceVariablesDescr"></ref> for a complete
	    attribute list) of the <verb>Table</verb> object gives us quite
	    a bit of information about the table data.
	  </p>

	  <p>You can interactively retrieve general information about
	    the public objects in PyTables by printing their internal
	    doc strings:
	  </p>

	  <verbatim>
>>> print table.__doc__
Represent a table in the object tree.
    It provides methods to create new tables or open existing ones, as
    well as to write/read data to/from table objects over the
    file. A method is also provided to iterate over the rows without
    loading the entire table or column in memory.

    Data can be written or read both as Row instances or numarray
    (NumArray or RecArray) objects or NestedRecArray objects.

    Methods:

        __getitem__(key)
        __iter__()
        __setitem__(key, value)
        append(rows)
        flushRowsToIndex()
        iterrows(start, stop, step)
        itersequence(sequence)
        modifyRows(start, rows)
        modifyColumn(columns, names, [start] [, stop] [, step])
        modifyColumns(columns, names, [start] [, stop] [, step])
        read([start] [, stop] [, step] [, field [, flavor]])
        reIndex()
        reIndexDirty()
        removeRows(start [, stop])
        removeIndex(column)
        where(condition [, start] [, stop] [, step])
        whereAppend(dstTable, condition [, start] [, stop] [, step])
        getWhereList(condition [, flavor])

    Instance variables:

        description -- the metaobject describing this table
        row -- a reference to the Row object associated with this table
        nrows -- the number of rows in this table
        rowsize -- the size, in bytes, of each row
        cols -- accessor to the columns using a natural name schema
        colnames -- the field names for the table (tuple)
        coltypes -- the type class for the table fields (dictionary)
        colshapes -- the shapes for the table fields (dictionary)
        colindexed -- whether the table fields are indexed (dictionary)
        indexed -- whether or not some field in Table is indexed
        indexprops -- properties of an indexed Table

	  </verbatim>

	  <p>The <verb>help</verb> function is also a handy way to see
	    <verb>PyTables</verb> reference documentation online.  Try
	    it yourself with other object docs:
	  </p>

	  <verbatim>
>>> help(table.__class__)
>>> help(table.removeRows)
	  </verbatim>

	  <p>To examine metadata in the <em>/columns/pressure</em>
	    <verb>Array</verb> object:
	  </p>

	  <verbatim>
>>> pressureObject = h5file.getNode("/columns", "pressure")
>>> print "Info on the object:", repr(pressureObject)
Info on the object: /columns/pressure (Array(3,)) 'Pressure column selection'
  type = Float64
  itemsize = 8
  flavor = 'numarray'
  byteorder = 'little'
>>> print "  shape: ==>", pressureObject.shape
  shape: ==> (3,)
>>> print "  title: ==>", pressureObject.title
  title: ==> Pressure column selection
>>> print "  type: ==>", pressureObject.type
  type: ==> Float64
	  </verbatim>

	  <p>Observe that we have used the <verb>getNode()</verb>
	    method of the <verb>File</verb> class to access a node in
	    the tree, instead of the natural naming method. Both are
	    useful, and depending on the context you will prefer one
	    or the other. <verb>getNode()</verb> has the advantage
	    that it can get a node from the pathname string (as in
	    this example) and can also act as a filter to show only
	    nodes in a particular location that are instances of class
	    <em>classname</em>. In general, however, I consider
	    natural naming to be more elegant and easier to use,
	    especially if you are using the name completion capability
	    present in interactive console. Try this powerful
	    combination of natural naming and completion capabilities
	    present in most Python consoles, and see how pleasant it
	    is to browse the object tree (well, as pleasant as such an
	    activity can be).
	  </p>
	  <p>If you look at the <verb>type</verb> attribute of the
	    <verb>pressureObject</verb> object, you can verify that it
	    is a "<visual markup="bf">Float64</visual>" array.  By
	    looking at its <verb>shape</verb> attribute, you can
	    deduce that the array on disk is unidimensional and has 3
	    elements. See <ref
	    refid="ArrayClassInstanceVariables"></ref> or the internal
	    doc strings for the complete <verb>Array</verb> attribute
	    list.
	  </p>
	</subsection>

	<subsection>
	  <heading>Reading data from <visual
	  markup="tt">Array</visual> objects</heading>

	  <p>Once you have found the desired <verb>Array</verb>, use
	    the <verb>read()</verb> method of the <verb>Array</verb>
	    object to retrieve its data:</p>

	  <verbatim>
>>> pressureArray = pressureObject.read()
>>> pressureArray
array([ 25.,  36.,  49.])
>>> print "pressureArray is an object of type:", type(pressureArray)
pressureArray is an object of type: &lt;class 'numarray.numarraycore.NumArray'>
>>> nameArray = h5file.root.columns.name.read()
>>> nameArray
['Particle:      5', 'Particle:      6', 'Particle:      7']
>>> print "nameArray is an object of type:", type(nameArray)
nameArray is an object of type: &lt;type 'list'>
>>>
>>> print "Data on arrays nameArray and pressureArray:"
Data on arrays nameArray and pressureArray:
>>> for i in range(pressureObject.shape[0]):
...   print nameArray[i], "-->", pressureArray[i]
...
Particle:      5 --> 25.0
Particle:      6 --> 36.0
Particle:      7 --> 49.0
>>> pressureObject.name
'pressure'
	  </verbatim>

	  <p>You can see that the <verb>read()</verb> method (see <ref
	    refid="readArrayDescr">section</ref>) returns an authentic
	    <verb>numarray</verb> object for the
	    <verb>pressureObject</verb> instance by looking at the
	    output of the <verb>type()</verb> call.  A
	    <verb>read()</verb> of the <verb>nameObject</verb> object
	    instance returns a native Python list (of strings).  The
	    type of the object saved is stored as an HDF5 attribute
	    (named <verb>FLAVOR</verb>) for objects on disk. This
	    attribute is then read as <verb>Array</verb>
	    meta-information (accessible through in the
	    <verb>Array.attrs.FLAVOR</verb> variable), enabling the
	    read array to be converted into the original object. This
	    provides a means to save a large variety of objects as
	    arrays with the guarantee that you will be able to later
	    recover them in their original form. See <ref
	    refid="createArrayDescr">section</ref> for a complete list
	    of supported objects for the <verb>Array</verb> object
	    class.
	  </p>

	</subsection>

      </section> <!-- Browsing the object tree -->

      <section>
	<heading>Commiting data to tables and arrays</heading>

	<p>We have seen how to create tables and arrays and how to
	  browse both data and metadata in the object tree. Let's
	  examine more closely now one of the most powerful
	  capabilities of <verb>PyTables</verb>, namely, how to modify
	  already created tables and arrays<footnote>Appending data to
	  arrays is also supported, but you need to create special
	  objects called <verb>EArray</verb> (see <ref
	  refid="EArrayClassDescr"></ref> for more info).</footnote>.
	</p>

	<subsection>
	  <heading>Appending data to an existing table</heading>

	  <p>Now, let's have a look at how we can add records to an
	    existing table on disk. Let's use our well-known
	    <em>readout</em> <verb>Table</verb> object and
	    append some new values to it:
	  </p>

	  <verbatim>
>>> table = h5file.root.detector.readout
>>> particle = table.row
>>> for i in xrange(10, 15):
...     particle['name']  = 'Particle: %6d' % (i)
...     particle['TDCcount'] = i % 256
...     particle['ADCcount'] = (i * 256) % (1 &lt;&lt; 16)
...     particle['grid_i'] = i
...     particle['grid_j'] = 10 - i
...     particle['pressure'] = float(i*i)
...     particle['energy'] = float(particle['pressure'] ** 4)
...     particle['idnumber'] = i * (2 ** 34)
...     particle.append()
...
>>> table.flush()
	  </verbatim>

	  <p>It's the same method we used to fill a new
	    table. <verb>PyTables</verb> knows that this table is on
	    disk, and when you add new records, they are appended to
	    the end of the table<footnote>Note that you can append not
	    only scalar values to tables, but also fully
	    multidimensional array objects.</footnote>.
	  </p>
	  <p>
	    If you look carefully at the code you will see that we
	    have used the <verb>table.row</verb> attribute to
	    create a table row and fill it with the new values.
	    Each time that its <verb>append()</verb> method is called,
	    the actual row is committed to the output buffer and the
	    row pointer is incremented to point to the next table
	    record. When the buffer is full, the data is saved on
	    disk, and the buffer is reused again for the next cycle.
	  </p>

	  <p><visual markup="bf">Caveat emptor</visual>: Do not
	    forget to always call the <verb>.flush()</verb>
	    method after a write operation, or else your tables
	    will not be updated!</p>

	  <p>Let's have a look at some rows in the modified table and
	    verify that our new data has been appended:
	  </p>

	  <verbatim>
>>> for r in table.iterrows():
...     print "%-16s | %11.1f | %11.4g | %6d | %6d | %8d |" % \
...        (r['name'], r['pressure'], r['energy'], r['grid_i'], r['grid_j'],
...         r['TDCcount'])
...
...
Particle:      0 |         0.0 |           0 |      0 |     10 |        0 |
Particle:      1 |         1.0 |           1 |      1 |      9 |        1 |
Particle:      2 |         4.0 |         256 |      2 |      8 |        2 |
Particle:      3 |         9.0 |        6561 |      3 |      7 |        3 |
Particle:      4 |        16.0 |   6.554e+04 |      4 |      6 |        4 |
Particle:      5 |        25.0 |   3.906e+05 |      5 |      5 |        5 |
Particle:      6 |        36.0 |    1.68e+06 |      6 |      4 |        6 |
Particle:      7 |        49.0 |   5.765e+06 |      7 |      3 |        7 |
Particle:      8 |        64.0 |   1.678e+07 |      8 |      2 |        8 |
Particle:      9 |        81.0 |   4.305e+07 |      9 |      1 |        9 |
Particle:     10 |       100.0 |       1e+08 |     10 |      0 |       10 |
Particle:     11 |       121.0 |   2.144e+08 |     11 |     -1 |       11 |
Particle:     12 |       144.0 |     4.3e+08 |     12 |     -2 |       12 |
Particle:     13 |       169.0 |   8.157e+08 |     13 |     -3 |       13 |
Particle:     14 |       196.0 |   1.476e+09 |     14 |     -4 |       14 |
	  </verbatim>

	</subsection>

	<subsection id="modifyingTableUsage">
	  <heading>Modifying data in tables</heading>

	  <p>Ok, until now, we've been only reading and writing
	    (appending) values to our tables. But there are times that
	    you need to modify your data once you have saved it on
	    disk (this is specially true when you need to modify the
	    real world data to adapt your goals ;). Let's see how we
	    can modify the values that were saved in our existing
	    tables. We will start modifying single cells in the first
	    row of the <verb>Particle</verb> table:
	  </p>

	  <verbatim>
>>> print "Before modif-->", table[0]
Before modif--> (0, 0, 0.0, 0, 10, 0L, 'Particle:      0', 0.0)
>>> table.cols.TDCcount[0] = 1
>>> print "After modif first row of ADCcount-->", table[0]
After modif first row of ADCcount--> (0, 1, 0.0, 0, 10, 0L, 'Particle: 0', 0.0)
>>> table.cols.energy[0] = 2
>>> print "After modif first row of energy-->", table[0]
After modif first row of energy--> (0, 1, 2.0, 0, 10, 0L, 'Particle: 0', 0.0)

	  </verbatim>

	  <p>We can modify complete ranges of columns as well:</p>

	  <verbatim>
>>> table.cols.TDCcount[2:5] = [2,3,4]
>>> print "After modifying slice [2:5] of ADCcount-->", table[0:5]
After modifying slice [2:5] of ADCcount--> RecArray[
(0, 1, 2.0, 0, 10, 0L, 'Particle:      0', 0.0),
(256, 1, 1.0, 1, 9, 17179869184L, 'Particle:      1', 1.0),
(512, 2, 256.0, 2, 8, 34359738368L, 'Particle:      2', 4.0),
(768, 3, 6561.0, 3, 7, 51539607552L, 'Particle:      3', 9.0),
(1024, 4, 65536.0, 4, 6, 68719476736L, 'Particle:      4', 16.0)
]
>>> table.cols.energy[1:9:3] = [2,3,4]
>>> print "After modifying slice [1:9:3] of energy-->", table[0:9]
After modifying slice [1:9:3] of energy--> RecArray[
(0, 1, 2.0, 0, 10, 0L, 'Particle:      0', 0.0),
(256, 1, 2.0, 1, 9, 17179869184L, 'Particle:      1', 1.0),
(512, 2, 256.0, 2, 8, 34359738368L, 'Particle:      2', 4.0),
(768, 3, 6561.0, 3, 7, 51539607552L, 'Particle:      3', 9.0),
(1024, 4, 3.0, 4, 6, 68719476736L, 'Particle:      4', 16.0),
(2560, 10, 100000000.0, 10, 0, 171798691840L, 'Particle:     10', 100.0),
(2816, 11, 214358881.0, 11, -1, 188978561024L, 'Particle:     11', 121.0),
(3072, 12, 4.0, 12, -2, 206158430208L, 'Particle:     12', 144.0),
(3328, 13, 815730721.0, 13, -3, 223338299392L, 'Particle:     13', 169.0)
]
	  </verbatim>

	  <p>Check that the values has been correctly
	    modified!. <visual markup="bf">Hint:</visual> remember
	    that column <verb>TDCcount</verb> is the first one, and
	    that <verb>energy</verb> is the third. Look for more info
	    on modifying columns in <ref
	    refid="Column.__setitem__">section</ref>.
	  </p>

	  <p>PyTables also let's you modify complete sets of rows at
	    the same time. As a demonstration of these capability, see
	    the next example:
	  </p>

	  <verbatim>
>>> table.modifyRows(start=1, step=3,
...                  rows=[(1, 2, 3.0, 4, 5, 6L, 'Particle:   None', 8.0),
...                        (2, 4, 6.0, 8, 10, 12L, 'Particle: None*2', 16.0)])
2
>>> print "After modifying the complete third row-->", table[0:5]
After modifying the complete third row--> RecArray[
(0, 1, 2.0, 0, 10, 0L, 'Particle:      0', 0.0),
(1, 2, 3.0, 4, 5, 6L, 'Particle:   None', 8.0),
(512, 2, 256.0, 2, 8, 34359738368L, 'Particle:      2', 4.0),
(768, 3, 6561.0, 3, 7, 51539607552L, 'Particle:      3', 9.0),
(2, 4, 6.0, 8, 10, 12L, 'Particle: None*2', 16.0)
]
	  </verbatim>

	  <p>As you can see, the <verb>modifyRows</verb> call has
	    modified the rows second and fifth, and it returned the
	    number of modified rows.
	  </p>

	  <p>Apart of <verb>modifyRows</verb>, there exists another
	    method, called <verb>modifyColumn</verb> to modify
	    specific columns as well. Please, check sections <ref
	    refid="Table.modifyRows"></ref> and <ref
	    refid="Table.modifyColumn"></ref> for a more in-depth
	    description of them.
	  </p>

	  <p>Finally, it exists another way of modifying tables that
	    is generally more handy than the described above. This new
	    way uses the method <verb>update()</verb> (see <ref
	    refid="RowMethods">section</ref>) of the <verb>Row</verb>
	    instance that is attached to every table, so it is meant
	    to be used in table iterators. Look at the next example:
	  </p>

	  <verbatim><![CDATA[
>>> for row in table.where(table.cols.TDCcount <= 2):
...    row['energy'] = row['TDCcount']*2
...    row.update()
...
>>> print "After modifying energy column (where TDCcount <=2)-->", table[0:4]
After modifying energy column (where TDCcount <=2)--> NestedRecArray[
(0, 1, 2.0, 0, 10, 0L, 'Particle:      0', 0.0),
(1, 2, 4.0, 4, 5, 6L, 'Particle:   None', 8.0),
(512, 2, 4.0, 2, 8, 34359738368L, 'Particle:      2', 4.0),
(768, 3, 6561.0, 3, 7, 51539607552L, 'Particle:      3', 9.0)
]
]]>
	  </verbatim>

	  <p><visual markup="bf">Note:</visual>The authors find this
	    way of updating tables (i.e. using
	    <verb>Row.update()</verb>) to be both convenient and
	    efficient. Please, make sure to use it extensively.
	  </p>

	</subsection>

	<subsection id="modifyingArrayUsage">
	  <heading>Modifying data in arrays</heading>

	  <p>We are going now to see how to modify data in array
	    objects. The basic way to do this is through the use of
	    <verb>__setitem__</verb> special method (see <ref
	    refid="Array.__setitem__"></ref>). Let's see at how modify
	    data on the <verb>pressureObject</verb> array:
	  </p>

	  <verbatim>
>>> print "Before modif-->", pressureObject[:]
Before modif--> [ 25.  36.  49.]
>>> pressureObject[0] = 2
>>> print "First modif-->", pressureObject[:]
First modif--> [  2.  36.  49.]
>>> pressureObject[1:3] = [2.1, 3.5]
>>> print "Second modif-->", pressureObject[:]
Second modif--> [ 2.   2.1  3.5]
>>> pressureObject[::2] = [1,2]
>>> print "Third modif-->", pressureObject[:]
Third modif--> [ 1.   2.1  2. ]

	  </verbatim>

	  <p>So, in general, you can use any combination of
	    (multidimensional) extended slicing<footnote>With the sole
	    exception that you cannot use negative values for
	    <verb>step</verb>.</footnote> to refer to indexes that you
	    want to modify. See <ref
	    refid="Array.__getitem__">section</ref> for more examples
	    on how to use extended slicing in PyTables objects.</p>

	  <p>Similarly, with and array of strings:</p>

	  <verbatim>
>>> print "Before modif-->", nameObject[:]
Before modif--> ['Particle:      5', 'Particle:      6', 'Particle:      7']
>>> nameObject[0] = 'Particle:   None'
>>> print "First modif-->", nameObject[:]
First modif--> ['Particle:   None', 'Particle:      6', 'Particle:      7']
>>> nameObject[1:3] = ['Particle:      0', 'Particle:      1']
>>> print "Second modif-->", nameObject[:]
Second modif--> ['Particle:   None', 'Particle:      0', 'Particle:      1']
>>> nameObject[::2] = ['Particle:     -3', 'Particle:     -5']
>>> print "Third modif-->", nameObject[:]
Third modif--> ['Particle:     -3', 'Particle:      0', 'Particle:     -5']

	  </verbatim>
	</subsection>

	<subsection>
	  <heading>And finally... how to delete rows from a table</heading>

	  <p>We'll finish this tutorial by deleting some
	    rows from the table we have. Suppose that we want to
	    delete the the 5th to 9th rows (inclusive):
	  </p>

	  <verbatim>
>>> table.removeRows(5,10)
5
	  </verbatim>

	  <p><verb>removeRows(start, stop)</verb> (<ref
	      refid="removeRowsDescr">see</ref>) deletes the rows in
	      the range (start, stop). It returns the number of rows
	      effectively removed.
	  </p>

	  <!-- SGP:  It would be cool if you could delete all the rows
	  returned by a selection. -->
	  <p>We have reached the end of this first tutorial.
	    Don't forget to close the file when you finish:
	  </p>

	  <verbatim>
>>> h5file.close()
>>> ^D
$
	  </verbatim>

	  <p>In <ref refid="tutorial1-2-tableview">figure</ref> you
	    can see a graphical view of the <verb>PyTables</verb> file
	    with the datasets we have just created. In <ref
	    refid="tutorial1-general">figure</ref> are displayed the
	    general properties of the table
	    <verb>/detector/readout</verb>.
	  </p>

	  <figure id="tutorial1-2-tableview">
	    <graphics file="tutorial1-2-tableview" scale="0.5" kind="bitmap">
	    </graphics>
	    <caption>The final version of the data file for tutorial 1.
	    </caption>
	  </figure>

	  <figure id="tutorial1-general">
	    <graphics file="tutorial1-general" scale="0.5" kind="bitmap">
	    </graphics>
	    <caption>General properties of the <visual
	    markup="tt">/detector/readout</visual> table.
	    </caption>
	  </figure>

	</subsection>

      </section> <!-- Commiting data -->

      <section id="secondExample">
	<heading>Multidimensional table cells and automatic
	  sanity checks</heading>

	<p>Now it's time for a more real-life example (i.e. with
	  errors in the code). We will create two groups that branch
	  directly from the <verb>root</verb> node,
	  <verb>Particles</verb> and <verb>Events</verb>. Then, we
	  will put three tables in each group. In
	  <verb>Particles</verb> we will put tables based on the
	  <verb>Particle</verb> descriptor and in <verb>Events</verb>,
	  the tables based the <verb>Event</verb> descriptor.
	</p>
	<p>
	  Afterwards, we will provision the tables with a number of
	  records. Finally, we will read the newly-created table
	  <verb>/Events/TEvent3</verb> and select some values from it,
	  using a comprehension list.
	</p>
	<p>Look at the next script (you can find it in
	  <verb>examples/tutorial2.py</verb>). It appears to do all of
	  the above, but it contains some small bugs. Note that this
	  <verb>Particle</verb> class is not directly related to the
	  one defined in last tutorial; this class is simpler (note,
	  however, the <em>multidimensional</em> columns called
	  <verb>pressure</verb> and <verb>temperature</verb>).
	</p>
	<p>We also introduce a new manner to describe a <verb>Table</verb>
	  as a dictionary, as you can see in the <verb>Event</verb>
	  description. See section <ref
	  refid="createTableDescr"></ref> about the different kinds of
	  descriptor objects that can be passed to the
	  <verb>createTable()</verb> method.
	</p>

	<verbatim>
from numarray import *
from tables import *

# Describe a particle record
class Particle(IsDescription):
    name        = StringCol(length=16) # 16-character String
    lati        = IntCol()             # integer
    longi       = IntCol()             # integer
    pressure    = Float32Col(shape=(2,3)) # array of floats (single-precision)
    temperature = FloatCol(shape=(2,3))   # array of doubles (double-precision)

# Another way to describe the columns of a table
Event = {
    "name"        : StringCol(length=16),
    "lati"        : IntCol(),
    "longi"       : IntCol(),
    "pressure"    : Float32Col(shape=(2,3)),
    "temperature" : FloatCol(shape=(2,3)),
    }

# Open a file in "w"rite mode
fileh = openFile("tutorial2.h5", mode = "w")
# Get the HDF5 root group
root = fileh.root
# Create the groups:
for groupname in ("Particles", "Events"):
    group = fileh.createGroup(root, groupname)
# Now, create and fill the tables in the Particles group
gparticles = root.Particles
# Create 3 new tables
for tablename in ("TParticle1", "TParticle2", "TParticle3"):
    # Create a table
    table = fileh.createTable("/Particles", tablename, Particle,
                           "Particles: "+tablename)
    # Get the record object associated with the table:
    particle = table.row
    # Fill the table with data for 257 particles
    for i in xrange(257):
        # First, assign the values to the Particle record
        particle['name'] = 'Particle: %6d' % (i)
        particle['lati'] = i
        particle['longi'] = 10 - i
        ########### Detectable errors start here. Play with them!
        particle['pressure'] = array(i*arange(2*3), shape=(2,4))  # Incorrect
        #particle['pressure'] = array(i*arange(2*3), shape=(2,3))  # Correct
        ########### End of errors
        particle['temperature'] = (i**2)     # Broadcasting
        # This injects the Record values
        particle.append()
    # Flush the table buffers
    table.flush()

# Now Events:
for tablename in ("TEvent1", "TEvent2", "TEvent3"):
    # Create a table in the Events group
    table = fileh.createTable(root.Events, tablename, Event,
                           "Events: "+tablename)
    # Get the record object associated with the table:
    event = table.row
    # Fill the table with data on 257 events
    for i in xrange(257):
        # First, assign the values to the Event record
        event['name']  = 'Event: %6d' % (i)
        event['TDCcount'] = i % (1&lt;&lt;8)   # Correct range
        ########### Detectable errors start here. Play with them!
        #event['xcoord'] = float(i**2)   # Correct spelling
        event['xcoor'] = float(i**2)     # Wrong spelling
        event['ADCcount'] = i * 2        # Correct type
        #event['ADCcount'] = "sss"          # Wrong type
        ########### End of errors
        event['ycoord'] = float(i)**4
        # This injects the Record values
        event.append()

    # Flush the buffers
    table.flush()

# Read the records from table "/Events/TEvent3" and select some
table = root.Events.TEvent3
e = [ p['TDCcount'] for p in table
      if p['ADCcount'] &lt; 20 and 4 &lt;= p['TDCcount'] &lt; 15 ]
print "Last record ==>", p
print "Selected values ==>", e
print "Total selected records ==> ", len(e)
# Finally, close the file (this also will flush all the remaining buffers)
fileh.close()
	</verbatim>

	<subsection>
	  <heading>Shape checking</heading>

	  <p>If you look at the code carefully, you'll see that it
	    won't work. You will get the following error:</p>

	<verbatim>
$ python tutorial2.py
Traceback (most recent call last):
  File "tutorial2.py", line 53, in ?
    particle['pressure'] = array(i*arange(2*3), shape=(2,4))  # Incorrect
  File  "/usr/local/lib/python2.2/site-packages/numarray/numarraycore.py",
 line 281, in array
  a.setshape(shape)
  File "/usr/local/lib/python2.2/site-packages/numarray/generic.py",
 line 530, in setshape
    raise ValueError("New shape is not consistent with the old shape")
ValueError: New shape is not consistent with the old shape
	</verbatim>

	  <p>This error indicates that you are trying to assign an
	    array with an incompatible shape to a table cell. Looking
	    at the source, we see that we were trying to assign an
	    array of shape <verb>(2,4)</verb> to a
	    <verb>pressure</verb> element, which was defined with the
	    shape <verb>(2,3)</verb>.
	  </p>
	  <p>In general, these kinds of operations are forbidden, with
	    one valid exception: when you assign a <em>scalar</em>
	    value to a multidimensional column cell, all the cell
	    elements are populated with the value of the scalar. For
	    example:
	  </p>

	  <verbatim>
        particle['temperature'] = (i**2)    # Broadcasting
	  </verbatim>

	  <p>The value <verb>i**2</verb> is assigned to all the
	    elements of the <verb>temperature</verb> table cell. This
	    capability is provided by the <verb>numarray</verb>
	    package and is known as <em>broadcasting</em>.
	  </p>

	</subsection>

	<subsection>
	  <heading>Field name checking</heading>

	  <p>After fixing the previous error and rerunning the
	  program, we encounter another error:
	  </p>
	  <verbatim>
$ python tutorial2.py
Traceback (most recent call last):
  File "tutorial2.py", line 74, in ?
    event['xcoor'] = float(i**2)     # Wrong spelling
  File "src/hdf5Extension.pyx",
 line 1812, in hdf5Extension.Row.__setitem__
    raise KeyError, "Error setting \"%s\" field.\n %s" % \
KeyError: Error setting "xcoor" field.
 Error was: "exceptions.KeyError: xcoor"
	  </verbatim>

	  <p>This error indicates that we are attempting to assign a
	    value to a non-existent field in the <em>event</em> table
	    object. By looking carefully at the <verb>Event</verb>
	    class attributes, we see that we misspelled the
	    <verb>xcoord</verb> field (we wrote <verb>xcoor</verb>
	    instead). This is unusual behavior for Python, as normally
	    when you assign a value to a non-existent instance
	    variable, Python creates a new variable with that
	    name. Such a feature can be dangerous when dealing with an
	    object that contains a fixed list of field names. PyTables
	    checks that the field exists and raises a
	    <verb>KeyError</verb> if the check fails.
	  </p>

	</subsection>

	<subsection>
	  <heading>Data type checking</heading>

	  <p>Finally, in order to test type checking, we will change
	    the next line:
	  </p>
	  <verbatim>
	    event.ADCcount = i * 2        # Correct type
	  </verbatim>

	  <p>to read:</p>

	  <verbatim>
	    event.ADCcount = "sss"          # Wrong type
	  </verbatim>

	  <p>This modification will cause the following <verb>TypeError</verb>
	    exception to be raised when the script is executed:
	  </p>

	  <verbatim>
$ python tutorial2.py
Traceback (most recent call last):
  File "tutorial2.py", line 76, in ?
    event['ADCcount'] = "sss"          # Wrong type
  File "src/hdf5Extension.pyx",
 line 1812, in hdf5Extension.Row.__setitem__
    raise KeyError, "Error setting \"%s\" field.\n %s" % \
KeyError: Error setting "ADCcount" field.
 Error was: "exceptions.TypeError: NA_setFromPythonScalar: bad value type."
	  </verbatim>

	  <p>You can see the structure created with this (corrected)
	    script in <ref refid="tutorial2-tableview">figure</ref>.
	    In particular, note the multidimensional column cells in
	    table <verb>/Particles/TParticle2</verb>.
	  </p>

	  <figure id="tutorial2-tableview">
	    <graphics file="tutorial2-tableview" scale="0.50" kind="bitmap">
	    </graphics>
	    <caption>Table hierarchy for tutorial 2.</caption>
	  </figure>

	</subsection>
      </section>
      <section id="ThirdExample">
	<heading>Exercising the Undo/Redo feature</heading>

	<p><verb>PyTables</verb> has integrated support for undoing
	  and/or redoing actions. This functionality lets you put
	  marks in specific places of your hierarchy manipulation operations, so that
	  you can make your HDF5 file pop back (<em>undo</em>) to a
	  specific mark (for example for inspecting how your hierarchy
	  looked at that point). You can also go forward to a more
	  recent marker (<em>redo</em>). You can even do jumps to the
	  marker you want using just one instruction as we will see
	  shortly.
	</p>

	<p>You can undo/redo all the operations that are related to
	  object tree management, like creating, deleting, moving
	  or renaming nodes (or complete sub-hierarchies) inside a
	  given object tree. You can also undo/redo operations
	  (i.e. creation, deletion or modification) of persistent node
	  attributes. However, when actions include <em>internal</em>
	  modifications of datasets (that includes
	  <verb>Table.append</verb>, <verb>Table.modifyRows</verb> or
	  <verb>Table.removeRows</verb> among others), they cannot be
	  undone/redone currently.
	</p>

	<p>This capability can be useful in many situations, like for
	  example when doing simulations with multiple branches. When
	  you have to choose a path to follow in such a situation, you
	  can put a mark there and, if the simulation is not going
	  well, you can go back to that mark and start another path.
	  Other possible application is defining coarse-grained operations
	  which operate in a transactional-like way,
	  i.e. which return the database to its previous state
	  if the operation finds some kind of problem while running.
	  You can probably devise many other
	  scenarios where the Undo/Redo feature can be useful to you
	  <footnote>You can even <em>hide</em> nodes temporarily.
	    Will you be able to find out how?
	  </footnote>.
	</p>

	<subsection>
	  <heading>A basic example</heading>

	  <p>In this section, we are going to show the basic behavior
	    of the Undo/Redo feature. You can find the code used in
	    this example in <verb>examples/tutorial3-1.py</verb>. A
	    somewhat more complex example will be explained in the
	    next section.
	  </p>

	  <p>First, let's create a file:</p>

	  <verbatim>
>>> import tables
>>> fileh = tables.openFile("tutorial3-1.h5", "w", title="Undo/Redo demo 1")
	  </verbatim>

	  <p>And now, activate the Undo/Redo feature with the method
	    <verb>enableUndo</verb> (see <pageref
	    refid="File.enableUndo">page</pageref>) of <verb>File</verb>:
	  </p>

	  <verbatim>
>>> fileh.enableUndo()
	  </verbatim>

	  <p>From now on, all our actions will be logged internally by
	    <verb>PyTables</verb>. Now, we are going to create a node
	    (in this case an <verb>Array</verb> object):
	  </p>

	  <verbatim>
>>> one = fileh.createArray('/', 'anarray', [3,4], "An array")
	  </verbatim>

	  <p>Now, mark this point:
	  </p>

	  <verbatim>
>>> fileh.mark()
1
>>>
	  </verbatim>

	  <p>We have marked the current point in the sequence of
	    actions. In addition, the <verb>mark()</verb> method has
	    returned the identifier assigned to this new mark, that is 1
	    (mark #0 is reserved for the implicit mark at the
	    beginning of the action log). In the next section we will
	    see that you can also assign a <em>name</em> to a mark
	    (see <pageref refid="File.mark">page</pageref> for more
	    info on <verb>mark()</verb>). Now, we are going to create
	    another array:
	  </p>

	  <verbatim>
>>> another = fileh.createArray('/', 'anotherarray', [4,5], "Another array")
	  </verbatim>

	  <p>Right. Now, we can start doing funny things. Let's say
	    that we want to pop back to the previous mark (that whose
	    value was 1, do you remember?). Let's introduce the
	    <verb>undo()</verb> method (see <pageref
	    refid="File.undo">page</pageref>):
	  </p>

	  <verbatim>
>>> fileh.undo()
>>>
	  </verbatim>

	  <p>Fine, what do you think it happened? Well, let's
	    have a look at the object tree:
	  </p>

	  <verbatim>
>>> print fileh
do-undo1.h5 (File) 'Undo/Redo demo 1'
Last modif.: 'Fri Mar  4 20:22:28 2005'
Object Tree:
/ (RootGroup) 'Undo/Redo demo 1'
/anarray (Array(2,)) 'An array'

>>>
	  </verbatim>

	  <p>What happened with the <verb>/anotherarray</verb> node
	    we've just created? You guess it, it has disappeared
	    because it was created <em>after</em> the mark 1. If you
	    are curious enough you may well ask where it has
	    gone. Well, it has not been deleted completely; it has
	    been just moved into a special, hidden, group of PyTables
	    that renders it invisible and waiting for a chance to
	    be reborn.
	  </p>

	  <p>Now, unwind once more, and look at the object tree:
	  </p>

	  <verbatim>
>>> fileh.undo()
>>> print fileh
do-undo1.h5 (File) 'Undo/Redo demo 1'
Last modif.: 'Fri Mar  4 20:22:28 2005'
Object Tree:
/ (RootGroup) 'Undo/Redo demo 1'

>>>
	  </verbatim>

	  <p>Oops, <verb>/anarray</verb> has disappeared as
	    well!. Don't worry, it will revisit us very shortly. So,
	    you might be somewhat lost right now; in which mark are
	    we?. Let's ask the <verb>getCurrentMark()</verb> method
	    (see <pageref refid="File.getCurrentMark">page</pageref>)
	    in the file handler:
	  </p>

	  <verbatim>
>>> print fileh.getCurrentMark()
0
	  </verbatim>

	  <p>So we are at mark #0, remember? Mark #0 is an implicit
	    mark that is created when you start the log of actions
	    when calling <verb>File.enableUndo()</verb>. Fine, but you
	    are missing your too-young-to-die arrays. What can we do
	    about that? <verb>File.redo()</verb> (see <pageref
	    refid="File.redo">page</pageref>) to the rescue:
	  </p>

	  <verbatim>
>>> fileh.redo()
>>> print fileh
do-undo1.h5 (File) 'Undo/Redo demo 1'
Last modif.: 'Fri Mar  4 20:22:28 2005'
Object Tree:
/ (RootGroup) 'Undo/Redo demo 1'
/anarray (Array(2,)) 'An array'

>>>
	  </verbatim>

	  <p>Great! The <verb>/anarray</verb> array has come into life
	    again. Just check that it is alive and well:
	  </p>

	  <verbatim>
>>> fileh.root.anarray.read()
[3, 4]
>>> fileh.root.anarray.title
'An array'
>>>
	  </verbatim>

	  <p>Well, it looks pretty similar than in its previous life;
	    what's more, it is exactly the same object!:
	  </p>

	  <verbatim>
>>> fileh.root.anarray is one
True
	  </verbatim>

	  <p>It just was moved to the the hidden group and back again,
	    but that's all! That's kind of fun, so we are going to do
	    the same with <verb>/anotherarray</verb>:
	  </p>

	  <verbatim>
>>> fileh.redo()
>>> print fileh
do-undo1.h5 (File) 'Undo/Redo demo 1'
Last modif.: 'Fri Mar  4 20:22:28 2005'
Object Tree:
/ (RootGroup) 'Undo/Redo demo 1'
/anarray (Array(2,)) 'An array'
/anotherarray (Array(2,)) 'Another array'

>>>
	  </verbatim>

	  <p>Welcome back, <verb>/anotherarray</verb>! Just a couple
	    of sanity checks:
	  </p>

	  <verbatim>
>>> assert fileh.root.anotherarray.read() == [4,5]
>>> assert fileh.root.anotherarray.title == "Another array"
>>> fileh.root.anotherarray is another
True
	  </verbatim>

	  <p>Nice, you managed to turn your data back into life.
	    Congratulations! But wait, do not forget to close
	    your action log when you don't need this feature anymore:
	  </p>

	  <verbatim>
>>> fileh.disableUndo()
	  </verbatim>

	  <p>That will allow you to continue working with your data
	    without actually requiring <verb>PyTables</verb> to keep
	    track of all your actions, and more importantly, allowing
	    your objects to die completely if they have to, not
	    requiring to keep them anywhere, and hence saving process
	    time and space in your database file.
	  </p>

	</subsection>

	<subsection>
	  <heading>A more complete example</heading>

	  <p>Now, time for a somewhat more sophisticated demonstration
	    of the Undo/Redo feature. In it, several marks will be set
	    in different parts of the code flow and we will see how to
	    jump between these marks with just one method call. You
	    can find the code used in this example in
	    <verb>examples/tutorial3-2.py</verb>
	  </p>

	  <p>Let's introduce the first part of the code:
	  </p>

	  <verbatim>
import tables

# Create an HDF5 file
fileh = tables.openFile('tutorial3-2.h5', 'w', title='Undo/Redo demo 2')

         #'-**-**-**-**-**-**- enable undo/redo log  -**-**-**-**-**-**-**-'
fileh.enableUndo()

# Start undoable operations
fileh.createArray('/', 'otherarray1', [3,4], 'Another array 1')
fileh.createGroup('/', 'agroup', 'Group 1')
# Create a 'first' mark
fileh.mark('first')
fileh.createArray('/agroup', 'otherarray2', [4,5], 'Another array 2')
fileh.createGroup('/agroup', 'agroup2', 'Group 2')
# Create a 'second' mark
fileh.mark('second')
fileh.createArray('/agroup/agroup2', 'otherarray3', [5,6], 'Another array 3')
# Create a 'third' mark
fileh.mark('third')
fileh.createArray('/', 'otherarray4', [6,7], 'Another array 4')
fileh.createArray('/agroup', 'otherarray5', [7,8], 'Another array 5')
	  </verbatim>

	  <p>You can see how we have set several marks interspersed in
	    the code flow, representing different states of the
	    database. Also, note that we have assigned <em>names</em>
	    to these marks, namely <verb>'first'</verb>,
	    <verb>'second'</verb> and <verb>'third'</verb>.
	  </p>

	  <p>Now, start doing some jumps back and forth in the states
	    of the database:
	  </p>

	  <verbatim>
# Now go to mark 'first'
fileh.goto('first')
assert '/otherarray1' in fileh
assert '/agroup' in fileh
assert '/agroup/agroup2' not in fileh
assert '/agroup/otherarray2' not in fileh
assert '/agroup/agroup2/otherarray3' not in fileh
assert '/otherarray4' not in fileh
assert '/agroup/otherarray5' not in fileh
# Go to mark 'third'
fileh.goto('third')
assert '/otherarray1' in fileh
assert '/agroup' in fileh
assert '/agroup/agroup2' in fileh
assert '/agroup/otherarray2' in fileh
assert '/agroup/agroup2/otherarray3' in fileh
assert '/otherarray4' not in fileh
assert '/agroup/otherarray5' not in fileh
# Now go to mark 'second'
fileh.goto('second')
assert '/otherarray1' in fileh
assert '/agroup' in fileh
assert '/agroup/agroup2' in fileh
assert '/agroup/otherarray2' in fileh
assert '/agroup/agroup2/otherarray3' not in fileh
assert '/otherarray4' not in fileh
assert '/agroup/otherarray5' not in fileh
	  </verbatim>

	  <p>Well, the code above shows how easy is to jump to a
	    certain mark in the database by using the
	    <verb>goto()</verb> method (see <pageref
	    refid="File.goto">page</pageref>).
	  </p>

	  <p>There are also a couple of implicit marks for going to
	   the beginning or the end of the saved states: 0 and
	   -1. Going to mark #0 means go to the beginning of the saved
	   actions, that is, when method
	   <verb>fileh.enableUndo()</verb> was called. Going to mark
	   #-1 means go to the last recorded action, that is the last
	   action in the code flow.
	  </p>

	  <p>Let's see what happens when going to the end of the
	    action log:
	  </p>

	  <verbatim>
# Go to the end
fileh.goto(-1)
assert '/otherarray1' in fileh
assert '/agroup' in fileh
assert '/agroup/agroup2' in fileh
assert '/agroup/otherarray2' in fileh
assert '/agroup/agroup2/otherarray3' in fileh
assert '/otherarray4' in fileh
assert '/agroup/otherarray5' in fileh
# Check that objects have come back to life in a sane state
assert fileh.root.otherarray1.read() == [3,4]
assert fileh.root.agroup.otherarray2.read() == [4,5]
assert fileh.root.agroup.agroup2.otherarray3.read() == [5,6]
assert fileh.root.otherarray4.read() == [6,7]
assert fileh.root.agroup.otherarray5.read() == [7,8]
	  </verbatim>

	  <p>Try yourself going to the beginning of the action log
	   (remember, the mark #0) and check the contents of the
	   object tree.
	  </p>

	  <p>We have nearly finished this demonstration. As always, do
	    not forget to close the action log as well as the database:
	  </p>

	  <verbatim>
         #'-**-**-**-**-**-**- disable undo/redo log  -**-**-**-**-**-**-**-'
fileh.disableUndo()

# Close the file
fileh.close()
	  </verbatim>

	  <p>You might want to check other examples on Undo/Redo
	    feature that appear in <verb>examples/undo-redo.py</verb>.
	  </p>

	</subsection>

      </section>     <!-- Undo/Redo -->

      <section>
	<heading>Using enumerated types</heading>

	<p>Beginning from version 1.1, PyTables supports the handling
	  of enumerated types.  Those types are defined by providing
	  an <em>exhaustive set or list</em> of possible, named values
	  for a variable of that type.  Enumerated variables of the
	  same type are usually compared between them for equality and
	  sometimes for order, but are not usually operated upon.
	</p>

	<p>Enumerated values have an associated <em>name</em> and
	  <em>concrete value</em>.  Every name is unique and so are
	  concrete values.  An enumerated variable always takes the
	  concrete value, not its name.  Usually, the concrete value
	  is not used directly, and frequently it is entirely
	  irrelevant.  For the same reason, an enumerated variable is
	  not usually compared with concrete values out of its
	  enumerated type.  For that kind of use, standard variables
	  and constants are more adequate.
	</p>

	<p>PyTables provides the <verb>Enum</verb> (see <ref
	  refid="EnumClassDescr"></ref>) class to provide support for
	  enumerated types.  Each instance of <verb>Enum</verb> is an
	  enumerated type (or <em>enumeration</em>).  For example, let
	  us create an enumeration of colors<footnote>All these
	  examples can be found in
	  <verb>examples/enum.py</verb>.</footnote>:
	</p>

	<verbatim>>>> import tables
>>> colorList = ['red', 'green', 'blue', 'white', 'black']
>>> colors = tables.Enum(colorList)
>>> </verbatim>

	<p>Here we used a simple list giving the names of enumerated
	  values, but we left the choice of concrete values up to the
	  <verb>Enum</verb> class.  Let us see the enumerated pairs to
	  check those values:
	</p>

	<verbatim>>>> print "Colors:", [v for v in colors]
Colors: [('blue', 2), ('black', 4), ('white', 3), ('green', 1), ('red', 0)]
>>> </verbatim>

	<p>Names have been given automatic integer concrete values.
	  We can iterate over the values in an enumeration,
	  but we will usually be more interested in accessing single values.
	  We can get the concrete value associated with a name by accessing it
	  as an attribute or as an item (the later can be useful for names
	  not resembling Python identifiers):
	</p>

	<verbatim><![CDATA[>>> print "Value of 'red' and 'white':", (colors.red, colors.white)
Value of 'red' and 'white': (0, 3)
>>> print "Value of 'yellow':", colors.yellow
Value of 'yellow':
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "enum.py", line 222, in __getattr__
AttributeError: no enumerated value with that name: 'yellow'
>>> 
>>> print "Value of 'red' and 'white':", (colors['red'], colors['white'])
Value of 'red' and 'white': (0, 3)
>>> print "Value of 'yellow':", colors['yellow']
Value of 'yellow':
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "enum.py", line 181, in __getitem__
KeyError: "no enumerated value with that name: 'yellow'"
>>> ]]></verbatim>

	<p>See how accessing a value that is not in the enumeration
	  raises the appropriate exception.  We can also do the
	  opposite action and get the name that matches a concrete
	  value by using the <verb>__call__()</verb> method of
	  <verb>Enum</verb>:
	</p>

	<verbatim><![CDATA[>>> print "Name of value %s:" % colors.red, colors(colors.red)
Name of value 0: red
>>> print "Name of value 1234:", colors(1234)
Name of value 1234:
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "enum.py", line 311, in __call__
ValueError: no enumerated value with that concrete value: 1234
>>> ]]></verbatim>

	<p>You can see what we made as using the enumerated type to
	  <em>convert</em> a concrete value into a name in the
	  enumeration.  Of course, values out of the enumeration can
	  not be converted.
	</p>

	<subsection>
	  <heading>Enumerated columns</heading>

	  <p>Columns of an enumerated type can be declared by using
	    the <verb>EnumCol</verb> (see <ref
	    refid="ColClassDescr"></ref>) class.  To see how this
	    works, let us open a new PyTables file and create a table
	    to collect the simulated results of a probabilistic
	    experiment.  In it, we have a bag full of colored balls;
	    we take a ball out and annotate the time of extraction and
	    the color of the ball.
	  </p>

	  <verbatim>>>> h5f = tables.openFile('enum.h5', 'w')
>>> 
>>> class BallExt(tables.IsDescription):
...     ballTime = tables.Time32Col()
...     ballColor = tables.EnumCol(colors, 'black', dtype='UInt8')
... 
>>> tbl = h5f.createTable(
...     '/', 'extractions', BallExt, title="Random ball extractions")
>>> </verbatim>

	  <p>We declared the <verb>ballColor</verb> column to be of
	    the enumerated type <verb>colors</verb>, with a default
	    value of <verb>black</verb>.  We also stated that we are
	    going to store concrete values as unsigned 8-bit integer
	    values<footnote>In fact, only integer values are supported
	    right now, but this may change in the future.</footnote>.
	  </p>

	  <p>Let us use some random values to fill the table:</p>

	  <verbatim>>>> import time
>>> import random
>>> now = time.time()
>>> row = tbl.row
>>> for i in range(10):
...     row['ballTime'] = now + i
...     row['ballColor'] = colors[random.choice(colorList)]  # notice this
...     row.append()
... 
>>> </verbatim>

	  <p>Notice how we used the <verb>__getitem()__</verb> call of
	    <verb>colors</verb> to get the concrete value to store in
	    <verb>ballColor</verb>.  You should know that this way of
	    appending values to a table does automatically check for
	    the validity on enumerated values.  For instance:
	  </p>

	  <verbatim><![CDATA[>>> row['ballTime'] = now + 42
>>> row['ballColor'] = 1234
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "hdf5Extension.pyx", line 2936, in hdf5Extension.Row.__setitem__
  File "enum.py", line 311, in __call__
ValueError: no enumerated value with that concrete value: 1234
>>> ]]></verbatim>

	  <p>But take care that this check is <em>only</em> performed
	    here and not in other methods such as
	    <verb>tbl.append()</verb> or
	    <verb>tbl.modifyRows()</verb>.  Now, after flushing the
	    table we can see the results of the insertions:
	  </p>

	  <verbatim>>>> tbl.flush()
>>>
>>> COMMENT("Now print them!")
>>> for r in tbl:
...     ballTime = r['ballTime']
...     ballColor = colors(r['ballColor'])  # notice this
...     print "Ball extracted on %d is of color %s." % (ballTime, ballColor)
...
Ball extracted on 1116501220 is of color white.
Ball extracted on 1116501221 is of color red.
Ball extracted on 1116501222 is of color blue.
Ball extracted on 1116501223 is of color white.
Ball extracted on 1116501224 is of color white.
Ball extracted on 1116501225 is of color green.
Ball extracted on 1116501226 is of color black.
Ball extracted on 1116501227 is of color red.
Ball extracted on 1116501228 is of color white.
Ball extracted on 1116501229 is of color white.
>>> </verbatim>

	  <p>As a last note, you may be wondering how to have access
	    to the enumeration associated with <verb>ballColor</verb>
	    once the file is closed and reopened.  You can call
	    <verb>tbl.getEnum('ballColor')</verb> (see <ref
	    refid="Table.getEnum"></ref>) to get the enumeration back.
	  </p>
	</subsection>

	<subsection>
	  <heading>Enumerated arrays</heading>

	  <p><verb>EArray</verb> and <verb>VLArray</verb> leaves can
	    also be declared to store enumerated values by means of
	    the <verb>EnumAtom</verb> (see <ref
	    refid="AtomClassDescr"></ref>) class, which works very
	    much like <verb>EnumCol</verb> for tables.  Also,
	    <verb>Array</verb> leaves can be used to open native HDF
	    enumerated arrays.
	  </p>

	  <p>Let us create a sample <verb>EArray</verb> containing
	    ranges of working days as bidimensional values:
	  </p>

	  <verbatim>>>> workingDays = {'Mon': 1, 'Tue': 2, 'Wed': 3, 'Thu': 4, 'Fri': 5}
>>> dayRange = tables.EnumAtom(workingDays, shape=(0, 2), flavor='Tuple')
>>> earr = h5f.createEArray('/', 'days', dayRange, title="Working day ranges")
>>> </verbatim>

	  <p>Nothing surprising, except for a pair of details.  In the
	    first place, we use a <em>dictionary</em> instead of a
	    list to explicitly set concrete values in the enumeration.
	    In the second place, there is no explicit
	    <verb>Enum</verb> instance created!  Instead, the
	    dictionary is passed as the first argument to the
	    constructor of <verb>EnumAtom</verb>.  If the constructor
	    gets a list or a dictionary instead of an enumeration, it
	    automatically builds the enumeration from it.
	  </p>

	  <p>Now let us feed some data to the array:</p>

	  <verbatim>>>> wdays = earr.getEnum()
>>> earr.append([(wdays.Mon, wdays.Fri), (wdays.Wed, wdays.Fri)])
>>> earr.append([(wdays.Mon, 1234)])
>>> </verbatim>

	  <p>Please note that, since we had no explicit
	    <verb>Enum</verb> instance, we were forced to use
	    <verb>getEnum()</verb> (see <ref
	    refid="EArray.getEnum"></ref>) to get it from the array
	    (we could also have used <verb>dayRange.enum</verb>).
	    Also note that we were able to append an invalid value
	    (1234).  Array methods do not check the validity of
	    enumerated values.
	  </p>

	  <p>Finally, we will print the contents of the array:</p>

	  <verbatim><![CDATA[>>> for (d1, d2) in earr:
...     print "From %s to %s (%d days)." % (wdays(d1), wdays(d2), d2-d1+1)
... 
From Mon to Fri (5 days).
From Wed to Fri (3 days).
Traceback (most recent call last):
  File "<stdin>", line 2, in ?
  File "enum.py", line 311, in __call__
ValueError: no enumerated value with that concrete value: 1234L
>>> ]]></verbatim>

	  <p>That was an example of operating on concrete values.
	    It also showed how the value-to-name conversion failed
	    because of the value not belonging to the enumeration.
	  </p>

	  <p>Now we will close and remove the file, and this little tutorial
	    on enumerated types is done:
	  </p>

	  <verbatim>>>> import os
>>> h5f.close()
>>> os.remove('enum.h5')
>>> </verbatim>
	</subsection>
      </section>  <!-- Using enumerated types -->

      <section>
	<heading>Dealing with nested structures in tables</heading>

	<p>PyTables supports the handling of nested structures (or
	  nested datatypes, as you prefer) in table objects, allowing
	  you to define arbitrarily nested columns.
	</p>
	<p>An example will clarify what this means. Let's suppose
	  that you want to group your data in pieces of information
	  that are more related than others pieces in your table, So
	  you may want to tie them up together in order to have your
	  table better structured but also be able to retrieve and
	  deal with these groups more easily.
	</p>

	<p>You can create such a nested substructures by just nesting
	  subclasses of <verb>IsDescription</verb>. Let's see one
	  example (okay, it's a bit silly, but will serve for
	  demonstration purposes):
	</p>

	<verbatim>
class Info(IsDescription):
    """A sub-structure of Test"""
    _v_pos = 2   # The position in the whole structure
    name = StringCol(10)
    value = Float64Col(pos=0)

colors = Enum(['red', 'green', 'blue'])  # An enumerated type

class NestedDescr(IsDescription):
    """A description that has several nested columns"""
    color = EnumCol(colors, 'red', dtype='UInt32', indexed=1) # indexed column
    info1 = Info()
    class info2(IsDescription):
        _v_pos = 1
        name = StringCol(10)
        value = Float64Col(pos=0)
        class info3(IsDescription):
            x = FloatCol(1)
            y = UInt8Col(1)

	</verbatim>

	<p>The root class is <verb>NestedDescr</verb> and both
	  <verb>info1</verb> and <verb>info2</verb> are
	  <em>substructures</em> of it. Note how <verb>info1</verb> is
	  actually an instance of the class <verb>Info</verb> that was
	  defined prior to <verb>NestedDescr</verb>. Also, there is a
	  third substructure, namely <verb>info3</verb> that hangs
	  from the substructure <verb>info2</verb>. You can also
	  define positions of substructures in the containing object
	  by declaring the special class attribute
	  <verb>_v_pos</verb>.
	</p>

	<subsection>
	  <heading>Nested table creation</heading>

	  <p>Now that we have defined our nested structure, let's
	    create a <em>nested</em> table, that is a table with
	    columns that contain other subcolumns.
	  </p>

	  <verbatim>
>>> from tables import *
>>> fileh = openFile("nested-tut.h5", "w")
>>> table = fileh.createTable(fileh.root, 'table', NestedDescr)
>>>
	  </verbatim>

	  <p>Done! Now, we have to feed the table with some
	    values. The problem is how we are going to reference to
	    the nested fields. That's easy, just use a
	    <verb>'/'</verb> character to separate names in different
	    nested levels. Look at this:
	  </p>

	  <verbatim>
>>> for i in range(10):
...     row['color'] = colors[['red', 'green', 'blue'][i%3]]
...     row['info1/name'] = "name1-%s" % i
...     row['info2/name'] = "name2-%s" % i
...     row['info2/info3/y'] =  i
...     # All the rest will be filled with defaults
...     row.append()
...
>>> table.flush()
>>> table.nrows
10L
>>>
	  </verbatim>

	  <p>You see? In order to fill the fields located in the
	    substructures, we just need to specify its full path
	    in the table hierarchy.
	  </p>

	</subsection>

	<subsection>
	  <heading>Reading nested tables: introducing <visual
	      markup="tt">NestedRecArray</visual> objects</heading>

	  <p>Now, what happens if we want to read the table? Which
	    data container will be used to keep the data? Well, it's
	    worth trying it:
	  </p>

	  <verbatim>
>>> nra = table[::4]
>>> print nra
NestedRecArray[
(((1.0, 0), 'name2-0', 0.0), ('name1-0', 0.0), 0L),
(((1.0, 4), 'name2-4', 0.0), ('name1-4', 0.0), 1L),
(((1.0, 8), 'name2-8', 0.0), ('name1-8', 0.0), 2L)
]
>>>
	  </verbatim>

	  <p>We have read one row for each four in the table, giving a
	    result of three rows. What about the container? Well, we
	    can see that it is a new mysterious object known as
	    <verb>NestedRecArray</verb>. If we ask for more info on
	    that:
	  </p>

	  <verbatim><![CDATA[
>>> type(nra)
<class 'tables.nestedrecords.NestedRecArray'>]]>
	  </verbatim>

	  <p>we see that it is an instance of the class
	    <verb>NestedRecArray</verb> that lives in the module
	    <verb>nestedrecords</verb> of <verb>tables</verb>
	    package. <verb>NestedRecArray</verb> is actually a
	    subclass of the <verb>RecArray</verb> object of the
	    <verb>records</verb> module of <verb>numarray</verb>
	    package. You can see more info about
	    <verb>NestedRecArray</verb> object in <ref
	    refid="NestedRecArrayClassDescr">appendix</ref>.
	  </p>

	  <p>You can make use of the above object in many different
	    ways. For example, you can use it to append new data to
	    the existing table object:
	  </p>

	  <verbatim>
>>> table.append(nra)
>>> table.nrows
13L
>>>
	  </verbatim>

	  <p>Or, to create new tables:
	  </p>

	  <verbatim>
>>> table2 = fileh.createTable(fileh.root, 'table2', nra)
>>> table2[:]
array(
[(((1.0, 0), 'name2-0', 0.0), ('name1-0', 0.0), 0L),
(((1.0, 4), 'name2-4', 0.0), ('name1-4', 0.0), 1L),
(((1.0, 8), 'name2-8', 0.0), ('name1-8', 0.0), 2L)],
descr=[('info2', [('info3', [('x', '1f8'), ('y', '1u1')]), ('name',
 '1a10'), ('value', '1f8')]), ('info1', [('name', '1a10'), ('value',
 '1f8')]), ('color', '1u4')], shape=3)
	  </verbatim>

	  <p>Finally, we can select nested values that fulfill some
	    condition:
	  </p>

	  <verbatim>
>>> names = [ x['info2/name'] for x in table if x['color'] == colors.red ]
>>> names
['name2-0', 'name2-3', 'name2-6', 'name2-9', 'name2-0']
>>>
	  </verbatim>

	  <p>Note that the row accessor does not provide the natural
	    naming feature, so you have to completely specify the path
	    of your desired columns in order to reach them.
	  </p>

	</subsection>

	<subsection>
	  <heading>Using Cols accessor</heading>

	  <p>We can use the <verb>cols</verb> attribute object (see
	    <ref refid="ColsClassDescr"></ref>) of the table so as to
	    quickly access the info located in the interesting
	    substructures:
	  </p>

	  <verbatim>
>>> table.cols.info2[1:5]
array(
[((1.0, 1), 'name2-1', 0.0),
((1.0, 2), 'name2-2', 0.0),
((1.0, 3), 'name2-3', 0.0),
((1.0, 4), 'name2-4', 0.0)],
descr=[('info3', [('x', '1f8'), ('y', '1u1')]), ('name', '1a10'),
 ('value', '1f8')],
shape=4)
>>>
	  </verbatim>

	  <p>Here, we have made use of the cols accessor to access to
	    the <em>info2</em> substructure and an slice operation to
	    get access to the subset of data we were interested in;
	    you probably have recognized the natural naming approach
	    here. We can continue and ask for data in <em>info3</em>
	    substructure:
	  </p>

	  <verbatim>
>>> table.cols.info2.info3[1:5]
array(
[(1.0, 1),
(1.0, 2),
(1.0, 3),
(1.0, 4)],
descr=[('x', '1f8'), ('y', '1u1')],
shape=4)
>>>
	  </verbatim>

	  <p>You can also use the <verb>_f_col</verb> method to get a
	    handler for a column:
	  </p>

	  <verbatim>
>>> table.cols._f_col('info2')
/table.cols.info2 (Cols), 3 columns
  info3 (Cols(1,), Description)
  name (Column(1,), CharType)
  value (Column(1,), Float64)
	  </verbatim>

	  <p>Here, you've got another <verb>Cols</verb> object handler
	    because <em>info2</em> was a nested column. If you select
	    a non-nested column, you will get a regular
	    <verb>Column</verb> instance:
	  </p>

	  <verbatim>
>>> ycol = table.cols._f_col('info2/info3/y')
>>> ycol
/table.cols.info2.info3.y (Column(1,), UInt8, idx=None)
>>>
	  </verbatim>

	  <p>To sum up, the <verb>cols</verb> accessor is a very handy
	    and powerful way to access data in your nested tables. Be
	    sure of using it, specially when doing interactive work.
	  </p>

	</subsection>

	<subsection>
	  <heading>Accessing meta-information of nested
	    tables</heading>

	  <p>Tables have an attribute called <verb>description</verb>
	    which points to an instance of the
	    <verb>Description</verb> class (see <ref
	    refid="DescriptionClassDescr"></ref>) and is useful to
	    discover different meta-information about table
	    data.
	  </p>

	  <p>Let's see how it looks like:
	  </p>

	  <verbatim>
>>> table.description
{
  "info2": {
    "info3": {
      "x": FloatCol(dflt=1, shape=1, itemsize=8, pos=0, indexed=False),
      "y": UInt8Col(dflt=1, shape=1, pos=1, indexed=False)},
    "name": StringCol(length=10, dflt=None, shape=1, pos=1, indexed=False),
    "value": Float64Col(dflt=0.0, shape=1, pos=2, indexed=False)},
  "info1": {
    "name": StringCol(length=10, dflt=None, shape=1, pos=0, indexed=False),
    "value": Float64Col(dflt=0.0, shape=1, pos=1, indexed=False)},
  "color": EnumCol(Enum({'blue': 2, 'green': 1, 'red': 0}), 'red',
 dtype='UInt32', shape=1, pos=2, indexed=1)}
>>>
	  </verbatim>

	  <p>As you can see, it provides very useful information on
	    both the formats and the structure of the columns in your
	    table.
	  </p>

	  <p>This object also provides a natural naming approach to
	    access to subcolumns metadata:
	  </p>

	  <verbatim>
>>> table.description.info1
{
    "name": StringCol(length=10, dflt=None, shape=1, pos=0, indexed=False),
    "value": Float64Col(dflt=0.0, shape=1, pos=1, indexed=False)}
>>> table.description.info2.info3
{
      "x": FloatCol(dflt=1, shape=1, itemsize=8, pos=0, indexed=False),
      "y": UInt8Col(dflt=1, shape=1, pos=1, indexed=False)}
>>>
	  </verbatim>

	  <p>There are other variables that can be interesting for you:
	  </p>

	  <verbatim>
>>> table.description._v_nestedNames
[('info2', [('info3', ['x', 'y']), 'name', 'value']), ('info1',
 ['name', 'value']), 'color']
>>> table.description.info1._v_nestedNames
['name', 'value']
>>>
	  </verbatim>

	  <p><verb>_v_nestedNames</verb> provides the names of the
	    columns as well as its structure. You can see that there
	    are the same attributes for the different levels of the
	    <verb>Description</verb> object, because the levels are
	    <em>also</em> <verb>Description</verb> objects themselves.
	  </p>


	  <p>There is a special attribute, called
	    <verb>_v_nestedDescr</verb> that can be useful to create
	    <verb>NestedRecArrays</verb> objects that imitate the
	    structure of the table (or a subtable!):
	  </p>
	  <verbatim>
>>> from tables import nestedrecords
>>> table.description._v_nestedDescr
[('info2', [('info3', [('x', '1f8'), ('y', '1u1')]), ('name', '1a10'),
 ('value', '1f8')]), ('info1', [('name', '1a10'), ('value', '1f8')]),
 ('color', '1u4')]
>>> nestedrecords.array(None, descr=table.description._v_nestedDescr)
array(
[],
descr=[('info2', [('info3', [('x', '1f8'), ('y', '1u1')]), ('name',
 '1a10'), ('value', '1f8')]), ('info1', [('name', '1a10'), ('value',
 '1f8')]),('color', '1u4')], shape=0)
>>> nestedrecords.array(None, descr=table.description.info2._v_nestedDescr)
array(
[],
descr=[('info3', [('x', '1f8'), ('y', '1u1')]), ('name', '1a10'),
('value', '1f8')], shape=0)
>>>
	  </verbatim>

	  <p>Look the section <ref refid="DescriptionClassDescr"></ref>
	    for the complete listing of attributes.
	  </p>

	  <p>Finally, there is a special iterator of the
	    <verb>Description</verb> class, called <verb>_v_walk</verb>
	    that is able to return you the different columns of the
	    table:
	  </p>

	  <verbatim>
>>> for coldescr in table.description._v_walk():
...     print "column-->",coldescr
...
column--> Description([('info2', [('info3', [('x', '1f8'), ('y',
 '1u1')]), ('name', '1a10'), ('value', '1f8')]), ('info1', [('name',
 '1a10'), ('value', '1f8')]), ('color', '1u4')])
column--> EnumCol(Enum({'blue': 2, 'green': 1, 'red': 0}), 'red',
 dtype='UInt32', shape=1, pos=2, indexed=1)
column--> Description([('info3', [('x', '1f8'), ('y', '1u1')]),
 ('name', '1a10'), ('value', '1f8')])
column--> StringCol(length=10, dflt=None, shape=1, pos=1, indexed=False)
column--> Float64Col(dflt=0.0, shape=1, pos=2, indexed=False)
column--> Description([('name', '1a10'), ('value', '1f8')])
column--> StringCol(length=10, dflt=None, shape=1, pos=0, indexed=False)
column--> Float64Col(dflt=0.0, shape=1, pos=1, indexed=False)
column--> Description([('x', '1f8'), ('y', '1u1')])
column--> FloatCol(dflt=1, shape=1, itemsize=8, pos=0, indexed=False)
column--> UInt8Col(dflt=1, shape=1, pos=1, indexed=False)
>>>
	  </verbatim>

	  <p>Well, this is the end of this tutorial. As always, do not
	  forget to close your files:
	  </p>

	  <verbatim>
>>> fileh.close()
>>>
	  </verbatim>

	  <p>Finally, you may want to have a look at your resulting
	    data file:
	  </p>

	  <verbatim>
$ ptdump -d nested-tut.h5
/ (RootGroup) ''
/table (Table(13L,)) ''
  Data dump:
[0] (((1.0, 0), 'name2-0', 0.0), ('name1-0', 0.0), 0L)
[1] (((1.0, 1), 'name2-1', 0.0), ('name1-1', 0.0), 1L)
[2] (((1.0, 2), 'name2-2', 0.0), ('name1-2', 0.0), 2L)
[3] (((1.0, 3), 'name2-3', 0.0), ('name1-3', 0.0), 0L)
[4] (((1.0, 4), 'name2-4', 0.0), ('name1-4', 0.0), 1L)
[5] (((1.0, 5), 'name2-5', 0.0), ('name1-5', 0.0), 2L)
[6] (((1.0, 6), 'name2-6', 0.0), ('name1-6', 0.0), 0L)
[7] (((1.0, 7), 'name2-7', 0.0), ('name1-7', 0.0), 1L)
[8] (((1.0, 8), 'name2-8', 0.0), ('name1-8', 0.0), 2L)
[9] (((1.0, 9), 'name2-9', 0.0), ('name1-9', 0.0), 0L)
[10] (((1.0, 0), 'name2-0', 0.0), ('name1-0', 0.0), 0L)
[11] (((1.0, 4), 'name2-4', 0.0), ('name1-4', 0.0), 1L)
[12] (((1.0, 8), 'name2-8', 0.0), ('name1-8', 0.0), 2L)
/table2 (Table(3L,)) ''
  Data dump:
[0] (((1.0, 0), 'name2-0', 0.0), ('name1-0', 0.0), 0L)
[1] (((1.0, 4), 'name2-4', 0.0), ('name1-4', 0.0), 1L)
[2] (((1.0, 8), 'name2-8', 0.0), ('name1-8', 0.0), 2L)
	  </verbatim>

	    <p>Most of the code in this section is also available in
	      <verb>examples/nested-tut.py</verb>.
	    </p>

	    <p>All in all, <verb>PyTables</verb> provides a quite
	      comprehensive toolset to cope with nested structures and
	      address your classification needs. However, caveat
	      emptor, be sure to not nest your data too deeply or you
	      will get inevitably messed interpreting too intertwined
	      lists, tuples and description objects.
	    </p>

	</subsection>

      </section> <!-- Nested structures in tables -->

      <section>
	<heading>Other examples in PyTables distribution</heading>

	<p>Feel free to examine the rest of examples in directory
	  <verb>examples/</verb>, and try to understand them. We have
	  written several practical sample scripts to give you an idea
	  of the <verb>PyTables</verb> capabilities, its way of
	  dealing with HDF5 objects, and how it can be used in the
	  real world.
	</p>

      </section>

    </chapter>

    <!-- SGP:  Ended here -->
    <chapter id="libraryReference">
      <heading>Library Reference</heading>

      <!-- FA: In case Scott finish the correction for 0.9.1 release,
      I've added the File.copyNodeAttrs method description as well as
      corrected the isPyTablesFile() function. -->

<!--       <aphorism>"Tenho pensamentos que, se pudesse revelá-los e -->
<!-- 	fazê-los viver, acrescentariam nova luminosidade às estrelas, -->
<!-- 	nova beleza ao mundo e maior amor ao coração dos homens." -->
<!-- 	<caption>Fernando Pessoa, in "O Eu Profundo"</caption> -->
<!--       </aphorism> -->

      <p><verb>PyTables</verb> implements several classes to represent
	the different nodes in the object tree. They are named
	<verb>File</verb>, <verb>Group</verb>, <verb>Leaf</verb>,
	<verb>Table</verb>, <verb>Array</verb>, <verb>CArray</verb>,
	<verb>EArray</verb>, <verb>VLArray</verb> and
	<verb>UnImplemented</verb>. Another one allows the user to
	complement the information on these different objects; its
	name is <verb>AttributeSet</verb>. Finally, another important
	class called <verb>IsDescription</verb> allows to build a
	<verb>Table</verb> record description by declaring a subclass
	of it. Many other classes are defined in
	<verb>PyTables</verb>, but they can be regarded as helpers
	whose goal is mainly to declare the <em>data type
	properties</em> of the different first class objects and will
	be described at the end of this chapter as well.
      </p>
      <p>An important function, called <verb>openFile</verb> is
	responsible to create, open or append to files. In addition, a
	few utility functions are defined to guess if the user
	supplied file is a <em>PyTables</em> or <em>HDF5</em>
	file. These are called <verb>isPyTablesFile()</verb> and
	<verb>isHDF5File()</verb>, respectively. Finally, there exists a
	function called <verb>whichLibVersion</verb> that informs
	about the versions of the underlying C libraries (for example,
	the <verb>HDF5</verb> or the <verb>Zlib</verb>).
      </p>

      <p>Let's start discussing the first-level variables and
	functions available to the user, then the different classes
	defined in <verb>PyTables</verb>.
      </p>

      <section>
	<heading><visual markup="tt">tables</visual> variables and
	  functions</heading>

	<subsection>
	  <heading>Global variables</heading>

	  <description>

	    <term>__version__</term> <item>The <verb>PyTables</verb>
	    version number.</item>

	    <term>hdf5Version</term>
	    <item>The underlying HDF5 library version number.</item>

	  </description>

	</subsection>

	<subsection id="GlobalFunctDescr">
	  <heading>Global functions</heading>

	  <subsubsection id="copyFileDescr">
	    <heading>copyFile(srcfilename, dstfilename,
	      overwrite=False, **kwargs)</heading>

	    <p>An easy way of copying one PyTables file to another.</p>

	    <p>This function allows you to copy an existing PyTables
	      file named <verb>srcfilename</verb> to another file
	      called <verb>dstfilename</verb>.  The source file must
	      exist and be readable.  The destination file can be
	      overwritten in place if existing by asserting the
	      <verb>overwrite</verb> argument.
	    </p>

	    <p>This function is a shorthand for the
	      <verb>File.copyFile()</verb> method, which acts on an
	      already opened file.  <verb>kwargs</verb> takes keyword
	      arguments used to customize the copying process.  See
	      the documentation of <verb>File.copyFile()</verb> (see
	      <ref refid="File.copyFile"></ref>) for a description of
	      those arguments.
	    </p>
	  </subsubsection>

	  <subsubsection id="isHDF5FileDescr">
	    <heading>isHDF5File(filename)</heading>

	    <p>Determine whether a file is in the HDF5 format.</p>
	    <p>When successful, it returns a true value if the file is
	      an HDF5 file, false otherwise.
	      If there were problems identifying the file,
	      an <verb>HDF5ExtError</verb> is raised.
	    </p>
	  </subsubsection>

	  <!-- isPyTablesFile modificada F. Altet 2004-11-18 -->

	  <subsubsection id="isPyTablesFileDescr">
	    <heading>isPyTablesFile(filename)</heading>

	    <p>Determine whether a file is in the PyTables format.</p>
	    <p>When successful, it returns a true value if the file is
	      a PyTables file, false otherwise.
	      The true value is the format version string of the file.
	      If there were problems identifying the file,
	      an <verb>HDF5ExtError</verb> is raised.
	    </p>
	  </subsubsection>

	  <!-- Final modificacio isPyTablesFile F. Altet 2004-11-18 -->

	  <subsubsection id="openFileDescr">
	    <heading>openFile(filename, mode='r', title='', trMap={},
	      rootUEP="/", filters=None)</heading>

	    <p>Open a <verb>PyTables</verb> (or generic
	      <verb>HDF5</verb>) file and returns a <verb>File</verb>
	      object.
	    </p>

	    <description>

	      <term>filename</term> <item>The name of the file
		(supports environment variable expansion). It is
		suggested that it should have any of
		<verb>".h5"</verb>, <verb>".hdf"</verb> or
		<verb>".hdf5"</verb> extensions, although this is
		not mandatory.
	      </item>

	      <term>mode</term> <item>The mode to open the file. It
		can be one of the following:

		<description>

		  <term>'r'</term> <item>read-only; no data can be
		    modified.</item>

		  <term>'w'</term> <item>write; a new file is
		    created (an existing file with the same name
		    would be deleted).</item>

		  <term>'a'</term> <item>append; an existing file is
		    opened for reading and writing, and if the file
		    does not exist it is created.</item>

		  <term>'r+'</term> <item>is similar to 'a', but the
		    file must already exist.</item>

		</description>
	      </item>

	      <term>title</term> <item>If filename is new, this will
		set a title for the root group in this file. If
		filename is not new, the title will be read from
		disk, and this will not have any effect.
	      </item>

	      <term>trMap</term> <item>A dictionary to map names in
		the object tree Python namespace into different HDF5
		names in file namespace. The keys are the Python
		names, while the values are the HDF5 names. This is
		useful when you need to use HDF5 node names with
		invalid or reserved words in Python.
	      </item>

	      <term>rootUEP</term> <item>The root User Entry
		Point. This is a group in the HDF5 hierarchy which
		will be taken as the starting point to create the
		object tree. The group has to be named after its
		HDF5 name and can be a path. If it does not exist, an
		<verb>HDF5ExtError</verb> exception is issued. Use
		this if you do not want to build the <visual
		  markup="bf">entire</visual> object tree, but rather
		only a <visual markup="bf">subtree</visual> of it.
	      </item>

	      <term>filters</term><item>An instance of the
		<verb>Filters</verb> class (see section <ref
		  refid="FiltersClassDescr"></ref>) that provides
		information about the desired I/O filters applicable
		to the leaves that hang directly from <em>root</em>
		(unless other filters properties are specified for
		these leaves). Besides, if you do not specify filter
		properties for its child groups, they will inherit
		these ones. So, if you open a new file with this
		parameter set, all the leaves that would be created
		in the file will recursively inherit this filtering
		properties (again, if you don't prevent that from
		happening by specifying other filters on the child
		groups or leaves).
	      </item>

	      <term>nodeCacheSize</term>
	      <item>The number of <em>unreferenced</em> nodes to be kept in memory.
		Least recently used nodes are unloaded from memory
		when this number of loaded nodes is reached.
		To load a node again, simply access it as usual.
		Nodes referenced by user variables
		are not taken into account nor unloaded.
	      </item>
	    </description>
	  </subsubsection>

	  <subsubsection id="whichLibVersionDescr">
	    <heading>whichLibVersion(name)</heading>

	    <p>Get version information about a C library.</p>

	    <p>If the library indicated by <verb>name</verb> is
	      available, this function returns a 3-tuple containing
	      the major library version as an integer, its full
	      version as a string, and the version date as a string.
	      If the library is not available, <verb>None</verb> is
	      returned.
	    </p>

	    <p>The currently supported library names are
	      <verb>hdf5</verb>, <verb>zlib</verb>, <verb>lzo</verb>,
	      <verb>ucl</verb> (in process of being
	      <em>deprecated</em>) and <verb>bzip2</verb>.  If another
	      name is given, a <verb>ValueError</verb> is raised.
	    </p>
	  </subsubsection>
	</subsection>
      </section>

      <section id="FileClassDescr">
	<heading>The <visual markup="tt">File</visual> class</heading>

	<p>An instance of this class is returned when a PyTables file
	  is opened with the <verb>openFile()</verb> function.
	  It offers methods to manipulate (create, rename, delete...) nodes
	  and handle their attributes,
	  as well as methods to traverse the object tree.
	  The <em>user entry point</em> to the object tree attached to the HDF5 file
	  is represented in the <verb>rootUEP</verb> attribute.
	  Other attributes are available.
	</p>

	<p><verb>File</verb> objects support an <em>Undo/Redo mechanism</em>
	  which can be enabled with the <verb>enableUndo()</verb> method.
	  Once the Undo/Redo mechanism is enabled,
	  explicit <em>marks</em> (with an optional unique name) can be set
	  on the state of the database using the <verb>mark()</verb> method.
	  There are two implicit marks which are always available:
	  the initial mark (0) and the final mark (-1).
	  Both the identifier of a mark and its name can be used
	  in <em>undo</em> and <em>redo</em> operations.
	</p>

	<p>Hierarchy manipulation operations (node creation, movement
	  and removal) and attribute handling operations (setting and
	  deleting) made after a mark can be undone by using the
	  <verb>undo()</verb> method, which returns the database to
	  the state of a past mark.  If <verb>undo()</verb> is not
	  followed by operations that modify the hierarchy or
	  attributes, the <verb>redo()</verb> method can be used to
	  return the database to the state of a future mark.  Else,
	  future states of the database are forgotten.
	</p>

	<p>Note that data handling operations can not be undone nor
	  redone by now.  Also, hierarchy manipulation operations on
	  nodes that do not support the Undo/Redo mechanism issue an
	  <verb>UndoRedoWarning</verb> <em>before</em> changing the
	  database.
	</p>

	<p>The Undo/Redo mechanism is persistent between sessions and
	  can only be disabled by calling the
	  <verb>disableUndo()</verb> method.
	</p>

	<subsection>
	  <heading><visual markup="tt">File</visual> instance
	    variables</heading>

	  <description>
	    <term id="File.filename">
	      filename
	    </term>
	    <item>The name of the opened file.</item>

	    <term id="File.format_version">
	      format_version
	    </term>
	    <item>The PyTables version number of this file.</item>

	    <term id="File.isopen">
	      isopen
	    </term>
	    <item>True if the underlying file is open, false otherwise.</item>

	    <term id="File.mode">
	      mode
	    </term>
	    <item>The mode in which the file was opened.</item>

	    <term id="File.title">
	      title
	    </term>
	    <item>The title of the root group in the file.</item>

	    <term id="File.trMap">
	      trMap
	    </term>
	    <item>A dictionary that maps node names
	      between PyTables and HDF5 domain names.
	      Its initial values are set from the <verb>trMap</verb> parameter
	      passed to the <verb>openFile</verb> function.
	      You can change its contents <em>after</em> a file is opened
	      and the new map will take effect over any new object
	      added to the tree.
	    </item>

	    <term id="File.rootUEP">
	      rootUEP
	    </term>
	    <item>The UEP (user entry point) group in the file
	      (see <ref refid="openFileDescr"></ref>).
	    </item>

	    <term id="File.filters">
	      filters
	    </term>
	    <item>Default filter properties for the root group
	      (see <ref refid="FiltersClassDescr">section</ref>).
	    </item>

	    <term id="File.root">
	      root
	    </term>
	    <item>The <em>root</em> of the object tree hierarchy
	      (a <verb>Group</verb> instance).
	    </item>

	    <term id="File.objects">
	      objects
	    </term>
	    <item>A dictionary which maps path names to objects,
	      for every visible node in the tree (deprecated, see note
	      below).
	    </item>

	    <term id="File.groups">
	      groups
	    </term>
	    <item>A dictionary which maps path names to objects,
	      for every visible group in the tree (deprecated, see
	      note below).
	    </item>

	    <term id="File.leaves">
	      leaves
	    </term>
	    <item>A dictionary which maps path names to objects,
	      for every visible leaf in the tree (deprecated, see note
	      below).
	    </item>
	  </description>

	  <p><visual markup="bf">Note:</visual> From PyTables 1.2 on,
	    the dictionaries <verb>objects</verb>, <verb>groups</verb>
	    and <verb>leaves</verb> are just instances of objects
	    faking the old functionality. Actually, they internally
	    use <verb>File.getNode()</verb>
	    (<ref refid="File.getNode">see</ref>)
	    and <verb>File.walknodes()</verb>
	    (<ref refid="File.walkNodes">see</ref>),
	    which are recommended instead.
	  </p>

	</subsection>

	<subsection id="FileMethodsDescr">
	  <heading><visual markup="tt">File</visual> methods</heading>

	  <subsubsection id="createGroupDescr">
	    <heading>createGroup(where, name, title='', filters=None)
	    </heading>

	    <p>Create a new Group instance with name <em>name</em> in
	      <em>where</em> location.
	    </p>

	    <description>
	      <term>where</term> <item>The parent group where the new
		group will hang from. <em>where</em> parameter can be
		a path string (for example
		<verb>"/level1/group5"</verb>), or another Group
		instance. </item>

	      <term>name</term>
	      <item>The name of the new group.</item>

	      <term>title</term> <item>A description for this
		group.</item>

	      <term>filters</term><item>An instance of the
		<verb>Filters</verb> class (see section<ref
		refid="FiltersClassDescr"></ref>) that provides
		information about the desired I/O filters applicable
		to the leaves that hangs directly from this new group
		(unless other filters properties are specified for
		these leaves). Besides, if you do not specify filter
		properties for its child groups, they will inherit
		these ones.
	      </item>

	    </description>

	  </subsubsection>

	  <subsubsection id="createTableDescr">
	    <heading>createTable(where, name,
	      description, title='', filters=None,
	      expectedrows=10000)
	    </heading>
	    <p>Create a new <verb>Table</verb> instance with name
	      <em>name</em> in <em>where</em> location.  See the <ref
	      refid="TableClassDescr">section</ref> for a description
	      of the <verb>Table</verb> class.
	    </p>
	    <description>
	      <term>where</term> <item>The parent group where the new
		table will hang from. <em>where</em> parameter can be
		a path string (for example
		<verb>"/level1/leaf5"</verb>), or Group instance.
               </item>
	      <term>name</term> <item>The name of the new table.
              </item>
	      <term>description</term> <item>

		<p>This is an object that describes the table, that
		  is, how many columns has it, and properties for each
		  column: the type, the shape, etc. as well as other
		  table properties.
		</p>

		<p><em>description</em> can be any of the next several
		  objects:
		</p>

		<description>

		  <term>A user-defined class</term> <item>This should
		    inherit from the <verb>IsDescription</verb> class
		    (see <ref refid="IsDescriptionClassDescr"></ref>)
		    where table fields are specified.
		  </item>

		  <term>A dictionary</term> <item>For example, when
		    you do not know beforehand which structure will
		    have your table). See <ref
		    refid="secondExample">section</ref> for an example
		    of use.
		  </item>

		  <term>A <verb>RecArray</verb></term> <item>This
		    object from the <verb>numarray</verb> package is
		    also accepted, and all the information about
		    columns and other metadata is used as a basis to
		    create the <verb>Table</verb> object. Moreover, if
		    the <verb>RecArray</verb> has actual data this is
		    also injected on the newly created
		    <verb>Table</verb> object.
		  </item>

		  <term>A <verb>NestedRecArray</verb></term>
		    <item>Finally, if you want to have nested columns
		    in your table, you can use this object (see <ref
		    refid="NestedRecArrayClassDescr">appendix</ref>)
		    and all the information about columns and other
		    metadata is used as a basis to create the
		    <verb>Table</verb> object. Moreover, if the
		    <verb>NestedRecArray</verb> has actual data this
		    is also injected on the newly created
		    <verb>Table</verb> object.
		  </item>

		</description>

	      </item> <!-- End of description parameter -->

	      <term>title</term> <item>A description for this object.
	      </item>
	      <term>filters</term> <item>An instance of the
		<verb>Filters</verb> class (see <ref
		refid="FiltersClassDescr">section</ref>) that provides
		information about the desired I/O filters to be
		applied during the life of this object.
	      </item>
	      <term>expectedrows</term> <item>An user estimate of the
		number of records that will be on table. If not
		provided, the default value is appropriate for tables
		until 10 MB in size (more or less). If you plan to
		save bigger tables you should provide a guess; this
		will optimize the HDF5 B-Tree creation and management
		process time and memory used. See <ref
		refid="expectedRowsOptim">section</ref> for a
		discussion on that issue.
	      </item>
	    </description>
	  </subsubsection>

	  <subsubsection id="createArrayDescr">
	    <heading>createArray(where, name,
	      object, title='')</heading>

	    <p>Create a new <verb>Array</verb> instance with name
	      <em>name</em> in <em>where</em> location.  See the <ref
	      refid="ArrayClassDescr">section</ref> for a
	      description of the <verb>Array</verb> class.
	    </p>
	    <description>
	      <term>object</term> <item>The regular array to be
		saved. Currently accepted values are:
		<verb>NumPy</verb>, <verb>Numeric</verb>,
		<verb>numarray</verb> arrays (including
		<verb>CharArray</verb> string numarrays) or other
		native Python types, provided that they are regular
		(i.e. they are not like <verb>[[1,2],2]</verb>) and
		homogeneous (i.e. all the elements are of the same
		type). Also, objects that have some of their
		dimensions equal to zero are not supported (use an
		<verb>EArray</verb> object if you want to create an
		array with one of its dimensions equal to 0).
	      </item>
	    </description>
	    <p>See <ref
	      refid="createTableDescr"><verb>createTable</verb>
	      description</ref> for more information on the
	      <em>where</em>, <em>name</em> and <em>title</em>,
	      parameters.
	    </p>
	  </subsubsection>

	  <subsubsection id="createCArrayDescr">
	    <heading>createCArray(where, name, shape, atom,
	      title='', filters=None)</heading>

	    <p>Create a new <verb>CArray</verb> instance with name
	      <em>name</em> in <em>where</em> location. See the <ref
	      refid="CArrayClassDescr">section</ref> for a
	      description of the <verb>CArray</verb> class.
	    </p>
	    <description>
	      <term>shape</term> <item>The <em>shape</em> of the
		objects to be saved.
	      </item>
	      <term>atom</term> <item>An <verb>Atom</verb> instance
		representing the <em>shape</em>, <em>type</em> and
		<em>flavor</em> of the chunk of the objects to be saved.
	      </item>
	    </description>
	    <p>See <ref
	      refid="createTableDescr"><verb>createTable</verb>
	      description</ref> for more information on the
	      <em>where</em>, <em>name</em> and <em>title</em>,
	      parameters.
	    </p>
	  </subsubsection>

	  <subsubsection id="createEArrayDescr">
	    <heading>createEArray(where, name,
	      atom, title='', filters=None, expectedrows=1000)
	    </heading>

	    <p>Create a new <verb>EArray</verb> instance with name
	      <em>name</em> in <em>where</em> location.  See the <ref
	      refid="EArrayClassDescr">section</ref> for a
	      description of the <verb>EArray</verb> class.
	    </p>

	    <description>

	      <term>atom</term> <item>An <verb>Atom</verb> instance
		representing the <em>shape</em>, <em>type</em> and
		<em>flavor</em> of the atomic objects to be saved.
		One (and only one) of the shape dimensions <visual
		markup="bf">must</visual> be 0. The dimension being 0
		means that the resulting <verb>EArray</verb> object
		can be extended along it. Multiple enlargeable
		dimensions are not supported right now.  See <ref
		refid="AtomClassDescr">section</ref> for the supported
		set of <verb>Atom</verb> class descendants.
              </item>
	      <term>expectedrows</term> <item>In the case of
                enlargeable arrays this represents an user estimate
                about the number of row elements that will be added to
                the growable dimension in the EArray object. If not
                provided, the default value is 1000 rows. If you plan
                to create both much smaller or much bigger EArrays try
                providing a guess; this will optimize the HDF5 B-Tree
                creation and management process time and the amount of
                memory used.
	      </item>

	    </description>

	    <p>See <ref
	      refid="createTableDescr"><verb>createTable</verb>
	      description</ref> for more information on the
	      <em>where</em>, <em>name</em>, <em>title</em>,
	      and <em>filters</em> parameters.
	    </p>

	  </subsubsection>

	  <subsubsection id="createVLArrayDescr">
	    <heading>createVLArray(where,
	      name, atom=None, title='', filters=None,
	      expectedsizeinMB=1.0)
	    </heading>

	    <p>Create a new <verb>VLArray</verb> instance with name
	      <em>name</em> in <em>where</em> location. See the <ref
	      refid="VLArrayClassDescr">section</ref> for a
	      description of the <verb>VLArray</verb> class.
	    </p>

	    <description>
	      <term>atom</term> <item>An <verb>Atom</verb> instance
                representing the shape, type and flavor of the atomic
                object to be saved. See <ref
                refid="AtomClassDescr">section</ref> for the supported set
                of <verb>Atom</verb> class descendants.
              </item>
              <term>expectedsizeinMB</term> <item>An user estimate
                about the size (in MB) in the final
                <verb>VLArray</verb> object. If not provided, the
                default value is 1 MB.  If you plan to create both
                much smaller or much bigger VLA's try providing a
                guess; this will optimize the HDF5 B-Tree creation and
                management process time and the amount of memory used.
              </item>
	    </description>
	    <p>See <ref
	      refid="createTableDescr"><verb>createTable</verb>
	      description</ref> for more information on the
	      <em>where</em>, <em>name</em>, <em>title</em>, and
	      <em>filters</em> parameters.
	    </p>
	  </subsubsection>

	  <subsubsection id="File.getNode">
	    <heading>getNode(where, name=None, classname=None)</heading>

	    <p>Get the node under <em>where</em> with the given <em>name</em>.</p>

	    <p><em>where</em> can be a <verb>Node</verb> instance
	      or a path string leading to a node.
	      If no <em>name</em> is specified, that node is returned.
	    </p>

	    <p>If a <em>name</em> is specified, this must be a string
	      with the name of a node under <em>where</em>.
	      In this case the <em>where</em> argument can only lead to
	      a <verb>Group</verb> instance
	      (else a <verb>TypeError</verb> is raised).
	      The node called <em>name</em> under the group <em>where</em>
	      is returned.
	    </p>

	    <p>In both cases, if the node to be returned does not exist, a
	      <verb>NoSuchNodeError</verb> is raised.
	      Please, note that hidden nodes are also considered.
	    </p>

	    <p>If the <em>classname</em> argument is specified,
	      it must be the name of a class derived from <verb>Node</verb>.
	      If the node is found but it is not an instance of that class,
	      a <verb>NoSuchNodeError</verb> is also raised.
	    </p>
	  </subsubsection>

	  <subsubsection id="File.isVisibleNode">
	    <heading>isVisibleNode(path)</heading>
	    <p>Is the node under <verb>path</verb> visible?</p>
	    <p>If the node does not exist,
	      a <verb>NoSuchNodeError</verb> is raised.</p>
	  </subsubsection>

	  <subsubsection id="File.getNodeAttr">
	    <heading>getNodeAttr(where, attrname, name=None)</heading>

	    <p>Returns the attribute <em>attrname</em> under
	      <em>where.name</em> location.
	    </p>
	    <description>
	      <term>where, name</term>
	      <item>These arguments work as in <verb>getNode()</verb>
		(see <pageref refid="File.getNode">page</pageref>),
		referencing the node to be acted upon.
	      </item>

	      <term>attrname</term>
	      <item>The name of the attribute to get.</item>
	    </description>
	  </subsubsection>

	  <subsubsection id="File.setNodeAttr">
	    <heading>setNodeAttr(where, attrname, attrvalue, name=None)</heading>

	    <p>Sets the attribute <em>attrname</em> with value
	      <em>attrvalue</em> under <em>where.name</em> location.
	      If the node already has a large number of attributes,
	      a <verb>PerformanceWarning</verb> will be issued.
	    </p>
	    <description>
	      <term>where, name</term>
	      <item>These arguments work as in <verb>getNode()</verb>
		(see <pageref refid="File.getNode">page</pageref>),
		referencing the node to be acted upon.
	      </item>

	      <term>attrname</term>
	      <item>The name of the attribute to set on disk.</item>

	      <term>attrvalue</term> <item>The value of the attribute
		to set.  Any kind of python object (like string, ints,
		floats, lists, tuples, dicts, small
		Numeric/NumPy/numarray objects...) can be stored as an
		attribute. However, if necessary,
		<verb>(c)Pickle</verb> is automatically used so as to
		serialize objects that you might want to save (see
		<ref refid="AttributeSetClassDescr"></ref> for
		details).
	      </item>
	    </description>
	  </subsubsection>

	  <subsubsection id="File.delNodeAttr">
	    <heading>delNodeAttr(where, attrname, name=None)</heading>

	    <p>Delete the attribute <em>attrname</em> in
	      <em>where.name</em> location.
	    </p>
	    <description>
	      <term>where, name</term>
	      <item>These arguments work as in <verb>getNode()</verb>
		(see <pageref refid="File.getNode">page</pageref>),
		referencing the node to be acted upon.
	      </item>

	      <term>attrname</term>
	      <item>The name of the attribute to delete on disk.</item>
	    </description>
	  </subsubsection>

	  <!-- copyNodeAttrs modificada F. Altet 2004-11-18 -->

	  <subsubsection id="File.copyNodeAttrs">

	    <heading>copyNodeAttrs(where, dstnode, name=None)</heading>

	    <p>Copy the attributes from node <em>where.name</em> to
	      <em>dstnode</em>.
	    </p>

	    <description>
	      <term>where, name</term>
	      <item>These arguments work as in <verb>getNode()</verb>
		(see <pageref refid="File.getNode">page</pageref>),
		referencing the node to be acted upon.
	      </item>

	      <term>dstnode</term> <item>This is the destination node
		where the attributes will be copied. It can be either
		a path string or a <verb>Node</verb> object.
	      </item>
            </description>
	  </subsubsection> <!-- copyNodeAttrs -->

	  <!-- Final modificacio copyNodeAttrs F. Altet 2004-11-18 -->

	  <subsubsection id="File.iterNodes">
	    <heading>iterNodes(where, classname=None)</heading>

	    <p>Returns an <em>iterator</em> yielding children nodes
	      hanging from <em>where</em>.  These nodes are
	      alpha-numerically sorted by its node name.
	    </p>
	    <description>
	      <term>where</term>
	      <item>This argument works as in <verb>getNode()</verb>
		(see <pageref refid="File.getNode">page</pageref>),
		referencing the node to be acted upon.
	      </item>

	      <term>classname</term> <item>If the name of a class
		derived from <verb>Node</verb> is supplied in the
		<em>classname</em> parameter, only instances of that
		class (or subclasses of it) will be returned.
	      </item>
	    </description>
	  </subsubsection>

	  <subsubsection id="File.listNodes">
	    <heading>listNodes(where, classname=None)</heading>

	    <p>Returns a <em>list</em> with children nodes hanging
	      from <em>where</em>.  The list is alpha-numerically
	      sorted by node name.
	    </p>
	    <description>
	      <term>where</term>
	      <item>This argument works as in <verb>getNode()</verb>
		(see <pageref refid="File.getNode">page</pageref>),
		referencing the node to be acted upon.
	      </item>

	      <term>classname</term> <item>If the name of a class
		derived from <verb>Node</verb> is supplied in the
		<em>classname</em> parameter, only instances of that
		class (or subclasses of it) will be returned.
	      </item>
	    </description>
	  </subsubsection>

	  <subsubsection id="File.removeNode">
	    <heading>removeNode(where, name=None, recursive=False)</heading>

	    <p>Removes the object node
	      <em>name</em> under <em>where</em> location.
	    </p>
	    <description>
	      <term>where, name</term>
	      <item>These arguments work as in <verb>getNode()</verb>
		(see <pageref refid="File.getNode">page</pageref>),
		referencing the node to be acted upon.
	      </item>

	      <term>recursive</term>
	      <item>If not supplied, the object will be removed
		only if it has no children;
		if it does, a <verb>NodeError</verb> will be raised.
		If supplied with a true value,
		the object and all its descendants will be completely removed.
	      </item>
	    </description>
	  </subsubsection>

	  <subsubsection id="File.copyNode">
	    <heading>
	      copyNode(where, newparent=None, newname=None, name=None,
	      overwrite=False, recursive=False, **kwargs)
	    </heading>

	    <p>Copy the node specified by <em>where</em> and <em>name</em>
	      to <em>newparent/newname</em>.</p>

	    <description>
	      <term>where, name</term>
	      <item>These arguments work as in <verb>getNode()</verb>
		(see <pageref refid="File.getNode">page</pageref>),
		referencing the node to be acted upon.
	      </item>

	      <term>newparent</term>
	      <item>The destination group that the node will be copied to
		(a path name or a <verb>Group</verb> instance).
		If <em>newparent</em> is <verb>None</verb>,
		the parent of the source node is selected as the new parent.
	      </item>

	      <term>newname</term> <item>The name to be assigned to
	      the new copy in its destination (a string).  If
	      <em>newname</em> is <verb>None</verb> or not specified,
	      the name of the source node is used.
	      </item>

	      <term>overwrite</term> <item>Whether the possibly
	      existing node <em>newparent/newname</em> should be
	      overwritten or not.  Note that trying to copy over an
	      existing node without overwriting it will issue a
	      <verb>NodeError</verb>.
	      </item>

	      <term>recursive</term>
	      <item>Specifies whether the copy should recurse
		into children of the copied node.
		This argument is ignored for leaf nodes.
		The default is not recurse.
	      </item>

	      <term>kwargs</term>
	      <item>
		Additional keyword arguments may be passed to customize
		the copying process.
		The supported arguments depend on the kind of node being copied.
		The following are some of them:
	      </item>

	      <term>title</term>
	      <item>The new title for the destination.
		If <verb>None</verb>, the original title is used.
		This only applies to the topmost node for recursive copies.
	      </item>

	      <term>filters</term>
	      <item>Specifying this parameter overrides the original
		filter properties in the source node.
		If specified, it must be an instance of the <verb>Filters</verb>
		class (see section <ref refid="FiltersClassDescr"></ref>).
		The default is to copy the filter attribute
		from the source node.
	      </item>

	      <term>copyuserattrs</term>
	      <item>You can prevent the user attributes from being copied
		by setting this parameter to <verb>False</verb>.
		The default is to copy them.
	      </item>

	      <term>start, stop, step</term>
	      <item>Specify the range of rows in child leaves to be copied;
		the default is to copy all the rows.
	      </item>

	      <term>stats</term>
	      <item>This argument may be used to collect statistics
		on the copy process.  When used, it should be a dictionary
		with keys <verb>groups</verb>, <verb>leaves</verb>
		and <verb>bytes</verb> having a numeric value.
		Their values will be incremented to reflect the number of
		groups, leaves and bytes, respectively,
		that have been copied in the operation.
	      </item>
	    </description>
	  </subsubsection>

	  <subsubsection id="File.renameNode">

	    <heading>
	      renameNode(where, newname, name=None)
	    </heading>

	    <p>Change the name of the node specified by <em>where</em> and <em>name</em>
	      to <em>newname</em>.</p>

	    <description>
	      <term>where, name</term>
	      <item>These arguments work as in <verb>getNode()</verb>
		(see <pageref refid="File.getNode">page</pageref>),
		referencing the node to be acted upon.
	      </item>

	      <term>newname</term>
	      <item>The new name to be assigned to the node (a string).
	      </item>
	    </description>
	  </subsubsection>

	  <subsubsection id="File.moveNode">
	    <heading>
	      moveNode(where, newparent=None, newname=None, name=None, overwrite=False)
	    </heading>

	    <p>Move the node specified by <em>where</em> and <em>name</em>
	      to <em>newparent/newname</em>.</p>

	    <description>
	      <term>where, name</term>
	      <item>These arguments work as in <verb>getNode()</verb>
		(see <pageref refid="File.getNode">page</pageref>),
		referencing the node to be acted upon.
	      </item>

	      <term>newparent</term>
	      <item>The destination group the node will be moved to
		(a path name or a <verb>Group</verb> instance).
		If <em>newparent</em> is <verb>None</verb>,
		the original node parent is selected as the new parent.
	      </item>

	      <term>newname</term>
	      <item>The new name to be assigned to the node in its destination (a string).
		If <em>newname</em> is <verb>None</verb> or not specified,
		the original node name is used.
	      </item>
	    </description>
	  </subsubsection>

	  <subsubsection id="walkGroupsDescr">
	    <heading>walkGroups(where='/')</heading>

	    <p><em>Iterator</em> that returns the list of Groups (not
	      Leaves) hanging from (and including) <em>where</em>.
	      The <em>where</em> Group is listed first (pre-order),
	      then each of its child Groups (following an
	      alpha-numerical order) is also traversed, following the
	      same procedure.  If <em>where</em> is not supplied, the
	      root object is used.
	    </p>
	    <description>

		<term>where</term> <item>The origin group. Can be a
		  path string or <verb>Group</verb> instance.
		</item>

	    </description>
	  </subsubsection>

	  <subsubsection id="File.walkNodes">
	    <heading>walkNodes(where="/", classname="")</heading>

	    <p>Recursively iterate over the nodes in the
	      <verb>File</verb> instance. It takes two parameters:</p>

	    <description>

	      <term>where</term> <item>If supplied, the iteration
	      starts from (and includes) this group.</item>

	      <term>classname</term> <item><em>(String)</em> If
	      supplied, only instances of this class are
	      returned.</item>

	    </description>

	    <p>Example of use:</p>

	    <verbatim>
	      # Recursively print all the nodes hanging from '/detector'
	      print "Nodes hanging from group '/detector':"
	      for node in h5file.walkNodes("/detector"):
	          print node
	    </verbatim>

	  </subsubsection> <!-- walkNodes -->

	  <subsubsection id="File.copyChildren">

	    <heading>copyChildren(srcgroup, dstgroup,
	      overwrite=False, recursive=False, **kwargs)
	    </heading>

	    <p>Copy the children of a group into another group.</p>

	    <p>This method copies the nodes hanging from the source group <verb>srcgroup</verb>
	      into the destination group <verb>dstgroup</verb>.
	      Existing destination nodes can be replaced by asserting
	      the <verb>overwrite</verb> argument.
	      If the <verb>recursive</verb> argument is true,
	      all descendant nodes of <verb>srcnode</verb> are recursively copied.
	    </p>

	    <p><verb>kwargs</verb> takes keyword arguments used to customize
	      the copying process.
	      See the documentation of <verb>Group._f_copyChildren()</verb>
	      (see <ref refid="Group._f_copyChildren"></ref>)
	      for a description of those arguments.
	    </p>
	  </subsubsection> <!-- copyChildren -->

	  <subsubsection id="File.copyFile">
	    <heading>copyFile(dstfilename, overwrite=False, **kwargs)</heading>

	    <p>Copy the contents of this file to <verb>dstfilename</verb>.</p>

	    <p><verb>dstfilename</verb> must be a path string
	      indicating the name of the destination file.  If it
	      already exists, the copy will fail with an
	      <verb>IOError</verb>, unless the <verb>overwrite</verb>
	      argument is true, in which case the destination file
	      will be overwritten in place. In this last case, the
	      destination file should be closed or ugly errors will
	      happen.
	    </p>

	    <p>Additional keyword arguments may be passed to customize the copying process.
	      For instance, title and filters may be changed,
	      user attributes may be or may not be copied, data may be sub-sampled,
	      stats may be collected, etc.
	      Arguments unknown to nodes are simply ignored.
	      Check the documentation for copying operations of nodes
	      to see which options they support.
	    </p>

	    <p>Copying a file usually has the beneficial side effect
	      of creating a more compact and cleaner version of the original file.
	    </p>
	  </subsubsection> <!-- copyFile -->

	  <subsubsection>
	    <heading>flush()</heading>

	    <p>Flush all the leaves in the object tree.
	    </p>
	  </subsubsection>

	  <subsubsection>
	    <heading>close()</heading>

	    <p>Flush all the leaves in object tree and close the file.
	    </p>
	  </subsubsection>


	  <subsubsection>
	    <heading>Undo/Redo support</heading>

	    <description>
	      <term id="File.isUndoEnabled">
		isUndoEnabled()
	      </term>
	      <item>
		<p>Is the Undo/Redo mechanism enabled?</p> <p>Returns
		<verb>True</verb> if the Undo/Redo mechanism has been
		enabled for this file, <verb>False</verb> otherwise.
		Please, note that this mechanism is persistent, so a
		newly opened PyTables file may already have Undo/Redo
		support.
		</p>
	      </item>

	      <term id="File.enableUndo">
		enableUndo(filters=Filters(complevel=1))
	      </term>
	      <item>
		<p>Enable the Undo/Redo mechanism.</p>
		<p>This operation prepares the database for undoing and redoing
		  modifications in the node hierarchy.
		  This allows <verb>mark()</verb>, <verb>undo()</verb>,
		  <verb>redo()</verb> and other methods to be called.
		</p>
		<p>The <verb>filters</verb> argument, when specified,
		  must be an instance of class <verb>Filters</verb>
		  (see <ref refid="FiltersClassDescr">section</ref>)
		  and is meant for setting the compression values for
		  the action log. The default is having compression
		  enabled, as the gains in terms of space can be
		  considerable. You may want to disable compression if
		  you want maximum speed for Undo/Redo operations.
		</p>
		<p>Calling <verb>enableUndo()</verb> when the
		  Undo/Redo mechanism is already enabled raises an
		  <verb>UndoRedoError</verb>.</p>
	      </item>

	      <term id="File.disableUndo">
		disableUndo()
	      </term>
	      <item>
		<p>Disable the Undo/Redo mechanism.</p>

		<p>Disabling the Undo/Redo mechanism leaves the
		  database in the current state and forgets past and
		  future database states.  This makes
		  <verb>mark()</verb>, <verb>undo()</verb>,
		  <verb>redo()</verb> and other methods fail with an
		  <verb>UndoRedoError</verb>.
		</p>

		<p>Calling <verb>disableUndo()</verb> when the
		  Undo/Redo mechanism is already disabled raises an
		  <verb>UndoRedoError</verb>.
		</p>
	      </item>

	      <term id="File.mark">
		mark(name=None)
	      </term>
	      <item>
		<p>Mark the state of the database.</p>
		<p>Creates a mark for the current state of the database.
		  A unique (and immutable) identifier for the mark is returned.
		  An optional <verb>name</verb> (a string) can be assigned to the mark.
		  Both the identifier of a mark and its name can be used
		  in <verb>undo()</verb> and <verb>redo()</verb> operations.
		  When the <verb>name</verb> has already been used for another mark,
		  an <verb>UndoRedoError</verb> is raised.
		</p>
		<p>This method can only be called when the Undo/Redo mechanism
		  has been enabled.
		  Otherwise, an <verb>UndoRedoError</verb> is raised.
		</p>
	      </item>

	      <term id="File.getCurrentMark">
		getCurrentMark()
	      </term>
	      <item>
		<p>Get the identifier of the current mark.</p>
		<p>Returns the identifier of the current mark.
		  This can be used to know the state of a database
		  after an application crash,
		  or to get the identifier of the initial implicit mark
		  after a call to <verb>enableUndo()</verb>.
		</p>
		<p>This method can only be called when the Undo/Redo mechanism
		  has been enabled.
		  Otherwise, an <verb>UndoRedoError</verb> is raised.
		</p>
	      </item>

	      <term id="File.undo">
		undo(mark=None)
	      </term>
	      <item>
		<p>Go to a past state of the database.</p>
		<p>Returns the database to the state associated with
		  the specified <verb>mark</verb>.
		  Both the identifier of a mark and its name can be used.
		  If the <verb>mark</verb> is omitted, the last created mark is used.
		  If there are no past marks, or the specified <verb>mark</verb>
		  is not older than the current one,
		  an <verb>UndoRedoError</verb> is raised.
		</p>
		<p>This method can only be called when the Undo/Redo mechanism
		  has been enabled.
		  Otherwise, an <verb>UndoRedoError</verb> is raised.
		</p>
	      </item>

	      <term id="File.redo">
		redo(mark=None)
	      </term>
	      <item>
		<p>Go to a future state of the database.</p>
		<p>Returns the database to the state associated with
		  the specified <verb>mark</verb>.
		  Both the identifier of a mark and its name can be used.
		  If the <verb>mark</verb> is omitted, the next created mark is used.
		  If there are no future marks, or the specified <verb>mark</verb>
		  is not newer than the current one,
		  an <verb>UndoRedoError</verb> is raised.
		</p>
		<p>This method can only be called when the Undo/Redo mechanism
		  has been enabled.
		  Otherwise, an <verb>UndoRedoError</verb> is raised.
		</p>
	      </item>

	      <term id="File.goto">
		goto(mark)
	      </term>
	      <item>
		<p>Go to a specific mark of the database.</p>
		<p>Returns the database to the state associated with
		  the specified <verb>mark</verb>.
		  Both the identifier of a mark and its name can be used.
		</p>
		<p>This method can only be called when the Undo/Redo mechanism
		  has been enabled.
		  Otherwise, an <verb>UndoRedoError</verb> is raised.
		</p>
	      </item>
	    </description>
	  </subsubsection>
	</subsection>

	<subsection>
	  <heading><visual markup="tt">File</visual> special
	      methods</heading>

	  <p>Following are described the methods that automatically
	    trigger actions when a <verb>File</verb> instance is
	    accessed in a special way.
	  </p>

	  <subsubsection id="File.__contains__">
	    <heading>__contains__(path)</heading>
	    <p>Is there a node with that <em>path</em>?</p>
	    <p>Returns <verb>True</verb> if the file has a node
	      with the given <em>path</em> (a string),
	      <verb>False</verb> otherwise.
	    </p>
	  </subsubsection>

	  <subsubsection id="File.__iter__">
	    <heading>__iter__()</heading>

	    <p>Iterate over the children on the <verb>File</verb>
	      instance. However, this does not accept parameters. This
	      iterator <em>is recursive</em>.</p>

	    <p>Example of use:</p>

	    <verbatim>
	      # Recursively list all the nodes in the object tree
	      h5file = tables.openFile("vlarray1.h5")
	      print "All nodes in the object tree:"
	      for node in h5file:
	          print node
	    </verbatim>

	  </subsubsection>

	  <subsubsection id="File.__str__">
	    <heading>__str__()</heading>

	    <p>Prints a short description of the <verb>File</verb>
	      object.</p>

	    <p>Example of use:</p>

	    <verbatim>
>>> f=tables.openFile("data/test.h5")
>>> print f
data/test.h5 (File) 'Table Benchmark'
Last modif.: 'Mon Sep 20 12:40:47 2004'
Object Tree:
/ (Group) 'Table Benchmark'
/tuple0 (Table(100L,)) 'This is the table title'
/group0 (Group) ''
/group0/tuple1 (Table(100L,)) 'This is the table title'
/group0/group1 (Group) ''
/group0/group1/tuple2 (Table(100L,)) 'This is the table title'
/group0/group1/group2 (Group) ''
	    </verbatim>
	  </subsubsection>

	  <subsubsection id="File.__repr__">
	    <heading>__repr__()</heading>

	    <p>Prints a detailed description of the <verb>File</verb>
	      object.</p>

	  </subsubsection>

	</subsection>


      </section>


      <section id="NodeClassDescr">
	<heading>The <visual markup="tt">Node</visual> class</heading>

	<!-- Exterminate! Annihilate! Destroy! M-q! (ivb ;) -->
	<p>This is the base class for <em>all</em> nodes in a PyTables hierarchy.
	  It is an abstract class, i.e. it may not be directly instantiated;
	  however, every node in the hierarchy is an instance of this class.
	</p>

	<p>A PyTables node is always hosted in a PyTables <em>file</em>,
	  under a <em>parent group</em>,
	  at a certain <em>depth</em> in the node hierarchy.
	  A node knows its own <em>name</em> in the parent group
	  and its own <em>path name</em> in the file.
	  When using a translation map (see <ref refid="FileClassDescr"></ref>),
	  its <em>HDF5 name</em> might differ from its PyTables name.
	</p>

	<p>All the previous information is location-dependent,
	  i.e. it may change when moving or renaming a node in the hierarchy.
	  A node also has location-independent information,
	  such as its <em>HDF5 object identifier</em> and its <em>attribute set</em>.
	</p>

	<p>This class gathers the operations and attributes
	  (both location-dependent and independent) which are common
	  to all PyTables nodes, whatever their type is.
	  Nonetheless, due to natural naming restrictions,
	  the names of all of these members start with a reserved prefix
	  (see <ref refid="GroupClassDescr"></ref>).
	</p>

	<p>Sub-classes with no children (i.e. leaf nodes) may define
	  new methods, attributes and properties to avoid natural naming restrictions.
	  For instance, <verb>_v_attrs</verb> may be shortened to <verb>attrs</verb>
	  and <verb>_f_rename</verb> to <verb>rename</verb>.
	  However, the original methods and attributes should still be available.
	</p>

	<subsection>
	  <heading><visual markup="tt">Node</visual> instance variables</heading>

	  <subsubsection>
	    <heading>Location dependent</heading>

	    <description>
	      <term id="Node._v_file">
		_v_file
	      </term>
	      <item>The hosting <verb>File</verb> instance
		(see <ref refid="FileClassDescr"></ref>).
	      </item>

	      <term id="Node._v_parent">
		_v_parent
	      </term>
	      <item>The parent <verb>Group</verb> instance
		(see <ref refid="GroupClassDescr"></ref>).
	      </item>

	      <term id="Node._v_depth">
		_v_depth
	      </term>
	      <item>The depth of this node in the tree
		(an non-negative integer value).
	      </item>

	      <term id="Node._v_name">
		_v_name
	      </term>
	      <item>The name of this node in its parent group
		(a string).
	      </item>

	      <term id="Node._v_hdf5name">
		_v_hdf5name
	      </term>
	      <item>The name of this node in the hosting HDF5 file
		(a string).
	      </item>

	      <term id="Node._v_pathname">
		_v_pathname
	      </term>
	      <item>The path of this node in the tree (a string).</item>

	      <term id="Node._v_rootgroup">
		_v_rootgroup
	      </term>
	      <item>The root group instance.
		This is deprecated; please use <verb>node._v_file.root</verb>.
	      </item>
	    </description>
	  </subsubsection>

	  <subsubsection>
	    <heading>Location independent</heading>

	    <description>
	      <term id="Node._v_objectID">
		_v_objectID
	      </term>
	      <item>The identifier of this node in the hosting HDF5 file.</item>

	      <term id="Node._v_attrs">
		_v_attrs
	      </term>
	      <item>The associated <verb>AttributeSet</verb> instance
		(see <ref refid="AttributeSetClassDescr"></ref>).
	      </item>
	    </description>
	  </subsubsection>

	  <subsubsection>
	    <heading>Attribute shorthands</heading>

	    <description>
	      <term id="Node._v_title">
		_v_title
	      </term>
	      <item>A description of this node.
		A shorthand for <verb>TITLE</verb> attribute.
	      </item>
	    </description>
	  </subsubsection>
	</subsection>

	<subsection>
	  <heading><visual markup="tt">Node</visual> methods</heading>

	  <subsubsection>
	    <heading>Hierarchy manipulation</heading>

	    <description>
	      <term id="Node._f_close">
		_f_close()
	      </term>
	      <item>
		<p>Close this node in the tree.</p>
		<p>This releases all resources held by the node,
		  so it should not be used again.
		  On nodes with data, it may be flushed to disk.
		</p>
		<p>The closing operation is <em>not</em> recursive,
		  i.e. closing a group does not close its children.
		</p>
	      </item>

	      <term id="Node._f_isOpen">
		_f_isOpen()
	      </term>
	      <item><p>Is this node open?</p></item>

	      <term id="Node._f_remove">
		_f_remove(recursive=False)
	      </term>
	      <item>
		<p>Remove this node from the hierarchy.</p>
		<p>If the node has children, recursive removal must be
		  stated by giving <verb>recursive</verb> a true value;
		  otherwise, a <verb>NodeError</verb> will be raised.
		</p>
	      </item>

	      <term id="Node._f_rename">
		_f_rename(newname)
	      </term>
	      <item>
		<p>Rename this node in place.</p>
		<p>Changes the name of a node to <em>newname</em> (a string).</p>
	      </item>

	      <term id="Node._f_move">
		_f_move(newparent=None, newname=None, overwrite=False)
	      </term>
	      <item>
		<p>Move or rename this node.</p>
		<p>Moves a node into a new parent group,
		  or changes the name of the node.
		  <verb>newparent</verb> can be a <verb>Group</verb> object
		  or a pathname in string form.
		  If it is not specified or <verb>None</verb>,
		  the current parent group is chosen as the new parent.
		  <verb>newname</verb> must be a string with a new name.
		  If it is not specified or <verb>None</verb>, the current name
		  is chosen as the new name.
		</p>
		<p>Moving a node across databases is not allowed, nor it is
		  moving a node <em>into</em> itself.
		  These result in a <verb>NodeError</verb>.
		  However, moving a node <em>over</em> itself is allowed
		  and simply does nothing.
		  Moving over another existing node is similarly not allowed,
		  unless the optional <verb>overwrite</verb> argument is true,
		  in which case that node is recursively removed before moving.
		</p>
		<p>Usually, only the first argument will be used, effectively
		  moving the node to a new location without changing its name.
		  Using only the second argument is equivalent to
		  renaming the node in place.
		</p>
	      </item>

	      <term id="Node._f_copy">
		_f_copy(newparent=None, newname=None,
		        overwrite=False, recursive=False, **kwargs)
	      </term>
	      <item>
		<p>Copy this node and return the new node.</p>
		<p>Creates and returns a copy of the node,
		  maybe in a different place in the hierarchy.
		  <verb>newparent</verb> can be a <verb>Group</verb> object
		  or a pathname in string form.
		  If it is not specified or <verb>None</verb>,
		  the current parent group is chosen as the new parent.
		  <verb>newname</verb> must be a string with a new name.
		  If it is not specified or <verb>None</verb>, the current name
		  is chosen as the new name.
		  If <verb>recursive</verb> copy is stated,
		  all descendants are copied as well.
		</p>
		<p>Copying a node across databases is supported but can not be undone.
		  Copying a node over itself is not allowed,
		  nor it is recursively copying a node into itself.
		  These result in a <verb>NodeError</verb>.
		  Copying over another existing node is similarly not allowed,
		  unless the optional <verb>overwrite</verb> argument is true,
		  in which case that node is recursively removed before copying.
		</p>
		<p>Additional keyword arguments may be passed to customize
		  the copying process.
		  For instance, title and filters may be changed, user attributes
		  may be or may not be copied, data may be sub-sampled,
		  stats may be collected, etc.
		  See the documentation for the particular node type.
		</p>
		<p>Using only the first argument is equivalent to copying the node
		  to a new location without changing its name.
		  Using only the second argument is equivalent to making a copy
		  of the node in the same group.
		</p>
	      </item>

	      <term id="Node._f_isVisible">
		_f_isVisible()
	      </term>
	      <item><p>Is this node visible?</p></item>
	    </description>
	  </subsubsection>

	  <subsubsection>
	    <heading>Attribute handling</heading>

	    <description>
	      <term id="Node._f_getAttr">
		_f_getAttr(name)
	      </term>
	      <item>
		<p>Get a PyTables attribute from this node.</p>
		<p>If the named attribute does not exist, an
		  <verb>AttributeError</verb> is raised.
		</p>
	      </item>

	      <term id="Node._f_setAttr">
		_f_setAttr(name, value)
	      </term>
	      <item>
		<p>Set a PyTables attribute for this node.</p>
		<p>If the node already has a large number of attributes,
		  a <verb>PerformanceWarning</verb> is issued.</p>
	      </item>

	      <term id="Node._f_delAttr">
		_f_delAttr(name)
	      </term>
	      <item>
		<p>Delete a PyTables attribute from this node.</p>
		<p>If the named attribute does not exist,
		  an <verb>AttributeError</verb> is raised.
		</p>
	      </item>
	    </description>
	  </subsubsection>
	</subsection>
      </section>


      <section id="GroupClassDescr">
	<heading>The <visual markup="tt">Group</visual> class</heading>

	<p>Instances of this class are a grouping structure containing
	  instances of zero or more groups or leaves, together with
	  supporting metadata.
	</p>

	<p>Working with groups and leaves is similar in many ways to
	  working with directories and files, respectively, in a Unix
	  filesystem. As with Unix directories and files, objects in
	  the object tree are often described by giving their full (or
	  absolute) path names. This full path can be specified either
	  as a string (like in <verb>'/group1/group2'</verb>) or as a
	  complete object path written in <em>natural name</em> schema
	  (like in <newline/><verb>file.root.group1.group2</verb>) as
	  discussed in the <ref
	  refid='ObjectTreeSection'>section</ref>.
	</p>

	<p>A collateral effect of the <em>natural naming</em> schema
	  is that names of <verb>Group</verb> members must be carefully chosen
	  to avoid colliding with existing children node names.
	  For this reason and not to pollute the children namespace,
	  it is explicitly forbidden to assign <em>normal</em> attributes
	  to Group instances, and all existing members start with some reserved prefixes,
	  like <verb>_f_</verb> (for methods) or <verb>_v_</verb> (for instance variables).
	  Any attempt to set a new child node whose name starts with one of these prefixes
	  will raise a <verb>ValueError</verb> exception.
	</p>

	<p>Another effect of natural naming is that
	  nodes having reserved Python names and other non-allowed Python names
	  (like for example <verb>$a</verb> or <verb>44</verb>)
	  can not be accessed using the <verb>node.child</verb> syntax.
	  You will be forced to use <verb>getattr(node, child)</verb>
	  and <verb>delattr(node, child)</verb> to access them.
	</p>

	<p>
	  You can also make use of the <verb>trMap</verb>
	  (translation map dictionary) parameter in the <verb>openFile</verb> function
	  (see section <ref refid="openFileDescr"></ref>) in order to
	  translate HDF5 names not suited for natural naming into more convenient ones.
	</p>

	<subsection>
	  <heading><visual markup="tt">Group</visual> instance
	    variables</heading>

	  <p>These instance variables are provided in addition to those
	    in <verb>Node</verb> (see <ref refid="NodeClassDescr"></ref>).
	  </p>

	  <description>
	    <term id="Group._v_nchildren">
	      _v_nchildren
	    </term>
	    <item>The number of children hanging from this group.</item>

	    <term id="Group._v_children">
	      _v_children
	    </term>
	    <item>Dictionary with all nodes hanging from this group.</item>

	    <term id="Group._v_groups">
	      _v_groups
	    </term>
	    <item>Dictionary with all groups hanging from this group.</item>

	    <term id="Group._v_leaves">
	      _v_leaves
	    </term>
	    <item>Dictionary with all leaves hanging from this group.</item>

	    <term id="Group._v_filters">
	      _v_filters
	    </term>
	    <item>Default filter properties for child nodes
	      &mdash;see <ref refid="FiltersClassDescr"></ref>.
	      A shorthand for <verb>FILTERS</verb> attribute.
	    </item>
	  </description>
	</subsection>

	<subsection>
	  <heading><visual markup="tt">Group</visual> methods</heading>

	  <p>This class defines the <verb>__setattr__</verb>,
	    <verb>__getattr__</verb> and <verb>__delattr__</verb> methods,
	    and they set, get and delete <em>ordinary Python attributes</em>
	    as normally intended.
	    In addition to that, <verb>__getattr__</verb> allows getting
	    <em>child nodes</em> by their name for the sake of
	    easy interaction on the command line,
	    as long as there is no Python attribute with the same name.
	    Groups also allow the interactive completion
	    (when using <verb>readline</verb>) of the names of child nodes.
	    For instance:

	    <verbatim>
	      nchild = group._v_nchildren  # get a Python attribute

	      # Add a Table child called "table" under "group".
	      h5file.createTable(group, 'table', myDescription)

	      table = group.table          # get the table child instance
	      group.table = 'foo'          # set a Python attribute
	      # (PyTables warns you here about using the name of a child node.)
	      foo = group.table            # get a Python attribute
	      del group.table              # delete a Python attribute
	      table = group.table          # get the table child instance again
	    </verbatim>
	  </p>

	  <p><visual markup="bf">Caveat: </visual>The following
	    methods are documented for completeness, and they can be
	    used without any problem. However, you should use the
	    high-level counterpart methods in the <verb>File</verb>
	    class, because these are most used in documentation and
	    examples, and are a bit more powerful than those exposed
	    here.
	  </p>

	  <p>These methods are provided in addition to those
	    in <verb>Node</verb> (see <ref refid="NodeClassDescr"></ref>).
	  </p>

	  <subsubsection id="Group._f_getChild">
	    <heading>_f_getChild(childname)
	    </heading>
	    <p>Get the child called <verb>childname</verb> of this group.</p>
	    <p>If the child exists (be it visible or not), it is returned.
	      Else, a <verb>NoSuchNodeError</verb> is raised.
	    </p>
	  </subsubsection>

	  <subsubsection id="Group._f_copy">
	    <heading>
	      _f_copy(newparent, newname, overwrite=False, recursive=False, **kwargs)
	    </heading>

	    <p>Copy this node and return the new one.</p>

	    <p>This method has the behavior described in <verb>Node._f_copy()</verb>
	      (see <pageref refid="Node._f_copy">page</pageref>).
	      In addition, it recognizes the following keyword arguments:
	    </p>

	    <description>
	      <term>title</term>
	      <item>The new title for the destination.
		If omitted or <verb>None</verb>, the original title is used.
		This only applies to the topmost node in recursive copies.
	      </item>

	      <term>filters</term>
	      <item>Specifying this parameter overrides the original
		filter properties in the source node.
		If specified, it must be an instance of the <verb>Filters</verb>
		class (see <ref refid="FiltersClassDescr">section</ref>).
		The default is to copy the filter properties
		from the source node.
	      </item>

	      <term>copyuserattrs</term>
	      <item>You can prevent the user attributes from being copied
		by setting this parameter to <verb>False</verb>.
		The default is to copy them.
	      </item>

	      <term>stats</term>
	      <item>This argument may be used to collect statistics
		on the copy process.  When used, it should be a dictionary
		with keys <verb>'groups'</verb>, <verb>'leaves'</verb>
		and <verb>'bytes'</verb> having a numeric value.
		Their values will be incremented to reflect the number of
		groups, leaves and bytes, respectively,
		that have been copied during the operation.
	      </item>
	    </description>
	  </subsubsection>

	  <subsubsection id="Group._f_iterNodes">
	    <heading>_f_iterNodes(classname=None)
	    </heading>
	    <p>Returns an <em>iterator</em> yielding all the object
	      nodes hanging from this instance. The nodes are
	      alpha-numerically sorted by its node name. If a
	      <em>classname</em> parameter is supplied, it will only
	      return instances of this class (or subclasses of it).
	    </p>
	  </subsubsection>

	  <subsubsection id="Group._f_listNodes">
	    <heading>_f_listNodes(classname=None)
	    </heading>
	    <p>Returns a <em>list</em> with all the object nodes
	      hanging from this instance. The list is
	      alpha-numerically sorted by node name. If a
	      <em>classname</em> parameter is supplied, it will only
	      return instances of this class (or subclasses of
	      it).
	    </p>
	  </subsubsection>

	  <subsubsection id="Group._f_walkGroups">
	    <heading>_f_walkGroups()
	    </heading>
	    <p>Iterate over the list of Groups (not Leaves) hanging
	      from (and including) <em>self</em>.  This Group is
	      listed first (pre-order), then each of its child Groups
	      (following an alpha-numerical order) is also traversed,
	      following the same procedure.
	    </p>
	  </subsubsection>

	  <subsubsection id="Group._f_walkNodes">
	    <heading>_f_walkNodes(classname=None, recursive=True)</heading>

	    <p>Iterate over the nodes in the <verb>Group</verb>
	      instance. It takes two parameters:</p>

	    <description>

	      <term>classname</term> <item><em>(String)</em> If
	      supplied, only instances of this class are
	      returned.</item>

	      <term>recursive</term> <item><em>(Integer)</em> If
	      false, only children hanging immediately after the group
	      are returned. If true, a recursion over all the groups
	      hanging from it is performed. </item>

	    </description>

	    <p>Example of use:</p>

	    <verbatim>
	      # Recursively print all the arrays hanging from '/'
	      print "Arrays the object tree '/':"
	      for array in h5file.root._f_walkNodes("Array", recursive=1):
	          print array
	    </verbatim>

	  </subsubsection>

	  <subsubsection id="Group._f_close">
	    <heading>
	      _f_close()
	    </heading>
	    <p>Close this node in the tree.</p>
	    <p>This method has the behavior described in <verb>Node._f_close()</verb>
	      (see <pageref refid="Node._f_close">page</pageref>).
	      It should be noted that this operation disables access
	      to nodes descending from this group.
	      Therefore, if you want to explicitly close them,
	      you will need to walk the nodes hanging from this group
	      <em>before</em> closing it.
	    </p>
	  </subsubsection>

	  <subsubsection id="Group._f_copyChildren">

	    <heading>_f_copyChildren(dstgroup,
	      overwrite=False, recursive=False, **kwargs)
	    </heading>

	    <p>Copy the children of this group into another group.</p>

	    <p>Children hanging directly from this group are copied into <verb>dstgroup</verb>,
	      which can be a <verb>Group</verb> (see <ref refid="GroupClassDescr"></ref>)
	      object or its pathname in string form.
	    </p>

	    <p>The operation will fail with a <verb>NodeError</verb>
	      if there is a child node in the destination group
	      with the same name as one of the copied children from this one,
	      unless <verb>overwrite</verb> is true;
	      in this case, the former child node is recursively removed
	      before copying the later.
	    </p>

	    <p>By default, nodes descending from children groups of this node
	      are not copied.  If the <verb>recursive</verb> argument is true,
	      all descendant nodes of this node are recursively copied.
	    </p>

	    <p>Additional keyword arguments may be passed to customize the copying process.
	      For instance, title and filters may be changed,
	      user attributes may be or may not be copied, data may be sub-sampled,
	      stats may be collected, etc.
	      Arguments unknown to nodes are simply ignored.
	      Check the documentation for copying operations of nodes
	      to see which options they support.
	    </p>
	  </subsubsection> <!-- _f_copyChildren -->

	</subsection>

	<subsection>
	  <heading><visual markup="tt">Group</visual> special
	      methods</heading>

	  <p>Following are described the methods that automatically
	    trigger actions when a <verb>Group</verb> instance is
	    accessed in a special way.
	  </p>

	  <subsubsection id="Group.__setattr__">
	    <heading>__setattr__(name, value)</heading>
	    <p>Set a Python attribute called <verb>name</verb>
	      with the given <verb>value</verb>.
	    </p>
	    <p>This method stores an <em>ordinary Python attribute</em> in the object.
	      It does <em>not</em> store new children nodes under this group;
	      for that, use the <verb>File.create*()</verb> methods
	      (see <ref refid="FileClassDescr"></ref>).
	      It does <em>neither</em> store a PyTables node attribute;
	      for that, use <verb>File.setNodeAttr()</verb>
	      (see <pageref refid="File.setNodeAttr">page</pageref>),
	      <verb>Node._f_setAttr()</verb>
	      (see <pageref refid="Node._f_setAttr">page</pageref>)
	      or <verb>Node._v_attrs</verb>
	      (see <pageref refid="Node._v_attrs">page</pageref>).
	    </p>
	    <p>If there is already a child node with the same <verb>name</verb>,
	      a <verb>NaturalNameWarning</verb> will be issued
	      and the child node will not be accessible via natural naming
	      nor <verb>getattr()</verb>.
	      It will still be available via <verb>File.getNode()</verb>
	      (see <pageref refid="File.getNode">page</pageref>),
	      <verb>Group._f_getChild()</verb>
	      (see <pageref refid="Group._f_getChild">page</pageref>)
	      and children dictionaries in the group (if visible).
	    </p>
	  </subsubsection>

	  <subsubsection id="Group.__getattr__">
	    <heading>__getattr__(name)</heading>
	    <p>Get a Python attribute or child node called <verb>name</verb>.</p>
	    <p>If the object has a Python attribute called <verb>name</verb>,
	      its value is returned.
	      Else, if the node has a child node called <verb>name</verb>,
	      it is returned.
	      Else, an <verb>AttributeError</verb> is raised.
	    </p>
	  </subsubsection>

	  <subsubsection id="Group.__delattr__">
	    <heading>__delattr__(name)</heading>
	    <p>Delete a Python attribute called <verb>name</verb>.</p>
	    <p>This method deletes an <em>ordinary Python attribute</em>
	      from the object.
	      It does <em>not</em> remove children nodes from this group;
	      for that, use <verb>File.removeNode()</verb>
	      (see <pageref refid="File.removeNode">page</pageref>)
	      or <verb>Node._f_remove()</verb>
	      (see <pageref refid="Node._f_remove">page</pageref>).
	      It does <em>neither</em> delete a PyTables node attribute;
	      for that, use <verb>File.delNodeAttr()</verb>
	      (see <pageref refid="File.delNodeAttr">page</pageref>),
	      <verb>Node._f_delAttr()</verb>
	      (see <pageref refid="Node._f_delAttr">page</pageref>)
	      or <verb>Node._v_attrs</verb>
	      (see <pageref refid="Node._v_attrs">page</pageref>).
	    </p>

	    <p>If there were an attribute and a child node
	      with the same <verb>name</verb>,
	      the child node will be made accessible again via natural naming.
	    </p>
	  </subsubsection>

	  <subsubsection id="Group.__contains__">
	    <heading>__contains__(name)</heading>
	    <p>Is there a child with that <em>name</em>?</p>
	    <p>Returns <verb>True</verb> if the group has a child node
	      (visible or hidden)
	      with the given <em>name</em> (a string),
	      <verb>False</verb> otherwise.
	    </p>
	  </subsubsection>

	  <subsubsection id="Group.__iter__">
	    <heading>__iter__()</heading>

	    <p>Iterate over the children on the group instance. However,
	      this does not accept parameters. This iterator is
	      <visual markup="bf">not</visual> recursive.</p>

	    <p>Example of use:</p>

	    <verbatim>
	      # Non-recursively list all the nodes hanging from '/detector'
	      print "Nodes in '/detector' group:"
	      for node in h5file.root.detector:
	          print node
	    </verbatim>

	  </subsubsection>

	  <subsubsection id="Group.__str__">
	    <heading>__str__()</heading>

	    <p>Prints a short description of the <verb>Group</verb>
	      object.</p>

	    <p>Example of use:</p>

	    <verbatim>
>>> f=tables.openFile("data/test.h5")
>>> print f.root.group0
/group0 (Group) 'First Group'
>>>
	    </verbatim>
	  </subsubsection>

	  <subsubsection id="Group.__repr__">
	    <heading>__repr__()</heading>

	    <p>Prints a detailed description of the <verb>Group</verb>
	      object.</p>

	    <p>Example of use:</p>

	    <verbatim>
>>> f=tables.openFile("data/test.h5")
>>> f.root.group0
/group0 (Group) 'First Group'
  children := ['tuple1' (Table), 'group1' (Group)]
>>>
	    </verbatim>
	  </subsubsection>
	</subsection>
      </section>

      <section id="LeafClassDescr">
	<heading>The <visual markup="tt">Leaf</visual> class</heading>

	<p>The goal of this class is to provide a place to put common
	  functionality of all its descendants as well as provide a
	  way to help classifying objects on the tree. A
	  <verb>Leaf</verb> object is an end-node, that is, a node
	  that can hang directly from a group object, but that is not
	  a group itself and, thus, it can not have descendants. Right
	  now, the set of end-nodes is composed by <verb>Table</verb>,
	  <verb>Array</verb>, <verb>CArray</verb>, <verb>EArray</verb>,
	  <verb>VLArray</verb> and <verb>UnImplemented</verb> class
	  instances. In fact, all the previous classes inherit from
	  the <verb>Leaf</verb> class.
	</p>

	<subsection>
	  <heading><visual markup="tt">Leaf</visual> instance
	    variables</heading>

	  <p>These instance variables are provided in addition to those
	    in <verb>Node</verb> (see <ref refid="NodeClassDescr"></ref>).
	  </p>

	  <description>
	    <term id="Leaf.shape">
	      shape
	    </term>
	    <item>The shape of data in the leaf.</item>

	    <term id="Leaf.byteorder">
	      byteorder
	    </term>
	    <item>The byte ordering of data in the leaf.</item>

	    <term id="Leaf.filters">
	      filters
	    </term>
	    <item>Filter properties for this leaf
	      &mdash;see <ref refid="FiltersClassDescr"></ref>.
	    </item>

	    <term id="Leaf.name">
	      name
	    </term>
	    <item>The name of this node in its parent group
	      (a string).
	      An alias for <verb>Node._v_name</verb>.
	    </item>

	    <term id="Leaf.hdf5name">
	      hdf5name
	    </term>
	    <item>The name of this node in the hosting HDF5 file
	      (a string).
	      An alias for <verb>Node._v_hdf5name</verb>.
	    </item>

	    <term id="Leaf.objectID">
	      objectID
	    </term>
	    <item>
	      The identifier of this node in the hosting HDF5 file.
	      An alias for <verb>Node._v_objectID</verb>.
	    </item>

	    <term id="Leaf.attrs">
	      attrs
	    </term>
	    <item>The associated <verb>AttributeSet</verb> instance
	      (see <ref refid="AttributeSetClassDescr"></ref>).
	      An alias for <verb>Node._v_attrs</verb>.
	    </item>

	    <term id="Leaf.title">
	      title
	    </term>
	    <item>
	      A description for this node.
	      An alias for <verb>Node._v_title</verb>.
	    </item>
	  </description>
	</subsection>

	<subsection>
	  <heading><visual markup="tt">Leaf</visual> methods</heading>

	  <subsubsection id="Leaf.flush">
	    <heading>
	      flush()
	    </heading>
	    <p>Flush pending data to disk.</p>

	    <p>Saves whatever remaining buffered data to disk. It also
	      releases I/O buffers, so, if you are filling many
	      objects (i.e. tables) in the same PyTables session,
	      please, call <verb>flush()</verb> extensively so as to
	      help PyTables to keep memory requirements low.
	    </p>

	  </subsubsection>

	  <!--
	  ivb (2005-03-07):
	  Why do I place this here?
	  Because there is one extra argument (it changes the interface)
	  and it still is a public method.
	  -->
	  <subsubsection id="Leaf._f_close">
	    <heading>
	      _f_close(flush=True)
	    </heading>
	    <p>Close this node in the tree.</p>
	    <p>This method has the behavior described in <verb>Node._f_close()</verb>
	      (see <pageref refid="Node._f_close">page</pageref>).
	      Besides that, the optional argument <verb>flush</verb> tells
	      whether to flush pending data to disk or not before closing.
	    </p>
	  </subsubsection>

	  <subsubsection id="Leaf.close">
	    <heading>
	      close(flush=True)
	    </heading>
	    <p>Close this node in the tree.</p>
	    <p>This method is completely equivalent to <verb>_f_close()</verb>.</p>
	  </subsubsection>

	  <subsubsection id="Leaf.isOpen">
	    <heading>
	      isOpen()
	    </heading>
	    <p>Is this node open?</p>
	    <p>This method is completely equivalent to <verb>_f_isOpen()</verb>.</p>
	  </subsubsection>

	  <subsubsection id="Leaf.remove">
	    <heading>
	      remove()
	    </heading>
	    <p>Remove this node from the hierarchy.</p> <p>This method
	    has the behavior described in
	    <verb>Node._f_remove()</verb> (see <pageref
	    refid="Node._f_remove">page</pageref>).  Please, note that
	    there is no <verb>recursive</verb> flag since leaves do
	    not have child nodes.
	    </p>
	  </subsubsection>

	  <subsubsection id="Leaf.copy">
	    <heading>
	      copy(newparent, newname, overwrite=False, **kwargs)
	    </heading>

	    <p>Copy this node and return the new one.</p>

	    <p>This method has the behavior described in
	      <verb>Node._f_copy()</verb> (see <pageref
	      refid="Node._f_copy">page</pageref>).  Please, note that
	      there is no <verb>recursive</verb> flag since leaves do
	      not have child nodes.  In addition, this method
	      recognizes the following keyword arguments:
	    </p>

	    <description>
	      <term>title</term>
	      <item>The new title for the destination.
		If omitted or <verb>None</verb>, the original title is used.
	      </item>

	      <term>filters</term>
	      <item>Specifying this parameter overrides the original
		filter properties in the source node.
		If specified, it must be an instance of the <verb>Filters</verb>
		class (see <ref refid="FiltersClassDescr">section</ref>).
		The default is to copy the filter properties
		from the source node.
	      </item>

	      <term>copyuserattrs</term>
	      <item>You can prevent the user attributes from being copied
		by setting this parameter to <verb>False</verb>.
		The default is to copy them.
	      </item>

	      <term>start, stop, step</term>
	      <item>Specify the range of rows in child leaves to be copied;
		the default is to copy all the rows.
	      </item>

	      <term>stats</term>
	      <item>This argument may be used to collect statistics
		on the copy process.  When used, it should be a dictionary
		with keys <verb>'groups'</verb>, <verb>'leaves'</verb>
		and <verb>'bytes'</verb> having a numeric value.
		Their values will be incremented to reflect the number of
		groups, leaves and bytes, respectively,
		that have been copied during the operation.
	      </item>
	    </description>
	  </subsubsection>

	  <subsubsection id="Leaf.rename">
	    <heading>
	      rename(newname)
	    </heading>
	    <p>Rename this node in place.</p>
	    <p>This method has the behavior described in <verb>Node._f_rename()</verb>
	      (see <pageref refid="Node._f_rename">page</pageref>).
	    </p>
	  </subsubsection>

	  <subsubsection id="Leaf.move">
	    <heading>
	      move(newparent=None, newname=None, overwrite=False)
	    </heading>
	    <p>Move or rename this node.</p>
	    <p>This method has the behavior described in <verb>Node._f_move()</verb>
	      (see <pageref refid="Node._f_move">page</pageref>).
	    </p>
	  </subsubsection>

	  <subsubsection id="Leaf.isVisible">
	    <heading>
	      _f_isVisible()
	    </heading>
	    <p>Is this node visible?</p>
	    <p>This method has the behavior described in <verb>Node._f_isVisible()</verb>
	      (see <pageref refid="Node._f_isVisible">page</pageref>).</p>
	  </subsubsection>

	  <subsubsection id="Leaf.getAttr">
	    <heading>
	      getAttr(name)
	    </heading>

	    <p>Get a PyTables attribute from this node.
	    </p>

	    <p>This method has the behavior described in
	      <verb>Node._f_getAttr()</verb> (see <pageref
	      refid="Node._f_getAttr">page</pageref>).
	    </p>
	  </subsubsection>

	  <subsubsection id="Leaf.setAttr">
	    <heading>
	      setAttr(name, value)
	    </heading>
	    <p>Set a PyTables attribute for this node.</p> <p>This
	      method has the behavior described in
	      <verb>Node._f_setAttr()</verb> (see <pageref
	      refid="Node._f_setAttr">page</pageref>).
	    </p>
	  </subsubsection>

	  <subsubsection id="Leaf.delAttr">
	    <heading>
	      delAttr(name)
	    </heading>

	    <p>Delete a PyTables attribute from this node.
	    </p>

	    <p>This method has the behavior described in
	      <verb>Node._f_delAttr()</verb> (see <pageref
	      refid="Node._f_delAttr">page</pageref>).
	    </p>
	  </subsubsection>
	</subsection>
      </section>

      <section id="TableClassDescr">
	<heading>The <visual markup="tt">Table</visual> class</heading>

	<p>Instances of this class represents table objects in the
	  object tree. It provides methods to read/write data and
	  from/to table objects in the file.
	</p>
	<p>Data can be read from or written to tables by accessing to
	  an special object that hangs from <verb>Table</verb>. This
	  object is an instance of the <verb>Row</verb> class (see
	  <ref refid="RowClassDescr"></ref>). See the tutorial
	  sections <ref refid="usage">chapter</ref> on how to use the
	  <verb>Row</verb> interface. The columns of the tables can
	  also be easily accessed (and more specifically, they can be
	  read but not written) by making use of the
	  <verb>Column</verb> class, through the use of an
	  <em>extension</em> of the natural naming schema applied
	  inside the tables. See the <ref
	  refid="ColumnClassDescr">section</ref> for some examples of
	  use of this capability.
	</p>

	<p>Note that this object inherits all the public attributes
	  and methods that <verb>Leaf</verb> already has.
	</p>

	<p>Finally, during the description of the different methods,
	  there will appear references to a particular object called
	  <verb>NestedRecArray</verb>. This inherits from
	  <verb>numarray.records.RecArray</verb> and is designed to
	  keep columns that have nested datatypes. Please, see <ref
	  refid="NestedRecArrayClassDescr">appendix</ref> for info on
	  these objects.
	</p>

	  <subsection id="TableInstanceVariablesDescr">
	    <heading><visual markup="tt">Table</visual> instance
	      variables</heading>
	    <description>
	      <term>description</term> <item>A <verb>Description</verb>
		(see <ref refid="DescriptionClassDescr"></ref>) instance
		describing the structure of this table.
	      </item>
	      <term>row</term>
	      <item>The associated <verb>Row</verb> instance
		(see <ref refid="RowClassDescr"></ref>).
	      </item>
	      <term>nrows</term> <item>The number of rows in this table.
	      </item>
	      <term>rowsize</term>
	      <item>The size in bytes of each row in the table.</item>

	      <term>cols</term>
	      <item>A <verb>Cols</verb>
		(see <ref refid="ColsClassDescr">section</ref>)
		instance that serves as an accessor to <verb>Column</verb>
		(see <ref refid="ColumnClassDescr">section</ref>) objects.
	      </item>
	      <term>colnames</term>
	      <item>A tuple containing the (possibly nested) names
		of the columns in the table.
	      </item>
	      <term>coltypes</term>
	      <item>Maps the name of a column to its data type.</item>
	      <term>colstypes</term>
	      <item>Maps the name of a column to its data string type.</item>
	      <term>colshapes</term>
	      <item>Maps the name of a column to it shape.</item>
	      <term>colitemsizes</term>
	      <item>Maps the name of a column to the size of its base
		items.</item>
	      <term>coldflts</term>
	      <item>Maps the name of a column to its default.</item>
	      <term>colindexed</term>
	      <item>Is the column which name is used as a key indexed?
		(dictionary)
	      </item>
	      <term>indexed</term> <item>Does this table have any
		indexed columns?
	      </item>
	      <term>indexprops</term> <item>Index properties for this
		table (an <verb>IndexProps</verb> instance, see <ref
		refid="IndexPropsClassDescr"></ref>).
	      </item>
	      <term>flavor</term> <item>The default flavor for this
		  table. This determines the type of objects returned
		  during input (i.e. read) operations. It can take the
		  <em>"numarray"</em> (default) or <em>"numpy"</em>
		  values. Its value is derived from the
		  <verb>_v_flavor</verb> attribute of the
		  <verb>IsDescription</verb> metaclass (see <ref
		  refid="IsDescriptionClassDescr"></ref>) or, if the
		  table has been created directly from a
		  <verb>numarray</verb> or <verb>NumPy</verb> object,
		  the flavor is set to the appropriate value.
	      </item>

	    </description>
	  </subsection>

	  <subsection>
	    <heading><visual markup="tt">Table</visual> methods
	    </heading>

	    <subsubsection id="Table.getEnum">
	      <heading>getEnum(colname)</heading>

	      <p>Get the enumerated type associated with the named
		column.
	      </p>

	      <p>If the column named <verb>colname</verb> (a string)
		exists and is of an enumerated type, the corresponding
		<verb>Enum</verb> instance (see <ref
		  refid="EnumClassDescr"></ref>) is returned.  If it is
		not of an enumerated type, a <verb>TypeError</verb> is
		raised.  If the column does not exist, a
		<verb>KeyError</verb> is raised.
	      </p>
	    </subsubsection>

	    <subsubsection id="Table.append">
	      <heading>append(rows)
	      </heading>
	      <p>Append a series of rows to this <verb>Table</verb>
		instance. <em>rows</em> is an object that can keep the
		rows to be append in several formats, like a
		<verb>NestedRecArray</verb> (see <ref
		refid="NestedRecArrayClassDescr">appendix</ref>), a
		<verb>RecArray</verb>, a <verb>NumPy</verb> object, a
		list of tuples, list of
		<verb>Numeric</verb>/<verb>numarray</verb>/<verb>NumPy</verb>
		objects, string, Python buffer or <verb>None</verb>
		(no append will result). Of course, this <em>rows</em>
		object has to be compliant with the underlying format
		of the <verb>Table</verb> instance or a
		<verb>ValueError</verb> will be issued.
	      </p>

	      <p>Example of use:
		<verbatim>
from tables import *
class Particle(IsDescription):
    name        = StringCol(16, pos=1)   # 16-character String
    lati        = IntCol(pos=2)        # integer
    longi       = IntCol(pos=3)        # integer
    pressure    = Float32Col(pos=4)    # float  (single-precision)
    temperature = FloatCol(pos=5)      # double (double-precision)

fileh = openFile("test4.h5", mode = "w")
table = fileh.createTable(fileh.root, 'table', Particle, "A table")
# Append several rows in only one call
table.append([("Particle:     10", 10, 0, 10*10, 10**2),
              ("Particle:     11", 11, -1, 11*11, 11**2),
              ("Particle:     12", 12, -2, 12*12, 12**2)])
fileh.close()
		</verbatim>
	      </p>
	    </subsubsection>


	    <subsubsection id="Table.col">
	      <heading>col(name)</heading>

	      <p>Get a column from the table.</p>

	      <p>If a column called <verb>name</verb> exists in the
		table, it is read and returned as a
		<verb>numarray</verb> object, or as a <verb>NumPy</verb>
		object (whatever is more appropriate depending on the
		flavor of the table).  If it does not exist, a
		<verb>KeyError</verb> is raised.
	      </p>

	      <p>Example of use:</p>

	      <verbatim>narray = table.col('var2')</verbatim>

	      <p>That statement is equivalent to:</p>

	      <verbatim>narray = table.read(field='var2')</verbatim>

	      <p>Here you can see how this method can be used as a
		shorthand for <verb>the read()</verb> (see <ref
		  refid="Table.read"></ref>) method.
	      </p>
	    </subsubsection>


	    <subsubsection id="Table.iterrows">
	      <heading>iterrows(start=None,
		stop=None, step=1)</heading>

	      <p>Returns an iterator yielding <verb>Row</verb> (see <ref
		  refid="RowClassDescr">section</ref>) instances built
		from rows in table. If a range is supplied (i.e. some of
		the <em>start</em>, <em>stop</em> or <em>step</em>
		parameters are passed), only the appropriate rows are
		returned. Else, all the rows are returned. See also the
		<verb>__iter__()</verb> special method in <ref
		  refid="TableSpecialMethods">section</ref> for a shorter
		way to call this iterator.
	      </p>
	      <p>The meaning of the <em>start</em>, <em>stop</em> and
		<em>step</em> parameters is the same as in the
		<verb>range()</verb> python function, except that
		negative values of <verb>step</verb> are not
		allowed. Moreover, if only <verb>start</verb> is
		specified, then <verb>stop</verb> will be set to
		<verb>start+1</verb>. If you do not specify neither
		<em>start</em> nor <em>stop</em>, then <visual
		  markup="bf">all the rows</visual> in the object are
		selected.
	      </p>

	      <p>Example of use:</p>
	      <verbatim>
		result = [ row['var2'] for row in table.iterrows(step=5)
		if row['var1'] &lt;= 20 ]
	      </verbatim>

	      <p><visual markup="bf">Note:</visual> This iterator can be
		nested (see example in <ref
		  refid="Table.where">section</ref>).
	      </p>

	    </subsubsection>

	    <subsubsection id="itersequenceDescr">
	      <heading>itersequence(sequence, sort=True)</heading>

	      <p>Iterate over a <em>sequence</em> of row coordinates.
	      </p>

	      <description>
		<term>sequence</term> <item>Can be any object that
		  supports the <verb>__getitem__</verb> special
		  method, like lists, tuples, Numeric/NumPy/numarray
		  objects, etc.
		</item>
		<term>sort</term> <item>If true, means that
		  <em>sequence</em> will be sorted out so that the I/O
		  process would get better performance. If your sequence
		  is already sorted or you don't want to sort it, put
		  this parameter to 0. The default is to sort the
		  <em>sequence</em>.
		</item>
	      </description>

	      <p><visual markup="bf">Note:</visual> This iterator can be
		nested (see example in <ref
		  refid="Table.where">section</ref>).
	      </p>

	    </subsubsection>

	    <subsubsection id="Table.read">
	      <heading>read(start=None, stop=None,
		step=1, field=None, flavor=None)</heading>

	      <p>Returns the actual data in <verb>Table</verb>. If
		<em>field</em> is not supplied, it returns the data as a
		<verb>NestedRecArray</verb> (see <ref
		  refid="NestedRecArrayClassDescr">appendix</ref>) object
		table.
	      </p>

	      <p>The meaning of the <em>start</em>, <em>stop</em> and
		<em>step</em> parameters is the same as in the
		<verb>range()</verb> python function, except that
		negative values of <em>step</em> are not
		allowed. Moreover, if only <em>start</em> is
		specified, then <em>stop</em> will be set to
		<em>start+1</em>. If you do not specify neither
		<em>start</em> nor <em>stop</em>, then all the rows in
		the object are selected.

	      </p>
	      <p>The rest of the parameters are described next:</p>

	      <description>
		<term>field</term> <item>If specified, only the column
		  <em>field</em> is returned as an homogeneous
		  <verb>numarray</verb>/<verb>NumPy</verb>/<verb>Numeric</verb>
		  object, depending on the <em>flavor</em>. If this is
		  not supplied, all the fields are selected and a
		  <verb>NestedRecArray</verb> (see <ref
		  refid="NestedRecArrayClassDescr">appendix</ref>) or
		  <verb>NumPy</verb> object is returned.  Nested
		  fields can be specified in the <em>field</em>
		  parameter by using a <verb>'/'</verb> character as a
		  separator between fields
		  (e.g. <verb>Info/value</verb>).
		</item>
		<term>flavor</term> <item>Passing a <em>flavor</em>
		  parameter make an additional conversion to happen in
		  the default returned object. <em>flavor</em> can
		  have any of the next values: <verb>"numarray"</verb>
		  <verb>"numpy"</verb>, <verb>"python"</verb> or
		  <verb>"numeric"</verb> (only if <em>field</em> has
		  been specified). If <em>flavor</em> is not
		  specified, then it will take the value of
		  <verb>self.flavor</verb>.
		</item>
	      </description>
	    </subsubsection>

	    <subsubsection id="Table.readCoordinates">
	      <heading>readCoordinates(coords, field=None,
		flavor=None)</heading>

	      <p>Read a set of rows given their indexes into an
		in-memory object.</p>

	      <p>This method works much like the <verb>read()</verb>
		method (see <ref refid="Table.read"></ref>), but it uses
		a sequence (<verb>coords</verb>) of row indexes to
		select the wanted columns, instead of a column range.
	      </p>

	      <p>It returns the selected rows in a
		<verb>NestedRecArray</verb> object (see <ref
		refid="NestedRecArrayClassDescr">appendix</ref>). If
		<verb>flavor</verb> is provided, an additional
		conversion to an object of this flavor is made, just
		as in <verb>read()</verb>.</p>
	    </subsubsection>

	    <subsubsection id="Table.modifyRows">
	      <heading>modifyRows(start=None, stop=None, step=1,
		rows=None)</heading>

	      <p>Modify a series of rows in the
		<verb>[start:stop:step]</verb> <em>extended slice</em>
		range. If you pass <verb>None</verb> to <em>stop</em>,
		all the rows existing in <em>rows</em> will be used.
	      </p>

	      <p><em>rows</em> can be either a <em>recarray</em> or a
		structure that is able to be converted to any of them
		and compliant with the table format.
	      </p>

	      <p>Returns the number of modified rows.
	      </p>

	      <p>It raises an <verb>ValueError</verb> in case the rows
		parameter could not be converted to an object compliant
		with table description.
	      </p>

	      <p>It raises an <verb>IndexError</verb> in case the
		modification will exceed the length of the table.
	      </p>

	    </subsubsection>

	    <subsubsection id="Table.modifyColumn">
	      <heading>modifyColumn(start=None, stop=None, step=1,
		column=None, colname=None)</heading>

	      <p>Modify a series of rows in the
		<verb>[start:stop:step]</verb> <em>extended slice</em>
		row range. If you pass <verb>None</verb> to <em>stop</em>,
		all the rows existing in <em>column</em> will be used.
	      </p>

	      <p><em>column</em> can be either a
		<verb>NestedRecArray</verb> (see <ref
		refid="NestedRecArrayClassDescr">appendix</ref>),
		<verb>RecArray</verb>, <verb>numarray</verb>,
		<verb>NumPy</verb> object, list or tuple that is able
		to be converted into a <verb>NestedRecArray</verb>
		compliant with the specified <em>colname</em> column
		of the table.
	      </p>

	      <p><em>colname</em> specifies the column name of the table
		to be modified.
	      </p>

	      <p>Returns the number of modified rows.
	      </p>

	      <p>It raises an <verb>ValueError</verb> in case the
		<em>column</em> parameter could not be converted into an
		object compliant with <verb>column</verb> description.
	      </p>

	      <p>It raises an <verb>IndexError</verb> in case the
		modification will exceed the length of the table.
	      </p>

	    </subsubsection>

	    <subsubsection id="Table.modifyColumns">
	      <heading>modifyColumns(start=None, stop=None, step=1,
		columns=None, names=None)</heading>

	      <p>Modify a series of rows in the
		<verb>[start:stop:step]</verb> <em>extended slice</em>
		row range. If you pass <verb>None</verb> to <em>stop</em>,
		all the rows existing in <em>columns</em> will be used.
	      </p>

	      <p><em>columns</em> can be either a
		<verb>NestedRecArray</verb> (see <ref
		refid="NestedRecArrayClassDescr">appendix</ref>),
		<verb>RecArray</verb>, a <verb>NumPy</verb> object, a
		list of arrays or list or tuples (the columns) that
		are able to be converted to a
		<verb>NestedRecArray</verb> compliant with the
		specified column <em>names</em> subset of the table
		format.
	      </p>

	      <p><em>names</em> specifies the column names of the table
		to be modified.
	      </p>

	      <p>Returns the number of modified rows.
	      </p>

	      <p>It raises an <verb>ValueError</verb> in case the
		<em>columns</em> parameter could not be converted to an
		object compliant with table description.
	      </p>

	      <p>It raises an <verb>IndexError</verb> in case the
		modification will exceed the length of the table.
	      </p>

	    </subsubsection>

	    <subsubsection id="removeRowsDescr">
	      <heading>removeRows(start, stop=None)</heading>

	      <p>Removes a range of rows in the table.  If only
		<em>start</em> is supplied, this row is to be
		deleted. If a range is supplied, i.e. both the
		<em>start</em> and <em>stop</em> parameters are passed,
		all the rows in the range are removed. A <em>step</em>
		parameter is not supported, and it is not foreseen to
		implement it anytime soon.
	      </p>
	      <description>
		<term>start</term> <item>Sets the starting row to
		  be removed. It accepts negative values meaning that
		  the count starts from the end. A value of 0 means
		  the first row.</item>

		<term>stop</term> <item>Sets the last row to be
		  removed to <em>stop</em> - 1, i.e. the end point is
		  omitted (in the Python <verb>range</verb>
		  tradition). It accepts, likewise <em>start</em>,
		  negative values. A special value of
		  <verb>None</verb> (the default) means removing just
		  the row supplied in start.
		</item>

	      </description>
	    </subsubsection>

	    <subsubsection id="removeIndexDescr">
	      <heading>removeIndex(index)</heading>

	      <p>Remove the index associated with the specified
		column. Only <verb>Index</verb> instances (see <ref
		  refid="IndexClassDescr"></ref>) are accepted as
		parameter.  This index can be recreated again by calling
		the <verb>createIndex</verb> (see <ref
		  refid="createIndexColumnDescr"></ref>) method of the
		appropriate <verb>Column</verb> object.
	      </p>
	    </subsubsection>

	    <subsubsection id="flushRowsToIndexDescr">
	      <heading>flushRowsToIndex()</heading>

	      <p> Add remaining rows in buffers to non-dirty
		indexes. This can be useful when you have chosen
		non-automatic indexing for the table (see <ref
		  refid="IndexPropsClassDescr">section</ref>) and want to
		update the indexes on it.
	      </p>
	    </subsubsection>

	    <subsubsection id="reIndexDescr">
	      <heading>reIndex()</heading>

	      <p>Recompute all the existing indexes in table. This can
		be useful when you suspect that, for any reason, the
		index information for columns is no longer valid and
		want to rebuild the indexes on it.
	      </p>
	    </subsubsection>

	    <subsubsection id="reIndexDirtyDescr">
	      <heading>reIndexDirty()</heading>

	      <p>Recompute the existing indexes in table, but
		<em>only</em> if they are dirty. This can be useful when
		you have set the <verb>reindex</verb> parameter to 0 in
		<verb>IndexProps</verb> constructor (see <ref
		  refid="IndexPropsInitDescr"></ref>) for the table and
		want to update the indexes after a invalidating index
		operation (<verb>Table.removeRows</verb>, for example).
	      </p>
	    </subsubsection>

	    <subsubsection id="Table.where">
	      <heading>where(condition, start=None, stop=None, step=None)</heading>

	      <p>Iterate over values fulfilling a <verb>condition</verb>.</p>

	      <p>This method returns an iterator yielding
		<verb>Row</verb> (see <ref refid="RowClassDescr"></ref>)
		instances built from rows in the table that satisfy the
		given <verb>condition</verb> over a column.  If that
		column is indexed, its index will be used in order to
		accelerate the search.  Else, the <em>in-kernel</em>
		iterator (with has still better performance than
		standard Python selections) will be chosen
		instead. Please, check the <ref
		  refid="searchOptim">section</ref> for more information
		about the performance of the different searching modes.
	      </p>

	      <p>Moreover, if a range is supplied (i.e. some of the
		<verb>start</verb>, <verb>stop</verb> or
		<verb>step</verb> parameters are passed), only the rows
		in that range <em>and</em> fulfilling the
		<verb>condition</verb> are returned.  The meaning of the
		<verb>start</verb>, <verb>stop</verb> and
		<verb>step</verb> parameters is the same as in the
		<verb>range()</verb> Python function, except that
		negative values of <verb>step</verb> are <em>not</em>
		allowed.  Moreover, if only <verb>start</verb> is
		specified, then <verb>stop</verb> will be set to
		<verb>start+1</verb>.
	      </p>

	      <p>You can mix this method with standard Python selections
		in order to have complex queries.  It is strongly
		recommended that you pass the most restrictive condition
		as the parameter to this method if you want to achieve
		maximum performance.
	      </p>

	      <p>Example of use:</p>

	      <verbatim>
<![CDATA[passvalues=[]
for row in table.where(0 < table.cols.col1 < 0.3, step=5):
    if row['col2'] <= 20:
        passvalues.append(row['col3'])
print "Values that pass the cuts:", passvalues]]>
	      </verbatim>

	      <p>Note that, from <verb>PyTables</verb> 1.1 on, you can
		nest several iterators over the same table. For example:
	      </p>

	      <verbatim>
for p in rout.where(rout.cols.pressure &lt; 16):
    for q in rout.where(rout.cols.pressure &lt; 9):
        for n in rout.where(rout.cols.energy &lt; 10):
            print "pressure, energy:", p['pressure'],n['energy']
	      </verbatim>

	      <p>In this example, the iterators returned by
		<verb>where()</verb> has been nested, but in fact, you
		can use any of the other reading iterators that the
		<verb>Table</verb> object offers. Look at
		<verb>examples/nested-iter.py</verb> for the full code.
	      </p>

	    </subsubsection>

	    <subsubsection id="Table.whereAppend">
	      <heading>whereAppend(dstTable, condition, start=None,
		stop=None, step=None)</heading>

	      <p>Append rows fulfilling the <em>condition</em> to the
		<em>dstTable</em> table.</p>

	      <p><em>dstTable</em> must be capable of taking the rows
		resulting from the query, i.e. it must have columns with
		the expected names and compatible types.  The meaning of
		the other arguments is the same as in the
		<verb>where()</verb> method (see <ref
		  refid="Table.where"></ref>).
	      </p>

	      <p>The number of rows appended to <em>dstTable</em> is
		returned as a result.
	      </p>
	    </subsubsection>

	    <subsubsection id="getWhereListTableDescr">
	      <heading>getWhereList(condition, flavor=None)</heading>

	      <p>Get the row coordinates that fulfill the
		<em>condition</em> parameter. This method will take
		advantage of an indexed column to speed-up the search.
	      </p>

	      <p><em>flavor</em> is the desired type of the returned
		list. It can take the <verb>"numarray"</verb>,
		<verb>"numpy"</verb>, <verb>"numeric"</verb> or
		<verb>"python"</verb> values. The default is returning
		an object of the same flavor than
		<verb>self.flavor</verb>.
	      </p>

	    </subsubsection>

	  </subsection>
	  <subsection id="TableSpecialMethods">
	    <heading><visual markup="tt">Table</visual> special methods
	    </heading>

	    <p>Following are described the methods that automatically
	      trigger actions when a <verb>Table</verb> instance is
	      accessed in a special way (e.g.,
	      <verb>table["var2"]</verb> will be equivalent to a call to
	      <verb>table.__getitem__("var2")</verb>).
	    </p>

	    <subsubsection id="Table.__iter__">
	      <heading>__iter__()</heading>

	      <p>It returns the same iterator than
		<verb>Table.iterrows(0,0,1)</verb>. However, this does not
		accept parameters.</p>

	      <p>Example of use:</p>

	      <verbatim>
result = [ row['var2'] for row in table if row['var1'] &lt;= 20 ]
	      </verbatim>

	      <p>Which is equivalent to:</p>

	      <verbatim>
result = [ row['var2'] for row in table.iterrows()
                       if row['var1'] &lt;= 20 ]
	      </verbatim>

	      <p><visual markup="bf">Note:</visual> This iterator can be
		nested (see example in <ref
		  refid="Table.where">section</ref>).
	      </p>

	    </subsubsection>

	    <subsubsection id="Table.__getitem__">
	      <heading>__getitem__(key)</heading>

	      <p>Get a row or a range of rows from the table.</p>

	      <p>If the <verb>key</verb> argument is an integer, the
		corresponding table row is returned as a
		<verb>tables.nestedrecords.NestedRecord</verb> object.
		If <verb>key</verb> is a slice, the range of rows
		determined by it is returned as a
		<verb>tables.nestedrecords.NestedRecArray</verb> object.
	      </p>

	      <p>Using a string as <verb>key</verb> to get a column is
		supported but deprecated.  Please use the
		<verb>col()</verb> (see <ref refid="Table.col"></ref>)
		method.
	      </p>

	      <p>Example of use:</p>

	      <verbatim>
<![CDATA[record = table[4]
recarray = table[4:1000:2]]]>
	      </verbatim>

	      <p>Those statements are equivalent to:</p>

	      <verbatim>
<![CDATA[record = table.read(start=4)[0]
recarray = table.read(start=4, stop=1000, step=2)]]>
	      </verbatim>

	      <p>Here you can see how indexing and slicing can be used
		as shorthands for the <verb>read()</verb> (see <ref
		  refid="Table.read"></ref>) method.
	      </p>
	    </subsubsection>


	    <subsubsection id="Table.__setitem__">
	      <heading>__setitem__(key, value)</heading>

	      <p>It takes different actions depending on the
		type of the <verb>key</verb> parameter:</p>

	      <description>
		<term><visual markup="tt">key</visual> is an
		  <verb>Integer</verb></term> <item>The corresponding
		  table row is set to <em>value</em>. <em>value</em>
		  must be a <verb>List</verb> or <verb>Tuple</verb>
		  capable of being converted to the table field format.
		</item>

		<term><visual markup="tt">key</visual> is a
		  <verb>Slice</verb></term><item>The row slice
		  determined by key is set to
		  <em>value</em>. <em>value</em> must be a
		  <verb>NestedRecArray</verb> object or a
		  <verb>RecArray</verb> object or a list of rows capable
		  of being converted to the table field format.
		</item>
	      </description>

	      <p>Example of use:</p>

	      <verbatim>
		# Modify just one existing row
		table[2] = [456,'db2',1.2]
		# Modify two existing rows
		rows = numarray.records.array([[457,'db1',1.2],[6,'de2',1.3]],
		formats="i4,a3,f8")
		table[1:3:2] = rows
	      </verbatim>

	      <p>Which is equivalent to:</p>

	      <verbatim>
		table.modifyRows(start=2, rows=[456,'db2',1.2])
		rows = numarray.records.array([[457,'db1',1.2],[6,'de2',1.3]],
		formats="i4,a3,f8")
		table.modifyRows(start=1, step=2, rows=rows)
	      </verbatim>

	    </subsubsection>
	  </subsection> <!-- Table special methods -->

	  <subsection id="RowClassDescr">
	    <heading>The <visual markup="tt">Row</visual> class</heading>

	    <p>This class is used to fetch and set values on the table
	      fields. It works very much like a dictionary, where the keys
	      are the field names of the associated table and the values
	      are the values of those fields in a specific row.
	    </p>
	    <p>This object turns out to actually be an extension type,
	      so you won't be able to access its documentation
	      interactively. However, you will be able to access some of
	      its internal attributes through the use of Python
	      properties. In addition, there are some important methods
	      that are useful for adding and modifying values in tables.
	    </p>

	    <subsubsection>
	      <heading><visual markup="tt">Row</visual> attributes
	      </heading>

	      <description>

		<term>nrow</term> <item>Property that returns the
		  current row number in the table. It is useful to know
		  which row is being dealt with in the middle of a
		  loop or iterator.</item>

	      </description>
	    </subsubsection>

	    <subsubsection id="RowMethods">
	      <heading><visual markup="tt">Row</visual> methods
	      </heading>

	      <description>

		<term id="Row.append">append()</term> <item>Once you
		  have filled the proper fields for the current row,
		  calling this method actually append these new data to
		  the disk (actually data are written to the output
		  buffer).

		  <p>Example of use:
		    <verbatim>
        row = table.row
        for i in xrange(nrows):
            row['col1'] = i-1
            row['col2'] = 'a'
            row['col3'] = -1.0
            row.append()
        table.flush()
		    </verbatim>

		    Please, note that, after the loop in which
		    <verb>Row.append()</verb> has been called, it is
		    always convenient to make a call to
		    <verb>Table.flush()</verb> in order to avoid losing
		    the last rows that can be in internal buffers.
		  </p>
		</item>

		<term>update()</term> <item>This allows
		  you to modify values of your tables when you are in
		  the middle of table iterators, like
		  <verb>Table.iterrows()</verb> (see <ref
		    refid="Table.iterrows"></ref>) or
		  <verb>Table.where()</verb> (see <ref
		    refid="Table.where"></ref>). Once you have filled the
		  proper fields for the current row, calling this method
		  actually commits these data to the disk (actually data
		  are written to the output buffer).

		  <p>Example of use:
		    <verbatim>
        for row in table.iterrows(step=10):
            row['col1'] = row.nrow
            row['col2'] = 'b'
            row['col3'] = 0.0
            row.update()
		    </verbatim>

		    which modifies every tenth row in table. Or:

		    <verbatim>
        for row in table.where(table.cols.col1 > 3):
            row['col1'] = row.nrow
            row['col2'] = 'b'
            row['col3'] = 0.0
            row.update()
		    </verbatim>

		    which just updates the rows with values in first
		    column bigger than 3.

		  </p>

		</item>

	      </description>
	    </subsubsection>
	  </subsection> <!-- Row -->
	</section> <!-- Table -->

	<section id="ColsClassDescr">
	  <heading>The <visual markup="tt">Cols</visual> class</heading>

	  <p>This class is used as an <em>accessor</em> to the table
	    columns following the natural name convention, so that you
	    can access the different columns because there exists one
	    attribute with the name of the columns for each associated
	    column, which can be a <verb>Column</verb> instance
	    (non-nested column) or another <verb>Cols</verb> instance
	    (nested column).
	  </p>

	  <p>Columns under a <verb>Cols</verb> accessor can be accessed
	    as attributes of it.  For instance, if
	    <verb>table.cols</verb> is a <verb>Cols</verb> instance with
	    a column named <verb>col1</verb> under it, the later can be
	    accessed as <verb>table.cols.col1</verb>.  If
	    <verb>col1</verb> is nested and contains a <verb>col2</verb>
	    column, this can be accessed as
	    <verb>table.cols.col1.col2</verb> and so on and so forth.
	  </p>

	  <subsection>
	    <heading><visual markup="tt">Cols</visual> instance
	      variables</heading>

	    <description>

	      <term>_v_colnames</term> <item>A list of the names of
		the columns (or nested columns) hanging directly from
		this <verb>Cols</verb> instance.  The order of the
		names matches the order of their respective columns in
		the containing table.
	      </item>

	      <term>_v_colpathnames</term> <item>A list of the
		complete pathnames of the columns hanging directly
		from this <verb>Cols</verb> instance.  If the table
		does not contain nested columns, this is exactly the
		same as <verb>_v_colnames</verb> attribute.
	      </item>

	      <term>_v_table</term> <item>The parent <verb>Table</verb>
		instance.</item>

	      <term>_v_desc</term> <item>The associated <ref
		  refid="ColumnClassDescr">Description</ref>
		instance.</item>

	    </description>
	  </subsection>

	  <subsection>
	    <heading><visual markup="tt">Cols</visual> methods</heading>

	    <subsubsection id="Cols._f_col">
	      <heading>_f_col(colname)</heading>

	      <p>Return a handler to the <em>colname</em> column. If
		<em>colname</em> is a nested column, a <verb>Cols</verb>
		instance is returned. If <em>colname</em> is a
		non-nested column a <verb>Column</verb> object is
		returned instead.
	      </p>

	    </subsubsection>

	    <subsubsection id="Cols.__getitem__">
	      <heading>__getitem__(key)</heading>

	      <p>Get a row or a range of rows from the
		<verb>Cols</verb> accessor.
	      </p>

	      <p>If the <verb>key</verb> argument is an integer, the
		corresponding <verb>Cols</verb> row is returned as a
		<verb>tables.nestedrecords.NestedRecord</verb> object.
		If <verb>key</verb> is a slice, the range of rows
		determined by it is returned as a
		<verb>tables.nestedrecords.NestedRecArray</verb> object.
	      </p>

	      <p>Using a string as <verb>key</verb> to get a column is
		supported but deprecated.  Please use the
		<verb>col()</verb> (see <ref refid="Table.col"></ref>)
		method.
	      </p>

	      <p>Example of use:</p>

	      <verbatim>
<![CDATA[record = table.cols[4]  # equivalent to table[4]
recarray = table.cols.Info[4:1000:2]]]>
	      </verbatim>

	      <p>Those statements are equivalent to:</p>

	      <verbatim>
<![CDATA[nrecord = table.read(start=4)[0]
nrecarray = table.read(start=4, stop=1000, step=2).field('Info')]]>
	      </verbatim>

	      <p>Here you can see how a mix of natural naming, indexing
		and slicing can be used as shorthands for the
		<verb>read()</verb> (see <ref refid="Table.read"></ref>)
		method.
	      </p>
	    </subsubsection> <!-- Cols.__setitem__ -->
	    <subsubsection>
	      <heading>__setitem__(key)</heading>

	      <p>Set a row or a range of rows to the <verb>Cols</verb>
		accessor.
	      </p>

	      <p>If the <verb>key</verb> argument is an integer, the
		corresponding <verb>Cols</verb> row is set to the
		<verb>value</verb> object.  If <verb>key</verb> is a
		slice, the range of rows determined by it is set to the
		<verb>value</verb> object.
	      </p>


	      <p>Example of use:</p>

	      <verbatim>
<![CDATA[
table.cols[4] = record
table.cols.Info[4:1000:2] = recarray
]]>
	      </verbatim>

	      <p>Those statements are equivalent to:</p>

	      <verbatim>
<![CDATA[
table.modifyRows(4, rows=record)
table.modifyColumn(4, 1000, 2, colname='Info', column=recarray)
]]>
	      </verbatim>

	      <p>Here you can see how a mix of natural naming, indexing
		and slicing can be used as shorthands for the
		<verb>modifyRows()</verb> and
		<verb>modifyColumn()</verb> (see <ref
		  refid="Table.modifyRows"></ref> and <ref
		  refid="Table.modifyColumn"></ref>) methods.
	      </p>

	    </subsubsection> <!-- Cols.__getitem__ -->
	  </subsection> <!-- Cols methods -->
	</section> <!-- Cols -->

	<section id="DescriptionClassDescr">
	  <heading>The <visual markup="tt">Description</visual> class</heading>

	  <p>The instances of the <verb>Description</verb> class provide
	    a description of the structure of a table.
	  </p>

	  <p>An instance of this class is automatically bound to
	    <verb>Table</verb> (see <ref refid="TableClassDescr"></ref>)
	    objects when they are created.  It provides a browseable
	    representation of the structure of the table, made of
	    non-nested (<verb>Col</verb> &mdash;see <ref
	      refid="ColClassDescr"></ref>) and nested
	    (<verb>Description</verb>) columns.  It also contains
	    information that will allow you to build
	    <verb>NestedRecArray</verb> (see <ref
	      refid="NestedRecArrayClassDescr">appendix</ref>) objects
	    suited for the different columns in a table (be they nested
	    or not).
	  </p>

	  <p>Column descriptions (see <verb>Col</verb> class in <ref
	      refid="ColClassDescr"></ref>) under a description can be
	    accessed as attributes of it.  For instance, if
	    <verb>table.description</verb> is a <verb>Description</verb>
	    instance with a column named <verb>col1</verb> under it, the
	    later can be accessed as
	    <verb>table.description.col1</verb>.  If <verb>col1</verb>
	    is nested and contains a <verb>col2</verb> column, this can
	    be accessed as <verb>table.description.col1.col2</verb>.
	  </p>

	  <subsection>
	    <heading><visual markup="tt">Description</visual> instance
	      variables </heading>

	    <description>
	      <term>_v_name</term> <item>The name of this description
		instance.  If description is the root of the nested type
		(or the description of a flat table), its name will be
		the empty string (<verb>''</verb>).
	      </item>

	      <term>_v_names</term> <item>A list of the names of the
		columns hanging directly from this description instance.
		The order of the names matches the order of their
		respective columns in the containing description.
	      </item>

	      <term>_v_pathnames</term> <item>A list of the pathnames of
		the columns hanging directly from this description.  If
		the table does not contain nested columns, this is
		exactly the same as <verb>_v_names</verb> attribute.
	      </item>

	      <term>_v_nestedNames</term> <item>A nested list of the
		names of all the columns hanging directly from this
		description instance.  You can use this for the
		<verb>names</verb> argument of
		<verb>NestedRecArray</verb> factory functions.
	      </item>

	      <term>_v_nestedFormats</term> <item>A nested list of the
		numarray string formats (and shapes) of all the columns
		hanging directly from this description instance.  You
		can use this for the <verb>formats</verb> argument of
		<verb>NestedRecArray</verb> factory functions.
	      </item>

	      <term>_v_nestedDescr</term> <item>A nested list of pairs
		of <verb>(name, format)</verb> tuples for all the
		columns under this table or nested column.  You can use
		this for the <verb>descr</verb> argument of
		<verb>NestedRecArray</verb> factory functions.
	      </item>

	      <term>_v_types</term> <item>A dictionary mapping the names
		of non-nested columns hanging directly from this
		description instance to their respective numarray types.
	      </item>

	      <term>_v_stypes</term> <item>A dictionary mapping the
		names of non-nested columns hanging directly from this
		description instance to their respective string types.
	      </item>

	      <term>_v_shapes</term> <item>A dictionary mapping the
		names of non-nested columns hanging directly from this
		description instance to their respective shapes.
	      </item>

	      <term>_v_dflts</term> <item>A dictionary mapping the names
		of non-nested columns hanging directly from this
		description instance to their respective default values.
		Please, note that all the default values are kept
		internally as numarray objects.
	      </item>

	      <term>_v_colObjects</term> <item>A dictionary mapping the
		names of the columns hanging directly from this
		description instance to their respective descriptions
		(<verb>Col</verb> &mdash;see <ref
		  refid="ColClassDescr"></ref>&mdash; or
		<verb>Description</verb> &mdash;see <ref
		  refid="DescriptionClassDescr"></ref>&mdash; instances).
	      </item>

	      <term>_v_itemsizes</term> <item>A dictionary mapping the
		names of non-nested columns hanging directly from this
		description instance to their respective item size (in
		bytes).
	      </item>

	      <term>_v_nestedlvl</term> <item>The level of the
		description in the nested datatype.
	      </item>
	    </description>
	  </subsection>

	  <subsection>
	    <heading><visual markup="tt">Description</visual> methods</heading>

	    <description>
	      <term>_v_walk(type='All')</term>
	      <item>
		<p>Iterate over nested columns.</p> <p>If
		  <verb>type</verb> is <verb>'All'</verb> (the default),
		  all column description objects (<verb>Col</verb> and
		  <verb>Description</verb> instances) are returned in
		  top-to-bottom order (pre-order).
		</p>
		<p>If <verb>type</verb> is <verb>'Col'</verb> or
		  <verb>'Description'</verb>, only column descriptions
		  of that type are returned.
		</p>
	      </item>
	    </description>
	  </subsection>
	</section> <!-- Description -->

	<section id="ColumnClassDescr">
	  <heading>The <visual markup="tt">Column</visual> class</heading>

	  <p>Each instance of this class is associated with one column
	    of every table. These instances are mainly used to fetch and
	    set actual data from the table columns, but there are a few
	    other associated methods to deal with indexes.
	  </p>

	  <subsection>
	    <heading><visual markup="tt">Column</visual> instance
	      variables
	    </heading>
	    <description>
	      <term>table</term> <item>The parent <verb>Table</verb>
		instance.
	      </item>
	      <term>name</term> <item>The name of the associated
		column.
	      </item>
	      <term>pathname</term> <item>The complete pathname of the
		associated column. This is mainly useful in nested
		columns; for non-nested ones this value is the same a
		<verb>name</verb>.
	      </item>
	      <term>type</term> <item>The data type of the column.
	      </item>
	      <term>shape</term> <item>The shape of the column.
	      </item>
	      <term>index</term> <item>The associated <verb>Index</verb>
		object (see <ref refid="IndexClassDescr"></ref>) to this
		column (<verb>None</verb> if does not exist).
	      </item>
	      <term>dirty</term> <item>Whether the index is dirty or not
		(property).
	      </item>
	    </description>
	  </subsection>

	  <subsection>
	    <heading><visual markup="tt">Column</visual> methods
	    </heading>

	    <subsubsection id="createIndexColumnDescr">
	      <heading><visual markup="tt">createIndex()</visual>
	      </heading>

	      <p>Create an <verb>Index</verb> (see <ref
		  refid="IndexClassDescr"></ref>) object for this
		column.
	      </p>

	    </subsubsection>

	    <subsubsection id="reIndexColumnDescr">
	      <heading><visual markup="tt">reIndex()</visual>
	      </heading>

	      <p>Recompute the index associated with this column. This
		can be useful when you suspect that, for any reason, the
		index information is no longer valid and want to rebuild
		it.
	      </p>

	    </subsubsection>

	    <subsubsection id="reIndexDirtyColumnDescr">
	      <heading><visual markup="tt">reIndexDirty()</visual>
	      </heading>

	      <p>Recompute the existing index only if it is dirty. This
		can be useful when you have set the <verb>reindex</verb>
		parameter to 0 in <verb>IndexProps</verb> constructor
		(see <ref refid="IndexPropsInitDescr"></ref>) for the
		table and want to update the column's index after a
		invalidating index operation
		(<verb>Table.removeRows</verb>, for example).
	      </p>

	    </subsubsection>

	    <subsubsection id="removeIndexColumnDescr">
	      <heading><visual markup="tt">removeIndex()</visual>
	      </heading>

	      <p>Delete the associated column's index. After doing that,
		you will loose the indexation information on
		disk. However, you can always re-create it using the
		<verb>createIndex()</verb> method (see <ref
		  refid="createIndexColumnDescr"></ref>).
	      </p>

	    </subsubsection>

	  </subsection> <!-- Column methods -->

	  <subsection>
	    <heading><visual markup="tt">Column</visual> special methods
	    </heading>

	    <subsubsection id="Column.__getitem__">
	      <heading><visual markup="tt">__getitem__(key)</visual>
	      </heading>

	      <p>Returns a column element or slice. It takes different
		actions depending on the type of the <em>key</em>
		parameter:
	      </p>

	      <description>
		<term><visual markup="tt">key</visual> is an
		  <verb>Integer</verb></term> <item>The corresponding
		  element in the column is returned as a scalar object
		  or as a <verb>numarray</verb> object, depending on
		  its shape.
		</item>

		<term><visual markup="tt">key</visual> is a
		  <verb>Slice</verb></term><item>The row range
		  determined by this slice is returned as a
		  <verb>numarray</verb> object.
		</item>

	      </description>

	      <p>Example of use:
		<verbatim>
print "Column handlers:"
for name in table.colnames:
    print table.cols[name]
print
print "Some selections:"
print "Select table.cols.name[1]-->", table.cols.name[1]
print "Select table.cols.name[1:2]-->", table.cols.name[1:2]
print "Select table.cols.lati[1:3]-->", table.cols.lati[1:3]
print "Select table.cols.pressure[:]-->", table.cols.pressure[:]
print "Select table.cols['temperature'][:]-->", table.cols['temperature'][:]
		</verbatim>
		and the output of this for a certain arbitrary table is:
		<verbatim>
Column handlers:
/table.cols.name (Column(1,), CharType)
/table.cols.lati (Column(2,), Int32)
/table.cols.longi (Column(1,), Int32)
/table.cols.pressure (Column(1,), Float32)
/table.cols.temperature (Column(1,), Float64)

Some selections:
Select table.cols.name[1]--> Particle:     11
Select table.cols.name[1:2]--> ['Particle:     11']
Select table.cols.lati[1:3]--> [[11 12]
 [12 13]]
Select table.cols.pressure[:]--> [  90.  110.  132.]
Select table.cols['temperature'][:]--> [ 100.  121.  144.]
		</verbatim>
		See the <verb>examples/table2.py</verb> for a more
		complete example.
	      </p>

	    </subsubsection> <!-- __getitem__ -->

	    <subsubsection id="Column.__setitem__">
	      <heading>__setitem__(key, value)</heading>

	      <p>It takes different actions depending on the
		type of the <verb>key</verb> parameter:</p>

	      <description>
		<term><visual markup="tt">key</visual> is an
		  <verb>Integer</verb></term> <item>The corresponding
		  element in the column is set to
		  <em>value</em>. <em>value</em> must be a scalar or
		  <verb>numarray</verb>/<verb>NumPy</verb> object,
		  depending on column's shape.
		</item>

		<term><visual markup="tt">key</visual> is a
		  <verb>Slice</verb></term><item>The row slice
		  determined by <em>key</em> is set to
		  <em>value</em>. <em>value</em> must be a list of
		  elements or a
		  <verb>numarray</verb>/<verb>NumPy</verb>.
		</item>
	      </description>

	      <p>Example of use:</p>

	      <verbatim>
		# Modify row 1
		table.cols.col1[1] = -1
		# Modify rows 1 and 3
		table.cols.col1[1::2] = [2,3]
	      </verbatim>

	      <p>Which is equivalent to:</p>

	      <verbatim>
		# Modify row 1
		table.modifyColumns(start=1, columns=[[-1]], names=["col1"])
		# Modify rows 1 and 3
		columns = numarray.records.fromarrays([[2,3]], formats="i4")
		table.modifyColumns(start=1, step=2, columns=columns, names=["col1"])
	      </verbatim>
	    </subsubsection> <!-- __setitem__ -->

	  </subsection> <!-- Column special methods -->
	</section> <!-- Column -->

	<section id="ArrayClassDescr">
	  <heading>The <visual markup="tt">Array</visual>
	    class</heading>

	  <p>Represents an array on file. It provides methods to
	    write/read data to/from array objects in the file. This
	    class does not allow you to enlarge the datasets on disk;
	    see the <verb>EArray</verb> descendant in <ref
	      refid="EArrayClassDescr">section</ref> if you want
	    enlargeable dataset support and/or compression features.
	    See also <verb>CArray</verb> in <ref
	      refid="CArrayClassDescr">section</ref>
	  </p>

	  <p>The array data types supported are the same as the set
	    provided by the <verb>numarray</verb> package.  For
	    details of these data types see <ref
	    refid="datatypesSupported">appendix</ref>, or the
	    <verb>numarray</verb> reference manual (<cite
	    refid="Numarray"></cite>).
	  </p>

	  <p>An interesting property of the <verb>Array</verb> class
	    is that it remembers the <em>flavor</em> of the object
	    that has been saved so that if you saved, for example, a
	    <verb>List</verb>, you will get a <verb>List</verb> during
	    readings afterwards, or if you saved a <verb>NumPy</verb>
	    array, you will get a <verb>NumPy</verb> object.
	  </p>

	  <p>Note that this object inherits all the public attributes
	    and methods that <verb>Leaf</verb> already provides.
	  </p>

	  <subsection id="ArrayClassInstanceVariables">
	    <heading><visual markup="tt">Array</visual> instance
	      variables</heading>
	    <description>
	      <term>flavor</term> <item>The object representation for
		this array. It can be any of <em>"numarray"</em>,
		<em>"numpy"</em>, <em>"numeric"</em> or
		<em>"python"</em> values.
	      </item>
	      <term>nrows</term> <item>The length of the first dimension
		of the array.
	      </item>
	      <term>nrow</term> <item>On iterators, this is the index of
		the current row.
	      </item>
	      <term>type</term> <item>The type class of the represented
		array.
	      </item>
	      <term>stype</term> <item>The string type of the represented
		array.
	      </item>
	      <term>itemsize</term> <item>The size of the base
		items. Specially useful for <verb>CharType</verb>
		objects.
	      </item>
	    </description>

	  </subsection>

	  <subsection>
	    <heading><visual markup="tt">Array</visual>
	      methods</heading>

	    <p>Note that, as this object has no internal I/O buffers, it
	      is not necessary to use the flush() method inherited from
	      <verb>Leaf</verb> in order to save its internal state to
	      disk. When a writing method call returns, all the data is
	      already on disk.
	    </p>

	    <subsubsection id="Array.getEnum">
	      <heading>getEnum()</heading>

	      <p>Get the enumerated type associated with this array.
	      </p>

	      <p>If this array is of an enumerated type, the
		corresponding <verb>Enum</verb> instance (see <ref
		  refid="EnumClassDescr"></ref>) is returned.  If it is
		not of an enumerated type, a <verb>TypeError</verb> is
		raised.
	      </p>
	    </subsubsection>

	    <subsubsection id="iterrowsArrayDescr">
	      <heading>iterrows(start=None,
		stop=None, step=1)</heading>

	      <p>Returns an iterator yielding <verb>numarray</verb>
		instances built from rows in array. The return rows are
		taken from the first dimension in case of an
		<verb>Array</verb> and <verb>CArray</verb> instance and
		the enlargeable dimension in case of an
		<verb>EArray</verb> instance. If a range is supplied
		(i.e. some of the <em>start</em>, <em>stop</em> or
		<em>step</em> parameters are passed), only the
		appropriate rows are returned. Else, all the rows are
		returned. See also the and <verb>__iter__()</verb>
		special methods in <ref
		  refid="ArraySpecialMethods">section</ref> for a shorter
		way to call this iterator.
	      </p>

	      <p>The meaning of the <em>start</em>, <em>stop</em> and
		<em>step</em> parameters is the same as in the
		<verb>range()</verb> python function, except that
		negative values of <verb>step</verb> are not
		allowed. Moreover, if only <verb>start</verb> is
		specified, then <verb>stop</verb> will be set to
		<verb>start+1</verb>. If you do not specify neither
		<em>start</em> nor <em>stop</em>, then all the rows in
		the object are selected.
	      </p>

	      <p>Example of use:</p>
	      <verbatim>
		result = [ row for row in arrayInstance.iterrows(step=4) ]
	      </verbatim>

	    </subsubsection>

	    <subsubsection id="readArrayDescr">
	      <heading>read(start=None, stop=None, step=1)</heading>

	      <p>Read the array from disk and return it as a
		<verb>numarray</verb> (default) object, or an object
		with the same original <em>flavor</em> that it was
		saved. It accepts <em>start</em>, <em>stop</em> and
		<em>step</em> parameters to select rows (the first
		dimension in the case of an <verb>Array</verb> and
		<verb>CArray</verb> instance and the
		<em>enlargeable</em> dimension in case of an
		<verb>EArray</verb>) for reading.
	      </p>
	      <p>The meaning of the <em>start</em>, <em>stop</em> and
		<em>step</em> parameters is the same as in the
		<verb>range()</verb> python function, except that
		negative values of <verb>step</verb> are not
		allowed. Moreover, if only <verb>start</verb> is
		specified, then <verb>stop</verb> will be set to
		<verb>start+1</verb>. If you do not specify neither
		<em>start</em> nor <em>stop</em>, then all the rows in
		the object are selected.
	      </p>
	    </subsubsection> <!-- read() -->
	  </subsection> <!-- Array methods -->

	  <subsection id="ArraySpecialMethods">
	    <heading><visual markup="tt">Array</visual> special
	      methods</heading>

	    <p>Following are described the methods that automatically
	      trigger actions when an <verb>Array</verb> instance is
	      accessed in a special way (e.g.,
	      <verb>array[2:3,...,::2]</verb> will be equivalent to a
	      call to <newline/><verb>array.__getitem__(slice(2,3, None),
		Ellipsis, slice(None, None, 2))</verb>).
	    </p>

	    <subsubsection id="Array.__iter__">
	      <heading>__iter__()</heading>

	      <p>It returns the same iterator than
		<verb>Array.iterrows(0,0,1)</verb>. However, this does not
		accept parameters.</p>

	      <p>Example of use:</p>

	      <verbatim>
		result = [ row[2] for row in array ]

	      </verbatim>

	      <p>Which is equivalent to:</p>

	      <verbatim>
		result = [ row[2] for row in array.iterrows(0, 0, 1) ]
	      </verbatim>

	    </subsubsection>

	    <subsubsection id="Array.__getitem__">
	      <heading>__getitem__(key)</heading>

	      <p>It returns a <verb>numarray</verb> (default) object (or
		an object with the same original <em>flavor</em> that it
		was saved) containing the slice of rows stated in the
		<verb>key</verb> parameter. The set of allowed tokens in
		<verb>key</verb> is the same as extended slicing in
		python (the <verb>Ellipsis</verb> token included).
	      </p>

	      <p>Example of use:</p>

	      <verbatim>
		array1 = array[4]   # array1.shape == array.shape[1:]
		array2 = array[4:1000:2]  # len(array2.shape) == len(array.shape)
		array3 = array[::2, 1:4, :]
		array4 = array[1, ..., ::2, 1:4, 4:] # General slice selection
	      </verbatim>

	    </subsubsection> <!-- __getitem__() -->

	    <subsubsection id="Array.__setitem__">
	      <heading>__setitem__(key, value)</heading>

	      <p>Sets an Array element, row or extended slice. It takes
		different actions depending on the type of the
		<verb>key</verb> parameter:
	      </p>
	      <description>
		<term><verb>key</verb> is an integer:</term> <item>The
		  corresponding row is assigned to value. If needed,
		  this <verb>value</verb> is broadcasted to fit the
		  specified row.</item>

		<term><verb>key</verb> is a slice:</term> <item>The row
		  slice determined by it is assigned to
		  <verb>value</verb>. If needed, this <verb>value</verb>
		  is broadcasted to fit in the desired range. If the
		  slice to be updated exceeds the actual shape of the
		  array, only the values in the existing range are
		  updated, i.e. the index error will be silently
		  ignored. If <verb>value</verb> is a multidimensional
		  object, then its shape must be compatible with the
		  slice specified in <verb>key</verb>, otherwise, a
		  <verb>ValueError</verb> will be issued.</item>

	      </description>

	      <p>Example of use:</p>

	      <verbatim>
		a1[0] = 333       # Assign an integer to a Integer Array row
		a2[0] = "b"       # Assign a string to a string Array row
		a3[1:4] = 5       # Broadcast 5 to slice 1:4
		a4[1:4:2] = "xXx" # Broadcast "xXx" to slice 1:4:2
		# General slice update (a5.shape = (4,3,2,8,5,10)
		a5[1, ..., ::2, 1:4, 4:] = arange(1728, shape=(4,3,2,4,3,6))
	      </verbatim>

	    </subsubsection> <!-- __setitem__() -->

	  </subsection> <!-- Special methods -->
	</section> <!-- Array class -->

	<section id="CArrayClassDescr">
	  <heading>The <visual markup="tt">CArray</visual> class</heading>

	  <p>This is a child of the <verb>Array</verb> class (see <ref
	      refid="ArrayClassDescr"></ref>) and as such,
	    <verb>CArray</verb> represents an array on the file. The
	    difference is that <verb>CArray</verb> has a chunked layout
	    and, as a consequence, it also supports compression.  You
	    can use this class to easily save or load array (or array
	    slices) objects to or from disk, with compression support
	    included.
	  </p>

	  <subsection id="CArrayClassInstanceVariables">
	    <heading><visual markup="tt">CArray</visual> instance
	      variables</heading>

	    <p>In addition to the attributes that <verb>CArray</verb>
	      inherits from <verb>Array</verb>, it supports some more
	      that provide information about the filters used.
	    </p>

	    <description>
	      <term>atom</term> <item>An <verb>Atom</verb> (see <ref
		  refid="AtomClassDescr"></ref>) instance representing the
		shape, type and flavor of the atomic objects to be saved.
	      </item>
	    </description>
	  </subsection>

	  <subsection>
	    <heading>Example of use</heading>

	    <p>See below a small example of <verb>CArray</verb>
	      class. The code is available in
	      <verb>examples/carray1.py</verb>.
	    </p>

	    <verbatim>
<![CDATA[import numarray
import tables

fileName = 'carray1.h5'
shape = (200,300)
atom = tables.UInt8Atom(shape = (128,128))
filters = tables.Filters(complevel=5, complib='zlib')

h5f = tables.openFile(fileName,'w')
ca = h5f.createCArray(h5f.root, 'carray', shape, atom, filters=filters)
# Fill a hyperslab in ca. The array will be converted to UInt8 elements
ca[10:60,20:70] = numarray.ones((50,50))
h5f.close()

# Re-open a read another hyperslab
h5f = tables.openFile(fileName)
print h5f
print h5f.root.carray[8:12, 18:22]
h5f.close()
]]>
	    </verbatim>

	    <p>The output for the previous script is something like:
	    </p>

	    <verbatim>
carray1.h5 (File) ''
Last modif.: 'Thu Jun 16 10:47:18 2005'
Object Tree:
/ (RootGroup) ''
/carray (CArray(200L, 300L)) ''

[[0 0 0 0]
 [0 0 0 0]
 [0 0 1 1]
 [0 0 1 1]]
	    </verbatim>

	  </subsection> <!-- Example of use -->

	</section> <!-- CArray class -->

	<section id="EArrayClassDescr">
	  <heading>The <visual markup="tt">EArray</visual> class</heading>

	  <p>This is a child of the <verb>Array</verb> class (see <ref
	      refid="ArrayClassDescr"></ref>) and as such,
	    <verb>EArray</verb> represents an array on the file. The
	    difference is that <verb>EArray</verb> allows to enlarge
	    datasets along any single dimension<footnote>In the future,
	      multiple enlargeable dimensions might be implemented as
	      well.</footnote> you select. Another important difference is
	    that it also supports compression.
	  </p>

	  <p>So, in addition to the attributes and methods that
	    <verb>EArray</verb> inherits from <verb>Array</verb>, it
	    supports a few more that provide a way to enlarge the
	    arrays on disk. Following are described the new variables
	    and methods as well as some that already exist in
	    <verb>Array</verb> but that differ somewhat on the meaning
	    and/or functionality in the <verb>EArray</verb> context.
	  </p>

	  <subsection id="EArrayClassInstanceVariables">
	    <heading><visual markup="tt">EArray</visual> instance
	      variables</heading>
	    <description>
	      <term>atom</term> <item>An <verb>Atom</verb> (see <ref
		  refid="AtomClassDescr"></ref>) instance representing
		the shape, type and flavor of the atomic objects to be
		saved.  One of the dimensions of the shape is 0,
		meaning that the array can be extended along it.
	      </item>
	      <term>extdim</term> <item>The enlargeable dimension,
		i.e. the dimension this array can be extended along.
	      </item>
	      <term>nrows</term>
	      <item>The length of the enlargeable dimension of the array.</item>
	    </description>
	  </subsection>

	  <subsection id="EArrayMethodsDescr">
	    <heading><visual markup="tt">EArray</visual>
	      methods</heading>

	    <subsubsection id="EArray.getEnum">
	      <heading>getEnum()</heading>

	      <p>Get the enumerated type associated with this array.</p>

	      <p>If this array is of an enumerated type, the
		corresponding <verb>Enum</verb> instance (see <ref
		  refid="EnumClassDescr"></ref>) is returned.  If it is
		not of an enumerated type, a <verb>TypeError</verb> is
		raised.
	      </p>

	    </subsubsection>

	    <subsubsection id="EArrayAppendDescr">
	      <heading>append(sequence)</heading>

	      <p>Appends a <verb>sequence</verb> to the underlying
		dataset. Obviously, this sequence must have the same
		type as the <verb>EArray</verb> instance; otherwise a
		<verb>TypeError</verb> is issued. In the same way, the
		dimensions of the <verb>sequence</verb> have to conform to
		those of <verb>EArray</verb>, that is, all the
		dimensions have to be the same except, of course, that of
		the enlargeable dimension which can be of any length
		(even 0!).
	      </p>

	      <p>Example of use (code available in
		<verb>examples/earray1.py</verb>):
	      </p>

	      <verbatim>
import tables
from numarray import strings

fileh = tables.openFile("earray1.h5", mode = "w")
a = tables.StringAtom(shape=(0,), length=8)
# Use 'a' as the object type for the enlargeable array
array_c = fileh.createEArray(fileh.root, 'array_c', a, "Chars")
array_c.append(strings.array(['a'*2, 'b'*4], itemsize=8))
array_c.append(strings.array(['a'*6, 'b'*8, 'c'*10], itemsize=8))

# Read the string EArray we have created on disk
for s in array_c:
    print "array_c[%s] => '%s'" % (array_c.nrow, s)
# Close the file
fileh.close()
	      </verbatim>

	      <p>and the output is:
	      </p>

	      <verbatim>
		array_c[0] => 'aa'
		array_c[1] => 'bbbb'
		array_c[2] => 'aaaaaa'
		array_c[3] => 'bbbbbbbb'
		array_c[4] => 'cccccccc'
	      </verbatim>

	    </subsubsection> <!-- EArray.append -->
	  </subsection> <!-- EArray methods -->
	</section> <!-- EArray class -->

	<section id="VLArrayClassDescr">
	  <heading>The <visual markup="tt">VLArray</visual> class</heading>

	  <p>Instances of this class represents array objects in the
	    object tree with the property that their rows can have a
	    <visual markup="bf">variable</visual> number of
	    (homogeneous) elements (called <em>atomic</em> objects, or
	    just <em>atoms</em>). Variable length arrays (or
	    <em>VLA's</em> for short), similarly to <verb>Table</verb>
	    instances, can have only one dimension, and likewise
	    <verb>Table</verb>, the compound elements (the
	    <em>atoms</em>) of the rows of <verb>VLArrays</verb> can be
	    fully multidimensional objects.
	  </p>
	  <p><verb>VLArray</verb> provides methods to read/write data
	    from/to variable length array objects residents on disk.
	    Also, note that this object inherits all the public
	    attributes and methods that <verb>Leaf</verb> already has.
	  </p>

	  <subsection>
	    <heading><visual markup="tt">VLArray</visual> instance
	      variables</heading>
	    <description>
	      <term>atom</term>
	      <item>
		An <verb>Atom</verb> (see <ref refid="AtomClassDescr"></ref>)
		instance representing the shape, type and flavor
		of the atomic objects to be saved.
	      </item>
	      <term>nrow</term> <item>On iterators, this is the index of
		the current row.
	      </item>
	      <term>nrows</term> <item>The total number of rows.
	      </item>
	    </description>
	  </subsection>

	  <subsection>
	    <heading><visual markup="tt">VLArray</visual> methods</heading>

	    <subsubsection id="VLArray.getEnum">
	      <heading>getEnum()</heading>

	      <p>Get the enumerated type associated with this array.
	      </p>

	      <p>If this array is of an enumerated type, the
		corresponding <verb>Enum</verb> instance (see <ref
		  refid="EnumClassDescr"></ref>) is returned.  If it is
		not of an enumerated type, a <verb>TypeError</verb> is
		raised.
	      </p>
	    </subsubsection>

	    <subsubsection id="VLArray.append">
	      <heading>append(sequence, *objects)</heading>

	      <p>Append objects in the <verb>sequence</verb> to the array.</p>

	      <p>This method appends the objects in the <verb>sequence</verb>
		to a <em>single row</em> in this array.
		The type of individual objects must be compliant with
		the type of atoms in the array.
		In the case of variable length strings, the very string to append
		is the <verb>sequence</verb>.
	      </p>

	      <p>Example of use (code available in
		<verb>examples/vlarray1.py</verb>):
	      </p>

	      <verbatim>
<![CDATA[
import tables
from numpy import *   # or, from numarray import *

# Create a VLArray:
fileh = tables.openFile("vlarray1.h5", mode = "w")
vlarray = fileh.createVLArray(fileh.root, 'vlarray1',
tables.Int32Atom(flavor="numpy"),
                 "ragged array of ints", Filters(complevel=1))
# Append some (variable length) rows:
vlarray.append(array([5, 6]))
vlarray.append(array([5, 6, 7]))
vlarray.append([5, 6, 9, 8])

# Now, read it through an iterator:
for x in vlarray:
    print vlarray.name+"["+str(vlarray.nrow)+"]-->", x

# Close the file
fileh.close()
]]>
	      </verbatim>

	      <p>The output of the previous program looks like this:</p>

	      <verbatim>
<![CDATA[vlarray1[0]--> [5 6]
vlarray1[1]--> [5 6 7]
vlarray1[2]--> [5 6 9 8]]]>
	      </verbatim>

	      <p>The <verb>objects</verb> argument is only retained
		for backwards compatibility; please do <em>not</em>
		use it.
	      </p>
	    </subsubsection> <!-- VLArray.append() -->

	    <subsubsection id="iterrowsVLArrayDescr">
	      <heading>iterrows(start=None,
		stop=None, step=1)</heading>

	      <p>Returns an iterator yielding one row per iteration. If
		a range is supplied (i.e. some of the <em>start</em>,
		<em>stop</em> or <em>step</em> parameters are passed),
		only the appropriate rows are returned. Else, all the
		rows are returned. See also the <verb>__iter__()</verb>
		special methods in <ref
		  refid="VLArraySpecialMethods">section</ref> for a
		shorter way to call this iterator.
	      </p>
	      <p>The meaning of the <em>start</em>, <em>stop</em> and
		<em>step</em> parameters is the same as in the
		<verb>range()</verb> python function, except that
		negative values of <verb>step</verb> are not
		allowed. Moreover, if only <verb>start</verb> is
		specified, then <verb>stop</verb> will be set to
		<verb>start+1</verb>. If you do not specify neither
		<em>start</em> nor <em>stop</em>, then all the rows in
		the object are selected.
	      </p>

	      <p>Example of use:</p>

	      <verbatim>
		for row in vlarray.iterrows(step=4):
		print vlarray.name+"["+str(vlarray.nrow)+"]-->", row
	      </verbatim>

	    </subsubsection> <!-- VLArray.iterrows -->

	    <subsubsection id="readVLArrayDescr">
	      <heading>read(start=None, stop=None,
		step=1)</heading>

	      <p>Returns the actual data in <verb>VLArray</verb>. As the
		lengths of the different rows are variable, the returned
		value is a python list, with as many entries as
		specified rows in the range parameters.
	      </p>
	      <p>The meaning of the <em>start</em>, <em>stop</em> and
		<em>step</em> parameters is the same as in the
		<verb>range()</verb> python function, except that
		negative values of <verb>step</verb> are not
		allowed. Moreover, if only <verb>start</verb> is
		specified, then <verb>stop</verb> will be set to
		<verb>start+1</verb>. If you do not specify neither
		<em>start</em> nor <em>stop</em>, then all the rows in
		the object are selected.
	      </p>
	    </subsubsection>

	  </subsection> <!-- VLArray methods -->

	  <subsection id="VLArraySpecialMethods">
	    <heading><visual markup="tt">VLArray</visual> special
	      methods</heading>

	    <p>Following are described the methods that automatically
	      trigger actions when a <verb>VLArray</verb> instance is
	      accessed in a special way (e.g., <verb>vlarray[2:5]</verb>
	      will be equivalent to a call to
	      <verb>vlarray.__getitem__(slice(2,5,None)</verb>).
	    </p>

	    <subsubsection id="VLArray.__iter__">
	      <heading>__iter__()</heading>

	      <p>It returns the same iterator than
		<verb>VLArray.iterrows(0,0,1)</verb>. However, this does
		not accept parameters.</p>

	      <p>Example of use:</p>

	      <verbatim>
		result = [ row for row in vlarray ]
	      </verbatim>

	      <p>Which is equivalent to:</p>

	      <verbatim>
		result = [ row for row in vlarray.iterrows() ]
	      </verbatim>

	    </subsubsection>

	    <subsubsection id="VLArray.__getitem__">
	      <heading>__getitem__(key)</heading>

	      <p>It returns the slice of rows determined by
		<verb>key</verb>, which can be an integer index or an
		extended slice. The returned value is a list of objects
		of type <verb>array.atom.type</verb>.
	      </p>

	      <p>Example of use:</p>

	      <verbatim>
		list1 = vlarray[4]
		list2 = vlarray[4:1000:2]
	      </verbatim>

	    </subsubsection> <!-- VLArray __getitem__ -->

	    <subsubsection id="VLArray.__setitem__">
	      <heading>__setitem__(keys, value)</heading>

	      <p>Updates a vlarray row described by <verb>keys</verb> by
		setting it to <verb>value</verb>. Depending on the value
		of <verb>keys</verb>, the action taken is different:
	      </p>
	      <description>
		<term><verb>keys</verb> is an integer:</term> <item>It
		  refers to the number of row to be modified. The
		  <verb>value</verb> object must be type and shape
		  compatible with the object that exists in the vlarray
		  row.</item>

		<term><verb>keys</verb> is a tuple:</term> <item>The
		  first element refers to the row to be modified, and
		  the second element to the range (so, it can be an
		  integer or an slice) of the row that will be
		  updated. As above, the <verb>value</verb> object must
		  be type and shape compatible with the object specified
		  in the vlarray row <em>and</em> range.</item>
	      </description>


	      <p><visual markup="bf">Note:</visual> When updating
		<verb>VLStrings</verb> (codification UTF-8) or
		<verb>Objects</verb> atoms, there is a problem: one can
		only update values with <em>exactly</em> the same bytes
		than in the original row. With UTF-8 encoding this is
		problematic because, for instance, '<verb>c</verb>'
		takes 1 byte, but '<verb>ç</verb>' takes two. The same
		applies when using <verb>Objects</verb> atoms, because
		when cPickle applies to a class instance (for example),
		it does not guarantee to return the same number of bytes
		than over other instance, even of the same class than
		the former. These facts effectively limit the number of
		objects than can be updated in <verb>VLArray</verb>s.
	      </p>

	      <p>Example of use:</p>

	      <verbatim>
		vlarray[0] = vlarray[0]*2+3
		vlarray[99,3:] = arange(96)*2+3
		# Negative values for start and stop (but not step) are supported
		vlarray[99,-99:-89:2] = vlarray[5]*2+3
	      </verbatim>

	    </subsubsection> <!-- VLArray __setitem__ -->

	  </subsection> <!-- VLArray special methods -->
	</section> <!-- VLArray class -->

	<section id="UnImplementedClassDescr">
	  <heading>The <visual markup="tt">UnImplemented</visual> class</heading>

	  <p>Instances of this class represents an unimplemented dataset
	    in a generic HDF5 file. When reading such a file (i.e. one
	    that has not been created with <verb>PyTables</verb>, but
	    with some other HDF5 library based tool), chances are that
	    the specific combination of <em>datatypes</em> and/or
	    <em>dataspaces</em> in some dataset might not be supported
	    by <verb>PyTables</verb> yet. In such a case, this dataset
	    will be mapped into the <verb>UnImplemented</verb> class and
	    hence, the user will still be able to build the complete
	    object tree of this generic HDF5 file, as well as enabling
	    the access (both read and <em>write</em>) of the attributes
	    of this dataset and some metadata. Of course, the user won't
	    be able to read the actual data on it.
	  </p>

	  <p>This is an elegant way to allow users to work with generic
	    HDF5 files despite the fact that some of its datasets would
	    not be supported by <verb>PyTables</verb>. However, if you
	    are really interested in having access to an unimplemented
	    dataset, please, get in contact with the developer team.
	  </p>
	  <p>This class does not have any public instance variables,
	    except those inherited from the <verb>Leaf</verb> class
	    (<ref refid="LeafClassDescr">see</ref>).
	  </p>
	</section> <!-- UnImplemented class -->

	<section id="AttributeSetClassDescr">
	  <heading>The <visual markup="tt">AttributeSet</visual>
	    class</heading>

	  <p>Represents the set of attributes of a node (Leaf or
	    Group). It provides methods to create new attributes, open,
	    rename or delete existing ones.
	  </p>

	  <p>Like in <verb>Group</verb> instances,
	    <verb>AttributeSet</verb> instances make use of the
	    <em>natural naming</em> convention, i.e. you can access the
	    attributes on disk like if they were <em>normal</em>
	    <verb>AttributeSet</verb> attributes. This offers the user
	    a very convenient way to access (but also to set and
	    delete) node attributes by simply specifying them like a
	    <em>normal</em> attribute class.
	  </p>

	  <p><visual markup="bf">Caveat emptor:</visual> All Python
	    data types are supported. In particular, multidimensional
	    <verb>numarray</verb> objects are saved natively as
	    multidimensional objects in the HDF5 file.  Python strings
	    are also saved natively as HDF5 strings, and loaded back
	    as Python strings.  However, the rest of the data types
	    including the Python scalar ones (i.e. Int, Long and
	    Float) and more general objects (like <verb>NumPy</verb>
	    or <verb>Numeric</verb>) are serialized using
	    <verb>cPickle</verb>, so you will be able to correctly
	    retrieve them only from a Python-aware HDF5 library. So,
	    if you want to save Python scalar values and be able to
	    read them with generic HDF5 tools, you should make use of
	    scalar <verb>numarray</verb> objects (for example
	    <verb>numarray.array(1, type=numarray.Int64)</verb>). In
	    the same way, attributes in HDF5 native files will be
	    always mapped into <verb>numarray</verb>
	    objects. Specifically, a multidimensional attribute will
	    be mapped into a multidimensional <verb>numarray</verb>
	    and an scalar will be mapped into a scalar
	    <verb>numarray</verb> (for example, an attribute of type
	    <verb>H5T_NATIVE_LLONG</verb> will be read and returned as
	    a <verb>numarray.array(X, type=numarray.Int64)</verb>
	    scalar).
	  </p>

	  <p>One more warning: because of the various potential difficulties
	    in restoring a Python object stored in an attribute,
	    you may end up getting a <verb>cPickle</verb> string
	    where a Python object is expected.
	    If this is the case, you may wish to run <verb>cPickle.loads()</verb>
	    on that string to get an idea of where things went wrong,
	    as shown in this example:
	  </p>

	  <verbatim>
<![CDATA[
>>> import tables
>>>
>>> class MyClass(object):
...   foo = 'bar'
...
>>> # An object of my custom class.
... myObject = MyClass()
>>>
>>> h5f = tables.openFile('test.h5', 'w')
>>> h5f.root._v_attrs.obj = myObject  # store the object
>>> print h5f.root._v_attrs.obj.foo  # retrieve it
bar
>>> h5f.close()
>>>
>>> # Delete class of stored object and reopen the file.
... del MyClass, myObject
>>>
>>> h5f = tables.openFile('test.h5', 'r')
>>> print h5f.root._v_attrs.obj.foo
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
AttributeError: 'str' object has no attribute 'foo'
>>> # Let us inspect the object to see what is happening.
... print repr(h5f.root._v_attrs.obj)
'ccopy_reg\n_reconstructor\np1\n(c__main__\nMyClass\np2\nc__builtin__\nobject\np3\nNtRp4\n.'
>>> # Maybe unpickling the string will yield more information:
... import cPickle
>>> cPickle.loads(h5f.root._v_attrs.obj)
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
AttributeError: 'module' object has no attribute 'MyClass'
>>> # So the problem was not in the stored object,
... # but in the *environment* where it was restored.
... h5f.close()
]]>
	  </verbatim>

	  <subsection id="AttributeSetClassInstanceVariables">
	    <heading><visual markup="tt">AttributeSet</visual> instance
	      variables</heading>
	    <description>

	      <term>_v_node</term> <item>The parent node instance.</item>

	      <term>_v_attrnames</term> <item>List with all attribute
		names.</item>

	      <term>_v_attrnamessys</term> <item>List with system attribute
		names.</item>

	      <term>_v_attrnamesuser</term> <item>List with user attribute
		names.</item>

	    </description>

	  </subsection>

	  <subsection>
	    <heading><visual markup="tt">AttributeSet</visual>
	      methods</heading>

	    <p>Note that this class defines the
	      <verb>__setattr__</verb>, <verb>__getattr__</verb> and
	      <verb>__delattr__</verb> and they work as normally
	      intended. Any scalar (string, ints or floats) attribute
	      is supported natively as an attribute. However,
	      <verb>(c)Pickle</verb> is automatically used so as to
	      serialize other kind of objects (like lists, tuples,
	      dicts, small NumPy/Numeric/numarray objects, ...)  that
	      you might want to save.  If an attribute is set on a
	      target node that already has a large number of
	      attributes, a <verb>PerformanceWarning</verb> will be
	      issued.
	    </p>
	    <p>With these special methods, you can access, assign or
	      delete attributes on disk by just using the next
	      constructs:

	      <verbatim>
		leaf.attrs.myattr = "str attr"  # Set a string (native support)
		leaf.attrs.myattr2 = 3          # Set an integer (native support)
		leaf.attrs.myattr3 = [3,(1,2)]  # A generic object (Pickled)
		attrib = leaf.attrs.myattr      # Get the attribute myattr
		del leaf.attrs.myattr           # Delete the attribute myattr
	      </verbatim>

	    </p>

	    <description>
	      <term id="copyAttrDescr">_f_copy(where)</term><item>Copy
		the user attributes (as well as <em>certain</em> system
		attributes) to <em>where</em> object. <em>where</em> has
		to be a <verb>Group</verb> or <verb>Leaf</verb>
		instance.
	      </item>
	      <term id="listAttrDescr">_f_list(attrset = "user")</term>
	      <item>Return a list of attribute names of the parent
		node. <em>attrset</em> selects the attribute set to be
		used. A <verb>user</verb> value returns only the
		user attributes and this is the
		default. <verb>sys</verb> returns only the system
		attributes.  <verb>all</verb> returns both the system
		and user attributes.
	      </item>

	      <term id="renameAttrDescr">_f_rename(oldattrname,
		newattrname)</term><item>Rename an attribute.</item>
	    </description>

	  </subsection>

	</section> <!-- AttributeSet class -->

	<section id="declarativeClasses">
	  <heading>Declarative classes
	  </heading>
	  <p>In this section a series of classes that are meant to
	    <em>declare</em> datatypes that are required for primary
	    <verb>PyTables</verb> (like <verb>Table</verb> or
	    <verb>VLArray</verb> ) objects are described.
	  </p>

	  <subsection id="IsDescriptionClassDescr">
	    <heading>The <visual markup="tt">IsDescription</visual>
	      class</heading>

	    <p>This class is designed to be used as an easy, yet
	      meaningful way to describe the properties of
	      <verb>Table</verb> objects through the definition of
	      <em>derived classes</em> that inherit properties from it.
	      In order to define such a class, you must declare it as
	      descendant of <em>IsDescription</em>, with as many
	      attributes as columns you want in your table.  The name of
	      each attribute will become the name of a column, and its
	      value will hold a description of it.
	    </p>
	    <p>Ordinary columns can be described using instances of the
	      <verb>Col</verb> (see <ref
		refid="ColClassDescr">section</ref>) class.  Nested
	      columns can be described by using classes derived from
	      <verb>IsDescription</verb> or instances of it.  Derived
	      classes can be declared in place (in which case the column
	      takes the name of the class) or referenced by name, and
	      they can have a <verb>_v_pos</verb> special attribute
	      which sets the position of the nested column among its
	      sibling columns.
	    </p>
	    <p>Once you have created a description object, you can pass
	      it to the <verb>Table</verb> constructor, where all the
	      information it contains will be used to define the table
	      structure. See the <ref
		refid="secondExample">section</ref> for an example on how
	      that works.
	    </p>
	    <p>See below for a complete list of the special attributes
	      that can be specified to complement the
	      <em>metadata</em> of an <verb>IsDescription</verb>
	      class.
	    </p>

	    <subsubsection id="IsDescription.specialAttrs">
	      <heading><visual markup="tt">IsDescription</visual> special
		attributes</heading>
	      <description>
		<term>_v_flavor</term> <item>The flavor of the
		  table. It can take <em>"numarray"</em> (default) or
		  <em>"numpy"</em> values. This determines the type of
		  objects returned during input (i.e. read) operations.
		</item>
		<term>_v_indexprops</term> <item>An instance of the
		  <verb>IndexProps</verb> class (see <ref
		  refid="IndexPropsClassDescr">section</ref>). You can
		  use this to alter the properties of the index
		  creation process for a table.
		</item>
		<term>_v_pos</term> <item>Sets the position of a
		  possible nested column description among its sibling
		  columns.
		</item>
	      </description>
	    </subsubsection> <!-- special attributes of IsDescription -->
	  </subsection> <!-- isDescription -->

	  <subsection id="ColClassDescr">
	    <heading>The <visual markup="tt">Col</visual> class and its
	      descendants
	    </heading>

	    <p>The <verb>Col</verb> class is used as a mean to declare
	      the different properties of a table column. In addition, a
	      series of descendant classes are offered in order to make
	      these column descriptions easier to the user. In general,
	      it is recommended to use these descendant classes, as they
	      are more meaningful when found in the middle of the code.
	    </p>

	    <subsubsection>
	      <heading><visual markup="tt">Col</visual> instance
		attributes</heading>

	      <description>

		<term>type</term> <item>The type class of the column.
		</item>

		<term>stype</term> <item>The string type of the column.
		</item>

		<term>recarrtype</term> <item>The string type, in
		  <verb>RecArray</verb> format, of the column.
		</item>

		<term>shape</term><item>The shape of the column.</item>

		<term>itemsize</term> <item>The size of the base
		  items. Specially useful for <verb>StringCol</verb>
		  objects.
		</item>

		<term>indexed</term><item>Whether this column is meant
		  to be indexed or not.</item>

		<term>_v_pos</term><item>The position of this column
		  with regard to its column siblings.</item>

		<term>_v_name</term><item>The name of this column</item>

		<term>_v_pathname</term> <item>The complete pathname of
		  the column. This is mainly useful in nested columns;
		  for non-nested ones this value is the same a
		  <verb>_v_name</verb>.
		</item>

	      </description>

	    </subsubsection>

	    <subsubsection>
	      <heading><visual markup="tt">Col</visual>
		methods</heading>

	      <p>None.</p>

	    </subsubsection>

	    <subsubsection>
	      <heading><visual markup="tt">Col</visual>
		constructors</heading>

	      <p>A description of the different constructors with their
		parameters follows:
	      </p>

	      <description>

		<term>Col(dtype="Float64", shape=1, dflt=None, pos=None,
		  indexed=0)
		</term>
		<item>Declare the properties of a <verb>Table</verb>
		  column.

		  <description>

		    <term>dtype</term> <item>The data type for the
		      column. All types listed in <ref
			refid="datatypesSupported"> appendix</ref> are valid
		      data types for columns.  The type description is
		      accepted both in string-type format and as a
		      numarray data type.</item>

		    <term>shape</term> <item>An integer or a tuple, that
		      specifies the number of <em>dtype</em> items for
		      each element (or shape, for multidimensional
		      elements) of this column. For <verb>CharType</verb>
		      columns, the last dimension is used as the length
		      of the character strings. However, for this kind of
		      objects, the use of <verb>StringCol</verb> subclass
		      is strongly recommended.</item>

		    <term>dflt</term> <item>The default value for
		      elements of this column. If the user does not
		      supply a value for an element while filling a
		      table, this default value will be written to
		      disk. If the user supplies an scalar value for a
		      multidimensional column, this value is
		      automatically <em>broadcasted</em> to all the
		      elements in the column cell. If <em>dflt</em> is
		      not supplied, an appropriate zero value (or
		      <em>null</em> string) will be chosen by default.
		      Please, note that all the default values are kept
		      internally as numarray objects. </item>

		    <term>pos</term> <item>By default, columns are
		      arranged in memory following an alpha-numerical
		      order of the column names. In some situations,
		      however, it is convenient to impose a user defined
		      ordering. <em>pos</em> parameter allows the user
		      to force the desired ordering.</item>

		    <term>indexed</term> <item>Whether this column should
		      be indexed for better performance in table
		      selections. </item>

		  </description>
		</item>

		<term>StringCol(length=None, dflt=None, shape=1, pos=None,
		  indexed=0)
		</term>
		<item>Declare a column to be of type
		  <verb>CharType</verb>. The <em>length</em> parameter
		  sets the length of the strings. The meaning of the other
		  parameters are like in the <verb>Col</verb> class.
		</item>

		<term>BoolCol(dflt=0, shape=1, pos=None, indexed=0) </term>
		<item>Define a column to be of type <verb>Bool</verb>.
		  The meaning of the parameters are the same of those in
		  the <verb>Col</verb> class.
		</item>

		<term>IntCol(dflt=0, shape=1, itemsize=4, sign=1,
		  pos=None, indexed=0)
		</term>
		<item>Declare a column to be of type <verb>IntXX</verb>,
		  depending on the value of <em>itemsize</em> parameter,
		  that sets the number of bytes of the integers in the
		  column. <em>sign</em> determines whether the integers
		  are signed or not. The meaning of the other parameters
		  are the same of those in the <verb>Col</verb> class.

		  <p>This class has several descendants:
		  </p>

		  <description>
		    <term>Int8Col(dflt=0, shape=1, pos=None,
		      indexed=0)</term> <item>Define a column of type
		      <verb>Int8</verb>.</item>

		    <term>UInt8Col(dflt=0, shape=1, pos=None,
		      indexed=0)</term> <item>Define a column of type
		      <verb>UInt8</verb>.</item>

		    <term>Int16Col(dflt=0, shape=1, pos=None, indexed=0)</term>
		    <item>Define a column of type <verb>Int16</verb>.</item>

		    <term>UInt16Col(dflt=0, shape=1, pos=None, indexed=0)</term>
		    <item>Define a column of type <verb>UInt16</verb>.</item>

		    <term>Int32Col(dflt=0, shape=1, pos=None, indexed=0)</term>
		    <item>Define a column of type <verb>Int32</verb>.</item>

		    <term>UInt32Col(dflt=0, shape=1, pos=None, indexed=0)</term>
		    <item>Define a column of type <verb>UInt32</verb>.</item>

		    <term>Int64Col(dflt=0, shape=1, pos=None, indexed=0)</term>
		    <item>Define a column of type <verb>Int64</verb>.</item>

		    <term>UInt64Col(dflt=0, shape=1, pos=None, indexed=0)</term>
		    <item>Define a column of type <verb>UInt64</verb>.</item>

		  </description>

		</item>

		<term>FloatCol(dflt=0.0, shape=1, itemsize=8, pos=None,
		  indexed=0)
		</term>
		<item>Define a column to be of type <verb>FloatXX</verb>,
		  depending on the value of <verb>itemsize</verb>. The
		  <verb>itemsize</verb> parameter sets the number of bytes
		  of the floats in the column and the default is 8 bytes
		  (double precision). The meaning of the other parameters
		  are the same as those in the <verb>Col</verb> class.

		  <p>This class has two descendants:
		  </p>

		  <description>
		    <term>Float32Col(dflt=0.0, shape=1, pos=None,
		      indexed=0)</term> <item>Define a column of type
		      <verb>Float32</verb>.</item>

		    <term>Float64Col(dflt=0.0, shape=1, pos=None,
		      indexed=0)</term> <item>Define a column of type
		      <verb>Float64</verb>.</item>

		  </description>
		</item>

		<term>ComplexCol(dflt=0.+0.j, shape=1, itemsize=16, pos=None)
		</term>
		<item>Define a column to be of type
		  <verb>ComplexXX</verb>, depending on the value of
		  <verb>itemsize</verb>. The <verb>itemsize</verb>
		  parameter sets the number of bytes of the complex types
		  in the column and the default is 16 bytes (double
		  precision complex). The meaning of the other parameters
		  are the same as those in the <verb>Col</verb> class.

		  <p>This class has two descendants:
		  </p>

		  <description>
		    <term>Complex32Col(dflt=0.+0.j, shape=1, pos=None)</term>
		    <item>Define a column of type <verb>Complex32</verb>.</item>

		    <term>Float64Col(dflt=0+0.j, shape=1, pos=None)</term>
		    <item>Define a column of type <verb>Complex64</verb>.</item>

		  </description>

		  <p><verb>ComplexCol</verb> columns and its descendants
		    do not support indexation.</p>
		</item>

		<term>TimeCol(dflt=0, shape=1, itemsize=8, pos=None,
		  indexed=0)
		</term>
		<item>Define a column to be of type <em>Time</em>.  Two
		  kinds of time columns are supported depending on the
		  value of <verb>itemsize</verb>: 4-byte signed integer
		  and 8-byte double precision floating point columns
		  (the default ones).  The meaning of the other
		  parameters are the same as those in the
		  <verb>Col</verb> class.

		  <p>Time columns have a special encoding in the HFD5 file.
		    See <ref refid="datatypesSupported">appendix</ref>
		    for more information on those types.
		  </p>

		  <p>This class has two descendants:
		  </p>

		  <description>
		    <term>Time32Col(dflt=0, shape=1, pos=None,
		      indexed=0)</term> <item>Define a column of type
		      <verb>Time32</verb>.</item>

		    <term>Time64Col(dflt=0.0, shape=1, pos=None,
		      indexed=0)</term> <item>Define a column of type
		      <verb>Time64</verb>.</item>
		  </description>
		</item>

		<term>EnumCol(enum, dflt, dtype='UInt32', shape=1, pos=None,
		  indexed=False)
		</term>
		<item>
		  <p>Description of a column of an enumerated type.</p>

		  <p>Instances of this class describe a table column which
		    stores enumerated values.  Those values belong to an
		    enumerated type, defined by the first argument
		    (<verb>enum</verb>) in the constructor of
		    <verb>EnumCol</verb>, which accepts the same kinds of
		    arguments as <verb>Enum</verb> (see <ref
		      refid="EnumClassDescr"></ref>).  The enumerated type
		    is stored in the <verb>enum</verb> attribute of the
		    column.
		  </p>

		  <p>A default value must be specified as the second
		    argument (<verb>dflt</verb>) in the constructor; it
		    must be the <em>name</em> (a string) of one of the
		    enumerated values in the enumerated type.  Once the
		    column is created, the corresponding concrete value
		    is stored in its <verb>dflt</verb> attribute.  If
		    the name does not match any value in the enumerated
		    type, a <verb>KeyError</verb> is raised.
		  </p>

		  <p>A numarray data type might be specified in order
		    to determine the base type used for storing the
		    values of enumerated values in memory and disk.
		    The data type must be able to represent each and
		    every concrete value in the enumeration.  If it is
		    not, a <verb>TypeError</verb> is raised.  The
		    default base type is unsigned 32-bit integer,
		    which is sufficient for most cases.
		  </p>

		  <p>The <verb>stype</verb> attribute of enumerated
		    columns is always <verb>'Enum'</verb>, while the
		    <verb>type</verb> attribute is the data type used
		    for storing concrete values.
		  </p>

		  <p>The shape, position and indexed attributes of the
		    column are treated as with other column description
		    objects (see <ref refid="ColClassDescr"></ref>).
		  </p>
		</item>

	      </description>
	    </subsubsection> <!-- Col constructors -->
	  </subsection> <!-- Col classes -->

	  <subsection id="AtomClassDescr">
	    <heading>The <visual markup="tt">Atom</visual> class and its
	      descendants.
	    </heading>
	    <p>The <verb>Atom</verb> class is a descendant of the
	      <verb>Col</verb> class (see <ref
		refid="ColClassDescr"></ref>) and is meant to declare the
	      different properties of the <em>base element</em> (also
	      known as <em>atom</em>) of <verb>CArray</verb>,
	      <verb>EArray</verb> and <verb>VLArray</verb> objects. The
	      <verb>Atom</verb> instances have the property that their
	      length is always the same. However, you can grow objects
	      along the extensible dimension in the case of
	      <verb>EArray</verb> or put a variable number of them on a
	      <verb>VLArray</verb> row. Moreover, the atoms are not
	      restricted to scalar values, and they can be fully
	      multidimensional objects.
	    </p>
	    <p>A series of descendant classes are offered in order to
	      make the use of these element descriptions easier. In
	      general, it is recommended to use these descendant
	      classes, as they are more meaningful when found in the
	      middle of the code.
	    </p>

	    <subsubsection>
	      <heading><visual markup="tt">Atom</visual>
		instance variables</heading>

	      <p>In addition to the variables that it inherits from the
		<verb>Col</verb> class, it has the next additional
		attributes:</p>

	      <description>

		<term>flavor</term> <item>The object representation for
		  this atom. See below on constructors description for
		  <verb>Atom</verb> class the possible values it can
		  take.
		</item>

	      </description>

	    </subsubsection>

	    <subsubsection>
	      <heading><visual markup="tt">Atom</visual>
		methods</heading>

	      <description>

		<term>atomsize()</term> <item>Returns the total length,
		  in bytes, of the element base atom. If its shape is
		  has one zero element on it (for use in
		  <verb>EArrays</verb>, for example), this is replaced
		  by an one in order to compute the atom size correctly.
		</item>

	      </description>
	    </subsubsection>
	    <subsubsection>
	      <heading><visual markup="tt">Atom</visual>
		constructors</heading>

	      <p>A description of the different constructors with their
		parameters follows:
	      </p>

	      <description>
		<term>Atom(dtype="Float64", shape=1, flavor="numarray")
		</term>
		<item>Define properties for the base elements of
		  <verb>CArray</verb>, <verb>EArray</verb> and
		  <verb>VLArray</verb> objects.
		  <description>
		    <term>dtype</term> <item>The data type for the base
		      element. See the <ref
			refid="datatypesSupported">appendix</ref> for a
		      relation of data types supported. The type
		      description is accepted both in string-type format
		      and as a numarray data type.
		    </item>

		    <term>shape</term> <item>In a <verb>EArray</verb>
		      context, it is a <visual markup="bf">tuple</visual>
		      specifying the shape of the object, and one (and only
		      one) of its dimensions <visual
			markup="bf">must</visual> be 0, meaning that the
		      <verb>EArray</verb> object will be enlarged along
		      this axis. In the case of a <verb>VLArray</verb>, it
		      can be an integer with a value of 1 (one) or a
		      tuple, that specifies whether the atom is an scalar
		      (in the case of a 1) or has multiple dimensions (in
		      the case of a tuple). For <!-- <verb>CharType</verb>
		      elements, the first dimension -->
		      <verb>CharType</verb> elements, the last dimension
		      is used as the length of the character
		      strings. However, for this kind of objects, the use
		      of <verb>StringAtom</verb> subclass is strongly
		      recommended.
		    </item>

		    <term>flavor</term> <item>The object
		      representation for this atom. It can be any of
		      <em>"numarray"</em>, <em>"numpy"</em> or
		      <em>"python"</em> for the character types and
		      <em>"numarray"</em>, <em>"numpy"</em>,
		      <em>"numeric"</em> or <em>"python"</em> for the
		      numerical types. If specified, the read atoms
		      will be converted to that specific flavor. If
		      not specified, the atoms will remain in their
		      native format (i.e. <verb>numarray</verb>).
		    </item>
		  </description>

		</item> <!-- Atom class -->

		<term>StringAtom(shape=1, length=None,
		  flavor="numarray")</term> <item>Define an atom to be
		  of <verb>CharType</verb> type. The meaning of the
		  <em>shape</em> parameter is the same as in the
		  <verb>Atom</verb> class. <em>length</em> sets the
		  length of the strings atoms. <em>flavor</em> can be
		  whether <verb>"numarray"</verb>,
		  <verb>"numpy"</verb> or
		  <verb>"python"</verb>. Unicode strings are not
		  supported by this type; see the
		  <verb>VLStringAtom</verb> class if you want Unicode
		  support (only available for <verb>VLAtom</verb>
		  objects).
		</item>

		<term>BoolAtom(shape=1, flavor="numarray") </term>
		<item>Define an atom to be of type <verb>Bool</verb>.
		  The meaning of the parameters are the same of those in
		  the <verb>Atom</verb> class.
		</item>

		<term>IntAtom(shape=1, itemsize=4, sign=1,
		  flavor="numarray") </term> <item>Define an atom to be of
		  type <verb>IntXX</verb>, depending on the value of
		  <em>itemsize</em> parameter, that sets the number of
		  bytes of the integers that conform the
		  atom. <em>sign</em> determines whether the integers are
		  signed or not. The meaning of the other parameters are
		  the same of those in the <verb>Atom</verb> class.

		  <p>This class has several descendants:
		  </p>

		  <description>
		    <term>Int8Atom(shape=1, flavor="numarray")</term>
		    <item>Define an atom of type <verb>Int8</verb>.</item>

		    <term>UInt8Atom(shape=1, flavor="numarray")</term>
		    <item>Define an atom of type <verb>UInt8</verb>.</item>

		    <term>Int16Atom(shape=1, flavor="numarray")</term>
		    <item>Define an atom of type <verb>Int16</verb>.</item>

		    <term>UInt16Atom(shape=1, flavor="numarray")</term>
		    <item>Define an atom of type <verb>UInt16</verb>.</item>

		    <term>Int32Atom(shape=1, flavor="numarray")</term>
		    <item>Define an atom of type <verb>Int32</verb>.</item>

		    <term>UInt32Atom(shape=1, flavor="numarray")</term>
		    <item>Define an atom of type <verb>UInt32</verb>.</item>

		    <term>Int64Atom(shape=1, flavor="numarray")</term>
		    <item>Define an atom of type <verb>Int64</verb>.</item>

		    <term>UInt64Atom(shape=1, flavor="numarray")</term>
		    <item>Define an atom of type <verb>UInt64</verb>.</item>

		  </description>

		</item>

		<term>FloatAtom(shape=1, itemsize=8, flavor="numarray")
		</term>
		<item>Define an atom to be of <verb>FloatXX</verb>
		  type, depending on the value of <verb>itemsize</verb>. The
		  <verb>itemsize</verb> parameter sets the number of bytes
		  of the floats in the atom and the default is 8 bytes
		  (double precision). The meaning of the other parameters
		  are the same as those in the <verb>Atom</verb> class.

		  <p>This class has two descendants:
		  </p>

		  <description>
		    <term>Float32Atom(shape=1, flavor="numarray")</term>
		    <item>Define an atom of type <verb>Float32</verb>.</item>

		    <term>Float64Atom(shape=1, flavor="numarray")</term>
		    <item>Define an atom of type <verb>Float64</verb>.</item>

		  </description>
		</item>

		<term>ComplexAtom(shape=1, itemsize=16, flavor="numarray")
		</term>
		<item>Define an atom to be of <verb>ComplexXX</verb> type,
		  depending on the value of <verb>itemsize</verb>. The
		  <verb>itemsize</verb> parameter sets the number of bytes
		  of the floats in the atom and the default is 16 bytes
		  (double precision complex). The meaning of the other
		  parameters are the same as those in the
		  <verb>Atom</verb> class.

		  <p>This class has two descendants:
		  </p>

		  <description>
		    <term>Complex32Atom(shape=1, flavor="numarray")</term>
		    <item>Define an atom of type <verb>Complex32</verb>.</item>

		    <term>Complex64Atom(shape=1, flavor="numarray")</term>
		    <item>Define an atom of type <verb>Complex64</verb>.</item>

		  </description>
		</item>

		<term>TimeAtom(shape=1, itemsize=8, flavor="numarray")
		</term>
		<item>Define an atom to be of type <em>Time</em>.  Two
		  kinds of time atoms are supported depending on the
		  value of <verb>itemsize</verb>: 4-byte signed integer
		  and 8-byte double precision floating point atoms (the
		  default ones).  The meaning of the other parameters
		  are the same as those in the <verb>Atom</verb> class.

		  <p>Time atoms have a special encoding in the HFD5 file.
		    See <ref refid="datatypesSupported">appendix</ref>
		    for more information on those types.
		  </p>

		  <p>This class has two descendants:
		  </p>

		  <description>
		    <term>Time32Atom(shape=1, flavor="numarray")</term>
		    <item>Define an atom of type <verb>Time32</verb>.</item>

		    <term>Time64Atom(shape=1, flavor="numarray")</term>
		    <item>Define an atom of type <verb>Time64</verb>.</item>
		  </description>
		</item>

		<term>EnumAtom(enum, dtype='UInt32', shape=1,
		  flavor='numarray')</term>
		<item>
		  <p>Description of an atom of an enumerated type.</p>

		  <p>Instances of this class describe the atom type used
		    by an array to store enumerated values.  Those
		    values belong to an enumerated type.
		  </p>

		  <p>The meaning of the <verb>enum</verb> and
		    <verb>dtype</verb> arguments is the same as in
		    <verb>EnumCol</verb> (see <ref
		      refid="ColClassDescr"></ref>).  The
		    <verb>shape</verb> and <verb>flavor</verb> arguments
		    have the usual meaning of other <verb>Atom</verb>
		    classes (the <verb>flavor</verb> applies to the
		    representation of concrete read values).
		  </p>

		  <p>Enumerated atoms also have <verb>stype</verb> and
		    <verb>type</verb> attributes with the same values as
		    in <verb>EnumCol</verb>.
		  </p>
		</item>
	      </description>

	      <p>Now, there come two special classes,
		<verb>ObjectAtom</verb> and <verb>VLString</verb>, that
		actually do not descend from <verb>Atom</verb>, but
		which goal is so similar that they should be described
		here. The difference between them and the
		<verb>Atom</verb> and descendants classes is that these
		special classes does not allow multidimensional atoms,
		nor multiple values per row. A <em>flavor</em> can not
		be specified neither as it is immutable (see below).
	      </p>
	      <p><visual markup="bf">Caveat emptor:</visual> You are
		only allowed to use these classes to create
		<verb>VLArray</verb> objects, not <verb>CArray</verb>
		and <verb>EArray</verb> objects.
	      </p>

	      <description>
		<term>ObjectAtom()</term> <item>This class is meant to
		  fit <em>any</em> kind of object in a row of an
		  <verb>VLArray</verb> instance by using
		  <verb>cPickle</verb> behind the scenes. Due to the
		  fact that you can not foresee how long will be the
		  output of the <verb>cPickle</verb> serialization
		  (i.e. the atom already has a <em>variable</em>
		  length), you can only fit a representant of it per
		  row. However, you can still pass several parameters to
		  the <verb>VLArray.append()</verb> method as they will
		  be regarded as a <em>tuple</em> of compound objects
		  (the parameters), so that we still have only one
		  object to be saved in a single row. It does not accept
		  parameters and its flavor is automatically set to
		  <verb>"Object"</verb>, so the reads of rows always
		  returns an arbitrary python object.

		  You can regard <verb>ObjectAtom</verb> types as an
		  easy way to save an arbitrary number of generic python
		  objects in a <verb>VLArray</verb> object.
		</item>

		<term>VLStringAtom()</term> <item>This class describes a
		  <em>row</em> of the <verb>VLArray</verb> class, rather
		  than an <em>atom</em>. It differs from the
		  <verb>StringAtom</verb> class in that you can only add
		  one instance of it to one specific row, i.e. the
		  <verb>VLArray.append()</verb> method only accepts one
		  object when the base atom is of this type. Besides, it
		  supports Unicode strings (contrarily to
		  <verb>StringAtom</verb>) because it uses the UTF-8
		  codification (this is why its <verb>atomsize()</verb>
		  method returns always 1) when serializing to disk. It
		  does not accept any parameter and because its
		  <em>flavor</em> is automatically set to
		  <verb>"VLString"</verb>, the reads of rows always
		  returns a python string. See the <ref
		    refid="VLArrayFormatDescr">appendix</ref> if you are
		  curious on how this is implemented at the low-level.

		  You can regard <verb>VLStringAtom</verb> types as an
		  easy way to save generic variable length strings.
		</item>
	      </description>

	      <p>See <verb>examples/vlarray1.py</verb> and
		<verb>examples/vlarray2.py</verb> for further examples
		on <verb>VLArray</verb>s, including object serialization
		and Unicode string management.
	      </p>

	    </subsubsection> <!-- Atom constructors -->
	  </subsection> <!-- Atom and its descendants -->
	</section> <!-- Declarative classes -->

	<section id="helperClasses">
	  <heading>Helper classes</heading>

	  <p>In this section are listed classes that does not fit in any
	    other section and that mainly serve for ancillary
	    purposes.</p>

	  <subsection id="FiltersClassDescr">
	    <heading>The <visual markup="tt">Filters</visual> class
	    </heading>

	    <p>This class is meant to serve as a container that keeps
	      information about the filter properties associated with
	      the enlargeable leaves, that is <verb>Table</verb>,
	      <verb>EArray</verb> and <verb>VLArray</verb> as well as
	      <verb>CArray</verb>.
	    </p>

	    <p>The public variables of <verb>Filters</verb> are listed
	      below:
	    </p>

	    <description>
	      <term>complevel</term> <item>The compression level (0
		means no compression).
	      </item>
	      <term>complib</term> <item>The compression filter used (in
		case of compressed dataset).
	      </item>
	      <term>shuffle</term> <item>Whether the shuffle filter is
		active or not.
	      </item>
	      <term>fletcher32</term> <item>Whether the fletcher32
		filter is active or not.
	      </item>
	    </description>

	    <p>There are no <verb>Filters</verb> public methods with the
	      exception of the constructor itself that is described
	      next.
	    </p>

	    <subsubsection id="FiltersInitDescr">
	      <heading><visual markup="tt">Filters(complevel=0,
		  complib="zlib", shuffle=1, fletcher32=0)</visual>
	      </heading>

	      <p>The parameters that can be passed to the
		<verb>Filters</verb> class constructor are:
	      </p>

	      <description>
		<term>complevel</term> <item>Specifies a compress level
		  for data. The allowed range is 0-9. A value of 0
		  disables compression. The default is that compression
		  is disabled, that balances between compression effort
		  and CPU consumption.
		</item>
		<term>complib</term> <item> Specifies the compression
		  library to be used. Right now, <verb>"zlib"</verb>
		  (default), <verb>"lzo"</verb>, <verb>"ucl"</verb>
		  and <verb>"bzip2"</verb> values are supported. See
		  <ref refid="compressionIssues">section</ref> for
		  some advice on which library is better suited to
		  your needs.
		</item>
		<term>shuffle</term><item>Whether or not to use the
		  <em>shuffle</em> filter present in the
		  <verb>HDF5</verb> library. This is normally used to
		  improve the compression ratio (at the cost of
		  consuming a little bit more CPU time). A value of 0
		  disables shuffling and 1 makes it active. The default
		  value depends on whether compression is enabled or
		  not; if compression is enabled, shuffling defaults to
		  be active, else shuffling is disabled.
		</item>
		<term>fletcher32</term> <item>Whether or not to use the
		  <em>fletcher32</em> filter in the HDF5 library. This
		  is used to add a checksum on each data chunk. A value
		  of 0 disables the checksum and it is the default.
		</item>
	      </description>

	      <p>Of course, you can also create an instance and then
		assign the ones you want to change. For example:
		<verbatim>
import numarray as na
from tables import *

fileh = openFile("test5.h5", mode = "w")
atom = Float32Atom(shape=(0,2))
filters = Filters(complevel=1, complib = "lzo")
filters.fletcher32 = 1
arr = fileh.createEArray(fileh.root, 'earray', atom, "A growable array",
                         filters = filters)
# Append several rows in only one call
arr.append(na.array([[1., 2.],
                     [2., 3.],
                     [3., 4.]], type=na.Float32))

# Print information on that enlargeable array
print "Result Array:"
print repr(arr)

fileh.close()
		</verbatim>
		This enforces the use of the <verb>LZO</verb> library, a
		compression level of 1 and a fletcher32 checksum filter
		as well. See the output of this example:
		<verbatim>
Result Array:
/earray (EArray(3L, 2), fletcher32, shuffle, lzo(1)) 'A growable array'
  type = Float32
  shape = (3L, 2)
  itemsize = 4
  nrows = 3
  extdim = 0
  flavor = 'numarray'
  byteorder = 'little'
		</verbatim>
	      </p>
	    </subsubsection> <!-- Filters constructor method -->
	  </subsection> <!-- Filters class -->

	  <subsection id="IndexPropsClassDescr">
	    <heading>The <visual markup="tt">IndexProps</visual> class
	    </heading>

	    <p>You can use this class to set/unset the properties in the
	      indexing process of a <verb>Table</verb> column. To use
	      it, create an instance, and assign it to the special
	      attribute <verb>_v_indexprops</verb> in a table description class
	      (<ref refid="IsDescriptionClassDescr">see</ref>) or dictionary.
	    </p>

	    <p>The public variables of <verb>IndexProps</verb> are listed
	      below:
	    </p>

	    <description>
	      <term>auto</term> <item>Whether an existing index should
		be updated or not after a table append operation.
	      </item>
	      <term>reindex</term> <item>Whether the table columns are
		to be re-indexed after an invalidating index operation.
	      </item>
	      <term>filters</term> <item>The filter settings for the
		different <verb>Table</verb> indexes.
	      </item>
	    </description>

	    <p>There are no <verb>IndexProps</verb> public methods with
	      the exception of the constructor itself that is described
	      next.
	    </p>

	    <subsubsection id="IndexPropsInitDescr">
	      <heading><visual markup="tt">IndexProps(auto=1, reindex=1,
		  filters=None)</visual>
	      </heading>

	      <p>The parameters that can be passed to the
		<verb>IndexProps</verb> class constructor are:
	      </p>

	      <description>
		<term>auto</term> <item>Specifies whether an existing
		  index should be updated or not after a table append
		  operation. The default is enable automatic index
		  updates.
		</item>
		<term>reindex</term> <item>Specifies whether the table
		  columns are to be re-indexed after an invalidating
		  index operation (like for example, after a
		  <verb>Table.removeRows</verb> call). The default is to
		  reindex after operations that invalidate indexes.
		</item>
		<term>filters</term><item>Sets the filter properties for
		  <verb>Column</verb> indexes. It has to be an instance
		  of the <verb>Filters</verb> (see <ref
		    refid="FiltersClassDescr">section</ref>) class. A
		  <verb>None</verb> value means that the default
		  settings for the <verb>Filters</verb> object are
		  selected.
		</item>
	      </description>

	    </subsubsection>
	  </subsection> <!-- IndexProps -->

	  <subsection id="IndexClassDescr">
	    <heading>The <visual markup="tt">Index</visual> class</heading>

	    <p>This class is used to keep the indexing information for
	      table columns. It is actually a descendant of the
	      <verb>Group</verb> class, with some added
	      functionality.
	    </p>
	    <p>It has no methods intended for programmer's use, but it
	      has some attributes that maybe interesting for him.
	    </p>

	    <subsubsection id="IndexClassInstanceVariables">
	      <heading><visual markup="tt">Index</visual> instance
		variables</heading>

	      <description>
		<term>column</term> <item>The column object this index
		  belongs to.
		</item>
		<term>type</term> <item>The type class for the
		  index.
		</item>
		<term>itemsize</term> <item>The size of the atomic
		  items. Specially useful for columns of
		  <verb>CharType</verb> type.
		</item>
		<term>nelements</term> <item>The total number of
		  elements in index.
		</item>
		<term>dirty</term> <item>Whether the index is dirty or
		  not.
		</item>
		<term>filters</term> <item>The <verb>Filters</verb> (see
		  <ref refid="FiltersClassDescr">section</ref>) instance
		  for this index.
		</item>
	      </description>

	    </subsubsection> <!-- Index attributes -->
	  </subsection> <!-- Index class -->

	  <subsection id="EnumClassDescr">
	    <heading>The <visual markup="tt">Enum</visual> class</heading>

	    <p>Each instance of this class represents an enumerated type.
	      The values of the type must be declared <em>exhaustively</em>
	      and named with <em>strings</em>, and they might be given
	      explicit concrete values, though this is not compulsory.
	      Once the type is defined, it can not be modified.
	    </p>

	    <p>There are three ways of defining an enumerated type.
	      Each one of them corresponds to the type of the only argument
	      in the constructor of <verb>Enum</verb>:
	    </p>

	    <itemize>
	      <item><em>Sequence of names</em>: each enumerated value is named
		using a string, and its order is determined by its position
		in the sequence; the concrete value is assigned automatically:

		<verbatim>>>> boolEnum = Enum(['True', 'False'])</verbatim>
	      </item>

	      <item><em>Mapping of names</em>: each enumerated value is named
		by a string and given an explicit concrete value.
		All of the concrete values must be different,
		or a <verb>ValueError</verb> will be raised.

	      <verbatim>>>> priority = Enum({'red': 20, 'orange': 10, 'green': 0})
>>> colors = Enum({'red': 1, 'blue': 1})
Traceback (most recent call last):
  ...
ValueError: enumerated values contain duplicate concrete values: 1</verbatim>
	      </item>

	      <item><em>Enumerated type</em>: in that case, a copy of
		the original enumerated type is created.
		Both enumerated types are considered equal.

		<verbatim>>>> prio2 = Enum(priority)
>>> priority == prio2
True</verbatim>
	      </item>
	    </itemize>

	    <p>Please, note that names starting with <verb>_</verb> are
	      not allowed, since they are reserved for internal
	      usage:</p>

	    <verbatim>>>> prio2 = Enum(['_xx'])
Traceback (most recent call last):
  ...
ValueError: name of enumerated value can not start with ``_``: '_xx'</verbatim>

	    <p>The concrete value of an enumerated value is obtained
	      by getting its name as an attribute of the <verb>Enum</verb> instance
	      (see <verb>__getattr__()</verb>) or as an item (see <verb>__getitem__()</verb>).
	      This allows comparisons between enumerated values
	      and assigning them to ordinary Python variables:
	    </p>

	    <verbatim>>>> redv = priority.red
>>> redv == priority['red']
True
>>> redv > priority.green
True
>>> priority.red == priority.orange
False</verbatim>

	    <p>The name of the enumerated value corresponding to a
	      concrete value can also be obtained by using the
	      <verb>__call__()</verb> method of the enumerated type.
	      In this way you get the symbolic name to use it later
	      with <verb>__getitem__()</verb>:
	    </p>

	    <verbatim>>>> priority(redv)
'red'
>>> priority.red == priority[priority(priority.red)]
True</verbatim>

	    <p>(If you ask, the <verb>__getitem__()</verb> method is
	      not used for this purpose to avoid ambiguity in the case
	      of using strings as concrete values.)
	    </p>

	    <subsubsection>
	      <heading>Special methods</heading>

	      <description>
		<term>__getitem__(name)</term>
		<item>
		  <p>Get the concrete value of the enumerated value with
		    that <verb>name</verb>.</p> <p>The
		    <verb>name</verb> of the enumerated value must be
		    a string.  If there is no value with that
		    <verb>name</verb> in the enumeration, a
		    <verb>KeyError</verb> is raised.
		  </p>
		</item>

		<term>__getattr__(name)</term>
		<item>
		  <p>Get the concrete value of the enumerated value
		    with that <verb>name</verb>.</p> <p>The
		    <verb>name</verb> of the enumerated value must be a
		    string.  If there is no value with that
		    <verb>name</verb> in the enumeration, an
		    <verb>AttributeError</verb> is raised.
		  </p>
		</item>

		<term>__contains__(name)</term>
		<item>
		  <p>Is there an enumerated value with that
		    <verb>name</verb> in the type?</p> <p>If the
		    enumerated type has an enumerated value with that
		    <verb>name</verb>, <verb>True</verb> is returned.
		    Otherwise, <verb>False</verb> is returned.  The
		    <verb>name</verb> must be a string.
		  </p>
		  <p>This method does <em>not</em> check for concrete
		    values matching a value in an enumerated type.  For
		    that, please use the <verb>__call__()</verb> method.
		  </p>
		</item>

		<term>__call__(value, *default)</term>
		<item>
		  <p>Get the name of the enumerated value with that
		    concrete <verb>value</verb>.</p> <p>If there is no
		    value with that concrete value in the enumeration and
		    a second argument is given as a <verb>default</verb>,
		    this is returned.  Else, a <verb>ValueError</verb> is
		    raised.
		  </p>
		  <p>This method can be used for checking that a concrete value
		    belongs to the set of concrete values in an enumerated type.
		  </p>
		</item>

		<term>__len__()</term> <item><p>Return the number of
		    enumerated values in the enumerated type.</p></item>

		<term>__iter__()</term>
		<item>
		  <p>Iterate over the enumerated values.</p>
		  <p>Enumerated values are returned as <verb>(name,
		      value)</verb> pairs <em>in no particular order</em>.
		  </p>
		</item>

		<term>__eq__(other)</term>
		<item>
		  <p>Is the <verb>other</verb> enumerated type
		    equivalent to this one?</p> <p>Two enumerated types
		    are equivalent if they have exactly the same
		    enumerated values (i.e. with the same names and
		    concrete values).
		  </p>
		</item>

		<term>__repr__()</term>
		<item>
		  <p>Return the canonical string representation of the
		    enumeration. The output of this method can be
		    evaluated to give a new enumeration object that
		    will compare equal to this one.
		  </p>
		</item>
	      </description>
	    </subsubsection>
	  </subsection> <!-- Enum class -->

	</section> <!-- Helper classes -->
      </chapter> <!-- Reference library -->

      <chapter id="optimizationTips">
	<heading>Optimization tips</heading>

	<aphorism>... durch planmässiges
	  Tattonieren. <newline/>[... through systematic, palpable
	  experimentation.] <caption>Johann Karl Friedrich Gauss
	    <newline/>[asked how he came upon his theorems]</caption>
	</aphorism>

	<p>On this chapter, you will get deeper knowledge of
	  <verb>PyTables</verb> internals. <verb>PyTables</verb> has
	  several places where the user can improve the performance of
	  his application. If you are planning to deal with really large
	  data, you should read carefully this section in order to learn
	  how to get an important efficiency boost for your code. But if your
	  dataset is small or medium size (say, up to 10 MB), you should
	  not worry about that as the default parameters in
	  <verb>PyTables</verb> are already tuned to handle that
	  perfectly.
	</p>

	<section id="expectedRowsOptim">
	  <heading>Informing <visual markup="tt">PyTables</visual>
	    about expected number of rows in tables</heading>

	  <p>The underlying HDF5 library that is used by
	    <verb>PyTables</verb> allows for certain datasets
	    (<em>chunked</em> datasets) to take the data in bunches of
	    a certain length, so-called <em>chunks</em>, to write them
	    on disk as a whole, i.e. the HDF5 library treats chunks as
	    atomic objects and disk I/O is always made in terms of
	    complete chunks. This allows data filters to be defined by
	    the application to perform tasks such as compression,
	    encryption, checksumming, etc. on entire chunks.
	  </p>

	  <p>An in-memory B-tree is used to map chunk structures on
	    disk. The more chunks that are allocated for a dataset the
	    larger the B-tree. Large B-trees take memory and cause
	    file storage overhead as well as more disk I/O and higher
	    contention for the metadata cache. Consequently, it's
	    important to balance between memory and I/O overhead
	    (small B-trees) and time to access data (big B-trees).
	  </p>

	  <p><verb>PyTables</verb> can determine an optimum chunk size
	    to make B-trees adequate to your dataset size if you help it
	    by providing an estimation of the number of rows for a
	    table. This must be made at table creation time by passing
	    this value to the <verb>expectedrows</verb> keyword of the
	    <verb>createTable</verb> method (see <ref
	      refid="createTableDescr"></ref>).
	  </p>

	  <p>When your table size is bigger than 10 MB (take this figure
	    only as a reference, not strictly), by providing this guess
	    of the number of rows you will be optimizing the access to
	    your data. When the table size is larger than, say 100MB,
	    you are <visual markup="bf">strongly</visual> suggested to
	    provide such a guess; failing to do that may cause your
	    application to do very slow I/O operations and to demand
	    <visual markup="bf">huge</visual> amounts of memory. You
	    have been warned!
	  </p>

	</section> <!-- Optimal rows -->

	<section id="searchOptim">
	  <heading>Accelerating your searches</heading>

	  <p>If you are going to use a lot of searches like the next one:

	    <verbatim>
row = table.row
result = [ row['var2'] for row in table if row['var1'] &lt;= 20 ]
	    </verbatim>

	    (for future reference, we will call this the
	    <em>standard</em> selection mode) and you want to improve the
	    time taken to run it, keep reading.
	  </p>


	  <subsection id="inkernelSearch">
	    <heading>In-kernel searches</heading>

	    <p><verb>PyTables</verb> provides a way to accelerate data
	      selections when they are simple, i.e. when only a column is
	      implied in the selection process, through the use of the
	      <verb>where</verb> iterator (see <ref
		refid="Table.where"></ref>). We will call this mode of
	      selecting data <em>in-kernel</em>. Let's see an example
	      of <em>in-kernel</em> selection based on the
	      <em>standard</em> selection mentioned above:

	      <verbatim>
row = table
result = [ row['var2'] for row in table.where(table.cols.var1 &lt;= 20)]
	      </verbatim>

	      This simple change of mode selection can account for an
	      improvement in search times up to a factor of 10 (see the
	      <ref refid="searchTimes-int">figure</ref>).
	    </p>

	    <figure id="searchTimes-int">
	      <graphics file="searchTimes-int-itanium" scale="0.40" kind="vector">
	      </graphics>
	      <caption>Times for different selection modes over <visual
		  markup="tt">Int32</visual> values. Benchmark made on a
		machine with Itanium (IA64) @ 900 MHz processors with
		SCSI disk @ 10K RPM.
	      </caption>
	    </figure>

	    <figure id="searchTimes-float">
	      <graphics file="searchTimes-float-itanium" scale="0.40" kind="vector">
	      </graphics>
	      <caption>Times for different selection modes over <visual
		  markup="tt">Float64</visual> values. Benchmark made on
		a machine with Itanium (IA64) @ 900 MHz processors
		with SCSI disk @ 10K RPM.
	      </caption>
	    </figure>

	    <p>So, where is the trick? It's easy. In the
	      <em>standard</em> selection mode the data for column
	      <verb>var1</verb> has to be carried up to Python space so
	      as to evaluate the condition and decide if the
	      <verb>var2</verb> value should be added to the
	      <verb>result</verb> list. On the contrary, in the
	      <em>in-kernel</em> mode, the <em>condition</em> is passed
	      to the <verb>PyTables</verb> kernel (hence the name),
	      written in C, and evaluated there at C speed (with some
	      help of the <verb>numarray</verb> package), so that the
	      only values that are brought to the Python space are the
	      references for <verb>rows</verb> that fulfilled the
	      condition.
	    </p>

	    <p>You should note, however, that currently the
	      <verb>where</verb> method only accepts conditions along a
	      single column<footnote>PyTables Pro will address this
		shortcoming.</footnote>. Fortunately, you can mix the
	      <em>in-kernel</em> and <em>standard</em> selection modes
	      for evaluating arbitrarily complex conditions along
	      several columns at once. Look at this example:

	      <verbatim>
row = table
result = [ row['var2'] for row in table.where(table.cols.var3 == "foo")
                                  if row['var1'] &lt;= 20 ]
	      </verbatim>

	      here, we have used a <em>in-kernel</em> selection to filter
	      the rows whose <verb>var3</verb> field is equal to string
	      <verb>"foo"</verb>. Then, we apply a <em>standard</em>
	      selection to complete the query.
	    </p>

	    <p>Of course, when you mix the <em>in-kernel</em> and
	      <em>standard</em> selection modes you should pass the most
	      restrictive condition to the <em>in-kernel</em> part,
	      i.e. to the <verb>where</verb> iterator. In situations
	      where it is not clear which is the most restrictive
	      condition, you might want to experiment a bit in order to
	      find the best combination.
	    </p>

	  </subsection> <!-- In-kernel searches -->

	  <subsection id="indexedSearches">
	    <heading>Indexed searches</heading>

	    <p>When you need more speed than <em>in-kernel</em>
	      selections can offer you, <verb>PyTables</verb> offers a
	      third selection method, the so-called <em>indexed</em>
	      mode. In this mode, you have to decide which column(s) you
	      are going to do your selections on, and index them. Indexing
	      is just a kind of sort operation, so that next searches
	      along a column will look at the sorted information using a
	      <em>binary search</em> which is much faster than a
	      <em>sequential search</em>.
	    </p>

	    <p>You can index your selected columns in several ways:
	    </p>

	    <description>
	      <term>Declaratively</term> <item>In this mode, you can
		declare a column as being indexed by passing the
		<em>indexed</em> parameter to the column
		descriptor. That is:

		<verbatim>
class Example(IsDescription):
    var1 = StringCol(length=4, dflt="", pos=1, indexed=1)
    var2 = BoolCol(0, indexed=1, pos = 2)
    var3 = IntCol(0, indexed=1, pos = 3)
    var4 = FloatCol(0, indexed=0, pos = 4)
		</verbatim>

		In this case, we are telling that <verb>var1</verb>,
		<verb>var2</verb> and <verb>var3</verb> columns will be
		indexed automatically when you add rows to the table
		with this description.

	      </item>

	      <term>Calling Column.createIndex()</term> <item>In this
		mode, you can create an index even on an already created
		table. For example:

		<verbatim>
indexrows = table.cols.var1.createIndex()
indexrows = table.cols.var2.createIndex()
indexrows = table.cols.var3.createIndex()
		</verbatim>

		will create indexes for all <verb>var1</verb>,
		<verb>var2</verb> and <verb>var3</verb> columns, and
		after doing that, they will behave as regular indexes.
	      </item>

	    </description>

	    <p>After you have indexed a column, you can proceed to use
	      it through the use of <verb>Table.where</verb> method:

	      <verbatim>
row = table
result = [ row['var2'] for row in table.where(table.cols.var1 == "foo") ]
	      </verbatim>

	      or, if you want to add more conditions, you can mix the
	      indexed selection with a standard one:

	      <verbatim>
row = table
result = [ row['var2'] for row in table.where(table.cols.var3 &lt;= 20)
                                  if row['var1'] == "foo" ]
	      </verbatim>

	      remember to pass the most restrictive condition to the
	      <verb>where</verb> iterator.
	    </p>

	    <p>You can see in figures <ref
		refid="searchTimes-int"></ref> and <ref
		refid="searchTimes-float"></ref> that indexing can
	      accelerate quite a lot your data selections in
	      tables. For moderately large tables (> one million
	      rows), you can get speedups in the order of 100x with
	      regard to <em>in-kernel</em> selections, and in the order
	      of 1000x with regard to <em>standard</em> selections.
	    </p>

	    <p>One important aspect of indexation in
	      <verb>PyTables</verb> is that it has been implemented with
	      the goal of being capable to manage effectively very large
	      tables. In <ref refid="indexTimes">figure</ref>, you can
	      see that the times to index columns in tables always grow
	      <em>linearly</em>. In particular, the time to index a
	      couple of columns with 1 billion of rows each is 40
	      min. (roughly 20 min. each), which is a quite reasonable
	      figure. This is because <verb>PyTables</verb> has chosen
	      an algorithm that does a <em>partial</em> sort of the
	      columns in order to ensure that the indexing time grows
	      <em>linearly</em>. On the contrary, most of relational
	      databases try to do a <em>complete</em> sort of
	      columns, and this makes the time to index grow much
	      faster with the number of rows.
	    </p>
	    <p>The fact that relational databases use a complete
	      sorting algorithm for indexes means that their index would
	      be more effective (but not by a large extent) for
	      searching purposes than the <verb>PyTables</verb>
	      approach. However, for relatively large tables (> 10
	      millions of rows) the time required for completing such a
	      sort can be so large, that indexing is not normally worth
	      the effort. In other words, <verb>PyTables</verb> indexing
	      scales much better than relational databases. So don't
	      worry if you have extremely large columns to index:
	      <verb>PyTables</verb> is designed to cope with that
	      perfectly.
	    </p>

	    <figure id="indexTimes">
	      <graphics file="indexTimes-itanium" scale="0.40" kind="vector">
	      </graphics>
	      <caption>Times for indexing a couple of columns of
		data type <visual markup="tt">Int32</visual> and
		<visual markup="tt">Float64</visual>. Benchmark made
		on a machine with Itanium (IA64) @ 900 MHz processors
		with SCSI disk @ 10K RPM.
	      </caption>
	    </figure>

	  </subsection>
	</section> <!-- search optimization -->

	<section id="compressionIssues">
	  <heading>Compression issues</heading>

	  <p>One of the beauties of <verb>PyTables</verb> is that it
	    supports compression on tables and arrays<footnote>More
	      precisely, it is supported in <verb>CArray</verb>,
	      <verb>EArray</verb> and <verb>VLArray</verb> objects, but
	      not in <verb>Array</verb> objects.</footnote>, although it
	    is not used by default. Compression of big amounts of data
	    might be a bit controversial feature, because compression
	    has a legend of being a very big consumer of CPU time resources.
	    However, if you are willing to check if
	    compression can help not only by reducing your dataset file
	    size but <visual markup="bf">also</visual> by improving
	    I/O efficiency, specially when dealing with very large
	    datasets, keep reading.
	  </p>

	  <p>There is a common scenario where users need to save
	    duplicated data in some record fields, while the others
	    have varying values. In a relational database approach
	    such redundant data can normally be moved to other
	    tables and a relationship between the rows on the separate
	    tables can be created. But that takes analysis and
	    implementation time, and makes the underlying libraries
	    more complex and slower.
	  </p>

	  <p><verb>PyTables</verb> transparent compression allows the
	    users to not worry about finding which is their optimum strategy
	    for data tables, but rather use less, not directly related,
	    tables with a larger number of columns while still not
	    cluttering the database too much with duplicated data
	    (compression is responsible to avoid that). As a side
	    effect, data selections can be made more easily because you
	    have more fields available in a single table, and they can
	    be referred in the same loop. This process may normally end
	    in a simpler, yet powerful manner to process your data
	    (although you should still be careful about in which kind of
	    scenarios the use of compression is convenient or not).
	  </p>

	  <p>The compression library used by default is the <visual
	      markup="bf">Zlib</visual> (see <cite
	      refid="zlibRef"></cite>). Since HDF5 <em>requires</em>
	    it, you can safely use it and expect that your HDF5 files
	    will be readable on any other platform that has HDF5 libraries
	    installed. Zlib provides good compression ratio, although
	    somewhat slow, and reasonably fast decompression. Because
	    of that, it is a good candidate to be used for compressing
	    you data.
	  </p>

	  <p>However, in some situations it is critical to have
	      <em>very good</em> decompression speed (at the expense
	      of lower compression ratios or more CPU wasted on
	      compression, as we will see soon). In others, the
	      emphasis is put in achieving the maximum compression
	      ratios, no matter which reading speed will result. This
	      is why support for two additional compressors has been
	      added to PyTables: LZO (see <cite
	      refid="lzoRef"></cite>) and bzip2 (see <cite
	      refid="bzip2Ref"></cite>). Following the author of LZO
	      (and checked by the author of this section, as you will
	      see soon), LZO offers pretty fast compression (though a
	      small compression ratio) and extremely fast
	      decompression.  In fact, LZO is so fast when
	      compressing/decompressing that it may well happen (that
	      depends on your data, of course) that writing or reading
	      a compressed dataset is sometimes faster than if it is
	      not compressed at all (specially when dealing with
	      extremely large datasets). This fact is very important,
	      specially if you have to deal with very large amounts of
	      data. Regarding bzip2, it has a reputation of achieving
	      excellent compression ratios, but at the price of
	      spending much more CPU time, which results in very low
	      compression/decompression speeds.
	  </p>

	  <p>Be aware that the LZO and bzip2 support in PyTables is
	    not standard on HDF5, so if you are going to use your
	    PyTables files in other contexts different from PyTables
	    you will not be able to read them. Still, see the <ref
	    refid="ptrepackDescr">appendix</ref> (where the
	    <verb>ptrepack</verb> utility is described) to find a way
	    to free your files from LZO or bzip2 dependencies, so that
	    you can use these compressors locally with the warranty
	    that you can replace them with Zlib (or even remove
	    compression completely) if you want to use these files
	    with other HDF5 tools or platforms afterwards.
	  </p>

	  <p>In order to allow you to grasp what amount of compression
	    can be achieved, and how this affects
	    performance, a series of experiments has been carried out.
	    All the results presented in this section (and in the next
	    one) have been obtained with synthetic data and using
	    PyTables 1.3. Also, the tests have been conducted on a IBM
	    OpenPower 720 (e-series) with a PowerPC G5 at 1.65 GHz and
	    a hard disk spinning at 15K RPM.  As your data and
	    platform may be totally different for your case, take this
	    just as a guide because your mileage will probably
	    vary. Finally, and to be able to play with tables with a
	    number of rows as large as possible, the record size has
	    been chosen to be small (16 bytes). Here is its
	    definition:
	  </p>

		<verbatim>
class Bench(IsDescription):
    var1 = StringCol(length=4)
    var2 = IntCol()
    var3 = FloatCol()
		</verbatim>

	  <p>With this setup, you can look at the compression ratios
	    that can be achieved in <ref
	    refid="comprTblComparison">plot</ref>. As you can see, LZO
	    is the compressor that performs worse in this sense, but,
	    curiosly enough, there is not much difference between Zlib
	    and bzip2.
	  </p>

	  <figure id="comprTblComparison">
	    <graphics file="compressed-recordsize" scale="0.50" kind="vector">
	    </graphics>
	    <caption>Comparison between different compression libraries.
	    </caption>
	  </figure>

	  <p>Also, PyTables lets you select different compression
	    levels for Zlib and bzip2, although you may get a bit
	    disappointed by the small improvement that show these
	    compressors when dealing with a combination of numbers and
	    strings as in our example. As a reference, see plot <ref
	    refid="comprZlibComparison"></ref> for a comparison of the
	    compression achieved by selecting different levels of
	    Zlib. Very oddly, the best compression ratio corresponds to
	    level 1 (!). It's difficult to explain that, but this
	    lesson will serve to reaffirm that there is no replacement
	    for experiments with your own data. In general, it is
	    recommended to select the <em>lowest</em> level of
	    compression in order to achieve best performance and
	    decent (if not the best!) compression ratio. See later for
	    more figures on this regard.
	  </p>

	  <figure id="comprZlibComparison">
	    <graphics file="compressed-recordsize-zlib" scale="0.50"
	      kind="vector">
	    </graphics>
	    <caption>Comparison between different compression levels of Zlib.
	    </caption>
	  </figure>

	  <p>Have also a look at <ref
	      refid="comprWriteComparison">graph</ref>. It shows how
	      the speed of writing rows evolves as the size (the row
	      number) of the table grows. Even though in these graphs
	      the size of one single row is 16 bytes, you can most
	      probably extrapolate these figures to other row sizes.
	  </p>

	  <figure id="comprWriteComparison">
	    <graphics file="compressed-writing" scale="0.50" kind="vector">
	    </graphics>
	    <caption>Writing tables with several compressors.
	    </caption>
	  </figure>

	  <p>In <ref refid="comprReadNoCacheComparison">plot</ref> you
	    can see how compression affects the reading performance.
	    In fact, what you see in the plot is an <em>in-kernel
	    selection</em> speed, but provided that this operation is
	    very fast (see <ref refid="inkernelSearch">section</ref>),
	    we can accept it as an actual read test. Compared with the
	    reference line without compression, the general trend here
	    is that LZO does not affect too much the reading
	    performance (and in some points it is actually better),
	    Zlib makes speed to drop to a half, while bzip2 is
	    performing very slow (up to 8x slower).
	  </p>

	  <p>Also, in the same <ref
	    refid="comprReadNoCacheComparison">figure</ref> you can
	    notice some strange peaks in the speed that we might be
	    tempted to attribute to libraries on which PyTables relies
	    (HDF5, compressors...), or to PyTables itself. However,
	    <ref refid="comprReadCacheComparison">graph</ref> reveals
	    that, if we put the file in the filesystem cache (by
	    reading it several times before, for example), the
	    evolution of the performance is much smoother. So, the
	    most probable explanation would be that such a peaks are a
	    consequence of the underlying OS filesystem, rather than a
	    flaw in PyTables (or any other library behind it).
	    Another consequence that can be derived from the above
	    plot is that LZO decompression performance is much better
	    than Zlib, allowing an improvement in overal speed of more
	    than 2x, and perhaps more important, the read performance
	    for really large datasets (i.e. when they do not fit in
	    the OS filesystem cache) can be actually <em>better</em>
	    than not using compression at all. Finally, one can see
	    that reading performance is very badly affected when bzip2
	    is used (it is 10x slower than LZO and 4x than Zlib), but
	    this is not too strange anyway.
	  </p>

	  <figure id="comprReadNoCacheComparison">
	    <graphics file="compressed-select-nocache"
	      scale="0.50" kind="vector">

	    </graphics>
	    <caption>Selecting values in tables with several
	    compressors. The file is not in the OS cache.
	    </caption>
	  </figure>

	  <figure id="comprReadCacheComparison">
	    <graphics file="compressed-select-cache"
	      scale="0.50" kind="vector">

	    </graphics>
	    <caption>Selecting values in tables with several
	    compressors. The file is in the OS cache.
	    </caption>
	  </figure>

	  <p>So, generally speaking and looking at the experiments above,
	     you can expect that LZO will be the fastest in both
	     compressing and decompressing, but the one that achieves
	     the worse compression ratio (although that may be just OK
	     for many situations, specially when used with the <ref
	     refid="ShufflingOptim">shuffle filter</ref>). bzip2 is
	     the slowest, by large, in both compressing and
	     decompressing, and besides, it does not achieve any
	     better compression ratio than Zlib. Zlib represents a
	     balance between them: it's somewhat slow compressing (2x)
	     and decompressing (3x) than LZO, but it normally achieves
	     fairly good compression ratios.
	  </p>

	  <p>Finally, by looking at the plots <ref
	    refid="comprWriteZlibComparison"></ref>, <ref
	    refid="comprReadZlibComparison"></ref>, and the
	    aforementioned <ref refid="comprZlibComparison"></ref> you
	    can see why the recommended compression level to use for
	    all compression libraries is 1. This is the lowest level
	    of compression, but if you take the approach suggested
	    above, the redundant data is to be found normally in the
	    same row, making redundancy locality very high so that a
	    small level of compression should be enough to achieve a
	    good compression ratio on your data tables, saving CPU
	    cycles for doing other things. Nonetheless, in some
	    situations you may want to check for your own how the
	    different compression levels affect your application.
	  </p>

	  <p>You can select the compression library and level by
	    setting the <verb>complib</verb> and <verb>complevel</verb>
	    keywords in the <verb>Filters</verb> class (see <ref
	      refid="FiltersClassDescr"></ref>). A compression level of 0
	    will completely disable compression (the default), 1 is the
	    less CPU time demanding level, while 9 is the maximum level
	    and most CPU intensive. Finally, have in mind that LZO is
	    not accepting a compression level right now, so, when using
	    LZO, 0 means that compression is not active, and any other
	    value means that LZO is active.
	  </p>

	  <p>So, in conclusion, if your ultimate goal is writing and
	    reading as fast as possible, choose LZO. If you want to
	    reduce as much as possible your data, while retaining
	    acceptable read speed, choose Zlib. Finally, if
	    portability is important for you, Zlib is your best
	    bet. So, when you want to use bzip2?  Well, looking at the
	    results, it is difficult to recommend its use in general,
	    but you may want to experiment with it in those cases
	    where you know that it is well suited for your data
	    pattern (for example, for dealing with repetitive string
	    datasets).
	  </p>

	  <figure id="comprWriteZlibComparison">
	    <graphics file="compressed-writing-zlib"
	      scale="0.50" kind="vector">

	    </graphics>
	    <caption>Writing in tables with different levels of
	      compression.
	    </caption>
	  </figure>

	  <figure id="comprReadZlibComparison">
	    <graphics file="compressed-select-cache-zlib"
	      scale="0.50" kind="vector">

	    </graphics>
	    <caption>Selecting values in tables with different levels
	      of compression. The file is in the OS cache.
	    </caption>
	  </figure>

	</section>

	<section id="ShufflingOptim">
	  <heading>Shuffling (or how to make the compression process
	    more effective)</heading>

	  <p>The <verb>HDF5</verb> library provides an interesting
	    filter that can leverage the results of your favorite
	    compressor. Its name is <em>shuffle</em>, and because it
	    can greatly benefit compression and it does not take many
	    CPU resources (see below for a justification), it is active
	    by <em>default</em> in <verb>PyTables</verb> whenever
	    compression is activated (independently of the chosen
	    compressor). It is of course deactivated when compression
	    is off (which is the default, as you already should
	    know). Of course, you can deactivate it if you want, but
	    this is not recommended.
	  </p>

	  <p>So, how exactly works this mysterious filter?  From the
	    HDF5 reference manual: <quote>The shuffle filter
	    de-interlaces a block of data by reordering the bytes. All
	    the bytes from one consistent byte position of each data
	    element are placed together in one block; all bytes from a
	    second consistent byte position of each data element are
	    placed together a second block; etc. For example, given
	    three data elements of a 4-byte datatype stored as
	    012301230123, shuffling will re-order data as
	    000111222333. This can be a valuable step in an effective
	    compression algorithm because the bytes in each byte
	    position are often closely related to each other and
	    putting them together can increase the compression ratio.
	    </quote>
	  </p>

	  <p>In <ref refid="comprShuffleComparison">graph</ref> you
	    can see a benchmark that shows how the <em>shuffle</em>
	    filter can help the different libraries in compressing
	    data. In this experiment, shuffle has made LZO to compress
	    almost 3x more (!), while Zlib and bzip2 are seeing
	    improvements of 2x. Once again, the data for this
	    experiment is synthetic, and <em>shuffle</em> seems to do
	    a great work with it, but in general, the results will
	    vary in each case<footnote>Some users reported that the
	    typical improvement with real data is between a factor
	    1.5x and 2.5x over the already compressed
	    datasets.</footnote>.
	  </p>

	  <figure id="comprShuffleComparison">
	    <graphics file="compressed-recordsize-shuffle"
	      scale="0.50" kind="vector">

	    </graphics>
	    <caption>Comparison between different compression
	      libraries with and without the <em>shuffle</em> filter.
	    </caption>
	  </figure>

	  <p>At any rate, the most remarkable fact about the
	    <em>shuffle</em> filter is the relatively high level of
	    compression that compressor filters can achieve when used
	    in combination with it. A curious thing to note is that
	    the Bzip2 compression rate does not seem very much
	    improved (less than a 40%), and what is more striking,
	    Bzip2+shuffle does compress quite <em>less</em> than
	    Zlib+shuffle or LZO+shuffle combinations, which is
	    kind of unexpected. The thing that seems clear is that
	    Bzip2 is not very good at compressing patterns that result
	    of shuffle application. As always, you may want to
	    experiment with your own data before widely applying the
	    Bzip2+shuffle combination in order to avoid surprises.
	  </p>

	  <p>Now, how does shuffling affect performance? Well, if you look
	    at plots <ref refid="comprWriteShuffleComparison"></ref>,
	    <ref refid="comprReadNoCacheShuffleComparison"></ref> and
	    <ref refid="comprReadCacheShuffleComparison"></ref>, you
	    will get a somewhat unexpected (but pleasant)
	    surprise. Roughly, <em>shuffle</em> makes the writing
	    process (shuffling+compressing) faster (aproximately a 15%
	    for LZO, 30% for Bzip2 and a 80% for Zlib), which is an
	    interesting result by itself. But perhaps more exciting is
	    the fact that the reading process
	    (unshuffling+decompressing) is also accelerated by a
	    similar extent (a 20% for LZO, 60% for Zlib and a 75% for
	    Bzip2, roughly).
	  </p>

	  <figure id="comprWriteShuffleComparison">
	    <graphics file="compressed-writing-shuffle"
	      scale="0.50" kind="vector">

	    </graphics>
	    <caption>Writing with different compression
	      libraries with and without the <em>shuffle</em> filter.
	    </caption>
	  </figure>

	  <figure id="comprReadNoCacheShuffleComparison">
	    <graphics file="compressed-select-nocache-shuffle-only"
	      scale="0.50" kind="vector">

	    </graphics>
	    <caption>Reading with different compression libraries with
	      the <em>shuffle</em> filter. The file is not in OS cache.
	    </caption>
	  </figure>

	  <figure id="comprReadCacheShuffleComparison">
	    <graphics file="compressed-select-cache-shuffle"
	      scale="0.50" kind="vector">

	    </graphics>
	    <caption>Reading with different compression libraries with
	      and without the <em>shuffle</em> filter.  The file is in
	      OS cache.
	    </caption>
	  </figure>

	  <p>You may wonder why introducing another filter in the
	    write/read pipelines does effectively accelerate the
	    throughput. Well, maybe data elements are more similar or
	    related column-wise than row-wise, i.e. contiguous
	    elements in the same column are more alike, so shuffling
	    makes the job of the compressor easier (faster) and more
	    effective (greater ratios). As a side effect, compressed
	    chunks do fit better in the CPU cache (at least,
	    the chunks are smaller!) so that the process of
	    unshuffle/decompress can make a better use of the cache
	    (i.e. reducing the number of CPU cache faults).
	  </p>

	  <p>So, given the potential gains (faster writing and
	    reading, but specially much improved compression level),
	    it is a good thing to have such a filter enabled by
	    default in the battle for discovering redundancy when you
	    want to compress your data, just as PyTables does.
	  </p>

	</section> <!-- Shuffle -->

	<section>
	  <heading>Using Psyco</heading>

	  <p>Psyco (see <cite refid="psycoRef"></cite>) is a kind of
	    specialized compiler for Python that typically accelerates
	    Python applications with no change in source code. You can
	    think of Psyco as a kind of just-in-time (JIT) compiler, a
	    little bit like Java's, that emits machine code on the fly
	    instead of interpreting your Python program step by
	    step. The result is that your unmodified Python programs run
	    faster.
	  </p>

	  <p>Psyco is very easy to install and use, so in most scenarios
	    it is worth to give it a try. However, it only runs on Intel
	    386 architectures, so if you are using other architectures,
	    you are out of luck (at least until Psyco will support
	    yours).
	  </p>

	  <p>As an example, imagine that you have a small script that
	    reads and selects data over a series of datasets, like this:
	  </p>

	  <verbatim>
	    def readFile(filename):
	    "Select data from all the tables in filename"

	    fileh = openFile(filename, mode = "r")
	    result = []
	    for table in fileh("/", 'Table'):
	    result = [ p['var3'] for p in table if p['var2'] &lt;= 20 ]

	    fileh.close()
	    return result

	    if __name__=="__main__":
	    print readFile("myfile.h5")
	  </verbatim>

	  <p>In order to accelerate this piece of code, you can rewrite
	    your main program to look like:
	  </p>

	  <verbatim>
	    if __name__=="__main__":
	    import psyco
	    psyco.bind(readFile)
	    print readFile("myfile.h5")
	  </verbatim>

	  <p>That's all!. From now on, each time that you execute your
	    Python script, Psyco will deploy its sophisticated
	    algorithms so as to accelerate your calculations.
	  </p>

	  <p>You can see in the graphs <ref
	      refid="psycoWriteComparison"></ref> and <ref
	      refid="psycoReadComparison"></ref> how much I/O speed
	    improvement you can get by using Psyco. By looking at this
	    figures you can get an idea if these improvements are of
	    your interest or not. In general, if you are not going to
	    use compression you will take advantage of Psyco if your
	    tables are medium sized (from a thousand to a million rows),
	    and this advantage will disappear progressively when the
	    number of rows grows well over one million. However if you
	    use compression, you will probably see improvements even
	    beyond this limit (see <ref
	      refid="compressionIssues">section</ref>). As always, there
	    is no substitute for experimentation with your own dataset.
	  </p>

	  <figure id="psycoWriteComparison">
	    <graphics file="write-medium-psyco-nopsyco-comparison"
	      scale="0.40" kind="vector">
	    </graphics>
	    <caption>Writing tables with/without Psyco.
	    </caption>
	  </figure>

	  <figure id="psycoReadComparison">
	    <graphics file="read-medium-psyco-nopsyco-comparison"
	      scale="0.40" kind="vector">
	    </graphics>
	    <caption>Reading tables with/without Psyco.
	    </caption>
	  </figure>

	</section> <!-- Psyco -->

	<section>
	  <heading>Getting the most from the node LRU cache</heading>

	  <p>Starting from PyTables 1.2 on, it has been introduced a new
	    LRU cache that prevents from loading all the nodes of the
	    <em>object tree</em> in memory. This cache is responsible of
	    loading just up to a certain amount of nodes and discard the
	    least recent used ones when there is a need to load new
	    ones. This represents a big advantage over the old schema,
	    specially in terms of memory usage (as there is no need to
	    load <em>every</em> node in memory), but it also adds very
	    convenient optimizations for working interactively like, for
	    example, speeding-up the opening times of files with lots of
	    nodes, allowing to open almost any kind of file in typically
	    less than one tenth of second (compare this with the more
	    than 10 seconds for files with more than 10000 nodes in
	    PyTables pre-1.2 era). See <cite
	      refid="NewObjectTreeCacheRef"></cite> for more info on the
	    advantages (and also drawbacks) of this approach.
	  </p>

	  <p>One thing that deserves some discussion is the election of
	    the parameter that sets the maximum amount of nodes to be
	    held in memory at any time. As PyTables is meant to be
	    deployed in machines that have potentially low memory, the
	    default for it is quite conservative (you can look at its
	    actual value in the <verb>NODE_CACHE_SIZE</verb> parameter
	    in module <verb>tables/constants.py</verb>). However, if you
	    usually have to deal with files that have much more nodes
	    than the maximum default, and you have a lot of free memory
	    in your system, then you may want to experiment which is the
	    appropriate value of <verb>NODE_CACHE_SIZE</verb> that fits
	    better your needs.
	  </p>

	  <p>As an example, look at the next code:
	  </p>

	  <verbatim>
	    def browse_tables(filename):
	    fileh = openFile(filename,'a')
	    group = fileh.root.newgroup
	    for j in range(10):
	    for tt in fileh.walkNodes(group, "Table"):
            title = tt.attrs.TITLE
            for row in tt:
	    pass
	    fileh.close()
	  </verbatim>

	  <p>We will be running the code above against a couple of files
	    having a <verb>/newgroup</verb> containing 100 tables and
	    1000 tables respectively. We will run this small benchmark
	    for different values of the LRU cache size, namely 256 and
	    1024. You can see the results in <ref
	      refid="LRUTblComparison">table</ref>.
	  </p>

	  <table id="LRUTblComparison">
	    <tabular preamble="llrrrrrrrr">
	      <tabhead>
		<row>
		  <cell colspan="2" align="left"></cell>
		  <cell colspan='4' align='center'>100 nodes</cell>
		  <cell colspan='4' align='center'>1000 nodes</cell>
		</row>
		<hline from="3"/>
		<row>
		  <cell colspan='2' align="left"></cell>
		  <cell colspan='2' align='center'>Memory (MB)</cell>
		  <cell colspan='2' align='center'>Time (ms)</cell>
		  <cell colspan='2' align='center'>Memory (MB)</cell>
		  <cell colspan='2' align='center'>Time (ms)</cell>
		</row>
		<hline from="3"/>
		<srow> Node is coming from... |Cache size  | 256 |1024 | 256 |1024 |256 |1024 |256 |1024 </srow>
	      </tabhead>
	      <tabbody>
		<srow>From disk   || 14 | 14 | 1.24 | 1.24 | 51 | 66 | 1.33| 1.31 </srow>
		<srow>From cache  || 14 | 14 | 0.53 | 0.52 | 65 | 73 | 1.35| 0.68 </srow>
	      </tabbody>
	    </tabular>
	    <caption>Retrieving speed and memory consumption
	      dependency of the number of nodes in LRU cache.
	    </caption>
	  </table>

	  <p>From the data in <ref refid="LRUTblComparison">table</ref>,
	    one can see that, when the number of objects that you are
	    dealing with does fit in cache, you will get better access
	    times to them. Also, incrementing the node cache size does
	    effectively consumes more memory <em>only</em> if the total
	    nodes exceeds the slots in cache; otherwise the memory
	    consumption remains the same. It is also worth noting that
	    incrementing the node cache size in the case you want to fit
	    all your nodes in cache, it does not take much more memory
	    than keeping too conservative. On another hand, it might
	    happen that the speed-up that you can achieve by allocating
	    more slots in your cache maybe is not worth the amount of
	    memory used.
	  </p>
	  <p>Anyway, if you feel that this issue is important for you,
	    setup your own experiments and proceed fine-tuning the
	    <verb>NODE_CACHE_SIZE</verb> parameter.
	  </p>

	</section> <!-- LRU cache -->

	<section>
	  <heading>Selecting an User Entry Point (UEP) in your
	    tree</heading>

	  <p><visual markup="bf">Note:</visual> After the introduction
	    of the new object tree cache in PyTables 1.2, this feature
	    is not very useful anymore and might become
	    <em>deprecated</em> in future versions.
	  </p>

	  <p>If you have a <visual markup="bf">huge</visual> tree in
	    your data file with many nodes on it, creating the object
	    tree would take long time. Many times, however, you are
	    interested only in access to a part of the complete tree, so
	    you won't strictly need PyTables to build the entire object
	    tree in-memory, but only the <em>interesting</em> part.
	  </p>

	  <p>This is where the <verb>rootUEP</verb> parameter of
	    <verb>openFile</verb> function (see <ref
	      refid="openFileDescr"></ref>) can be helpful. Imagine that
	    you have a file called <verb>"test.h5"</verb> with the
	    associated tree that you can see in figure <ref
	      refid="rootUEPfig1"></ref>, and you are interested only in
	    the section marked in red.  You can avoid the build of all
	    the object tree by saying to <verb>openFile</verb> that your
	    root will be the <verb>/Group2/Group3</verb> group. That is:
	  </p>
	  <verbatim>
	    fileh = openFile("test.h5", rootUEP="/Group2/Group3")
	  </verbatim>

	  <p>As a result, the actual object tree built will be like the
	    one that can be seen in <ref
	      refid="rootUEPfig2">figure</ref>.
	  </p>

	  <p>Of course this has been a simple example and the use
	    of the <verb>rootUEP</verb> parameter was not very
	    necessary. But when you have <em>thousands</em> of nodes on
	    a tree, you will certainly appreciate the
	    <verb>rootUEP</verb> parameter.
	  </p>

	  <figure id="rootUEPfig1">
	    <graphics file="rootUEP1" scale="0.6" kind="vector">
	    </graphics>
	    <caption>Complete tree in file <visual
		markup="tt">test.h5</visual>, and subtree of interest for
	      the user.
	    </caption>
	  </figure>

	  <figure id="rootUEPfig2">
	    <graphics file="rootUEP2" scale="0.75" kind="vector">
	    </graphics>
	    <caption>Resulting object tree derived from the use of the
	      <visual markup="tt">rootUEP</visual> parameter.
	    </caption>
	  </figure>

	</section>

	<section>
	  <heading>Compacting your <visual markup="tt">PyTables</visual>
	    files
	  </heading>

	  <p>Let's suppose that you have a file on which you have made a
	    lot of row deletions on one or more tables, or deleted many
	    leaves or even entire subtrees. These operations might leave
	    <em>holes</em> (i.e. space that is not used anymore) in your
	    files, that may potentially affect not only the size of the
	    files but, more importantly, the performance of I/O. This is
	    because when you delete a lot of rows on a table, the space
	    is not automatically recovered on-the-flight. In addition,
	    if you add many more rows to a table than specified in the
	    <verb>expectedrows</verb> keyword in creation time this may
	    affect performance as well, as explained in <ref
	      refid="expectedRowsOptim">section</ref>.
	  </p>

	  <p>In order to cope with these issues, you should be aware
	    that a handy <verb>PyTables</verb> utility called
	    <verb>ptrepack</verb> can be very useful, not only to
	    compact your already existing <em>leaky</em> files, but also
	    to adjust some internal parameters (both in memory and in
	    file) in order to create adequate buffer sizes and chunk
	    sizes for optimum I/O speed. Please, check the <ref
	      refid="ptrepackDescr">appendix</ref> for a brief tutorial on
	    its use.
	  </p>

	  <p>Another thing that you might want to use
	    <verb>ptrepack</verb> for is changing the compression
	    filters or compression levels on your existing data for
	    different goals, like checking how this can affect both
	    final size and I/O performance, or getting rid of the
	    optional compressors like <verb>LZO</verb>,
	    <verb>UCL</verb> or <verb>bzip2</verb> in your existing
	    files in case you want to use them with generic HDF5 tools
	    that do not have support for these filters.
	  </p>

	</section>

      </chapter>

    </part>

    <part>
      <heading>Complementary modules</heading>

      <chapter id="FileNode">
	<heading>FileNode - simulating a filesystem with <visual
	    markup="tt">PyTables</visual></heading>

	<section>
	  <heading>What is <visual
	      markup="tt">FileNode</visual>?</heading>

	  <p><verb>FileNode</verb> is a module which enables you to
	    create a <verb>PyTables</verb> database of nodes which can
	    be used like regular opened files in Python.  In other
	    words, you can store a file in a <verb>PyTables</verb>
	    database, and read and write it as you would do with any
	    other file in Python.  Used in conjunction with
	    <verb>PyTables</verb> hierarchical database organization,
	    you can have your database turned into an open, extensible,
	    efficient, high capacity, portable and metadata-rich
	    filesystem for data exchange with other systems (including
	    backup purposes).
	  </p>

	  <p>Between the main features of <verb>FileNode</verb>, one can
	    list:</p>

	  <itemize>
	    <item><em>Open:</em> Since it relies on
	      <verb>PyTables</verb>, which in turn, sits over HDF5 (see
	      <cite refid="HDFWhatIs"></cite>), a standard hierarchical
	      data format from NCSA.</item>

	    <item><em>Extensible:</em> You can define new types of
	      nodes, and their instances will be safely preserved (as
	      are normal groups, leafs and attributes) by
	      <verb>PyTables</verb> applications having no knowledge of
	      their types.  Moreover, the set of possible attributes for
	      a node is not fixed, so you can define your own node
	      attributes.
	    </item>

	    <item><em>Efficient:</em> Thanks to PyTables' proven
	      extreme efficiency on handling huge amounts of data.
	      FileNode can make use of PyTables' on-the-fly compression
	      and decompression of data.</item>

	    <item><em>High capacity:</em> Since PyTables and HDF5 are
	      designed for massive data storage (they use 64-bit
	      addressing even where the platform does not support it
	      natively).</item>

	    <item><em>Portable:</em> Since the HDF5 format has an
	      architecture-neutral design, and the HDF5 libraries and
	      PyTables are known to run under a variety of platforms.
	      Besides that, a PyTables database fits into a single file,
	      which poses no trouble for transportation.</item>

	    <item><em>Metadata-rich:</em> Since PyTables can store
	      arbitrary key-value pairs (even Python objects!) for every
	      database node.  Metadata may include authorship, keywords,
	      MIME types and encodings, ownership information, access
	      control lists (ACL), decoding functions and anything you
	      can imagine!</item>

	  </itemize>
	</section>

	<section>
	  <heading>Finding a <visual markup="tt">FileNode</visual>
	    node</heading>

	  <p><visual markup="tt">FileNode</visual> nodes can be
	    recognized because they have a <verb>NODE_TYPE</verb> system attribute
	    with a <verb>'file'</verb> value.  It is recommended that
	    you use the <verb>getNodeAttr()</verb> method (see <ref
	      refid="File.getNodeAttr"></ref>) of <verb>tables.File</verb>
	    class to get the <verb>NODE_TYPE</verb> attribute independently
	    of the nature (group or leaf) of the node, so you do not
	    need to care about.
	  </p>
	</section>

	<section>
	  <heading><visual markup="tt">FileNode</visual> - simulating
	    files inside <visual markup="tt">PyTables</visual></heading>

	  <p>The <visual markup="tt">FileNode</visual> module is part of
	    the <verb>nodes</verb> sub-package of <verb>PyTables</verb>.
	    The recommended way to import the module is:
	  </p>
	  <verbatim>
	    >>> from tables.nodes import FileNode
	  </verbatim>

	  <p>However, <visual markup="tt">FileNode</visual> exports very
	    few symbols, so you can import <verb>*</verb> for
	    interactive usage.  In fact, you will most probably only use
	    the <verb>NodeType</verb> constant and the
	    <verb>newNode()</verb> and <verb>openNode()</verb> calls.
	  </p>
	  <p>The <verb>NodeType</verb> constant contains the value that
	    the <verb>NODE_TYPE</verb> system attribute of a node file is expected
	    to contain (<verb>'file'</verb>, as we have seen).  Although
	    this is not expected to change, you should use <visual
	      markup="tt">FileNode.NodeType</visual> instead of the
	    literal <verb>'file'</verb> when possible.
	  </p>
	  <p><verb>newNode()</verb> and <verb>openNode()</verb> are the
	    equivalent to the Python <verb>file()</verb> call (alias
	    <verb>open()</verb>) for ordinary files.  Their arguments
	    differ from that of <verb>file()</verb>, but this is the
	    only point where you will note the difference between
	    working with a node file and working with an ordinary file.
	  </p>

	  <p>For this little tutorial, we will assume that we have a
	    <verb>PyTables</verb> database opened for writing. Also, if
	    you are somewhat lazy at typing sentences, the code that we
	    are going to explain is included in the
	    <verb>examples/filenodes1.py</verb> file.
	  </p>
	  <p>You can create a brand new file with these sentences:
	  </p>
	  <verbatim>
	    >>> import tables
	    >>> h5file = tables.openFile('fnode.h5', 'w')
	  </verbatim>

	  <subsection>
	    <heading>Creating a new file node</heading>

	    <p>Creation of a new file node is achieved with the
	      <verb>newNode()</verb> call.  You must tell it in which
	      <verb>PyTables</verb> file you want to create it, where in
	      the <verb>PyTables</verb> hierarchy you want to create the
	      node and which will be its name.  The
	      <verb>PyTables</verb> file is the first argument to
	      <verb>newNode()</verb>; it will be also called the
	      <verb>'host PyTables file'</verb>.  The other two
	      arguments must be given as keyword arguments
	      <verb>where</verb> and <verb>name</verb>, respectively.
	      As a result of the call, a brand new appendable and
	      readable file node object is returned.
	    </p>
	    <p>So let us create a new node file in the previously opened
	      <verb>h5file</verb> <verb>PyTables</verb> file, named
	      <verb>'fnode_test'</verb> and placed right under the root
	      of the database hierarchy.  This is that command:
	    </p>
	    <verbatim>
	      >>> fnode = FileNode.newNode(h5file, where='/', name='fnode_test')
	    </verbatim>

	    <p>That is basically all you need to create a file node.
	      Simple, isn't it?  From that point on, you can use
	      <verb>fnode</verb> as any opened Python file (i.e. you can
	      write data, read data, lines of text and so on).
	    </p>

	    <p><verb>newNode()</verb> accepts some more keyword
	      arguments.  You can give a title to your file with the
	      <verb>title</verb> argument.  You can use
	      <verb>PyTables</verb>' compression features with the
	      <verb>filters</verb> argument.  If you know beforehand the
	      size that your file will have, you can give its final file
	      size in bytes to the <verb>expectedsize</verb> argument so
	      that the <verb>PyTables</verb> library would be able to
	      optimize the data access.
	    </p>

	    <p><verb>newNode()</verb> creates a <verb>PyTables</verb>
	      node where it is told to.  To prove it, we will try to get
	      the <verb>NODE_TYPE</verb> attribute from the newly created
	      node.
	    </p>
	    <verbatim>
	      >>> print h5file.getNodeAttr('/fnode_test', 'NODE_TYPE')
	      file
	    </verbatim>
	  </subsection>

	  <subsection>
	    <heading>Using a file node</heading>

	    <p>As stated above, you can use the new node file as any
	      other opened file.  Let us try to write some text in and
	      read it.
	    </p>
	    <verbatim>
	      >>> print >> fnode, "This is a test text line."
	      >>> print >> fnode, "And this is another one."
	      >>> print >> fnode
	      >>> fnode.write("Of course, file methods can also be used.")
	      >>>
	      >>> fnode.seek(0)  # Go back to the beginning of file.
	      >>>
	      >>> for line in fnode:
	      ...   print repr(line)
	      'This is a test text line.\n'
	      'And this is another one.\n'
	      '\n'
	      'Of course, file methods can also be used.'
	    </verbatim>

	    <p>This was run on a Unix system, so newlines are expressed
	      as <verb>'\n'</verb>.  In fact, you can override the line
	      separator for a file by setting its
	      <verb>lineSeparator</verb> property to any string you
	      want.
	    </p>

	    <p>While using a file node, you should take care of closing
	      it <visual markup="bf">before</visual> you close the
	      <verb>PyTables</verb> host file.  Because of the way
	      <verb>PyTables</verb> works, your data it will not be at a
	      risk, but every operation you execute after closing the
	      host file will fail with a <verb>ValueError</verb>.  To
	      close a file node, simply delete the corresponding reference
	      it or call its <verb>close()</verb> method.
	    </p>
	    <verbatim>
	      >>> fnode.close()
	      >>> print fnode.closed
	      True
	    </verbatim>
	  </subsection>

	  <subsection>
	    <heading>Opening an existing file node</heading>

	    <p>If you have a file node that you created using
	      <verb>newNode()</verb>, you can open it later by calling
	      <verb>openNode()</verb>.  Its arguments are similar to
	      that of <verb>file()</verb> or <verb>open()</verb>: the
	      first argument is the <verb>PyTables</verb> node that you
	      want to open (i.e. a node with a <verb>NODE_TYPE</verb>
	      attribute having a <verb>'file'</verb> value), and the
	      second argument is a mode string indicating how to open
	      the file.  Contrary to <verb>file()</verb>,
	      <verb>openNode()</verb> can not be used to create a new
	      file node.
	    </p>

	    <p>File nodes can be opened in read-only mode
	      (<verb>'r'</verb>) or in read-and-append mode
	      (<verb>'a+'</verb>).  Reading from a file node is allowed
	      in both modes, but appending is only allowed in the second
	      one.  Just like Python files do, writing data to an
	      appendable file places it after the file pointer if it is
	      on or beyond the end of the file, or otherwise after the
	      existing data.  Let us see an example:
	    </p>
	    <verbatim>
	      >>> node = h5file.root.fnode_test
	      >>> fnode = FileNode.openNode(node, 'a+')
	      >>> print repr(fnode.readline())
	      'This is a test text line.\n'
	      >>> print fnode.tell()
	      26
	      >>> print >> fnode, "This is a new line."
	      >>> print repr(fnode.readline())
	      ''
	    </verbatim>

	    <p>Of course, the data append process places the pointer at
	      the end of the file, so the last <verb>readline()</verb>
	      call hit <verb>EOF</verb>.  Let us seek to the beginning
	      of the file to see the whole contents of our file.
	    </p>
	    <verbatim>
	      >>> fnode.seek(0)
	      >>> for line in fnode:
	      ...   print repr(line)
	      'This is a test text line.\n'
	      'And this is another one.\n'
	      '\n'
	      'Of course, file methods can also be used.This is a new line.\n'
	    </verbatim>

	    <p>As you can check, the last string we wrote was correctly
	      appended at the end of the file, instead of overwriting
	      the second line, where the file pointer was positioned by
	      the time of the appending.
	    </p>
	  </subsection>

	  <subsection>
	    <heading>Adding metadata to a file node</heading>

	    <p>You can associate arbitrary metadata to any open node
	      file, regardless of its mode, as long as the host
	      <verb>PyTables</verb> file is writable.  Of course, you
	      could use the <verb>setNodeAttr()</verb> method of
	      <verb>tables.File</verb> to do it directly on the proper
	      node, but <visual markup="tt">FileNode</visual> offers a
	      much more comfortable way to do it.  <verb>FileNode</verb>
	      objects have an <verb>attrs</verb> property which gives
	      you direct access to their corresponding
	      <verb>AttributeSet</verb> object.
	    </p>

	    <p>For instance, let us see how to associate MIME type
	      metadata to our file node:
	    </p>
	    <verbatim>
	      >>> fnode.attrs.content_type = 'text/plain; charset=us-ascii'
	    </verbatim>

	    <p>As simple as A-B-C.  You can put nearly anything in an
	      attribute, which opens the way to authorship, keywords,
	      permissions and more.  Moreover, there is not a fixed list
	      of attributes.  However, you should avoid names in all
	      caps or starting with <verb>'_'</verb>, since
	      <verb>PyTables</verb> and <visual
		markup="tt">FileNode</visual> may use them internally.
	      Some valid examples:
	    </p>

	    <verbatim>
	      >>> fnode.attrs.author = "Ivan Vilata i Balaguer"
	      >>> fnode.attrs.creation_date = '2004-10-20T13:25:25+0200'
	      >>> fnode.attrs.keywords_en = ["FileNode", "test", "metadata"]
	      >>> fnode.attrs.keywords_ca = ["FileNode", "prova", "metadades"]
	      >>> fnode.attrs.owner = 'ivan'
	      >>> fnode.attrs.acl = {'ivan': 'rw', '@users': 'r'}
	    </verbatim>

	    <p>You can check that these attributes get stored by running
	      the <verb>ptdump</verb> command on the host
	      <verb>PyTables</verb> file:
	    </p>
	    <verbatim>
	      $ ptdump -a fnode.h5:/fnode_test
	      /fnode_test (EArray(113,)) ''
	      /fnode_test.attrs (AttributeSet), 14 attributes:
	      [CLASS := 'EARRAY',
	      EXTDIM := 0,
	      FLAVOR := 'numarray',
	      NODE_TYPE := 'file',
	      NODE_TYPE_VERSION := 2,
	      TITLE := '',
	      VERSION := '1.2',
	      acl := {'ivan': 'rw', '@users': 'r'},
	      author := 'Ivan Vilata i Balaguer',
	      content_type := 'text/plain; charset=us-ascii',
	      creation_date := '2004-10-20T13:25:25+0200',
	      keywords_ca := ['FileNode', 'prova', 'metadades'],
	      keywords_en := ['FileNode', 'test', 'metadata'],
	      owner := 'ivan']
	    </verbatim>

	    <p>Note that <visual markup="tt">FileNode</visual> makes no
	      assumptions about the meaning of your metadata, so its
	      handling is entirely left to your needs and imagination.
	    </p>
	  </subsection>
	</section>

	<section>
	  <heading>Complementary notes</heading>

	  <p>You can use <verb>FileNode</verb> objects and
	    <verb>PyTables</verb> groups to mimic a filesystem with
	    files and directories.  Since you can store nearly anything
	    you want as file metadata, this enables you to use a
	    <verb>PyTables</verb> file as a portable compressed backup,
	    even between radically different platforms.
	    However, you may need to devise some
	    strategy to represent special files such as devices, sockets
	    and such (not necessarily using <visual
	      markup="tt">FileNode</visual>).
	  </p>

	  <p>We are eager to hear your opinion about <visual
	      markup="tt">FileNode</visual> and its potential uses.
	    Suggestions to improve <visual markup="tt">FileNode</visual>
	    and create other node types are also welcome.  Do not
	    hesitate to contact us!
	  </p>

	</section>

	<section>
	  <heading>Current limitations</heading>

	  <p><verb>FileNode</verb> is still a young piece of software,
	    so it lacks some functionality.  This is a list of known
	    current limitations:
	  </p>

	  <enumerate>
	    <item>Node files can only be opened for read-only or read
	      and append mode.  This will be enhanced in the future.
	    </item>
	    <item>There is no universal newline support yet.  This is
	      likely to be implemented in a near future.
	    </item>
	    <item>Sparse files (files with lots of zeros) are not
	      treated specially; if you want them to take less space,
	      you should be better off using compression.
	    </item>
	    <item>Node file and attribute names are constrained to
	      valid <verb>PyTables</verb> node names,
	      which happen to be more or less those accepted
	      by Unix filesystems (see <ref refid="GroupClassDescr"></ref>).
	    </item>
	  </enumerate>

	  <p>These limitations still make <verb>FileNode</verb> entirely
	    adequate to work with most binary and text files.  Of
	    course, suggestions and patches are welcome.
	  </p>

	</section>

	<section>
	  <heading><visual markup="tt">FileNode</visual> module
	    reference</heading>

	  <subsection>
	    <heading>Global constants</heading>

	    <description>
	      <term>NodeType</term> <item>Value for <verb>NODE_TYPE</verb>
		node system attribute.</item>

	      <term>NodeTypeVersions</term> <item>Supported values for
		<verb>NODE_TYPE_VERSION</verb> node system attribute.</item>
	    </description>

	  </subsection>

	  <subsection>
	    <heading>Global functions</heading>

	    <subsubsection>
	      <heading>newNode(h5file, where, name, title="",
		filters=None, expectedsize=1000)</heading>

	      <p>Creates a new file node object in the specified
		<verb>PyTables</verb> file object.  Additional named
		arguments <verb>where</verb> and <verb>name</verb> must
		be passed to specify where the file node is to be
		created.  Other named arguments such as
		<verb>title</verb> and <verb>filters</verb> may also be
		passed. The special named argument
		<verb>expectedsize</verb>, indicating an estimate of the
		file size in bytes, may also be passed. It returns the
		file node object.
	      </p>
	    </subsubsection>

	    <subsubsection>
	      <heading>openNode(node, mode = 'r')</heading>

	      <p>Opens an existing file node. Returns a file node object
		from the existing specified PyTables node.  If mode is
		not specified or it is <verb>'r'</verb>, the file can
		only be read, and the pointer is positioned at the
		beginning of the file.  If mode is <verb>'a+'</verb>,
		the file can be read and appended, and the pointer is
		positioned at the end of the file.
	      </p>

	    </subsubsection>
	  </subsection> <!-- Global functions -->

	  <subsection>
	    <heading>The <visual markup="tt">FileNode</visual>
	      abstract class</heading>

	    <p>This is the ancestor of <verb>ROFileNode</verb> and
	      <verb>RAFileNode</verb> (see below). Instances of these
	      classes are returned when <verb>newNode()</verb> or
	      <verb>openNode()</verb> are called. It represents a new
	      file node associated with a <verb>PyTables</verb> node,
	      providing a standard Python file interface to it.
	    </p>
	    <p>This abstract class provides only an implementation of
	      the reading methods needed to implement a file-like object
	      over a <verb>PyTables</verb> node.  The attribute set of
	      the node becomes available via the <verb>attrs</verb>
	      property.  You can add attributes there, but try to avoid
	      attribute names in all caps or starting with
	      <verb>'_'</verb>, since they may clash with internal
	      attributes.
	    </p>
	    <p>The node used as storage is also made available via the
	      read-only attribute <verb>node</verb>.  Please do not
	      tamper with this object unless unavoidably, since you may
	      break the operation of the file node object.
	    </p>
	    <p>The <verb>lineSeparator</verb> property contains the
	      string used as a line separator, and defaults to
	      <verb>os.linesep</verb>.  It can be set to any
	      reasonably-sized string you want.
	    </p>
	    <p>The constructor sets the <verb>closed</verb>,
	      <verb>softspace</verb> and <verb>_lineSeparator</verb>
	      attributes to their initial values, as well as the
	      <verb>node</verb> attribute to <verb>None</verb>.
	      Sub-classes should set the <verb>node</verb>,
	      <verb>mode</verb> and <verb>offset</verb> attributes.
	    </p>
	    <p>Version 1 implements the file storage as a
	      <verb>UInt8</verb> uni-dimensional <verb>EArray</verb>.
	    </p>

	    <subsubsection>
	      <heading><visual markup="tt">FileNode</visual>
		methods</heading>

	      <description>

		<term>getLineSeparator()</term> <item>Returns the line
		  separator string.</item>

		<term>setLineSeparator()</term> <item>Sets the line
		  separator string.</item>

		<term>getAttrs()</term> <item>Returns the attribute set
		  of the file node.</item>

		<term>close()</term> <item>Flushes the file and closes
		  it.  The <verb>node</verb> attribute becomes
		  <verb>None</verb> and the <verb>attrs</verb> property
		  becomes no longer available.</item>

		<term>next()</term> <item>Returns the next line of text.
		  Raises <verb>StopIteration</verb> when lines are
		  exhausted.  See <verb>file.next.__doc__</verb> for
		  more information.</item>

		<term>read(size=None)</term> <item>Reads at most
		  <verb>size</verb> bytes.  See
		  <verb>file.read.__doc__</verb> for more
		  information</item>

		<term>readline(size=-1)</term> <item>Reads the next text
		  line.  See <verb>file.readline.__doc__</verb> for more
		  information</item>

		<term>readlines(sizehint=-1)</term> <item>Reads the text
		  lines.  See <verb>file.readlines.__doc__</verb> for
		  more information.</item>

		<term>seek(offset, whence=0)</term> <item>Moves to a new
		  file position.  See <verb>file.seek.__doc__</verb> for
		  more information.</item>

		<term>tell()</term> <item> Gets the current file
		  position.  See <verb>file.tell.__doc__</verb> for more
		  information.</item>

		<term>xreadlines()</term> <item>For backward
		  compatibility.  See
		  <verb>file.xreadlines.__doc__</verb> for more
		  information.</item>

	      </description>

	    </subsubsection>

	  </subsection>

	  <subsection>
	    <heading>The <visual markup="tt">ROFileNode</visual>
	      class</heading>

	    <p>Instances of this class are returned when
	      <verb>openNode()</verb> is called in read-only mode
	      (<verb>'r'</verb>). This is a descendant of
	      <verb>FileNode</verb> class, so it inherits all its
	      methods. Moreover, it does not define any other useful
	      method, just some protections against users intents to
	      write on file.
	    </p>

	  </subsection>

	  <subsection>
	    <heading>The <visual markup="tt">RAFileNode</visual>
	      class</heading>

	    <p>Instances of this class are returned when either
	      <verb>newNode()</verb> is called or when
	      <verb>openNode()</verb> is called in append mode
	      (<verb>'a+'</verb>). This is a descendant of
	      <verb>FileNode</verb> class, so it inherits all its
	      methods. It provides additional methods that allow to
	      write on file nodes.
	    </p>

	    <description>

	      <term>flush()</term> <item>Flushes the file node.  See
		<verb>file.flush.__doc__</verb> for more
		information.</item>

	      <term>truncate(size=None)</term> <item>Truncates the file
		node to at most <verb>size</verb> bytes. Currently, this
		method only makes sense to grow the file node, since
		data can not be rewritten nor deleted.  See
		<verb>file.truncate.__doc__</verb> for more
		information.</item>

	      <term>write(string)</term> <item>Writes the string to the
		file.  Writing an empty string does nothing, but
		requires the file to be open.  See
		<verb>file.write.__doc__</verb> for more
		information.</item>

	      <term>writelines(sequence)</term> <item>Writes the
		sequence of strings to the file.  See
		<verb>file.writelines.__doc__</verb> for more
		information.</item>

	    </description>

	  </subsection>

	</section> <!-- FileNode module reference -->

      </chapter> <!-- FileNodes -->

      <chapter>
	<heading>NetCDF - a <visual markup="tt">PyTables</visual>
	  NetCDF3 emulation API</heading>

	<section>
	  <heading>What is NetCDF?</heading>

	  <p>The netCDF format is a popular format for binary files. It
	    is portable between machines and self-describing, i.e. it
	    contains the information necessary to interpret its
	    contents. A free library provides convenient access to these
	    files (see <cite refid="NetCDFRef"></cite>). A very nice python
	    interface to that library is available in the
	    <verb>Scientific Python NetCDF</verb> module (see <cite
	      refid="scientificpythonRef"></cite>). Although it is somewhat
	    less efficient and flexible than HDF5, netCDF is geared for
	    storing gridded data and is quite easy to use.  It has
	    become a de facto standard for gridded data, especially in
	    meteorology and oceanography.  The next version of netCDF
	    (netCDF 4) will actually be a software layer on top of HDF5
	    (see <cite refid="NetCDF4Ref"></cite>).  The
	    <verb>tables.NetCDF</verb> module does not create HDF5 files
	    that are compatible with netCDF 4 (although this is a
	    long-term goal).
	  </p>
	</section>

	<section>
	  <heading>Using the <visual
	      markup="tt">tables.NetCDF</visual> module
	  </heading>

	  <p>The module <verb>tables.NetCDF</verb> emulates the
	    <verb>Scientific.IO.NetCDF</verb> API using PyTables. It
	    presents the data in the form of objects that behave very
	    much like arrays. A <verb>tables.NetCDF</verb> file
	    contains any number of dimensions and variables, both of
	    which have unique names. Each variable has a shape defined
	    by a set of dimensions, and optionally attributes whose
	    values can be numbers, number sequences, or strings. One
	    dimension of a file can be defined as <em>unlimited</em>,
	    meaning that the file can grow along that direction.  In
	    the sections that follow, a step-by-step tutorial shows
	    how to create and modify a <verb>tables.NetCDF</verb>
	    file.  All of the code snippets presented here are
	    included in <verb>examples/netCDF_example.py</verb>.  The
	    <verb>tables.NetCDF</verb> module is designed to be used
	    as a drop-in replacement for
	    <verb>Scientific.IO.NetCDF</verb>, with only minor
	    modifications to existing code.  The differences between
	    <verb>table.NetCDF</verb> and
	    <verb>Scientific.IO.NetCDF</verb> are summarized in the
	    last section of this chapter.
	  </p>

	  <subsection>
	    <heading>Creating/Opening/Closing a <visual
		markup="tt">tables.NetCDF</visual> file
	    </heading>

	    <p>To create a <verb>tables.netCDF</verb> file from
	      python, you simply call the <verb>NetCDFFile</verb>
	      constructor.  This is also the method used to open an
	      existing <verb>tables.netCDF</verb> file.  The object
	      returned is an instance of the <verb>NetCDFFile</verb>
	      class and all future access must be done through this
	      object.  If the file is open for write access
	      (<verb>'w'</verb> or <verb>'a'</verb>), you may write
	      any type of new data including new dimensions, variables
	      and attributes.  The optional <verb>history</verb>
	      keyword argument can be used to set the
	      <verb>history</verb> <verb>NetCDFFile</verb> global file
	      attribute.  Closing the <verb>tables.NetCDF</verb> file
	      is accomplished via the <verb>close</verb> method of
	      <verb>NetCDFFile</verb> object.
	    </p>
	    <p>Here's an example:
	    </p>
	    <verbatim>
>>> import tables.NetCDF as NetCDF
>>> import time
>>> history = 'Created ' + time.ctime(time.time())
>>> file = NetCDF.NetCDFFile('test.h5', 'w', history=history)
>>> file.close()
	    </verbatim>

	  </subsection>

	  <subsection>
	    <heading>Dimensions in a <visual
		markup="tt">tables.NetCDF</visual> file
	    </heading>

	    <p>NetCDF defines the sizes of all variables in terms of
	      dimensions, so before any variables can be created the
	      dimensions they use must be created first.  A dimension
	      is created using the <verb>createDimension</verb> method
	      of the <verb>NetCDFFile</verb> object.  A Python string
	      is used to set the name of the dimension, and an integer
	      value is used to set the size.  To create an
	      <em>unlimited</em> dimension (a dimension that can be
	      appended to), the size value is set to
	      <verb>None</verb>.
	    </p>

	    <verbatim>
>>> import tables.NetCDF as NetCDF
>>> file = NetCDF.NetCDFFile('test.h5', 'a')
>>> file.NetCDFFile.createDimension('level', 12)
>>> file.NetCDFFile.createDimension('time', None)
>>> file.NetCDFFile.createDimension('lat', 90)
	    </verbatim>

	    <p>
	      All of the dimension names and their associated sizes
	      are stored in a Python dictionary.
	    </p>
	    <verbatim>
>>> print file.dimensions
{'lat': 90, 'time': None, 'level': 12}
	    </verbatim>

	  </subsection>

	  <subsection>
	    <heading>Variables in a <visual
		markup="tt">tables.NetCDF</visual> file
	  </heading>

	    <p>Most of the data in a <verb>tables.NetCDF</verb> file
	      is stored in a netCDF variable (except for global
	      attributes).  To create a netCDF variable, use the
	      <verb>createVariable</verb> method of the
	      <verb>NetCDFFile</verb> object. The
	      <verb>createVariable</verb> method has three mandatory
	      arguments, the variable name (a Python string), the
	      variable datatype described by a single character
	      Numeric typecode string which can be one of
	      <verb>f</verb> (Float32), <verb>d</verb> (Float64),
	      <verb>i</verb> (Int32), <verb>l</verb> (Int32),
	      <verb>s</verb> (Int16), <verb>c</verb> (CharType -
	      length 1), <verb>F</verb> (Complex32), <verb>D</verb>
	      (Complex64) or <verb>1</verb> (Int8), and a tuple
	      containing the variable's dimension names (defined
	      previously with <verb>createDimension</verb>).  The
	      dimensions themselves are usually defined as variables,
	      called coordinate variables. The
	      <verb>createVariable</verb> method returns an instance
	      of the <verb>NetCDFVariable</verb> class whose methods
	      can be used later to access and set variable data and
	      attributes.
	    </p>

	    <verbatim>
>>> times = file.createVariable('time','d',('time',))
>>> levels = file.createVariable('level','i',('level',))
>>> latitudes = file.createVariable('latitude','f',('lat',))
>>> temp = file.createVariable('temp','f',('time','level','lat',))
>>> pressure = file.createVariable('pressure','i',('level','lat',))
	    </verbatim>

	    <p>All of the variables in the file are stored in a Python
	      dictionary, in the same way as the dimensions:
	    </p>

	    <verbatim>
<![CDATA[
>>> print file.variables
{'latitude': <tables.NetCDF.NetCDFVariable instance at 0x244f350>,
 'pressure': <tables.NetCDF.NetCDFVariable instance at 0x244f508>,
 'level': <tables.NetCDF.NetCDFVariable instance at 0x244f0d0>,
 'temp': <tables.NetCDF.NetCDFVariable instance at 0x244f3a0>,
 'time': <tables.NetCDF.NetCDFVariable instance at 0x2564c88>}
]]>
	    </verbatim>
	  </subsection>

	  <subsection>
	    <heading>Attributes in a <visual
		markup="tt">tables.NetCDF</visual> file
	    </heading>

	    <p>There are two types of attributes in a
	      <verb>tables.NetCDF</verb> file, global (or file) and
	      variable.  Global attributes provide information about
	      the dataset, or file, as a whole.  Variable attributes
	      provide information about one of the variables in the
	      file.  Global attributes are set by assigning values to
	      <verb>NetCDFFile</verb> instance variables.  Variable
	      attributes are set by assigning values to
	      <verb>NetCDFVariable</verb> instance variables.
	    </p>

	    <p>Attributes can be strings, numbers or sequences.
	      Returning to our example,
	    </p>

  <verbatim>
>>> file.description = 'bogus example to illustrate the use of tables.NetCDF'
>>> file.source = 'PyTables Users Guide'
>>> latitudes.units = 'degrees north'
>>> pressure.units = 'hPa'
>>> temp.units = 'K'
>>> times.units = 'days since January 1, 2005'
>>> times.scale_factor = 1
	    </verbatim>

	    <p>The <verb>ncattrs</verb> method of the
	      <verb>NetCDFFile</verb> object can be used to retrieve
	      the names of all the global attributes.  This method is
	      provided as a convenience, since using the built-in
	      <verb> dir</verb> Python function will return a bunch of
	      private methods and attributes that cannot (or should
	      not) be modified by the user.  Similarly, the
	      <verb>ncattrs</verb> method of a
	      <verb>NetCDFVariable</verb> object returns all of the
	      netCDF variable attribute names.  These functions can be
	      used to easily print all of the attributes currently
	      defined, like this
	    </p>

	    <verbatim>
>>> for name in file.ncattrs():
>>>     print 'Global attr', name, '=', getattr(file,name)
Global attr description = bogus example to illustrate the use of tables.NetCDF
Global attr history = Created Mon Nov  7 10:30:56 2005
Global attr source = PyTables Users Guide
	</verbatim>

	    <p>Note that the <verb>ncattrs</verb> function is not part
	      of the <verb> Scientific.IO.NetCDF</verb> interface.
	    </p>

	  </subsection>

	  <subsection>
	    <heading>Writing data to and retrieving data from a
	      <visual markup="tt">tables.NetCDF</visual> variable
	    </heading>

	    <p>Now that you have a netCDF variable object, how do you
	      put data into it?  If the variable has no
	      <em>unlimited</em> dimension, you just treat it like a
	      Numeric array object and assign data to a slice.
	    </p>

       <verbatim>

>>> import numarray
>>> levels[:] = numarray.arange(12)+1
>>> latitudes[:] = numarray.arange(-89,90,2)
>>> for lev in levels[:]:
>>>     pressure[:,:] = 1000.-100.*lev
>>> print 'levels = ',levels[:]
levels =  [ 1  2  3  4  5  6  7  8  9 10 11 12]
>>> print 'latitudes =\n',latitudes[:]
latitudes =
[-89. -87. -85. -83. -81. -79. -77. -75. -73. -71. -69. -67. -65. -63.
 -61. -59. -57. -55. -53. -51. -49. -47. -45. -43. -41. -39. -37. -35.
 -33. -31. -29. -27. -25. -23. -21. -19. -17. -15. -13. -11.  -9.  -7.
  -5.  -3.  -1.   1.   3.   5.   7.   9.  11.  13.  15.  17.  19.  21.
  23.  25.  27.  29.  31.  33.  35.  37.  39.  41.  43.  45.  47.  49.
  51.  53.  55.  57.  59.  61.  63.  65.  67.  69.  71.  73.  75.  77.
  79.  81.  83.  85.  87.  89.]

       </verbatim>

	    <p> Note that retrieving data from the netCDF variable
	      object works just like a Numeric array too.  If the
	      netCDF variable has an <em>unlimited</em> dimension, and
	      there is not yet an entry for the data along that
	      dimension, the <verb>append</verb> method must be used.
	  </p>

	    <verbatim>

>>> for n in range(10):
>>>     times.append(n)
>>> print 'times = ',times[:]
times =  [ 0.  1.  2.  3.  4.  5.  6.  7.  8.  9.]

	    </verbatim>

	    <p> The data you append must have either the same number
	      of dimensions as the <verb>NetCDFVariable</verb>, or one
	      less.  The shape of the data you append must be the same
	      as the <verb>NetCDFVariable</verb> for all of the
	      dimensions except the <em>unlimited</em> dimension.  The
	      length of the data long the <em>unlimited</em> dimension
	      controls how may entries along the <em>unlimited</em>
	      dimension are appended.  If the data you append has one
	      fewer number of dimensions than the
	      <verb>NetCDFVariable</verb>, it is assumed that you are
	      appending one entry along the <em>unlimited</em>
	      dimension.  For example, if the
	      <verb>NetCDFVariable</verb> has shape
	      <verb>(10,50,100)</verb> (where the dimension length of
	      length <verb>10</verb> is the <em>unlimited</em>
	      dimension), and you append an array of shape
	      <verb>(50,100)</verb>, the <verb>NetCDFVariable</verb>
	      will subsequently have a shape of
	      <verb>(11,50,100)</verb>.  If you append an array with
	      shape <verb>(5,50,100)</verb>, the
	      <verb>NetCDFVariable</verb> will have a new shape of
	      <verb>(15,50,100)</verb>.  Appending an array whose last
	      two dimensions do not have a shape <verb>(50,100)</verb>
	      will raise an exception.  This <verb>append</verb>
	      method does not exist in the
	      <verb>Scientific.IO.NetCDF</verb> interface, instead
	      entries are appended along the <em>unlimited</em>
	      dimension one at a time by assigning to a slice.  This
	      is the biggest difference between the
	      <verb>tables.NetCDF</verb> and
	      <verb>Scientific.IO.NetCDF</verb> interfaces.
	    </p>

	    <!-- This must be really here?? -->
	    <newline vspace="0.25cm"/>

	    <p> Once data has been appended to any variable with an
	      <em>unlimited</em> dimension, the <verb>sync</verb>
	      method can be used to synchronize the sizes of all the
	      other variables with an <em>unlimited</em> dimension.
	      This is done by filling in missing values (given by the
	      default netCDF <verb>_FillValue</verb>, which is
	      intended to indicate that the data was never defined).
	      The <verb>sync</verb> method is automatically invoked
	      with a <verb>NetCDFFile</verb> object is closed.  Once
	      the <verb>sync</verb> method has been invoked, the
	      filled-in values can be assigned real data with slices.
	    </p>

	    <verbatim>

>>> print 'temp.shape before sync = ',temp.shape
temp.shape before sync =  (0, 12, 90)
>>> file.sync()
>>> print 'temp.shape after sync = ',temp.shape
temp.shape after sync =  (10L, 12, 90)
>>> import numarray.random_array as random_array
>>> for n in range(10):
>>>     temp[n] = 10.*random_array.random(pressure.shape)
>>>     print 'time, min/max temp, temp[n,0,0] = ',\
               times[n],min(temp[n].flat),max(temp[n].flat),temp[n,0,0]
time, min/max temp, temp[n,0,0] = 0.0 0.0122650898993 9.99259281158 6.13053750992
time, min/max temp, temp[n,0,0] = 1.0 0.00115821603686 9.9915933609 6.68516159058
time, min/max temp, temp[n,0,0] = 2.0 0.0152112031356 9.98737239838 3.60537290573
time, min/max temp, temp[n,0,0] = 3.0 0.0112022599205 9.99535560608 6.24249696732
time, min/max temp, temp[n,0,0] = 4.0 0.00519315246493 9.99831295013 0.225010097027
time, min/max temp, temp[n,0,0] = 5.0 0.00978941563517 9.9843454361 4.56814193726
time, min/max temp, temp[n,0,0] = 6.0 0.0159023851156 9.99160385132 6.36837291718
time, min/max temp, temp[n,0,0] = 7.0 0.0019518379122 9.99939727783 1.42762875557
time, min/max temp, temp[n,0,0] = 8.0 0.00390585977584 9.9909954071 2.79601073265
time, min/max temp, temp[n,0,0] = 9.0 0.0106026884168 9.99195957184 8.18835449219

	    </verbatim>

	    <p>Note that appending data along an <em>unlimited</em>
	      dimension always increases the length of the variable
	      along that dimension.  Assigning data to a variable with
	      an <em>unlimited</em> dimension with a slice operation
	      does not change its shape.  Finally, before closing the
	      file we can get a summary of its contents simply by
	      printing the <verb>NetCDFFile</verb> object.  This
	      produces output very similar to running 'ncdump -h' on a
	      netCDF file.
	    </p>

	    <verbatim>
>>> print file
test.h5 {
dimensions:
    lat = 90 ;
    time = UNLIMITED ; // (10 currently)
    level = 12 ;
variables:
    float latitude('lat',) ;
        latitude:units = 'degrees north' ;
    int pressure('level', 'lat') ;
        pressure:units = 'hPa' ;
    int level('level',) ;
    float temp('time', 'level', 'lat') ;
        temp:units = 'K' ;
    double time('time',) ;
        time:scale_factor = 1 ;
        time:units = 'days since January 1, 2005' ;
// global attributes:
        :description = 'bogus example to illustrate the use of tables.NetCDF' ;
        :history = 'Created Wed Nov  9 12:29:13 2005' ;
        :source = 'PyTables Users Guilde' ;
}
	    </verbatim>

	  </subsection>

	  <subsection>
	    <heading>Efficient compression of <visual
		markup="tt">tables.NetCDF</visual> variables
	    </heading>

	    <p>Data stored in <verb>NetCDFVariable</verb> objects is
	      compressed on disk by default.  The parameters for the
	      default compression are determined from a
	      <verb>Filters</verb> class instance (see section <ref
	      refid="FiltersClassDescr"></ref>) with
	      <verb>complevel=6, complib='zlib' and shuffle=1</verb>.
	      To change the default compression, simply pass a
	      <verb>Filters</verb> instance to
	      <verb>createVariable</verb> with the
	      <verb>filters</verb> keyword.  If your data only has a
	      certain number of digits of precision (say for example,
	      it is temperature data that was measured with a
	      precision of <verb>0.1</verb> degrees), you can
	      dramatically improve compression by quantizing (or
	      truncating) the data using the
	      <verb>least_significant_digit</verb> keyword argument to
	      <verb>createVariable</verb>.  The <em>least significant
	      digit</em> is the power of ten of the smallest decimal
	      place in the data that is a reliable value.  For example
	      if the data has a precision of <verb>0.1</verb>, then
	      setting <verb>least_significant_digit=1</verb> will
	      cause data the data to be quantized using
	      <verb>numarray.around(scale*data)/scale</verb>, where
	      <verb>scale = 2**bits</verb>, and bits is determined so
	      that a precision of <verb>0.1</verb> is retained (in
	      this case <verb>bits=4</verb>).
	    </p>
	    <!-- This must be here?? -->
	    <newline vspace="0.25cm"/>

	    <p>In our example, try replacing the line
	    </p>
	    <verbatim>
>>> temp = file.createVariable('temp','f',('time','level','lat',))
	    </verbatim>
	    <p>with
	    </p>
	    <verbatim>
>>> temp = file.createVariable('temp','f',('time','level','lat',),
                               least_significant_digit=1)
	    </verbatim>

	    <p>and see how much smaller the resulting file is.
	    </p>

	    <p>The <verb>least_significant_digit</verb> keyword
	      argument is not allowed in
	      <verb>Scientific.IO.NetCDF</verb>, since netCDF version
	      3 does not support compression.  The flexible, fast and
	      efficient compression available in HDF5 is the main
	      reason I wrote the <verb>tables.NetCDF</verb> module -
	      my netCDF files were just getting too big.
	    </p>

	    <!-- XXX -->
	    <newline vspace="0.25cm"/>

	    <p>The <verb>createVariable</verb> method has one other
	      keyword argument not found in
	      <verb>Scientific.IO.NetCDF</verb> -
	      <verb>expectedsize</verb>.  The
	      <verb>expectedsize</verb> keyword can be used to set the
	      expected number of entries along the <em>unlimited</em>
	      dimension (default 10000).  If you expect that your data
	      with have an order of magnitude more or less than 10000
	      entries along the <em>unlimited</em> dimension, you may
	      consider setting this keyword to improve efficiency (see
	      section <ref refid="expectedRowsOptim"></ref> for
	      details).
	    </p>

	  </subsection>
	</section> <!-- End of the tutorial -->

	<section>
	  <heading><visual markup="tt">tables.NetCDF</visual> module
	    reference</heading>

	  <subsection>
	    <heading>Global constants</heading>

	    <description>

	      <term>_fillvalue_dict</term> <item>Dictionary whose keys
		are <verb>NetCDFVariable</verb> single character
		typecodes and whose values are the netCDF _FillValue
		for that typecode.</item>

	      <term>ScientificIONetCDF_imported</term>
		<item><verb>True</verb> if
		<verb>Scientific.IO.NetCDF</verb> is installed and can
		be imported.</item>

	    </description>

	  </subsection>

	  <subsection>
	    <heading>The <visual markup="tt">NetCDFFile</visual>
	      class</heading>

	    <p><visual markup="bf">NetCDFFile(filename, mode='r',
	      history=None)</visual>
	    </p>

	    <newline vspace="0.25cm"/>

	    <p>Opens an existing <verb>tables.NetCDF</verb> file (mode
	      = <verb>'r'</verb> or <verb>'a'</verb>) or creates a new
	      one (mode = <verb>'w'</verb>). The <verb>history</verb>
	      keyword can be used to set the
	      <verb>NetCDFFile.history</verb> global attribute (if
	      mode = <verb>'a'</verb> or <verb>'w'</verb>).
	    </p>

	    <p>A <verb>NetCDFFile</verb> object has two standard
	      attributes: <verb>dimensions</verb> and
	      <verb>variables</verb>. The values of both are
	      dictionaries, mapping dimension names to their
	      associated lengths and variable names to variables.  All
	      other attributes correspond to global attributes defined
	      in a netCDF file. Global file attributes are created by
	      assigning to an attribute of the <verb>NetCDFFile</verb>
	      object.
	    </p>

	    <subsubsection>
	      <heading><visual markup="tt">NetCDFFile</visual>
		methods</heading>

	      <description>

		<term>close()</term> <item>Closes the file (after
		  invoking the <verb>sync</verb> method).
		</item>

		<term>sync()</term> <item>Synchronizes the size of
		  variables along the <em>unlimited</em> dimension, by
		  filling in data with default netCDF
		  _FillValue. Returns the length of the
		  <em>unlimited</em> dimension. Invoked automatically
		  when the <verb>NetCDFFile</verb> object is
		  closed.
		</item>

		<term>ncattrs()</term> <item>Returns a list with the
		  names of all currently defined netCDF global file
		  attributes.
		</item>

		<term>createDimension(name, length)</term><item>
		  Creates a netCDF dimension with a name given by the
		  Python string <verb>name</verb> and a size given by
		  the integer <verb>size</verb>.  If <verb>size =
		  None</verb>, the dimension is <em>unlimited</em>
		  (i.e. it can grow dynamically).  There can be only
		  one <em>unlimited</em> dimension in a file.
		</item>

		<term>createVariable(name, type, dimensions,
		  least_significant_digit=None, expectedsize=10000,
		  filters=None)</term><item>Creates a new variable
		  with the given <verb>name, type, and
		  dimensions</verb>.  The type is a one-letter Numeric
		  typecode string which can be one of <verb>f</verb>
		  (Float32), <verb>d</verb> (Float64), <verb>i</verb>
		  (Int32), <verb>l</verb> (Int32), <verb>s</verb>
		  (Int16), <verb>c</verb> (CharType - length 1),
		  <verb>F</verb> (Complex32), <verb>D</verb>
		  (Complex64) or <verb>1</verb> (Int8); the predefined
		  type constants from Numeric can also be used.  The
		  <verb>F</verb> and <verb>D</verb> types are not
		  supported in netCDF or Scientific.IO.NetCDF, if they
		  are used in a <verb>tables.NetCDF</verb> file, that
		  file cannot be converted to a true netCDF file nor
		  can it be shared over the internet with
		  OPeNDAP. Dimensions must be a tuple containing
		  dimension names (strings) that have been defined
		  previously by <verb>createDimensions</verb>. The
		  <verb>least_significant_digit</verb> is the power of
		  ten of the smallest decimal place in the variable's
		  data that is a reliable value.  If this keyword is
		  specified, the variable's data truncated to this
		  precision to improve compression.  The
		  <verb>expectedsize</verb> keyword can be used to set
		  the expected number of entries along the
		  <em>unlimited</em> dimension (default 10000).  If
		  you expect that your data with have an order of
		  magnitude more or less than 10000 entries along the
		  <em>unlimited</em> dimension, you may consider
		  setting this keyword to improve efficiency (see
		  section <ref refid="expectedRowsOptim"></ref> for
		  details). The <verb>filters</verb> keyword is a
		  PyTables <verb>Filters</verb> instance that
		  describes how to store the data on disk.  The
		  default corresponds to <verb>complevel=6</verb>,
		  <verb>complib='zlib'</verb>, <verb>shuffle=1</verb>
		  and <verb>fletcher32=0</verb>.
		</item>

		<term>nctoh5(filename, unpackshort=True,
		  filters=None)</term> <item>Imports the data in a
		  netCDF version 3 file (<verb>filename</verb>) into a
		  <verb>NetCDFFile</verb> object using
		  <verb>Scientific.IO.NetCDF</verb>
		  (<verb>ScientificIONetCDF_imported</verb> must be
		  <verb>True</verb>). If
		  <verb>unpackshort=True</verb>, data packed as short
		  integers (type <verb>s</verb>) in the netCDF file
		  will be unpacked to type <verb>f</verb> using the
		  <verb>scale_factor</verb> and
		  <verb>add_offset</verb> netCDF variable
		  attributes. The <verb>filters</verb> keyword can be
		  set to a PyTables <verb>Filters</verb> instance to
		  change the default parameters used to compress the
		  data in the <verb>tables.NetCDF</verb> file. The
		  default corresponds to <verb>complevel=6</verb>,
		  <verb>complib='zlib'</verb>, <verb>shuffle=1</verb>
		  and <verb>fletcher32=0</verb>.
		</item>

		<term>h5tonc(filename, packshort=False,
		  scale_factor=None, add_offset=None)</term>
		  <item>Exports the data in a
		  <verb>tables.NetCDF</verb> file defined by the
		  <verb>NetCDFFile</verb> instance into a netCDF
		  version 3 file using
		  <verb>Scientific.IO.NetCDF</verb>
		  (<verb>ScientificIONetCDF_imported</verb> must be
		  <verb>True</verb>).  If <verb>packshort=True></verb>
		  the dictionaries <verb>scale_factor</verb> and
		  <verb>add_offset</verb> are used to pack data of
		  type <verb>f</verb> as short integers (of type
		  <verb>s</verb>) in the netCDF file.  Since netCDF
		  version 3 does not provide automatic compression,
		  packing as short integers is a commonly used way of
		  saving disk space (see this <url
		  name="http://www.cdc.noaa.gov/cdc/conventions/cdc_netcdf_standard.shtml">page</url>
		  for more details).  The keys of these dictionaries
		  are the variable names to pack, the values are the
		  scale_factors and offsets to use in the packing. The
		  data are packed so that the original Float32 values
		  can be reconstructed by multiplying the
		  <verb>scale_factor</verb> and adding
		  <verb>add_offset</verb>.  The resulting netCDF file
		  will have the <verb>scale_factor</verb> and
		  <verb>add_offset</verb> variable attributes set
		  appropriately.
		</item>

	      </description>

	    </subsubsection>
	  </subsection>

	  <subsection>
	    <heading>The <visual markup="tt">NetCDFVariable</visual>
	      class</heading>

	    <p>The <verb>NetCDFVariable</verb> constructor is not
	      called explicitly, rather an <verb>NetCDFVarible</verb>
	      instance is returned by an invocation of
	      <verb>NetCDFFile.createVariable</verb>.
	      <verb>NetCDFVariable</verb> objects behave like arrays,
	      and have the standard attributes of arrays (such as
	      <verb>shape</verb>).  Data can be assigned or extracted
	      from <verb>NetCDFVariable</verb> objects via slices.
	    </p>

	    <subsubsection>
	      <heading><visual markup="tt">NetCDFVariable</visual>
		methods</heading>

	      <description>

		<term>typecode()</term> <item> Returns a single
		  character typecode describing the type of the
		  variable, one of <verb>f</verb> (Float32),
		  <verb>d</verb> (Float64), <verb>i</verb> (Int32),
		  <verb>l</verb> (Int32), <verb>s</verb> (Int16),
		  <verb>c</verb> (CharType - length 1), <verb>F</verb>
		  (Complex32), <verb>D</verb> (Complex64) or
		  <verb>1</verb> (Int8).
		</item>

		<term>append(data)</term><item> Append data to a
		  variable along its <em>unlimited</em> dimension.
		  The data you append must have either the same number
		  of dimensions as the <verb>NetCDFVariable</verb>, or
		  one less.  The shape of the data you append must be
		  the same as the <verb>NetCDFVariable</verb> for all
		  of the dimensions except the <em>unlimited</em>
		  dimension.  The length of the data long the
		  <em>unlimited</em> dimension controls how may
		  entries along the <em>unlimited</em> dimension are
		  appended.  If the data you append has one fewer
		  number of dimensions than the
		  <verb>NetCDFVariable</verb>, it is assumed that you
		  are appending one entry along the <em>unlimited</em>
		  dimension.  For variables without an
		  <em>unlimited</em> dimension, data can simply be
		  assigned to a slice without using the
		  <verb>append</verb> method.
		</item>

		<term>ncattrs()</term><item>Returns a list with all
		  the names of the currently defined netCDF variable
		  attributes.
		</item>

		<term>assignValue(data)</term> <item>Provided for
		  compatiblity with <verb>Scientific.IO.NetCDF</verb>.
		  Assigns data to the variable.  If the variable has
		  an <em>unlimited</em> dimension, it is equivalent to
		  <verb>append(data)</verb>.  If the variable has no
		  <em>unlimited</em> dimension, it is equivalent to
		  assigning data to the variable with the slice
		  <verb>[:]</verb>.
		</item>

		<term>getValue()</term> <item>Provided for
		  compatiblity with <verb>Scientific.IO.NetCDF</verb>.
		  Returns all the data in the variable.  Equivalent to
		  extracting the slice <verb>[:]</verb> from the
		  variable.
		</item>

	      </description>

	    </subsubsection>
	  </subsection>

	</section> <!-- NetCDF module reference -->

	<section>
	  <heading>Converting between true netCDF files and <visual
	      markup="tt">tables.NetCDF</visual> files
	  </heading>

	  <p>If <verb>Scientific.IO.NetCDF</verb> is installed,
	    <verb>tables.NetCDF</verb> provides facilities for
	    converting between true netCDF version 3 files and
	    <verb>tables.NetCDF</verb> hdf5 files via the
	    <verb>NetCDFFile.h5tonc()</verb> and
	    <verb>NetCDFFile.nctoh5()</verb> class methods.  Also, the
	    <verb>nctoh5</verb> command-line utility (see <ref
	    refid="nctoh5Descr">Appendix</ref>) uses the
	    <verb>NetCDFFile.nctoh5()</verb> class method.
	  </p>
	  <p>As an example, look how to convert a
	    <verb>tables.NetCDF</verb> hdf5 file to a true netCDF
	    version 3 file (named <verb>test.nc</verb>)
	  </p>
	  <verbatim>
>>> scale_factor = {'temp': 1.75e-4}
>>> add_offset = {'temp': 5.}
>>> file.h5tonc('test.nc',packshort=True, \
                 scale_factor=scale_factor,add_offset=add_offset)
packing temp as short integers ...
>>> file.close()
	  </verbatim>

	  <p>The dictionaries <verb>scale_factor</verb> and
	    <verb>add_offset</verb> are used to optionally pack the
	    data as short integers in the netCDF file.  Since netCDF
	    version 3 does not provide automatic compression, packing
	    as short integers is a commonly used way of saving disk
	    space (see this <url
	    name="http://www.cdc.noaa.gov/cdc/conventions/cdc_netcdf_standard.shtml">page</url>
	    for more details).  The keys of these dictionaries are the
	    variable names to pack, the values are the scale_factors
	    and offsets to use in the packing.  The resulting netCDF
	    file will have the <verb>scale_factor</verb> and
	    <verb>add_offset</verb> variable attributes set
	    appropriately.
	  </p>

	  <p>To convert the netCDF file back to a
	    <verb>tables.NetCDF</verb> hdf5 file:
	  </p>

	  <verbatim>
>>> history = 'Convert from netCDF ' + time.ctime(time.time())
>>> file = NetCDF.NetCDFFile('test2.h5', 'w', history=history)
>>> nobjects, nbytes = file.nctoh5('test.nc',unpackshort=True)
>>> print nobjects,' objects converted from netCDF, totaling',nbytes,'bytes'
5  objects converted from netCDF, totaling 48008 bytes
>>> temp = file.variables['temp']
>>> times = file.variables['time']
>>> print 'temp.shape after h5 --> netCDF --> h5 conversion = ',temp.shape
temp.shape after h5 --> netCDF --> h5 conversion =  (10L, 12, 90)
>>> for n in range(10):
>>>     print 'time, min/max temp, temp[n,0,0] = ',\
               times[n],min(temp[n].flat),max(temp[n].flat),temp[n,0,0]
time, min/max temp, temp[n,0,0] = 0.0 0.0123250000179 9.99257469177 6.13049983978
time, min/max temp, temp[n,0,0] = 1.0 0.00130000000354 9.99152469635 6.68507480621
time, min/max temp, temp[n,0,0] = 2.0 0.0153000000864 9.98732471466 3.60542488098
time, min/max temp, temp[n,0,0] = 3.0 0.0112749999389 9.99520015717 6.2423248291
time, min/max temp, temp[n,0,0] = 4.0 0.00532499980181 9.99817466736 0.225124999881
time, min/max temp, temp[n,0,0] = 5.0 0.00987500045449 9.98417472839 4.56827497482
time, min/max temp, temp[n,0,0] = 6.0 0.01600000076 9.99152469635 6.36832523346
time, min/max temp, temp[n,0,0] = 7.0 0.00200000009499 9.99922466278 1.42772495747
time, min/max temp, temp[n,0,0] = 8.0 0.00392499985173 9.9908246994 2.79605007172
time, min/max temp, temp[n,0,0] = 9.0 0.0107500003651 9.99187469482 8.18832492828
>>> file.close()
	  </verbatim>

	  <p>Setting <verb>unpackshort=True</verb> tells
	    <verb>nctoh5</verb> to unpack all of the variables which
	    have the <verb>scale_factor</verb> and
	    <verb>add_offset</verb> attributes back to floating point
	    arrays. Note that <verb>tables.NetCDF</verb> files have
	    some features not supported in netCDF (such as Complex
	    data types and the ability to make any dimension
	    <em>unlimited</em>).  <verb>tables.NetCDF</verb> files
	    which utilize these features cannot be converted to netCDF
	    using <verb>NetCDFFile.h5tonc</verb>.
	  </p>

	</section>

	<section>
	  <heading><visual markup="tt">tables.NetCDF</visual> file structure
	  </heading>

	  <p> A <verb>tables.NetCDF</verb> file consists of array
	    objects (either <verb> EArrays</verb> or
	    <verb>CArrays</verb>) located in the root group of a
	    pytables hdf5 file. Each of the array objects must have a
	    <verb>dimensions</verb> attribute, consisting of a tuple
	    of dimension names (the length of this tuple should be the
	    same as the rank of the array object). Any array objects
	    with one of the supported datatypes in a pytables file
	    that conforms to this simple structure can be read with
	    the <verb>tables.NetCDF</verb> module.
	  </p>

	</section>

	<section>
	  <heading>Sharing data in <visual
	      markup="tt">tables.NetCDF</visual> files over the
	      internet with OPeNDAP
	  </heading>

	  <p><verb>tables.NetCDF</verb> datasets can be shared over
	    the internet with the OPeNDAP protocol (<url
	    name="http://opendap.org">http://opendap.org</url>), via
	    the python opendap module (<url
	    name="http://opendap.oceanografia.org">http://opendap.oceanografia.org</url>).
	    A plugin for the python opendap server is included with
	    the pytables distribution
	    (<verb>contrib/h5_dap_plugin.py</verb>).  Simply copy that
	    file into the <verb>plugins</verb> directory of the
	    opendap python module source distribution, run
	    <verb>python setup.py install</verb>, point the opendap
	    server to the directory containing your
	    <verb>tables.NetCDF</verb> files, and away you go. Any
	    OPeNDAP aware client (such as Matlab or IDL) will now be
	    able to access your data over http as if it were a local
	    disk file.  The only restriction is that your
	    <verb>tables.NetCDF</verb> files must have the extension
	    <verb>.h5</verb> or <verb>.hdf5</verb>. Unfortunately,
	    <verb>tables.NetCDF</verb> itself cannot act as an OPeNDAP
	    client, although there is a client included in the opendap
	    python module, and <verb>Scientific.IO.NetCDF</verb> can
	    act as an OPeNDAP client if it is linked with the OPeNDAP
	    netCDF client library.  Either of these python modules can
	    be used to remotely acess <verb>tables.NetCDF</verb>
	    datasets with OPeNDAP.
	  </p>
	</section>

	<section>
	  <heading>Differences between the <visual
	      markup="tt">Scientific.IO.NetCDF</visual> API and the
	      <visual markup="tt">tables.NetCDF</visual> API
	  </heading>

	  <enumerate>

	    <item><verb>tables.NetCDF</verb> data is stored in an
	      HDF5 file instead of a netCDF file.
	    </item>

	    <item>Although each variable can have only one
	      <em>unlimited</em> dimension in a
	      <verb>tables.NetCDF</verb> file, it need not be the
	      first as in a true NetCDF file.  Complex data types
	      <verb>F</verb> (Complex32) and <verb>D</verb>
	      (Complex64) are supported in <verb>tables.NetCDF</verb>,
	      but are not supported in netCDF (or
	      <verb>Scientific.IO.NetCDF</verb>). Files with variables
	      that have these datatypes, or an <em>unlimited</em>
	      dimension other than the first, cannot be converted to
	      netCDF using <verb>h5tonc</verb>.
	    </item>

	    <item>Variables in a <verb>tables.NetCDF</verb> file are
	      compressed on disk by default using HDF5 zlib
	      compression with the <em>shuffle</em> filter.  If the
	      <em>least_significant_digit</em> keyword is used when a
	      variable is created with the <verb>createVariable
	      method</verb>, data will be truncated (quantized) before
	      being written to the file.  This can significantly
	      improve compression.  For example, if
	      <verb>least_significant_digit=1</verb>, data will be
	      quantized using
	      <verb>numarray.around(scale*data)/scale</verb>, where
	      <verb>scale = 2**bits</verb>, and bits is determined so
	      that a precision of 0.1 is retained (in this case
	      <verb>bits=4</verb>).  From <url
	      name="http://www.cdc.noaa.gov/cdc/conventions/cdc_netcdf_standard.shtml">http://www.cdc.noaa.gov/cdc/conventions/cdc_netcdf_standard.shtml</url>:
	      <quote>least_significant_digit -- power of ten of the
	      smallest decimal place in unpacked data that is a
	      reliable value.</quote> Automatic data compression is
	      not available in netCDF version 3, and hence is not
	      available in the <verb>Scientific.IO.NetCDF</verb>
	      module.
	    </item>

	    <item>In <verb>tables.NetCDF</verb>, data must be appended
	      to a variable with an <em>unlimited</em> dimension using
	      the <verb>append</verb> method of the
	      <verb>netCDF</verb> variable object. In
	      <verb>Scientific.IO.NetCDF</verb>, data can be added
	      along an <em>unlimited</em> dimension by assigning it to
	      a slice (there is no append method).  The
	      <verb>sync</verb> method of a <verb>tables.NetCDF
	      NetCDFVariable</verb> object synchronizes the size of
	      all variables with an <em>unlimited</em> dimension by
	      filling in data using the default netCDF
	      <verb>_FillValue</verb>. The <verb>sync</verb> method is
	      automatically invoked with a <verb>NetCDFFile</verb>
	      object is closed. In <verb>Scientific.IO.NetCDF</verb>,
	      the <verb>sync()</verb> method flushes the data to disk.
	    </item>

	    <item>The <verb>tables.NetCDF createVariable()</verb>
	      method has three extra optional keyword arguments not
	      found in the <verb>Scientific.IO.NetCDF</verb>
	      interface, <em>least_significant_digit</em> (see item
	      (2) above), <em>expectedsize</em> and <em>filters</em>.
	      The <em>expectedsize</em> keyword applies only to
	      variables with an <em>unlimited</em> dimension, and is
	      an estimate of the number of entries that will be added
	      along that dimension (default 1000). This estimate is
	      used to optimize HDF5 file access and memory usage.  The
	      <em>filters</em> keyword is a PyTables filters instance
	      that describes how to store the data on disk.  The
	      default corresponds to <verb>complevel=6</verb>,
	      <verb>complib='zlib'</verb>, <verb>shuffle=1</verb> and
	      <verb>fletcher32=0</verb>.
	    </item>

	    <item><verb>tables.NetCDF</verb> data can be saved to a
	      true netCDF file using the <verb>NetCDFFile</verb> class
	      method <verb>h5tonc</verb> (if
	      <verb>Scientific.IO.NetCDF</verb> is installed). The
	      <em>unlimited</em> dimension must be the first (for all
	      variables in the file) in order to use the
	      <verb>h5tonc</verb> method.  Data can also be imported
	      from a true netCDF file and saved in an HDF5
	      <verb>tables.NetCDF</verb> file using the
	      <verb>nctoh5</verb> class method.
	    </item>

	    <item>In <verb>tables.NetCDF</verb> a list of attributes
	      corresponding to global netCDF attributes defined in
	      the file can be obtained with the <verb>NetCDFFile
		ncattrs </verb>method.  Similarly, netCDF variable
	      attributes can be obtained with the
	      <verb>NetCDFVariable</verb> <verb>ncattrs</verb>
	      method.  These functions are not available in the
	      <verb>Scientific.IO.NetCDF</verb> API.
	    </item>

	    <item>You should not define <verb>tables.NetCDF</verb>
	      global or variable attributes that start with
	      <verb>_NetCDF_</verb>. Those names are reserved for
	      internal use.
	    </item>

	    <item>Output similar to 'ncdump -h' can be obtained by
	      simply printing a <verb>tables.NetCDF</verb>
	      <verb>NetCDFFile</verb> instance.
	    </item>

	  </enumerate>

	</section> <!-- Differences tables.NetCDF and Scientific.IO.NetCDF -->

      </chapter> <!-- NetCDF module -->


    </part> <!-- PyTables Complementary Modules -->

    <part>
      <heading>Appendixes</heading>

    <appendix>
      <chapter id="datatypesSupported">
	<heading>Supported data types in <visual
	markup="tt">PyTables</visual></heading>

	<p>The Table, Array, CArray, VLArray and EArray classes can
	  all handle the complete set of data types supported by the
	  <visual markup="tt">numarray</visual> package (see <cite
	  refid="Numarray"></cite>), <visual
	  markup="tt">NumPy</visual> (see <cite refid="NumPy"></cite>)
	  and <visual markup="tt">Numeric</visual> (see <cite
	  refid="Numeric"></cite>) in Python.  The data types for
	  table fields can be set via the constructor for the
	  <verb>Col</verb> class and its descendants (<ref
	  refid="ColClassDescr">see</ref>) while array elements can be
	  set through the use of the <verb>Atom</verb> class and its
	  descendants (<ref refid="AtomClassDescr">see</ref>).
	</p>
	<p>In addition to those data types, PyTables' Table, VLArray
	  and EArray classes do support some <em>aliasing</em> data
	  types for their columns and atoms.  Each one of these
	  aliasing types corresponds to one <visual
	  markup="tt">numarray</visual> type, but they also have
	  special meanings for PyTables.  They can be seen as the
	  ordinary types they are associated with, plus some
	  additional meaning.  Since they do not exist as <visual
	  markup="tt">numarray</visual> types, they can only be
	  specified to PyTables using strings.
	</p>
	<p>Currently, the only supported aliasing data type is <em>Time</em>.
	  Two kinds of time values can be handled:
	  4-byte signed integer and 8-byte double precision floating point.
	  Both of them reflect the number of seconds since the Unix Epoch,
	  i.e. Jan 1 00:00:00 UTC 1970.
	  Their types correspond to <visual markup="tt">numarray</visual>'s
	  Int32 and Float64, respectively.
	  Time values are stored in the HDF5 file
	  using the <visual markup="tt">H5T_TIME</visual> class.
	  Integer times are stored as is,
	  while floating point times are split into
	  two signed integer values representing seconds and microseconds
	  (beware: smaller decimals will be lost!).
	</p>
	<p>PyTables also supports HDF5 <visual
	  markup="tt">H5T_ENUM</visual> <em>enumerations</em>
	  (restricted sets of unique name and unique value pairs).
	  The <visual markup="tt">numarray</visual> representation of
	  an enumerated value depends on the concrete base type used
	  to store the enumeration in the HDF5 file.  Enumerations are
	  similar to aliasing data types in the sense that enumerated
	  data is handled as regular <visual
	  markup="tt">numarray</visual> data.  Enumerations are also
	  specified to PyTables using a string type, with an additional
	  <verb>Enum</verb> (see <ref refid="EnumClassDescr"></ref>)
	  instance.
	</p>
	<p>Currently, only scalar integer values (both signed and
	  unsigned) are supported in enumerations.  This restriction
	  may be lifted when HDF5 supports other kinds on enumerated
	  values.
	</p>
	<p>A quick reference to the complete set of data types supported by
	  PyTables is given in <ref refid="datatypesSupported">table</ref>.
	</p>

	<table id="datatypesSupportedTable">
	  <tabular preamble="lllcl">
	  <tabhead>
	    <srow>Type Code | Description | C Type | Size (in bytes) |
	      Python Counterpart </srow>
	  </tabhead>
	  <tabbody>
	    <srow>Bool | boolean | unsigned char | 1 | Boolean </srow>
	    <srow>Int8 | 8-bit integer | signed char | 1 | Integer </srow>
	    <srow>UInt8 | 8-bit unsigned integer | unsigned char | 1 | Integer </srow>
	    <srow>Int16 | 16-bit integer | short | 2 | Integer </srow>
	    <srow>UInt16 | 16-bit unsigned integer | unsigned short | 2 | Integer </srow>
	    <srow>Int32 | integer | int | 4  | Integer </srow>
	    <srow>UInt32 | unsigned integer | unsigned int | 4 | Long </srow>
	    <srow>Int64 | 64-bit integer | long long | 8 | Long </srow>
	    <srow>UInt64 | unsigned 64-bit integer | unsigned long long | 8 | Long </srow>
	    <srow>Float32 | single-precision float | float | 4 | Float </srow>
	    <srow>Float64 | double-precision float | double | 8 | Float </srow>
	    <srow>Complex32 | single-precision complex | struct {float r, i;} | 8 | Complex </srow>
	    <srow>Complex64 | double-precision complex | struct {double r, i;} | 16 | Complex </srow>
	    <srow>CharType | arbitrary length string | char[] | * | String </srow>
	    <srow>Time32 | integer time | POSIX's time_t | 4 | Integer </srow>
	    <srow>Time64 | floating point time | POSIX's struct timeval | 8 | Float </srow>
	    <srow>Enum | enumerated value | enum | - | - </srow>
	  </tabbody>
	</tabular>
	  <caption>Data types supported for array elements and
	      tables columns in PyTables.
	  </caption>
	</table>

      </chapter>  <!-- Data types supported -->

      <chapter  id="NestedRecArrayClassDescr">
	<heading>Using nested record arrays</heading>

	<!-- WARNING: Remember to keep in sync with doc/text/nestedrecords.txt! -->

	<section>
	  <heading>Introduction</heading>

	  <p>Nested record arrays are a generalization of the record
	    array concept.  Basically, a nested record array is a
	    record array that supports nested datatypes.  It means
	    that columns can contain not only regular datatypes but
	    also nested datatypes.
	  </p>

	  <p>Each nested record array is a <verb>NestedRecArray</verb>
	    object in the <verb>tables.nestedrecords</verb> module.
	    Nested record arrays are intended to be as compatible as
	    possible with ordinary record arrays (in fact the
	    <verb>NestedRecArray</verb> class inherits from
	    <verb>RecArray</verb>).  As a consequence, the user can
	    deal with nested record arrays nearly in the same way that
	    he does with ordinary record arrays.
	  </p>

	  <p>The easiest way to create a nested record array is to use
	    the <verb>array()</verb> function in the
	    <verb>tables.nestedrecords</verb> module.  The only
	    difference between this function and its non-nested
	    capable analogous is that now, we <em>must</em> provide an
	    structure for the buffer being stored.  For instance:
	  </p>

	  <verbatim>
>>> from tables.nestedrecords import array
>>> nra1 = array(
...     [(1, (0.5, 1.0), ('a1', 1j)), (2, (0, 0), ('a2', 1+.1j))],
...     formats=['Int64', '(2,)Float32', ['a2', 'Complex64']])
</verbatim>

	  <p>will create a two rows nested record array with two
	    regular fields (columns), and one nested field with two
	    sub-fields.
	  </p>

	  <p>The field structure of the nested record array is
	    specified by the keyword argument <verb>formats</verb>.
	    This argument only supports sequences of strings and other
	    sequences.  Each string defines the shape and type of a
	    non-nested field.  Each sequence contains the formats of
	    the sub-fields of a nested field.  Optionally, we can also
	    pass an additional <verb>names</verb> keyword argument
	    containing the names of fields and sub-fields:
	  </p>

	  <verbatim>>>> nra2 = array(
...     [(1, (0.5, 1.0), ('a1', 1j)), (2, (0, 0), ('a2', 1+.1j))],
...     names=['id', 'pos', ('info', ['name', 'value'])],
...     formats=['Int64', '(2,)Float32', ['a2', 'Complex64']])</verbatim>

	  <p>The names argument only supports lists of strings and
	    2-tuples.  Each string defines the name of a non-nested
	    field.  Each 2-tuple contains the name of a nested field
	    and a list describing the names of its sub-fields.  If the
	    <verb>names</verb> argument is not passed then all fields
	    are automatically named (<verb>c1</verb>, <verb>c2</verb>
	    etc. on each nested field) so, in our first example, the
	    fields will be named as <verb>['c1', 'c2', ('c3', ['c1',
	    'c2'])]</verb>.
	  </p>

	  <p>Another way to specify the nested record array structure
	    is to use the <verb>descr</verb> keyword argument:
	  </p>

	  <verbatim>>>> nra3 = array(
...     [(1, (0.5, 1.0), ('a1', 1j)), (2, (0, 0), ('a2', 1+.1j))],
...     descr=[('id', 'Int64'), ('pos', '(2,)Float32'),
...            ('info', [('name', 'a2'), ('value', 'Complex64')])])
>>>
>>> nra3
array(
[(1L, array([ 0.5,  1. ], type=Float32), ('a1', 1j)),
(2L, array([ 0.,  0.], type=Float32), ('a2', (1+0.10000000000000001j)))],
descr=[('id', 'Int64'), ('pos', '(2,)Float32'), ('info', [('name', 'a2'),
('value', 'Complex64')])],
shape=2)
>>>
]</verbatim>

	  <p>The <verb>descr</verb> argument is a list of 2-tuples,
	    each of them describing a field.  The first value in a
	    tuple is the name of the field, while the second one is a
	    description of its structure.  If the second value is a
	    string, it defines the format (shape and type) of a
	    non-nested field.  Else, it is a list of 2-tuples
	    describing the sub-fields of a nested field.
	  </p>

	  <p>As you can see, the <verb>descr</verb> list is a mix of
	    the <verb>names</verb> and <verb>formats</verb> arguments.
	    In fact, this argument is intended to replace
	    <verb>formats</verb> and <verb>names</verb>, so they
	    cannot be used at the same time.
	  </p>

	  <p>Of course the structure of all three keyword arguments
	    must match that of the elements (rows) in the
	    <verb>buffer</verb> being stored.
	  </p>
	  
	  <p>Sometimes it is convenient to create nested arrays by
	    processing a set of columns. In these cases the function
	    <verb>fromarrays</verb> comes handy. This function works in a very
	    similar way to the array function, but the passed buffer
	    is a list of columns. For instance:
	  </p>

	  <verbatim>
>>> from tables.nestedrecords import fromarrays
>>> nra = fromarrays([[1, 2], [4, 5]], descr=[('x', 'f8'),('y', 'f4')])
>>>
>>> nra
array(
[(1.0, 4.0),
(2.0, 5.0)],
descr=[('x', 'f8'), ('y', 'f4')],
shape=2)
	  </verbatim>

	  <p>Columns can be passed as nested arrays, what makes really
	  	straightforward to combine different nested arrays to get a new one, as
		you can see in the following examples:
	  </p>

	  <verbatim>>>> nra1 = fromarrays([nra, [7, 8]], descr=[('2D', [('x', 'f8'), ('y', 'f4')]),
>>> ... ('z', 'f4')])
>>>
>>> nra1
array(
[((1.0, 4.0), 7.0),
((2.0, 5.0), 8.0)],
descr=[('2D', [('x', 'f8'), ('y', 'f4')]), ('z', 'f4')],
shape=2)
>>>
>>> nra2 = fromarrays([nra1.field('2D/x'), nra1.field('z')], descr=[('x', 'f8'),
('z', 'f4')])
>>>
>>> nra2
array(
[(1.0, 7.0),
(2.0, 8.0)],
descr=[('x', 'f8'), ('z', 'f4')],
shape=2)
	  </verbatim>

	  <p>Finally it's worth to mention a small group of utility
	    functions, makeFormats, makeNames and makeDescr, that can
	    be useful to obtain the structure specification to be used
	    with array and fromarrays functions.  Given a description
	    list, makeFormats gets the corresponding formats list. In
	    the same way makeNames gets the names list. On the other
	    hand the descr list can be obtained from formats and names
	    lists using the makeDescr function. For example:
	  </p>

	  <verbatim>
>>> from tables.nestedrecords import makeDescr, makeFormats, makeNames
>>> descr =[('2D', [('x', 'f8'), ('y', 'f4')]),('z', 'f4')]
>>>
>>> formats = makeFormats(descr)
>>> formats
[['f8', 'f4'], 'f4']
>>> names = makeNames(descr)
>>> names
[('2D', ['x', 'y']), 'z']
>>> d1 = makeDescr(formats, names)
>>> d1
[('2D', [('x', 'f8'), ('y', 'f4')]), ('z', 'f4')]
>>> # If no names are passed then they are automatically generated
>>> d2 = makeDescr(formats)
>>> d2
[('c1', [('c1', 'f8'), ('c2', 'f4')]),('c2', 'f4')]
	  </verbatim>

	</section>

	<section>
	  <heading><visual markup="tt">NestedRecArray</visual>
	    methods</heading>

	  <p>To access the fields in the nested record array use the
	    <verb>field()</verb> method:
	  </p>

	  <verbatim>
>>> print nra2.field('id')
[1, 2]
>>>
	  </verbatim>

	  <p>The <verb>field()</verb> method accepts also names of
	    sub-fields.  It will consist of several field name
	    components separated by the string <verb>'/'</verb>, for
	    instance:
	  </p>

	  <verbatim>
>>> print nra2.field('info/name')
['a1', 'a2']
>>>
	  </verbatim>

	  <p>Eventually, the top level fields of the nested recarray
	    can be accessed passing an integer argument to the
	    <verb>field()</verb> method:
	  </p>

	  <verbatim>
>>> print nra2.field(1)
[[ 0.5 1. ] [ 0.  0. ]]
>>>
	  </verbatim>

	  <p>An alternative to the <verb>field()</verb> method is the
	    use of the <verb>fields</verb> attribute.  It is intended
	    mainly for interactive usage in the Python console.  For
	    example:
	  </p>

	  <verbatim>
>>> nra2.fields.id
[1, 2]
>>> nra2.fields.info.fields.name
['a1', 'a2']
>>>
	  </verbatim>

	  <p>Rows of nested recarrays can be read using the typical
	    index syntax.  The rows are retrieved as
	    <verb>NestedRecord</verb> objects:
	  </p>

	  <verbatim>
<![CDATA[>>> print nra2[0]
(1L, array([ 0.5,  1. ], type=Float32), ('a1', 1j))
>>>
>>> nra2[0].__class__
<class tables.nestedrecords.NestedRecord at 0x413cbb9c>
]]>
	  </verbatim>

	  <p>Slicing is also supported in the usual way:</p>

	  <verbatim>
>>> print nra2[0:2]
NestedRecArray[
(1L, array([ 0.5,  1. ], type=Float32), ('a1', 1j)),
(2L, array([ 0.,  0.], type=Float32), ('a2', (1+0.10000000000000001j)))
]
>>>
	  </verbatim>

	  <p>Another useful method is <verb>asRecArray()</verb>.
	    It converts a nested array to a non-nested equivalent array.
	  </p>

	  <p>This method creates a new vanilla <verb>RecArray</verb>
	    instance equivalent to this one by flattening its fields.
	    Only bottom-level fields included in the array.
	    Sub-fields are named by pre-pending the names of their
	    parent fields up to the top-level fields, using
	    <verb>'/'</verb> as a separator.  The data area of the
	    array is copied into the new one.  For example, calling
	    <verb>nra3.asRecArray()</verb> would return the same array
	    as calling:
	  </p>

	  <verbatim>
>>> ra = numarray.records.array(
...     [(1, (0.5, 1.0), 'a1', 1j), (2, (0, 0), 'a2', 1+.1j)],
...     names=['id', 'pos', 'info/name', 'info/value'],
...     formats=['Int64', '(2,)Float32', 'a2', 'Complex64'])
	  </verbatim>

	  <p>Note that the shape of multidimensional fields is kept.
	  </p>

	</section>

	<section>
	  <heading><visual markup="tt">NestedRecord</visual> objects</heading>

	  <p>Each element of the nested record array is a
	    <verb>NestedRecord</verb>, i.e. a <verb>Record</verb> with
	    support for nested datatypes.  As said before, we can do
	    indexing as usual:
	  </p>

	  <verbatim>
>>> print nra1[0]
(1, (0.5, 1.0), ('a1', 1j))
>>>
	  </verbatim>

	  <p>Using <verb>NestedRecord</verb> objects is quite similar
	    to using <verb>Record</verb> objects.  To get the data of
	    a field we use the <verb>field()</verb> method.  As an
	    argument to this method we pass a field name.  Sub-field
	    names can be passed in the way described for
	    <verb>NestedRecArray.field()</verb>.  The
	    <verb>fields</verb> attribute is also present and works as
	    it does in <verb>NestedRecArray</verb>.
	  </p>

	  <p>Field data can be set with the <verb>setField()</verb>
	    method.  It takes two arguments, the field name and its
	    value.  Sub-field names can be passed as usual.  Finally,
	    the <verb>asRecord()</verb> method converts a nested
	    record into a non-nested equivalent record.
	  </p>
	</section>
      </chapter>

      <chapter id="PTutilities">
	<heading>Utilities</heading>

	<p><verb>PyTables</verb> comes with a couple of utilities that
	  make the life easier to the user. One is called
	  <verb>ptdump</verb> and lets you see the contents of a
	  <verb>PyTables</verb> file (or generic <verb>HDF5</verb>
	  file, if supported). The other one is named
	  <verb>ptrepack</verb> that allows to (recursively) copy
	  sub-hierarchies of objects present in a file into another
	  one, changing, if desired, some of the filters applied to
	  the leaves during the copy process.
	</p>

	<p>Normally, these utilities will be installed somewhere in
	  your PATH during the process of installation of the
	  <verb>PyTables</verb> package, so that you can invoke them
	  from any place in your file system after the installation has
	  successfully finished.
	</p>

	<section id="ptdumpDescr">
	  <heading>ptdump</heading>

	  <p>As has been said before, <verb>ptdump</verb> utility
	    allows you look into the contents of your
	    <verb>PyTables</verb> files. It lets you see not only the
	    data but also the metadata (that is, the
	    <em>structure</em> and additional information in the form
	    of <em>attributes</em>).
	  </p>

	  <subsection>
	    <heading>Usage</heading>

	    <p>For instructions on how to use it, just pass the
	      <verb>-h</verb> flag to the command:

	      <verbatim>
$ ptdump -h
	      </verbatim>

	      to see the message usage:

	      <verbatim>
usage: ptdump [-R start,stop,step] [-a] [-h] [-d] [-v] file[:nodepath]
  -R RANGE -- Select a RANGE of rows in the form "start,stop,step"
  -a -- Show attributes in nodes (only useful when -v or -d are active)
  -c -- Show info of columns in tables (only useful when -v or -d are active)
  -i -- Show info of indexed columns (only useful when -v or -d are active)
  -d -- Dump data information on leaves
  -h -- Print help on usage
  -v -- Dump more meta-information on nodes
	      </verbatim>
	    </p>

	  </subsection>
	  <subsection>
	    <heading>A small tutorial on <visual
		markup="tt">ptdump</visual>
	    </heading>

	    <p>Let's suppose that we want to know only the
	      <em>structure</em> of a file. In order to do that, just
	      don't pass any flag, just the file as parameter:

	      <verbatim>
$ ptdump vlarray1.h5
Filename: 'vlarray1.h5' Title: '' , Last modif.: 'Fri Feb  6 19:33:28 2004' ,
 rootUEP='/', filters=Filters(), Format version: 1.2
/ (Group) ''
/vlarray1 (VLArray(4,), shuffle, zlib(1)) 'ragged array of ints'

	      </verbatim>

	      we can see that the file contains a just a leaf object
	      called <verb>vlarray1</verb>, that is an instance of
	      <verb>VLArray</verb>, has 4 rows, and two filters has
	      been used in order to create it: <verb>shuffle</verb>
	      and <verb>zlib</verb> (with a compression level of 1).
	    </p>

	    <p>Let's say we want more meta-information. Just add the
	      <verb>-v</verb> (verbose) flag:

	      <verbatim>
$ ptdump -v vlarray1.h5
/ (Group) ''
  children := ['vlarray1' (VLArray)]
/vlarray1 (VLArray(4,), shuffle, zlib(1)) 'ragged array of ints'
  atom = Atom(type=Int32, shape=1, flavor='Numeric')
  nrows = 4
  flavor = 'Numeric'
  byteorder = 'little'
	      </verbatim>
	      so we can see more info about the atoms that are the
	      components of the <verb>vlarray1</verb> dataset,
	      i.e. they are scalars of type <verb>Int32</verb> and
	      with <verb>Numeric</verb> <em>flavor</em>.
	    </p>

	    <p>If we want information about the attributes on the
	      nodes, we must add the <verb>-a</verb> flag:

	      <verbatim>
$ ptdump -va vlarray1.h5
/ (Group) ''
  children := ['vlarray1' (VLArray)]
  /._v_attrs (AttributeSet), 5 attributes:
   [CLASS := 'GROUP',
    FILTERS := None,
    PYTABLES_FORMAT_VERSION := '1.2',
    TITLE := '',
    VERSION := '1.0']
/vlarray1 (VLArray(4,), shuffle, zlib(1)) 'ragged array of ints'
  atom = Atom(type=Int32, shape=1, flavor='Numeric')
  nrows = 4
  flavor = 'Numeric'
  byteorder = 'little'
  /vlarray1.attrs (AttributeSet), 4 attributes:
   [CLASS := 'VLARRAY',
    FLAVOR := 'Numeric',
    TITLE := 'ragged array of ints',
    VERSION := '1.0']
	      </verbatim>
	    </p>

	    <p>Let's have a look at the real data:

	      <verbatim>
$ ptdump -d vlarray1.h5
/ (Group) ''
/vlarray1 (VLArray(4,), shuffle, zlib(1)) 'ragged array of ints'
  Data dump:
[array([5, 6]), array([5, 6, 7]), array([5, 6, 9, 8]), array([ 5,  6,  9, 10, 12])]
	      </verbatim>
	      we see here a data dump of the 4 rows in
	      <verb>vlarray1</verb> object, in the form of a
	      list. Because the object is a VLA, we see a different
	      number of integers on each row.
	    </p>

	    <p>Say that we are interested only on a specific <em>row
		range</em> of the <verb>/vlarray1</verb> object:
	      <verbatim>
ptdump -R2,4 -d vlarray1.h5:/vlarray1
/vlarray1 (VLArray(4,), shuffle, zlib(1)) 'ragged array of ints'
  Data dump:
[array([5, 6, 9, 8]), array([ 5,  6,  9, 10, 12])]
	      </verbatim>
	      Here, we have specified the range of rows between 2 and
	      4 (the upper limit excluded, as usual in Python). See
	      how we have selected only the <verb>/vlarray1</verb>
	      object for doing the dump
	      (<verb>vlarray1.h5:/vlarray1</verb>).
	    </p>

	    <p>Finally, you can mix several information at once:
	      <verbatim>
$ ptdump -R2,4 -vad vlarray1.h5:/vlarray1
/vlarray1 (VLArray(4,), shuffle, zlib(1)) 'ragged array of ints'
  atom = Atom(type=Int32, shape=1, flavor='Numeric')
  nrows = 4
  flavor = 'Numeric'
  byteorder = 'little'
  /vlarray1.attrs (AttributeSet), 4 attributes:
   [CLASS := 'VLARRAY',
    FLAVOR := 'Numeric',
    TITLE := 'ragged array of ints',
    VERSION := '1.0']
  Data dump:
[array([5, 6, 9, 8]), array([ 5,  6,  9, 10, 12])]
	      </verbatim>
	    </p>
	  </subsection>
	</section> <!-- ptdump -->

	<section id="ptrepackDescr">
	  <heading>ptrepack</heading>

	  <p>This utility is a very powerful one and lets you copy any
	    leaf, group or complete subtree into another file. During
	    the copy process you are allowed to change the filter
	    properties if you want so. Also, in the case of duplicated
	    pathnames, you can decide if you want to overwrite already
	    existing nodes on the destination file. Generally
	    speaking, <verb>ptrepack</verb> can be useful in may
	    situations, like replicating a subtree in another file,
	    change the filters in objects and see how affect this to
	    the compression degree or I/O performance, consolidating
	    specific data in repositories or even <em>importing</em>
	    generic <verb>HDF5</verb> files and create true
	    <verb>PyTables</verb> counterparts.
	  </p>

	  <subsection>
	    <heading>Usage</heading>

	    <p>For instructions on how to use it, just pass the
	      <verb>-h</verb> flag to the command:

	      <verbatim>
$ ptrepack -h
	      </verbatim>

	      to see the message usage:

	      <verbatim>
usage: ptrepack [-h] [-v] [-o] [-R start,stop,step] [--non-recursive]
  [--dest-title=title] [--dont-copyuser-attrs] [--overwrite-nodes]
  [--complevel=(0-9)] [--complib=lib] [--shuffle=(0|1)]
  [--fletcher32=(0|1)] [--keep-source-filters] 
  sourcefile:sourcegroup destfile:destgroup
 -h -- Print usage message.
 -v -- Show more information.
 -o -- Overwite destination file.
 -R RANGE -- Select a RANGE of rows (in the form "start,stop,step")
     during the copy of *all* the leaves.
 --non-recursive -- Do not do a recursive copy. Default is to do it.
 --dest-title=title -- Title for the new file (if not specified,
     the source is copied).
 --dont-copy-userattrs -- Do not copy the user attrs (default is to do it)
 --overwrite-nodes -- Overwrite destination nodes if they exist. Default is
     to not overwrite them.
 --complevel=(0-9) -- Set a compression level (0 for no compression, which
     is the default).
 --complib=lib -- Set the compression library to be used during the copy.
     lib can be set to "zlib", "lzo", "ucl" or "bzip2". Defaults to "zlib".
 --shuffle=(0|1) -- Activate or not the shuffling filter (default is active
     if complevel>0).
 --fletcher32=(0|1) -- Whether to activate or not the fletcher32 filter (not
     active by default).
 --keep-source-filters -- Use the original filters in source files. The
     default is not doing that if any of --complevel, --complib, --shuffle
     or --fletcher32 option is specified.
	      </verbatim>
	    </p>
	  </subsection>
	  <subsection>
	    <heading>A small tutorial on <visual
		markup="tt">ptrepack</visual>
	    </heading>

	    <p>Imagine that we have ended the tutorial 1 (see the
	      output of <verb>examples/tutorial1-1.py</verb>), and we
	      want to copy our reduced data (i.e. those datasets that
	      hangs from the <verb>/column</verb> group) to another
	      file. First, let's remember the content of the
	      <verb>examples/tutorial1.h5</verb>:

	      <verbatim>
$ ptdump tutorial1.h5
Filename: 'tutorial1.h5' Title: 'Test file' , Last modif.: 'Fri Feb  6
  19:33:28 2004' , rootUEP='/', filters=Filters(), Format version: 1.2
/ (Group) 'Test file'
/columns (Group) 'Pressure and Name'
/columns/name (Array(3,)) 'Name column selection'
/columns/pressure (Array(3,)) 'Pressure column selection'
/detector (Group) 'Detector information'
/detector/readout (Table(10L,)) 'Readout example'
	      </verbatim>

	      Now, copy the <verb>/columns</verb> to other
	      non-existing file. That's easy:

	      <verbatim>
$ ptrepack tutorial1.h5:/columns reduced.h5
	      </verbatim>
	      That's all. Let's see the contents of the newly created
	      <verb>reduced.h5</verb> file:

	      <verbatim>
$ ptdump reduced.h5
Filename: 'reduced.h5' Title: '' , Last modif.: 'Fri Feb 20 15:26:47 2004' ,
 rootUEP='/', filters=Filters(), Format version: 1.2
/ (Group) ''
/name (Array(3,)) 'Name column selection'
/pressure (Array(3,)) 'Pressure column selection'
	      </verbatim>
	      so, you have copied the children of
	      <verb>/columns</verb> group into the <em>root</em> of
	      the <verb>reduced.h5</verb> file.
	    </p>

	    <p>Now, you suddenly realized that what you intended to do
	      was to copy all the hierarchy, the group
	      <verb>/columns</verb> itself included. You can do that
	      by just specifying the destination group:

	      <verbatim>
$ ptrepack tutorial1.h5:/columns reduced.h5:/columns
ptdump reduced.h5
Filename: 'reduced.h5' Title: '' , Last modif.: 'Fri Feb 20 15:39:15 2004' ,
 rootUEP='/', filters=Filters(), Format version: 1.2
/ (Group) ''
/name (Array(3,)) 'Name column selection'
/pressure (Array(3,)) 'Pressure column selection'
/columns (Group) ''
/columns/name (Array(3,)) 'Name column selection'
/columns/pressure (Array(3,)) 'Pressure column selection'
	      </verbatim>
	      OK. Much better. But you want to get rid of the existing
	      nodes on the new file. You can achieve this by adding
	      the -o flag:

	      <verbatim>
$ ptrepack -o tutorial1.h5:/columns reduced.h5:/columns
$ ptdump reduced.h5
Filename: 'reduced.h5' Title: '' , Last modif.: 'Fri Feb 20 15:41:57 2004' ,
 rootUEP='/', filters=Filters(), Format version: 1.2
/ (Group) ''
/columns (Group) ''
/columns/name (Array(3,)) 'Name column selection'
/columns/pressure (Array(3,)) 'Pressure column selection'
	      </verbatim>
	      where you can see how the old contents of the
	      <verb>reduced.h5</verb> file has been overwritten.
	    </p>

	    <p>You can copy just one single node in the repacking
	      operation and change its name in destination:

	      <verbatim>
$ ptrepack tutorial1.h5:/detector/readout reduced.h5:/rawdata
$ ptdump reduced.h5
Filename: 'reduced.h5' Title: '' , Last modif.: 'Fri Feb 20 15:52:22 2004',
 rootUEP='/', filters=Filters(), Format version: 1.2
/ (Group) ''
/rawdata (Table(10L,)) 'Readout example'
/columns (Group) ''
/columns/name (Array(3,)) 'Name column selection'
/columns/pressure (Array(3,)) 'Pressure column selection'
	      </verbatim>
	      where the <verb>/detector/readout</verb> has been copied
	      to <verb>/rawdata</verb> in destination.
	    </p>

	    <p>We can change the filter properties as well:
	      <verbatim>
$ ptrepack --complevel=1 tutorial1.h5:/detector/readout reduced.h5:/rawdata
Problems doing the copy from 'tutorial1.h5:/detector/readout' to
'reduced.h5:/rawdata'
The error was --> exceptions.ValueError: The destination
 (/rawdata (Table(10L,)) 'Readout example') already exists.
 Assert the overwrite parameter if you really want to overwrite it.
The destination file looks like:
Filename: 'reduced.h5' Title: ''; Last modif.: 'Fri Feb 20 15:52:22 2004';
 rootUEP='/'; filters=Filters(), Format version: 1.2
/ (Group) ''
/rawdata (Table(10L,)) 'Readout example'
/columns (Group) ''
/columns/name (Array(3,)) 'Name column selection'
/columns/pressure (Array(3,)) 'Pressure column selection'

Traceback (most recent call last):
  File "../utils/ptrepack", line 358, in ?
    start=start, stop=stop, step=step)
  File "../utils/ptrepack", line 111, in copyLeaf
    raise RuntimeError, "Please, check that the node names are not
  duplicated in destination, and if so, add the --overwrite-nodes flag
  if desired."
RuntimeError: Please, check that the node names are not duplicated in
 destination, and if so, add the --overwrite-nodes flag if desired.
	    </verbatim>
	      ooops!. We ran into problems: we forgot that
	      <verb>/rawdata</verb> pathname already existed
	      in destination file. Let's add the
	      <verb>--overwrite-nodes</verb>, as the verbose
	      error suggested:
	      <verbatim>
$ ptrepack --overwrite-nodes --complevel=1 tutorial1.h5:/detector/readout 
reduced.h5:/rawdata
$ ptdump reduced.h5
Filename: 'reduced.h5' Title: ''; Last modif.: 'Fri Feb 20 16:02:20 2004';
 rootUEP='/'; filters=Filters(), Format version: 1.2
/ (Group) ''
/rawdata (Table(10L,), shuffle, zlib(1)) 'Readout example'
/columns (Group) ''
/columns/name (Array(3,)) 'Name column selection'
/columns/pressure (Array(3,)) 'Pressure column selection'
	      </verbatim>
	      you can check how the filter properties has been changed
	      for the <verb>/rawdata</verb> table. Check as the other
	      nodes still exists.
	    </p>

	    <p>Finally, let's copy a <em>slice</em> of the
	      <verb>readout</verb> table in origin to destination,
	      under a new group called <verb>/slices</verb> and with
	      the name, for example, <verb>aslice</verb>:
	      <verbatim>
$ ptrepack -R1,8,3 tutorial1.h5:/detector/readout reduced.h5:/slices/aslice
$ ptdump reduced.h5
Filename: 'reduced.h5' Title: ''; Last modif.: 'Fri Feb 20 16:17:13 2004';
 rootUEP='/'; filters=Filters(); Format version: 1.2
/ (Group) ''
/rawdata (Table(10L,), shuffle, zlib(1)) 'Readout example'
/columns (Group) ''
/columns/name (Array(3,)) 'Name column selection'
/columns/pressure (Array(3,)) 'Pressure column selection'
/slices (Group) ''
/slices/aslice (Table(3L,)) 'Readout example'
	      </verbatim>
	      note how only 3 rows of the original
	      <verb>readout</verb> table has been copied to the new
	      <verb>aslice</verb> destination. Note as well how the
	      previously inexistent <verb>slices</verb> group has been
	      created in the same operation.
	    </p>
	  </subsection>
	</section> <!-- ptrepack -->

	<section id="nctoh5Descr">
	  <heading>nctoh5</heading>

	  <p>This tool is able to convert a file in <url
	    name="http://www.unidata.ucar.edu/packages/netcdf/"><verb>NetCDF</verb></url>
	    format to a <verb>PyTables</verb> file (and hence, to a
	    HDF5 file). However, for this to work, you will need the
	    NetCDF interface for Python that comes with the excellent
	    <verb>Scientific Python</verb> (see <cite
	    refid="scientificpythonRef"></cite>) package. This script
	    was initially contributed by Jeff Whitaker. It has been
	    updated to support selectable filters from the command
	    line and some other small improvements.
	  </p>

	  <p>If you want other file formats to be converted to
	    <verb>PyTables</verb>, have a look at the
	    <verb>SciPy</verb> (see <cite refid="scipyRef"></cite>)
	    project (subpackage <verb>io</verb>), and look for
	    different methods to import them into
	    <verb>NumPy/Numeric/numarray</verb> objects. Following the
	    <verb>SciPy</verb> documentation, you can read, among
	    other formats, ASCII files (<verb>read_array</verb>),
	    binary files in C or Fortran (<verb>fopen</verb>) and
	    <verb>MATLAB</verb> (version 4, 5 or 6) files
	    (<verb>loadmat</verb>). Once you have the content of your
	    files as <verb>NumPy/Numeric/numarray</verb> objects, you
	    can save them as regular <verb>(E)Arrays</verb> in
	    <verb>PyTables</verb> files. Remember, if you end with a
	    nice conversor, do not forget to contribute it back to the
	    community. Thanks!
	  </p>

	  <subsection>
	    <heading>Usage</heading>

	    <p>For instructions on how to use it, just pass the
	      <verb>-h</verb> flag to the command:

	      <verbatim>
$ nctoh5 -h
	      </verbatim>

	      to see the message usage:

	      <verbatim>
usage: nctoh5 [-h] [-v] [-o] [--complevel=(0-9)] [--complib=lib]
 [--shuffle=(0|1)] [--fletcher32=(0|1)] [--unpackshort=(0|1)]
 [--quantize=(0|1)] netcdffilename hdf5filename
 -h -- Print usage message.
 -v -- Show more information.
 -o -- Overwite destination file.
 --complevel=(0-9) -- Set a compression level (0 for no compression, which
     is the default).
 --complib=lib -- Set the compression library to be used during the copy.
     lib can be set to "zlib", "lzo", "ucl" or "bzip2". Defaults to "zlib".
 --shuffle=(0|1) -- Activate or not the shuffling filter (default is active
     if complevel>0).
 --fletcher32=(0|1) -- Whether to activate or not the fletcher32 filter (not
     active by default).
 --unpackshort=(0|1) -- unpack short integer variables to float variables
     using scale_factor and add_offset netCDF variable attributes
     (not active by default).
 --quantize=(0|1) -- quantize data to improve compression using
     least_significant_digit netCDF variable attribute (not active by default).
     See http://www.cdc.noaa.gov/cdc/conventions/cdc_netcdf_standard.shtml
     for further explanation of what this attribute means.

	      </verbatim>

	      If you have followed the small tutorial on the
	      <verb>ptrepack</verb> utility (see <ref
	      refid="ptrepackDescr"></ref>), you should easily realize
	      what most of the different flags would mean.

	    </p>
	  </subsection>
	</section> <!-- nctoh5 -->
      </chapter>

      <chapter id="PyTablesInternalFormat">
	<heading><visual markup="tt">PyTables</visual> File
	  Format</heading>

	<p><verb>PyTables</verb> has a powerful capability to deal
	  with native HDF5 files created with another tools. However,
	  there are situations were you may want to create truly
	  native <verb>PyTables</verb> files with those tools while
	  retaining fully compatibility with <verb>PyTables</verb>
	  format. That is perfectly possible, and in this appendix is
	  presented the format that you should endow to your
	  own-generated files in order to get a fully
	  <verb>PyTables</verb> compatible file.
	</p>

	<p>We are going to describe the <visual markup="bf">1.6
	  version of <verb>PyTables</verb> file format</visual>
	  (introduced in <verb>PyTables</verb> version 1.3). At this
	  stage, this file format is considered stable enough to do
	  not introduce significant changes during a reasonable amount
	  of time. As time goes by, some changes will be introduced
	  (and documented here) in order to cope with new
	  necessities. However, the changes will be carefully pondered
	  so as to ensure backward compatibility whenever is possible.
	</p>

	<p>A <verb>PyTables</verb> file is composed with arbitrarily
	  large amounts of HDF5 groups (<verb>Groups</verb> in
	  <verb>PyTables</verb> naming scheme) and datasets
	  (<verb>Leaves</verb> in <verb>PyTables</verb> naming
	  scheme). For groups, the only requirements are that they
	  must have some <em>system attributes</em> available. By
	  convention, system attributes in <verb>PyTables</verb> are
	  written in upper case, and user attributes in lower case but
	  this is not enforced by the software. In the case of
	  datasets, besides the mandatory system attributes, some
	  conditions are further needed in their storage layout, as
	  well as in the datatypes used in there, as we will see
	  shortly.
	</p>

	<p>As a final remark, you can use any filter as you want to
	  create a <verb>PyTables</verb> file, provided that the
	  filter is a standard one in HDF5, like <em>zlib</em>,
	  <em>shuffle</em> or <em>szip</em> (although the last one
	  can not be used from within <verb>PyTables</verb> to create a
	  new file, datasets compressed with szip can be read, because
	  it is the HDF5 library which do the decompression
	  transparently).
	</p>

	<section>
	  <heading>Mandatory attributes for a <visual
	      markup="tt">File</visual></heading>

	  <p>The <verb>File</verb> object is, in fact, an special HDF5
	    <em>group</em> structure that is <em>root</em> for the
	    rest of the objects on the object tree. The next
	    attributes are mandatory for the HDF5 <em>root group</em>
	    structure in <verb>PyTables</verb> files:
	  </p>

	  <description>
	    <term>CLASS</term> <item>This attribute should always be
	    set to <verb>'GROUP'</verb> for group structures.</item>
	    <term>PYTABLES_FORMAT_VERSION</term> <item>It represents
	    the internal format version, and currently should be set
	    to the <verb>'1.6'</verb> string.</item>
	    <term>TITLE</term> <item>A string where the user can put
	    some description on what is this group used for. </item>
	    <term>VERSION</term> <item>Should contains the string
	    <verb>'1.0'</verb>.</item>
	  </description>

	</section>
	<section>
	  <heading>Mandatory attributes for a <visual
	      markup="tt">Group</visual></heading>

	  <p>The next attributes are mandatory for <em>group</em>
	    structures:
	  </p>

	  <description>
	    <term>CLASS</term> <item>This attribute should always be
	    set to <verb>'GROUP'</verb> for group structures.</item>
	    <term>TITLE</term> <item>A string where the user can put
	    some description on what is this group used for. </item>
	    <term>VERSION</term> <item>Should contains the string
	    <verb>'1.0'</verb>.</item>
	  </description>

	</section>

	<section>
	  <heading>Mandatory attributes, storage layout and supported
	      data types for <visual markup="tt">Leaves</visual>
	      </heading>

	  <p>This depends on the kind of <verb>Leaf</verb>. The
	    format for each type follows.
	  </p>

	  <subsection id="TableFormatDescr">
	    <heading><visual markup="tt">Table</visual>
	    format</heading>

	    <subsubsection>
	      <heading>Mandatory attributes</heading>

	      <p>The next attributes are mandatory for <em>table</em>
		structures:
	      </p>

	      <description>
		<term>CLASS</term> <item>Must be set to
		  <verb>'TABLE'</verb>.</item>

		<term>TITLE</term> <item>A string where the user can
		  put some description on what is this dataset used
		  for. </item>

		<term>VERSION</term> <item>Should contain the string
		  <verb>'2.6'</verb>.</item>

		<term>FLAVOR</term> <item>This is meant to provide the
		  information about the kind of object kept in the
		  <verb>Table</verb>, i.e. when the dataset is read,
		  it will be converted to the indicated flavor. It can
		  take one the next string values:

		  <description>
		    <term>"numarray"</term> <item>The read operations
		      will return a <verb>numarray</verb> object.
		    </item>
		    <term>"numpy"</term> <item>The read operations
		      will be return as a <verb>NumPy</verb> object.
		    </item>
		  </description>

		</item>

		<term>FIELD_X_NAME</term> <item>It contains the names
		  of the different fields. The <verb>X</verb> means
		  the number of the field, zero-based (beware, order
		  do matter). You should add as many attributes of
		  this kind as fields you have in your records.
		</item>

		<term>FIELD_X_FILL</term> <item>It contains the
		  default values of the different fields. All the
		  datatypes are suported natively, except for complex
		  types that are currently serialized using
		  Pickle. The <verb>X</verb> means the number of the
		  field, zero-based (beware, order do matter). You
		  should add as many attributes of this kind as fields
		  you have in your records. These fields are meant for
		  saving the default values persistently and their
		  existence is optional.
		</item>

		<term>NROWS</term> <item>This should contain the
		  number of <em>compound</em> data type entries in the
		  dataset. It must be an <em>int</em> data type.
		</item>

	      </description>

	    </subsubsection>

	    <subsubsection>
	      <heading>Storage Layout</heading>

	      <p>A <verb>Table</verb> has a <em>dataspace</em> with a
		<em>1-dimensional chunked</em> layout.
	      </p>
	    </subsubsection>

	    <subsubsection>
	      <heading>Datatypes supported</heading>

	      <p>The datatype of the elements (rows) of
		<verb>Table</verb> must be the H5T_COMPOUND
		<em>compound</em> data type, and each of these compound
		components must be built with only the next HDF5
		data types <em>classes</em>:
	      </p>

	      <description>
		<term>H5T_BITFIELD</term> <item>This class is used to
		  represent the <verb>Bool</verb> type. Such a type
		  must be build using a H5T_NATIVE_B8 datatype,
		  followed by a HDF5 <verb>H5Tset_precision</verb>
		  call to set its precision to be just 1 bit.
		</item>
		<term>H5T_INTEGER</term> <item>This includes the next
		  data types:
		  <description>
		    <term>H5T_NATIVE_SCHAR</term> <item>This
		      represents a <em>signed char</em> C type, but it
		      is effectively used to represent an
		      <verb>Int8</verb> type.
		    </item>
		    <term>H5T_NATIVE_UCHAR</term> <item>This
		      represents an <em>unsigned char</em> C type, but
		      it is effectively used to represent an
		      <verb>UInt8</verb> type.
		    </item>
		    <term>H5T_NATIVE_SHORT</term> <item>This
		      represents a <em>short</em> C type, and
		      it is effectively used to represent an
		      <verb>Int16</verb> type.
		    </item>
		    <term>H5T_NATIVE_USHORT</term> <item>This
		      represents an <em>unsigned short</em> C type, and
		      it is effectively used to represent an
		      <verb>UInt16</verb> type.
		    </item>
		    <term>H5T_NATIVE_INT</term> <item>This represents
		      an <em>int</em> C type, and it is effectively
		      used to represent an <verb>Int32</verb> type.
		    </item>
		    <term>H5T_NATIVE_UINT</term> <item>This
		      represents an <em>unsigned int</em> C type, and
		      it is effectively used to represent an
		      <verb>UInt32</verb> type.
		    </item>
		    <term>H5T_NATIVE_LONG</term> <item>This represents
		      a <em>long</em> C type, and it is effectively
		      used to represent an <verb>Int32</verb> or an
		      <verb>Int64</verb>, depending on whether you are
		      running a 32-bit or 64-bit architecture.
		    </item>
		    <term>H5T_NATIVE_ULONG</term> <item>This
		      represents an <em>unsigned long</em> C type, and
		      it is effectively used to represent an
		      <verb>UInt32</verb> or an <verb>UInt64</verb>,
		      depending on whether you are running a 32-bit or
		      64-bit architecture.
		    </item>
		    <term>H5T_NATIVE_LLONG</term> <item>This
		      represents a <em>long long</em> C type
		      (<verb>__int64</verb>, if you are using a
		      Windows system) and it is effectively
		      used to represent an <verb>Int64</verb> type.
		    </item>
		    <term>H5T_NATIVE_ULLONG</term> <item>This
		      represents an <em>unsigned long long</em> C type
		      (beware: this type does not have a
		      correspondence on Windows systems) and it is
		      effectively used to represent an
		      <verb>UInt64</verb> type.
		    </item>
		  </description>
		</item>	<!-- H5T_INTEGER -->
		<term>H5T_FLOAT</term> <item>This includes the next
		  datatypes:
		  <description>
		    <term>H5T_NATIVE_FLOAT</term> <item>This
		      represents a <em>float</em> C type and it is
		      effectively used to represent an
		      <verb>Float32</verb> type.
		    </item>
		    <term>H5T_NATIVE_DOUBLE</term> <item>This
		      represents a <em>double</em> C type and it is
		      effectively used to represent an
		      <verb>Float64</verb> type.
		    </item>
		  </description>
		</item>
		<term>H5T_TIME</term> <item>This includes the next
		  datatypes:
		  <description>
		    <term>H5T_UNIX_D32BE</term>
		    <item>This represents a POSIX <em>time_t</em> C type
		      and it is effectively used to represent
		      a <verb>'Time32'</verb> aliasing type,
		      which corresponds to an <verb>Int32</verb> type.
		    </item>
		    <term>H5T_UNIX_D64BE</term>
		    <item>This represents a POSIX <em>struct timeval</em> C type
		      and it is effectively used to represent
		      a <verb>'Time64'</verb> aliasing type,
		      which corresponds to a <verb>Float64</verb> type.
		    </item>
		  </description>
		</item>
		<term>H5T_STRING</term> <item>The datatype used to
		  describe strings in PyTables is H5T_C_S1 (i.e. a
		  <em>string</em> C type) followed with a call to the
		  HDF5 <verb>H5Tset_size()</verb> function to set
		  their length.
		</item>
		<term>H5T_ARRAY</term> <item>This allows the
		  construction of homogeneous, multidimensional
		  arrays, so that you can include such objects in
		  compound records. The types supported as elements of
		  H5T_ARRAY data types are the ones described
		  above. Currently, <verb>PyTables</verb> does not
		  support nested H5T_ARRAY types.
		</item>
		<term>H5T_COMPOUND</term> <item>This allows the
		  support of complex numbers. Its format is described
		  below:
		  <p>
		    The H5T_COMPOUND type class contains two
		    members. Both members must have the H5T_FLOAT
		    atomic datatype class. The name of the first
		    member should be "r" and represents the real
		    part. The name of the second member should be "i"
		    and represents the imaginary part.  The
		    <em>precision</em> property of both of the
		    H5T_FLOAT members must be either 32 significant
		    bits (e.g. H5T_NATIVE_FLOAT) or 64 significant
		    bits (e.g. H5T_NATIVE_DOUBLE).  They represent
		    Complex32 and Complex64 types respectively.
		  </p>
		</item>
	      </description>

	      <p>Currently, <verb>PyTables</verb> does not support
		nested H5T_COMPOUND types, the only exception being
		supporting complex numbers in <verb>Table</verb>
		objects as described above.
	      </p>

	    </subsubsection>

	  </subsection>	<!-- Table -->

	  <subsection id="ArrayFormatDescr">
	    <heading><visual markup="tt">Array</visual>
	      format</heading>

	    <subsubsection>
	      <heading>Mandatory attributes</heading>

	      <p>The next attributes are mandatory for <em>array</em>
		structures:
	      </p>

	      <description>
		<term>CLASS</term> <item>Must be set to
		  <verb>'ARRAY'</verb>.
		</item>

		<term>FLAVOR</term> <item>This is meant to provide the
		  information about the kind of object kept in the
		  <verb>Array</verb>, i.e. when the dataset is read,
		  it will be converted to the indicated flavor. It can
		  take one the next string values:

		  <description>
		    <term>"numarray"</term> <item>The read operations
		      will return a <verb>numarray</verb> object.
		    </item>
		    <term>"numpy"</term> <item>The read operations
		      will return a <verb>NumPy</verb> object.
		    </item>
		    <term>"numeric"</term> <item>The read operations
		      will return a <verb>Numeric</verb> object.
		    </item>
		    <term>"python"</term> <item>The read operations
		      will return a Python <verb>list</verb> object in
		      case the dataset has dimensionality. If the
		      dataset is an scalar, then an appropriate Python
		      <verb>scalar</verb> will be returned instead.
		    </item>
		  </description>
		</item>

		<term>TITLE</term> <item>A string where the user can
		  put some description on what is this dataset used
		  for.
		</item>

		<term>VERSION</term> <item>Should contain the string
		  <verb>'2.3'</verb>.
		</item>
	      </description>
	    </subsubsection>

	    <subsubsection>
	      <heading>Storage Layout</heading>

	      <p>An <verb>Array</verb> has a <em>dataspace</em> with a
		<em>N-dimensional contiguous</em> layout (if you
		prefer a <em>chunked</em> layout see
		<verb>EArray</verb> below).
	      </p>
	    </subsubsection>

	    <subsubsection>
	      <heading>Datatypes supported</heading>

	      <p>The elements of <verb>Array</verb> must have either
		HDF5 <em>atomic</em> data types or a <em>compound</em>
		data type representing a complex number.  The atomic
		data types can currently be one of the next HDF5
		data type <em>classes</em>: H5T_BITFIELD, H5T_INTEGER,
		H5T_FLOAT and H5T_STRING.
		The H5T_TIME class is also supported
		for reading existing <verb>Array</verb> objects,
		but not for creating them.
		See the <verb>Table</verb> format description in <ref
		refid="TableFormatDescr">section</ref> for more info
		about these types.
	      </p>

	      <p>In addition to the HDF5 atomic data types, the Array
		format supports complex numbers with the H5T_COMPOUND
		data type class. See the <verb>Table</verb> format
		description in <ref
		refid="TableFormatDescr">section</ref> for more info
		about this special type.
	      </p>

	      <p>You should note that H5T_ARRAY class datatypes are
		not allowed in <verb>Array</verb> objects.
	      </p>

	    </subsubsection>
	  </subsection>	<!-- Array -->

	  <subsection id="CArrayFormatDescr">
	    <heading><visual markup="tt">CArray</visual>
	      format</heading>

	    <subsubsection>
	      <heading>Mandatory attributes</heading>

	      <p>The next attributes are mandatory for <em>carray</em>
		structures:
	      </p>

	      <description>
		<term>CLASS</term> <item>Must be set to
		  <verb>'CARRAY'</verb>.
		</item>

		<term>FLAVOR</term> <item>This is meant to provide the
		  information about the kind of objects kept in the
		  <verb>CArray</verb>, i.e. when the dataset is read,
		  it will be converted to the indicated flavor. It can
		  take the same values as the <verb>Array</verb> object.
		</item>

		<term>TITLE</term> <item>A string where the user can
		  put some description on what is this dataset used
		  for.
		</item>

		<term>VERSION</term> <item>Should contain the string
		  <verb>'1.0'</verb>.
		</item>
	      </description>
	    </subsubsection>

	    <subsubsection>
	      <heading>Storage Layout</heading>

	      <p>An <verb>CArray</verb> has a <em>dataspace</em> with a
		<em>N-dimensional chunked</em> layout.
	      </p>
	    </subsubsection>

	    <subsubsection>
	      <heading>Datatypes supported</heading>

	      <p>The elements of <verb>CArray</verb> must have either
		HDF5 <em>atomic</em> data types or a <em>compound</em>
		data type representing a complex number.  The atomic
		data types can currently be one of the next HDF5
		data type <em>classes</em>: H5T_BITFIELD, H5T_INTEGER,
		H5T_FLOAT and H5T_STRING.
		The H5T_TIME class is also supported
		for reading existing <verb>CArray</verb> objects,
		but not for creating them.
		See the <verb>Table</verb> format description in <ref
		refid="TableFormatDescr">section</ref> for more info
		about these types.
	      </p>

	      <p>In addition to the HDF5 atomic data types, the CArray
		format supports complex numbers with the H5T_COMPOUND
		data type class. See the <verb>Table</verb> format
		description in <ref
		refid="TableFormatDescr">section</ref> for more info
		about this special type.
	      </p>

	      <p>You should note that H5T_ARRAY class datatypes are
		not allowed in <verb>Array</verb> objects.
	      </p>

	    </subsubsection>
	  </subsection>	<!-- CArray -->

	  <subsection id="EArrayFormatDescr">
	    <heading><visual markup="tt">EArray</visual>
	      format</heading>

	    <subsubsection>
	      <heading>Mandatory attributes</heading>

	      <p>The next attributes are mandatory for <em>earray</em>
		structures:
	      </p>

	      <description>
		<term>CLASS</term> <item>Must be set to
		  <verb>'EARRAY'</verb>.
		</item>

		<term>EXTDIM</term> <item>(<em>Integer</em>) Must be
		  set to the extensible dimension. Only one extensible
		  dimension is supported right now.
		</item>

		<term>FLAVOR</term> <item>This is meant to provide the
		  information about the kind of objects kept in the
		  <verb>EArray</verb>, i.e. when the dataset is read,
		  it will be converted to the indicated flavor. It can
		  take the same values as the <verb>Array</verb>
		  object (see <ref refid="ArrayFormatDescr"></ref>),
		  except <verb>"Int"</verb> and <verb>"Float"</verb>.

		</item>
		<term>TITLE</term> <item>A string where the user can
		  put some description on what is this dataset used
		  for.
		</item>

		<term>VERSION</term> <item>Should contain the string
		  <verb>'1.3'</verb>.
		</item>

	      </description>
	    </subsubsection>

	    <subsubsection>
	      <heading>Storage Layout</heading>

	      <p>An <verb>EArray</verb> has a <em>dataspace</em> with
		a <em>N-dimensional chunked</em> layout.
	      </p>
	    </subsubsection>

	    <subsubsection>
	      <heading>Datatypes supported</heading>

	      <p>The elements of <verb>EArray</verb> are allowed to
		have the same data types as for the elements in the
		Array format.  They can be one of the HDF5
		<em>atomic</em> data type <em>classes</em>:
		H5T_BITFIELD, H5T_INTEGER, H5T_FLOAT, H5T_TIME or H5T_STRING,
		see the <verb>Table</verb> format description in <ref
		refid="TableFormatDescr">section</ref> for more info
		about these types.  They can also be a H5T_COMPOUND
		datatype representing a complex number, see the
		<verb>Table</verb> format description in <ref
		refid="TableFormatDescr">section</ref>.
	      </p>

	      <p>You should note that H5T_ARRAY class data types are
		not allowed in <verb>EArray</verb> objects.
	      </p>

	    </subsubsection>
	  </subsection>	<!-- EArray -->

	  <subsection id="VLArrayFormatDescr">
	    <heading><visual markup="tt">VLArray</visual>
	      format</heading>

	    <subsubsection>
	      <heading>Mandatory attributes</heading>

	      <p>The next attributes are mandatory for
		<em>vlarray</em> structures:
	      </p>

	      <description>
		<term>CLASS</term> <item>Must be set to
		  <verb>'VLARRAY'</verb>.
		</item>

		<term>FLAVOR</term> <item>This is meant to provide the
		  information about the kind of objects kept in the
		  <verb>VLArray</verb>, i.e. when the dataset is read,
		  it will be converted to the indicated flavor. It can
		  take one of the next values:

		  <description>
		    <term>"numarray"</term> <item>The dataset will be
		      returned as a <verb>numarray</verb> object.
		    </item>
		    <term>"numpy"</term> <item>The dataset will be
		      returned as a <verb>NumPy</verb> object.
		    </item>
		    <term>"numeric"</term> <item>The dataset will be
		      returned as an <verb>Numeric</verb> object.
		    </item>
		    <term>"python"</term> <item>The dataset will be
		      returned as a Python <verb>List</verb> object in
		      case the dataset has dimensionality. If the
		      dataset is an scalar, then an appropriate Python
		      scalar will be returned instead.
		    </item>
		    <term>"Object"</term> <item>The elements in the
		      dataset will be interpreted as pickled
		      (i.e. serialized objects through the use of the
		      <verb>Pickle</verb> Python module) objects and
		      returned as Python <em>generic</em>
		      objects. Only one of such objects will be
		      deserialized per entry. As the
		      <verb>Pickle</verb> module is not normally
		      available in other languages, this flavor won't
		      be useful in general.
		    </item>
		    <term>"VLString"</term> <item>The elements in the
		      dataset will be returned as Python
		      <verb>String</verb> objects of <em>any</em>
		      length, with the twist that <visual
		      markup="bf">Unicode</visual> strings are
		      supported as well (provided you use the <visual
		      markup="bf">UTF-8</visual> codification, see
		      below). However, only one of such objects will
		      be deserialized per entry.
		    </item>
		  </description>
		</item>

		<term>TITLE</term> <item>A string where the user can
		  put some description on what is this dataset used
		  for.
		</item>

		<term>VERSION</term> <item>Should contain the string
		  <verb>'1.2'</verb>.
		</item>

	      </description>
	    </subsubsection>

	    <subsubsection>
	      <heading>Storage Layout</heading>

	      <p>An <verb>VLArray</verb> has a <em>dataspace</em> with
		a <em>1-dimensional chunked</em> layout.
	      </p>
	    </subsubsection>

	    <subsubsection>
	      <heading>Data types supported</heading>

	      <p>The data type of the elements (rows) of
		<verb>VLArray</verb> objects must be the H5T_VLEN
		<em>variable-length</em> (or VL for short) datatype,
		and the base datatype specified for the VL datatype
		can be of any <em>atomic</em> HDF5 datatype that is
		listed in the <verb>Table</verb> format <ref
		refid="TableFormatDescr">description
		section</ref>. That includes the classes:
	      </p>
	      <itemize>
		<item>H5T_BITFIELD</item>
		<item>H5T_INTEGER</item>
		<item>H5T_FLOAT</item>
		<item>H5T_TIME</item>
		<item>H5T_STRING</item>
		<item>H5T_ARRAY</item>
	      </itemize>
	      <p>They can also be a H5T_COMPOUND data type representing
		a complex number, see the <verb>Table</verb> format
		description in <ref
		refid="TableFormatDescr">section</ref> for a detailed
		description.
	      </p>
	      <p>You should note that this does not include another VL
		datatype, or a compound datatype that does not fit the
		description of a complex number. Note as well that,
		for <verb>Object</verb> and <verb>VLString</verb>
		special flavors, the base for the VL datatype is
		always a H5T_NATIVE_UCHAR. That means that the
		complete row entry in the dataset has to be used in
		order to fully serialize the object or the variable
		length string.
	      </p>
	      <p>In addition, if you plan to use a
		<verb>VLString</verb> flavor for your text data and
		you are using ascii-7 (7 bits ASCII) codification for
		your strings, but you don't know (or just don't want)
		to convert it to the required UTF-8 codification, you
		should not worry too much about that because the ASCII
		characters with values in the range [0x00, 0x7f] are
		directly mapped to Unicode characters in the range
		[U+0000, U+007F] and the UTF-8 encoding has the useful
		property that an UTF-8 encoded ascii-7 string is
		indistinguishable from a traditional ascii-7 string.
		So, you will not need any further conversion in order
		to save your ascii-7 strings and have an
		<verb>VLString</verb> flavor.
	      </p>

	    </subsubsection>
	  </subsection>	<!-- VLArray -->

	</section> <!-- Leaf -->

      </chapter> <!-- PyTables Internal Format -->

    </appendix>

    </part>

  </mainmatter>

  <backmatter>

    <!-- latex code="\renewcommand{\refname}{References}"/> -->


    <references bibfile="usersguide"/>

  </backmatter>
</book>

<!-- Keep this comment at the end of the file
Local variables:
mode: xml
sgml-omittag:nil
sgml-shorttag:nil
sgml-namecase-general:nil
sgml-general-insert-case:lower
sgml-minimize-attributes:nil
sgml-always-quote-attributes:t
sgml-indent-step:2
sgml-indent-data:t
sgml-tab-width:4
sgml-parent-document:nil
sgml-exposed-tags:nil
sgml-local-catalogs:nil
sgml-local-ecat-files:nil
End:
-->
