PyTables User's Guide: Hierarchical datasets in Python - Release 1.3.3
Prev | Chapter 4. Library Reference | Next

# 4.16. Declarative classes

In this section a series of classes that are meant to declare datatypes that are required for primary `PyTables` (like `Table` or `VLArray` ) objects are described.

## 4.16.1. The IsDescription class

This class is designed to be used as an easy, yet meaningful way to describe the properties of `Table` objects through the definition of derived classes that inherit properties from it. In order to define such a class, you must declare it as descendant of IsDescription, with as many attributes as columns you want in your table. The name of each attribute will become the name of a column, and its value will hold a description of it.

Ordinary columns can be described using instances of the `Col` (see section 4.16.2) class. Nested columns can be described by using classes derived from `IsDescription` or instances of it. Derived classes can be declared in place (in which case the column takes the name of the class) or referenced by name, and they can have a `_v_pos` special attribute which sets the position of the nested column among its sibling columns.

Once you have created a description object, you can pass it to the `Table` constructor, where all the information it contains will be used to define the table structure. See the section 3.4 for an example on how that works.

See below for a complete list of the special attributes that can be specified to complement the metadata of an `IsDescription` class.

### 4.16.1.1. IsDescription special attributes

_v_flavor

  The flavor of the table. It can take "numarray" (default) or "numpy" values. This determines the type of objects returned during input (i.e. read) operations.

_v_indexprops

  An instance of the `IndexProps` class (see section 4.17.2). You can use this to alter the properties of the index creation process for a table.

_v_pos

  Sets the position of a possible nested column description among its sibling columns.

## 4.16.2. The Col class and its descendants

The `Col` class is used as a mean to declare the different properties of a table column. In addition, a series of descendant classes are offered in order to make these column descriptions easier to the user. In general, it is recommended to use these descendant classes, as they are more meaningful when found in the middle of the code.

### 4.16.2.1. Col instance attributes

type

  The type class of the column.

stype

  The string type of the column.

recarrtype

  The string type, in `RecArray` format, of the column.

shape

  The shape of the column.

itemsize

  The size of the base items. Specially useful for `StringCol` objects.

indexed

  Whether this column is meant to be indexed or not.

_v_pos

  The position of this column with regard to its column siblings.

_v_name

  The name of this column

_v_pathname

  The complete pathname of the column. This is mainly useful in nested columns; for non-nested ones this value is the same a `_v_name`.

### 4.16.2.2. Col methods

None.

### 4.16.2.3. Col constructors

A description of the different constructors with their parameters follows:

Col(dtype="Float64", shape=1, dflt=None, pos=None, indexed=0)

  Declare the properties of a `Table` column.

  dtype

    The data type for the column. All types listed in appendix A are valid data types for columns. The type description is accepted both in string-type format and as a numarray data type.

  shape

    An integer or a tuple, that specifies the number of dtype items for each element (or shape, for multidimensional elements) of this column. For `CharType` columns, the last dimension is used as the length of the character strings. However, for this kind of objects, the use of `StringCol` subclass is strongly recommended.

  dflt

    The default value for elements of this column. If the user does not supply a value for an element while filling a table, this default value will be written to disk. If the user supplies an scalar value for a multidimensional column, this value is automatically broadcasted to all the elements in the column cell. If dflt is not supplied, an appropriate zero value (or null string) will be chosen by default. Please, note that all the default values are kept internally as numarray objects.

  pos

    By default, columns are arranged in memory following an alpha-numerical order of the column names. In some situations, however, it is convenient to impose a user defined ordering. pos parameter allows the user to force the desired ordering.

  indexed

    Whether this column should be indexed for better performance in table selections.

StringCol(length=None, dflt=None, shape=1, pos=None, indexed=0)

  Declare a column to be of type `CharType`. The length parameter sets the length of the strings. The meaning of the other parameters are like in the `Col` class.

BoolCol(dflt=0, shape=1, pos=None, indexed=0)

  Define a column to be of type `Bool`. The meaning of the parameters are the same of those in the `Col` class.

IntCol(dflt=0, shape=1, itemsize=4, sign=1, pos=None, indexed=0)

  Declare a column to be of type `IntXX`, depending on the value of itemsize parameter, that sets the number of bytes of the integers in the column. sign determines whether the integers are signed or not. The meaning of the other parameters are the same of those in the `Col` class.

  This class has several descendants:

  Int8Col(dflt=0, shape=1, pos=None, indexed=0)

    Define a column of type `Int8`.

  UInt8Col(dflt=0, shape=1, pos=None, indexed=0)

    Define a column of type `UInt8`.

  Int16Col(dflt=0, shape=1, pos=None, indexed=0)

    Define a column of type `Int16`.

  UInt16Col(dflt=0, shape=1, pos=None, indexed=0)

    Define a column of type `UInt16`.

  Int32Col(dflt=0, shape=1, pos=None, indexed=0)

    Define a column of type `Int32`.

  UInt32Col(dflt=0, shape=1, pos=None, indexed=0)

    Define a column of type `UInt32`.

  Int64Col(dflt=0, shape=1, pos=None, indexed=0)

    Define a column of type `Int64`.

  UInt64Col(dflt=0, shape=1, pos=None, indexed=0)

    Define a column of type `UInt64`.

FloatCol(dflt=0.0, shape=1, itemsize=8, pos=None, indexed=0)

  Define a column to be of type `FloatXX`, depending on the value of `itemsize`. The `itemsize` parameter sets the number of bytes of the floats in the column and the default is 8 bytes (double precision). The meaning of the other parameters are the same as those in the `Col` class.

  This class has two descendants:

  Float32Col(dflt=0.0, shape=1, pos=None, indexed=0)

    Define a column of type `Float32`.

  Float64Col(dflt=0.0, shape=1, pos=None, indexed=0)

    Define a column of type `Float64`.

ComplexCol(dflt=0.+0.j, shape=1, itemsize=16, pos=None)

  Define a column to be of type `ComplexXX`, depending on the value of `itemsize`. The `itemsize` parameter sets the number of bytes of the complex types in the column and the default is 16 bytes (double precision complex). The meaning of the other parameters are the same as those in the `Col` class.

  This class has two descendants:

  Complex32Col(dflt=0.+0.j, shape=1, pos=None)

    Define a column of type `Complex32`.

  Float64Col(dflt=0+0.j, shape=1, pos=None)

    Define a column of type `Complex64`.

  `ComplexCol` columns and its descendants do not support indexation.

TimeCol(dflt=0, shape=1, itemsize=8, pos=None, indexed=0)

  Define a column to be of type Time. Two kinds of time columns are supported depending on the value of `itemsize`: 4-byte signed integer and 8-byte double precision floating point columns (the default ones). The meaning of the other parameters are the same as those in the `Col` class.

  Time columns have a special encoding in the HFD5 file. See appendix A for more information on those types.

  This class has two descendants:

  Time32Col(dflt=0, shape=1, pos=None, indexed=0)

    Define a column of type `Time32`.

  Time64Col(dflt=0.0, shape=1, pos=None, indexed=0)

    Define a column of type `Time64`.

EnumCol(enum, dflt, dtype='UInt32', shape=1, pos=None, indexed=False)

  Description of a column of an enumerated type.

  Instances of this class describe a table column which stores enumerated values. Those values belong to an enumerated type, defined by the first argument (`enum`) in the constructor of `EnumCol`, which accepts the same kinds of arguments as `Enum` (see 4.17.4). The enumerated type is stored in the `enum` attribute of the column.

  A default value must be specified as the second argument (`dflt`) in the constructor; it must be the name (a string) of one of the enumerated values in the enumerated type. Once the column is created, the corresponding concrete value is stored in its `dflt` attribute. If the name does not match any value in the enumerated type, a `KeyError` is raised.

  A numarray data type might be specified in order to determine the base type used for storing the values of enumerated values in memory and disk. The data type must be able to represent each and every concrete value in the enumeration. If it is not, a `TypeError` is raised. The default base type is unsigned 32-bit integer, which is sufficient for most cases.

  The `stype` attribute of enumerated columns is always `'Enum'`, while the `type` attribute is the data type used for storing concrete values.

  The shape, position and indexed attributes of the column are treated as with other column description objects (see 4.16.2).

## 4.16.3. The Atom class and its descendants.

The `Atom` class is a descendant of the `Col` class (see 4.16.2) and is meant to declare the different properties of the base element (also known as atom) of `CArray`, `EArray` and `VLArray` objects. The `Atom` instances have the property that their length is always the same. However, you can grow objects along the extensible dimension in the case of `EArray` or put a variable number of them on a `VLArray` row. Moreover, the atoms are not restricted to scalar values, and they can be fully multidimensional objects.

A series of descendant classes are offered in order to make the use of these element descriptions easier. In general, it is recommended to use these descendant classes, as they are more meaningful when found in the middle of the code.

### 4.16.3.1. Atom instance variables

In addition to the variables that it inherits from the `Col` class, it has the next additional attributes:

flavor

  The object representation for this atom. See below on constructors description for `Atom` class the possible values it can take.

### 4.16.3.2. Atom methods

atomsize()

  Returns the total length, in bytes, of the element base atom. If its shape is has one zero element on it (for use in `EArrays`, for example), this is replaced by an one in order to compute the atom size correctly.

### 4.16.3.3. Atom constructors

A description of the different constructors with their parameters follows:

Atom(dtype="Float64", shape=1, flavor="numarray")

  Define properties for the base elements of `CArray`, `EArray` and `VLArray` objects.

  dtype

    The data type for the base element. See the appendix A for a relation of data types supported. The type description is accepted both in string-type format and as a numarray data type.

  shape

    In a `EArray` context, it is a tuple specifying the shape of the object, and one (and only one) of its dimensions must be 0, meaning that the `EArray` object will be enlarged along this axis. In the case of a `VLArray`, it can be an integer with a value of 1 (one) or a tuple, that specifies whether the atom is an scalar (in the case of a 1) or has multiple dimensions (in the case of a tuple). For `CharType` elements, the last dimension is used as the length of the character strings. However, for this kind of objects, the use of `StringAtom` subclass is strongly recommended.

  flavor

    The object representation for this atom. It can be any of "numarray", "numpy" or "python" for the character types and "numarray", "numpy", "numeric" or "python" for the numerical types. If specified, the read atoms will be converted to that specific flavor. If not specified, the atoms will remain in their native format (i.e. `numarray`).

StringAtom(shape=1, length=None, flavor="numarray")

  Define an atom to be of `CharType` type. The meaning of the shape parameter is the same as in the `Atom` class. length sets the length of the strings atoms. flavor can be whether `"numarray"`, `"numpy"` or `"python"`. Unicode strings are not supported by this type; see the `VLStringAtom` class if you want Unicode support (only available for `VLAtom` objects).

BoolAtom(shape=1, flavor="numarray")

  Define an atom to be of type `Bool`. The meaning of the parameters are the same of those in the `Atom` class.

IntAtom(shape=1, itemsize=4, sign=1, flavor="numarray")

  Define an atom to be of type `IntXX`, depending on the value of itemsize parameter, that sets the number of bytes of the integers that conform the atom. sign determines whether the integers are signed or not. The meaning of the other parameters are the same of those in the `Atom` class.

  This class has several descendants:

  Int8Atom(shape=1, flavor="numarray")

    Define an atom of type `Int8`.

  UInt8Atom(shape=1, flavor="numarray")

    Define an atom of type `UInt8`.

  Int16Atom(shape=1, flavor="numarray")

    Define an atom of type `Int16`.

  UInt16Atom(shape=1, flavor="numarray")

    Define an atom of type `UInt16`.

  Int32Atom(shape=1, flavor="numarray")

    Define an atom of type `Int32`.

  UInt32Atom(shape=1, flavor="numarray")

    Define an atom of type `UInt32`.

  Int64Atom(shape=1, flavor="numarray")

    Define an atom of type `Int64`.

  UInt64Atom(shape=1, flavor="numarray")

    Define an atom of type `UInt64`.

FloatAtom(shape=1, itemsize=8, flavor="numarray")

  Define an atom to be of `FloatXX` type, depending on the value of `itemsize`. The `itemsize` parameter sets the number of bytes of the floats in the atom and the default is 8 bytes (double precision). The meaning of the other parameters are the same as those in the `Atom` class.

  This class has two descendants:

  Float32Atom(shape=1, flavor="numarray")

    Define an atom of type `Float32`.

  Float64Atom(shape=1, flavor="numarray")

    Define an atom of type `Float64`.

ComplexAtom(shape=1, itemsize=16, flavor="numarray")

  Define an atom to be of `ComplexXX` type, depending on the value of `itemsize`. The `itemsize` parameter sets the number of bytes of the floats in the atom and the default is 16 bytes (double precision complex). The meaning of the other parameters are the same as those in the `Atom` class.

  This class has two descendants:

  Complex32Atom(shape=1, flavor="numarray")

    Define an atom of type `Complex32`.

  Complex64Atom(shape=1, flavor="numarray")

    Define an atom of type `Complex64`.

TimeAtom(shape=1, itemsize=8, flavor="numarray")

  Define an atom to be of type Time. Two kinds of time atoms are supported depending on the value of `itemsize`: 4-byte signed integer and 8-byte double precision floating point atoms (the default ones). The meaning of the other parameters are the same as those in the `Atom` class.

  Time atoms have a special encoding in the HFD5 file. See appendix A for more information on those types.

  This class has two descendants:

  Time32Atom(shape=1, flavor="numarray")

    Define an atom of type `Time32`.

  Time64Atom(shape=1, flavor="numarray")

    Define an atom of type `Time64`.

EnumAtom(enum, dtype='UInt32', shape=1, flavor='numarray')

  Description of an atom of an enumerated type.

  Instances of this class describe the atom type used by an array to store enumerated values. Those values belong to an enumerated type.

  The meaning of the `enum` and `dtype` arguments is the same as in `EnumCol` (see 4.16.2). The `shape` and `flavor` arguments have the usual meaning of other `Atom` classes (the `flavor` applies to the representation of concrete read values).

  Enumerated atoms also have `stype` and `type` attributes with the same values as in `EnumCol`.

Now, there come two special classes, `ObjectAtom` and `VLString`, that actually do not descend from `Atom`, but which goal is so similar that they should be described here. The difference between them and the `Atom` and descendants classes is that these special classes does not allow multidimensional atoms, nor multiple values per row. A flavor can not be specified neither as it is immutable (see below).

Caveat emptor: You are only allowed to use these classes to create `VLArray` objects, not `CArray` and `EArray` objects.

ObjectAtom()

  This class is meant to fit any kind of object in a row of an `VLArray` instance by using `cPickle` behind the scenes. Due to the fact that you can not foresee how long will be the output of the `cPickle` serialization (i.e. the atom already has a variable length), you can only fit a representant of it per row. However, you can still pass several parameters to the `VLArray.append()` method as they will be regarded as a tuple of compound objects (the parameters), so that we still have only one object to be saved in a single row. It does not accept parameters and its flavor is automatically set to `"Object"`, so the reads of rows always returns an arbitrary python object. You can regard `ObjectAtom` types as an easy way to save an arbitrary number of generic python objects in a `VLArray` object.

VLStringAtom()

  This class describes a row of the `VLArray` class, rather than an atom. It differs from the `StringAtom` class in that you can only add one instance of it to one specific row, i.e. the `VLArray.append()` method only accepts one object when the base atom is of this type. Besides, it supports Unicode strings (contrarily to `StringAtom`) because it uses the UTF-8 codification (this is why its `atomsize()` method returns always 1) when serializing to disk. It does not accept any parameter and because its flavor is automatically set to `"VLString"`, the reads of rows always returns a python string. See the appendix D.3.5 if you are curious on how this is implemented at the low-level. You can regard `VLStringAtom` types as an easy way to save generic variable length strings.

See `examples/vlarray1.py` and `examples/vlarray2.py` for further examples on `VLArray`s, including object serialization and Unicode string management.

Prev | Home | Next
The AttributeSet class | Up | Helper classes