# PyPy[coding-guide]

PyPy[coding-guide]
news getting-started documentation svn issues contact EU/project

# PyPy - Coding Guide

Contents

This document describes coding requirements and conventions for working with the PyPy code base. Please read it carefully and ask back any questions you might have. The document does not talk very much about coding style issues. We mostly follow PEP 8 though. If in doubt, follow the style that is already present in the code base.

# 1 Overview and motivation

We are writing a Python interpreter in Python, using Python's well known ability to step behind the algorithmic problems as a language. At first glance, one might think this achieves nothing but a better understanding how the interpreter works. This alone would make it worth doing, but we have much larger goals.

## 1.1 CPython vs. PyPy

Compared to the CPython implementation, Python takes the role of the C Code. We rewrite the CPython interpreter in Python itself. We could also aim at writing a more flexible interpreter at C level but we want to use Python to give an alternative description of the interpreter.

The clear advantage is that such a description is shorter and simpler to read, and many implementation details vanish. The drawback of this approach is that this interpreter will be unbearably slow as long as it is run on top of CPython.

To get to a useful interpreter again, we need to translate our high-level description of Python to a lower level one. One rather straight-forward way is to do a whole program analysis of the PyPy interpreter and create a C source, again. There are many other ways, but let's stick with this somewhat canonical approach.

## 1.2 Application-level and interpreter-level execution and objects

Since Python is used for implementing all of our code base, there is a crucial distinction to be aware of: that between interpreter-level objects and application-level objects. The latter are the ones that you deal with when you write normal python programs. Interpreter-level code, however, cannot invoke operations nor access attributes from application-level objects. You will immediately recognize any interpreter level code in PyPy, because half the variable and object names start with a `w_`, which indicates that they are wrapped application-level values.

Let's show the difference with a simple example. To sum the contents of two variables `a` and `b`, one would write the simple application-level `a+b` -- in contrast, the equivalent interpreter-level code is `space.add(w_a, w_b)`, where `space` is an instance of an object space, and `w_a` and `w_b` are typical names for the wrapped versions of the two variables.

It helps to remember how CPython deals with the same issue: interpreter level code, in CPython, is written in C and thus typical code for the addition is `PyNumber_Add(p_a, p_b)` where `p_a` and `p_b` are C variables of type `PyObject*`. This is conceptually similar to how we write our interpreter-level code in Python.

Moreover, in PyPy we have to make a sharp distinction between interpreter- and application-level exceptions: application exceptions are always contained inside an instance of `OperationError`. This makes it easy to distinguish failures (or bugs) in our interpreter-level code from failures appearing in a python application level program that we are interpreting.

## 1.3 Application level is often preferable

Application-level code is substantially higher-level, and therefore correspondingly easier to write and debug. For example, suppose we want to implement the `update` method of dict objects. Programming at application level, we can write an obvious, simple implementation, one that looks like an executable definition of `update`, for example:

    def update(self, other):
        for k in other.keys():
            self[k] = other[k]

If we had to code only at interpreter level, we would have to code something much lower-level and involved, say something like:

    def update(space, w_self, w_other):
        w_keys = space.call_method(w_other, 'keys')
        w_iter = space.iter(w_keys)
        while True:
            try:
                w_key = space.next(w_iter)
            except OperationError, e:
                if not e.match(space, space.w_StopIteration):
                    raise       # re-raise other app-level exceptions
                break
            w_value = space.getitem(w_other, w_key)
            space.setitem(w_self, w_key, w_value)

This interpreter-level implementation looks much more similar to the C source code. It is still more readable than its C counterpart because it doesn't contain memory management details and can use Python's native exception mechanism.

In any case, it should be obvious that the application-level implementation is definitely more readable, more elegant and more maintainable than the interpreter-level one (and indeed, dict.update is really implemented at appleve in PyPy).

In fact, in almost all parts of PyPy, you find application level code in the middle of interpreter-level code. Apart from some bootstrapping problems (application level functions need a certain initialization level of the object space before they can be executed), application level code is usually preferable. We have an abstraction (called the 'Gateway') which allows the caller of a function to remain ignorant of whether a particular function is implemented at application or interpreter level.

## 1.4 our runtime interpreter is "restricted python"

In order to make a C code generator feasible all code on interpreter level has to restrict itself to a subset of the Python language, and we adhere to some rules which make translation to lower level languages feasible. Code on application level can still use the full expressivity of Python.

Unlike source-to-source translations (like e.g. Starkiller or more recently ShedSkin) we start translation from live python code objects which constitute our Python interpreter. When doing its work of interpreting bytecode our Python implementation must behave in a static way often referenced as "RPythonic".

However, when the PyPy interpreter is started as a Python program, it can use all of the Python language until it reaches a certain point in time, from which on everything that is being executed must be static. That is, during initialization our program is free to use the full dynamism of Python, including dynamic code generation.

An example can be found in the current implementation which is quite elegant: For the definition of all the opcodes of the Python interpreter, the module `dis` is imported and used to initialize our bytecode interpreter. (See `__initclass__` in pyopcode.py). This saves us from adding extra modules to PyPy. The import code is run at startup time, and we are allowed to use the CPython builtin import function.

After the startup code is finished, all resulting objects, functions, code blocks etc. must adhere to certain runtime restrictions which we describe further below. Here is some background for why this is so: during translation, a whole program analysis ("type inference") is performed, which makes use of the restrictions defined in RPython. This enables the code generator to emit efficient machine level replacements for pure integer objects, for instance.

# 2 Restricted Python

## 2.1 RPython Definition, not

The list and exact details of the "RPython" restrictions are a somewhat evolving topic. In particular, we have no formal language definition as we find it more practical to discuss and evolve the set of restrictions while working on the whole program analysis. If you have any questions about the restrictions below then please feel free to mail us at pypy-dev at codespeak net.

## 2.2 Flow restrictions

variables

> variables should contain values of at most one type as described in Object restrictions at each control flow point, that means for example that joining control paths using the same variable to contain both a string and a int must be avoided. It is allowed to mix None (basically with the role of a null pointer) with many other types: wrapped objects, class instances, lists, dicts, strings, etc. but not with int and floats.

constants

> all module globals are considered constants. Their binding must not be changed at run-time. Moreover, global (i.e. prebuilt) lists and dictionaries are supposed to be immutable: modifying e.g. a global list will give inconsistent results. However, global instances don't have this restriction, so if you need mutable global state, store it in the attributes of some prebuilt singleton instance.

control structures

> all allowed but yield

range

> `range` and `xrange` are identical. `range` does not necessarily create an array, only if the result is modified. It is allowed everywhere and completely implemented. The only visible difference to CPython is the inaccessability of the `xrange` fields start, stop and step.

definitions

> run-time definition of classes or functions is not allowed.

generators

> generators are not supported.

exceptions

- fully supported
- see below Exception rules for restrictions on exceptions raised by built-in operations

## 2.3 Object restrictions

We are using

integer, float, string, boolean

> a lot of, but not all string methods are supported. When slicing a string it is necessary to prove that the slice start and stop indexes are non-negative.

tuples

> no variable-length tuples; use them to store or return pairs or n-tuples of values. Each combination of types for elements and length constitute a separate and not mixable type.

lists

> lists are used as an allocated array. Lists are over-allocated, so list.append() is reasonably fast. Negative or out-of-bound indexes are only allowed for the most common operations, as follows:

> - indexing: positive and negative indexes are allowed. Indexes are checked when requested by an IndexError exception clause.
> - slicing: the slice start must be within bounds. The stop doesn't need to, but it must not be smaller than the start. All negative indexes are disallowed, except for the [:-1] special case. No step.
> - other operators: `+`, `+=`, `in`, `*`, `*=`, `==`, `!=` work as expected.
> - methods: append, index, insert, extend, reverse, pop. The index used in pop() follows the same rules as for indexing above. The index used in insert() must be within bounds and not negative.

dicts

> dicts with a unique key type only, provided it is hashable. String keys have been the only allowed key types for a while, but this was generalized. After some re-optimization, the implementation could safely decide that all string dict keys should be interned.

list comprehensions

> may be used to create allocated, initialized arrays. After list over-allocation was introduced, there is no longer any restriction.

functions

- statically called functions may use defaults and a variable number of arguments (which may be passed as a list instead of a tuple, so write code that does not depend on it being a tuple).
- dynamic dispatch enforces the use of signatures that are equal for all possible called function, or at least "compatible enough". This concerns mainly method calls, when the method is overridden or in any way given different definitions in different classes. It also concerns the less common case of explicitly manipulated function objects. Describing the exact compability rules is rather involved (but if you break them, you should get explicit errors from the rtyper and not obscure crashes.)

builtin functions

> A number of builtin functions can be used. The precise set can be found in pypy/annotation/builtin.py (see `def builtin_xxx()`). Some builtin functions may be limited in what they support, though.

> `int, float, str, ord, chr`... are available as simple conversion functions. Note that `int, float, str`... have a special meaning as a type inside of isinstance only.

classes

- methods and other class attributes do not change after startup
- single inheritance is fully supported
- simple mixins work too, but the mixed in class needs a `_mixin_ = True` class attribute
- classes are first-class objects too

objects

> in PyPy, wrapped objects are borrowed from the object space. Just like in CPython, code that needs e.g. a dictionary can use a wrapped dict and the object space operations on it.

This layout makes the number of types to take care about quite limited.

## 2.4 Integer Types

While implementing the integer type, we stumbled over the problem that integers are quite in flux in CPython right now. Starting on Python 2.2, integers mutate into longs on overflow. However, shifting to the left truncates up to 2.3 but extends to longs as well in 2.4. By contrast, we need a way to perform wrap-around machine-sized arithmetic by default, while still being able to check for overflow when we need it explicitly. Moreover, we need a consistent behavior before and after translation.

We use normal integers for signed arithmetic. It means that before translation we get longs in case of overflow, and after translation we get a silent wrap-around. Whenever we need more control, we use the following helpers (which live the pypy/rlib/rarithmetic.py):

ovfcheck()

> This special function should only be used with a single arithmetic operation as its argument, e.g. `z = ovfcheck(x+y)`. Its intended meaning is to perform the given operation in overflow-checking mode.

> At run-time, in Python, the ovfcheck() function itself checks the result and raises OverflowError if it is a `long`. But the code generators use ovfcheck() as a hint: they replace the whole `ovfcheck(x+y)` expression with a single overflow-checking addition in C.

ovfcheck_lshift()

> ovfcheck_lshift(x, y) is a workaround for ovfcheck(x<<y), because the latter doesn't quite work in Python prior to 2.4, where the expression `x<<y` will never return a long if the input arguments are ints. There is a specific function ovfcheck_lshift() to use instead of some convoluted expression like `x*2**y` so that code generators can still recognize it as a single simple operation.

intmask()

> This function is used for wrap-around arithmetic. It returns the lower bits of its argument, masking away anything that doesn't fit in a C "signed long int". Its purpose is, in Python, to convert from a Python `long` that resulted from a previous operation back to a Python `int`. The code generators ignore intmask() entirely, as they are doing wrap-around signed arithmetic all the time by default anyway. (We have no equivalent of the "int" versus "long int" distinction of C at the moment and assume "long ints" everywhere.)

r_uint

> In a few cases (e.g. hash table manipulation), we need machine-sized unsigned arithmetic. For these cases there is the r_uint class, which is a pure Python implementation of word-sized unsigned integers that silently wrap around. The purpose of this class (as opposed to helper functions as above) is consistent typing: both Python and the annotator will propagate r_uint instances in the program and interpret all the operations between them as unsigned. Instances of r_uint are special-cased by the code generators to use the appropriate low-level type and operations. Mixing of (signed) integers and r_uint in operations produces r_uint that means unsigned results. To convert back from r_uint to signed integers, use intmask().

## 2.5 Exception rules

Exceptions are by default not generated for simple cases.:

    #!/usr/bin/python

        lst = [1,2,3,4,5]
        item = lst[i]    # this code is not checked for out-of-bound access

        try:
            item = lst[i]
        except IndexError:
            # complain

Code with no exception handlers does not raise exceptions (after it has been translated, that is. When you run it on top of CPython, it may raise exceptions, of course). By supplying an exception handler, you ask for error checking. Without, you assure the system that the operation cannot fail. This rule does not apply to function calls: any called function is assumed to be allowed to raise any exception.

For example:

    x = 5.1
    x = x + 1.2       # not checked for float overflow
    try:
        x = x + 1.2
    except OverflowError:
        # float result too big

But:

    z = some_function(x, y)    # can raise any exception
    try:
        z = some_other_function(x, y)
    except IndexError:
        # only catches explicitly-raised IndexErrors in some_other_function()
        # other exceptions can be raised, too, and will not be caught here.

The ovfcheck() function described above follows the same rule: in case of overflow, it explicitly raise OverflowError, which can be caught anywhere.

Exceptions explicitly raised or re-raised will always be generated.

## 2.6 PyPy is debuggable on top of CPython

PyPy has the advantage that it is runnable on standard CPython. That means, we can run all of PyPy with all exception handling enabled, so we might catch cases where we failed to adhere to our implicit assertions.

## 2.7 RPylint

Pylint is a static code checker for Python. Recent versions (>=0.13.0) can be run with the `--rpython-mode` command line option. This option enables the RPython checker which will checks for some of the restrictions RPython adds on standard Python code (and uses a more agressive type inference than the one used by default by pylint). The full list of checks is available in the documentation of Pylin.

RPylint can be a nice tool to get some information about how much work will be needed to convert a piece of Python code to RPython, or to get started with RPython. While this tool will not guarantee that the code it checks will be translate successfully, it offers a few nice advantages over running a translation:

- it is faster and therefore provides feedback faster than `translate.py`
- it does not stop at the first problem it finds, so you can get more feedback on the code in one run
- the messages tend to be a bit less cryptic
- you can easily run it from emacs, vi, eclipse or visual studio.

Note: if pylint is not prepackaged for your OS/distribution, or if only an older version is available, you will need to install from source. In that case, there are a couple of dependencies, logilab-common and astng that you will need to install too before you can use the tool.

# 3 Wrapping rules

## 3.1 Wrapping

PyPy is made of Python source code at two levels: there is on the one hand application-level code that looks like normal Python code, and that implements some functionalities as one would expect from Python code (e.g. one can give a pure Python implementation of some built-in functions like `zip()`). There is also interpreter-level code for the functionalities that must more directly manipulate interpreter data and objects (e.g. the main loop of the interpreter, and the various object spaces).

Application-level code doesn't see object spaces explicitly: it runs using an object space to support the objects it manipulates, but this is implicit. There is no need for particular conventions for application-level code. The sequel is only about interpreter-level code. (Ideally, no application-level variable should be called `space` or `w_xxx` to avoid confusion.)

The `w_` prefixes so lavishly used in the example above indicate, by PyPy coding convention, that we are dealing with wrapped (or boxed) objects, that is, interpreter-level objects which the object space constructs to implement corresponding application-level objects. Each object space supplies `wrap`, `unwrap`, `int_w`, `interpclass_w`, etc. operations that move between the two levels for objects of simple built-in types; each object space also implements other Python types with suitable interpreter-level classes with some amount of internal structure.

For example, an application-level Python `list` is implemented by the standard object space as an instance of `W_ListObject`, which has an instance attribute `wrappeditems` (an interpreter-level list which contains the application-level list's items as wrapped objects).

The rules are described in more details below.

## 3.2 Naming conventions

- `space`: the object space is only visible at interpreter-level code, where it is by convention passed around by the name `space`.
- `w_xxx`: any object seen by application-level code is an object explicitly managed by the object space. From the interpreter-level point of view, this is called a wrapped object. The `w_` prefix is used for any type of application-level object.
- `xxx_w`: an interpreter-level container for wrapped objects, for example a list or a dict containing wrapped objects. Not to be confused with a wrapped object that would be a list or a dict: these are normal wrapped objects, so they use the `w_` prefix.

## 3.3 Operations on `w_xxx`

The core bytecode interpreter considers wrapped objects as black boxes. It is not allowed to inspect them directly. The allowed operations are all implemented on the object space: they are called `space.xxx()`, where `xxx` is a standard operation name (`add`, `getattr`, `call`, `eq`...). They are documented in the object space document.

A short warning: don't do `w_x == w_y` or `w_x is w_y`! rationale for this rule is that there is no reason that two wrappers are related in any way even if they contain what looks like the same object at application-level. To check for equality, use `space.is_true(space.eq(w_x, w_y))` or even better the short-cut `space.eq_w(w_x, w_y)` returning directly a interpreter-level bool. To check for identity, use `space.is_true(space.is_(w_x, w_y))` or better `space.is_w(w_x, w_y)`.

## 3.4 Application-level exceptions

Interpreter-level code can use exceptions freely. However, all application-level exceptions are represented as an `OperationError` at interpreter-level. In other words, all exceptions that are potentially visible at application-level are internally an `OperationError`. This is the case of all errors reported by the object space operations (`space.add()` etc.).

To raise an application-level exception:

    raise OperationError(space.w_XxxError, space.wrap("message"))

To catch a specific application-level exception:

    try:
        ...
    except OperationError, e:
        if not e.match(space, space.w_XxxError):
            raise
        ...

This construct catches all application-level exceptions, so we have to match it against the particular `w_XxxError` we are interested in and re-raise other exceptions. The exception instance `e` holds two attributes that you can inspect: `e.w_type` and `e.w_value`. Do not use `e.w_type` to match an exception, as this will miss exceptions that are instances of subclasses.

We are thinking about replacing `OperationError` with a family of common exception classes (e.g. `AppKeyError`, `AppIndexError`...) so that we can more easily catch them. The generic `AppError` would stand for all other application-level classes.

# 4 Modules in PyPy

Modules visible from application programs are imported from interpreter or application level files. PyPy reuses almost all python modules of CPython's standard library, currently from version 2.4.1. We sometimes need to modify modules and - more often - regression tests because they rely on implementation details of CPython.

If we don't just modify an original CPython module but need to rewrite it from scratch we put it into pypy/lib/ as a pure application level module.

When we need access to interpreter-level objects we put the module into pypy/module. Such modules use a mixed module mechanism which makes it convenient to use both interpreter- and applicationlevel parts for the implementation. Note that there is no extra facility for pure-interpreter level modules, you just write a mixed module and leave the application-level part empty.

## 4.1 Determining the location of a module implementation

You can interactively find out where a module comes from, when running py.py. here are examples for the possible locations:

    >>>> import sys
    >>>> sys.__file__
    '/home/hpk/pypy-dist/pypy/module/sys/*.py'

    >>>> import operator
    >>>> operator.__file__
    '/home/hpk/pypy-dist/pypy/lib/operator.py'

    >>>> import types
    >>>> types.__file__
    '/home/hpk/pypy-dist/lib-python/modified-2.4.1/types.py'

    >>>> import os
    faking <type 'posix.stat_result'>
    faking <type 'posix.statvfs_result'>
    >>>> os.__file__
    '/home/hpk/pypy-dist/lib-python/2.4.1/os.py'
    >>>>

## 4.2 Module directories / Import order

Here is the order in which PyPy looks up Python modules:

pypy/modules

> mixed interpreter/app-level builtin modules, such as the `sys` and `__builtin__` module.

contents of PYTHONPATH

> lookup application level modules in each of the `:` separated list of directories, specified in the `PYTHONPATH` environment variable.

pypy/lib/

> contains pure Python reimplementation of modules.

lib-python/modified-2.4.1/

> The files and tests that we have modified from the CPython library.

lib-python/2.4.1/

> The unmodified CPython library. Never ever check anything in there.

## 4.3 Modifying a CPython library module or regression test

Although PyPy is very compatible with CPython we sometimes need to change modules contained in our copy of the standard library, often due to the fact that PyPy works with all new-style classes by default and CPython has a number of places where it relies on some classes being old-style.

If you want to change a module or test contained in `lib-python/2.4.1` then make sure that you copy the file to our `lib-python/modified-2.4.1` directory first. In subversion commandline terms this reads:

    svn cp lib-python/2.4.1/somemodule.py lib-python/modified-2.4.1/

and subsequently you edit and commit `lib-python/modified-2.4.1/somemodule.py`. This copying operation is important because it keeps the original CPython tree clean and makes it obvious what we had to change.

## 4.4 Implementing a mixed interpreter/application level Module

If a module needs to access PyPy's interpreter level then it is implemented as a mixed module.

Mixed modules are directories in pypy/module with an __init__.py file containing specifications where each name in a module comes from. Only specified names will be exported to a Mixed Module's applevel namespace.

Sometimes it is necessary to really write some functions in C (or whatever target language). See rctypes and external functions documentation for details. The latter approach is cumbersome and being phased out and former has currently quite a few rough edges.

### 4.4.1 application level definitions

Application level specifications are found in the appleveldefs dictionary found in `__init__.py` files of directories in `pypy/module`. For example, in pypy/module/__builtin__/__init__.py you find the following entry specifying where `__builtin__.locals` comes from:

    ...
    'locals'        : 'app_inspect.locals',
    ...

The `app_` prefix indicates that the submodule `app_inspect` is interpreted at application level and the wrapped function value for `locals` will be extracted accordingly.

### 4.4.2 interpreter level definitions

Interpreter level specifications are found in the `interpleveldefs` dictionary found in `__init__.py` files of directories in `pypy/module`. For example, in pypy/module/__builtin__/__init__.py the following entry specifies where `__builtin__.len` comes from:

    ...
    'len'       : 'operation.len',
    ...

The `operation` submodule lives at interpreter level and `len` is expected to be exposable to application level. Here is the definition for `operation.len()`:

    def len(space, w_obj):
        "len(object) -> integer\n\nReturn the number of items of a sequence or mapping."
        return space.len(w_obj)

Exposed interpreter level functions usually take a `space` argument and some wrapped values (see wrapping rules) .

You can also use a convenient shortcut in `interpleveldefs` dictionaries: namely an expression in parentheses to specify an interpreter level expression directly (instead of pulling it indirectly from a file):

    ...
    'None'          : '(space.w_None)',
    'False'         : '(space.w_False)',
    ...

The interpreter level expression has a `space` binding when it is executed.

Adding an entry under pypy/module (e.g. mymodule) entails automatic creation of a new config option (such as --withmod-mymodule and --withoutmod-mymodule (the later being the default)) for py.py and translate.py.

## 4.5 Testing modules in `pypy/lib`

You can go to the pypy/lib/test2 directory and invoke the testing tool ("py.test" or "python ../../test_all.py") to run tests against the pypy/lib hierarchy. Note, that tests in pypy/lib/test2 are allowed and encouraged to let their tests run at interpreter level although pypy/lib/ modules eventually live at PyPy's application level. This allows us to quickly test our python-coded reimplementations against CPython.

## 4.6 Testing modules in `pypy/module`

Simply change to `pypy/module` or to a subdirectory and run the tests as usual.

## 4.7 Testing modules in `lib-python`

In order to let CPython's regression tests run against PyPy you can switch to the lib-python/ directory and run the testing tool in order to start compliance tests. (XXX check windows compatibility for producing test reports).

# 5 Naming conventions and directory layout

## 5.1 Directory and File Naming

- directories/modules/namespaces are always lowercase
- never use plural names in directory and file names
- `__init__.py` is usually empty except for `pypy/objspace/*` and `pypy/module/*/__init__.py`.
- don't use more than 4 directory nesting levels
- keep filenames concise and completion-friendly.

## 5.2 Naming of python objects

- class names are CamelCase
- functions/methods are lowercase and `_` separated
- objectspace classes are spelled `XyzObjSpace`. e.g.
  - StdObjSpace
  - FlowObjSpace
- at interpreter level and in ObjSpace all boxed values have a leading `w_` to indicate "wrapped values". This includes w_self. Don't use `w_` in application level python only code.

## 5.3 Committing & Branching to the repository

- write good log messages because several people are reading the diffs.

- if you add (text/py) files to the repository then please run pypy/tool/fixeol in that directory. This will make sure that the property 'svn:eol-style' is set to native which allows checkin/checkout in native line-ending format.

- branching (aka "svn copy") of source code should usually happen at `svn/pypy/dist` level in order to have a full self-contained pypy checkout for each branch. For branching a `try1` branch you would for example do:

      svn cp http://codespeak.net/svn/pypy/dist \
             http://codespeak.net/svn/pypy/branch/try1

  This allows to checkout the `try1` branch and receive a self-contained working-copy for the branch. Note that branching/copying is a cheap operation with subversion, as it takes constant time irrespective of the size of the tree.

- To learn more about how to use subversion read this document.

# 6 Using the development bug/feature tracker

We have a development tracker, based on Richard Jones' roundup application. You can file bugs, feature requests or see what's going on for the next milestone, both from an E-Mail and from a web interface.

## 6.1 use your codespeak login or register

If you already committed to the PyPy source code, chances are that you can simply use your codespeak login that you use for subversion or for shell access.

If you are not a commiter then you can still register with the tracker easily.

## 6.2 modifying Issues from svn commit messages

If you are committing something related to an issue in the development tracker you can correlate your login message to a tracker item by following these rules:

- put the content of `issueN STATUS` on a single new line

- N must be an existing issue number from the development tracker.

- STATUS is one of:

      unread
      chatting
      in-progress
      testing
      duplicate
      resolved

# 7 Testing in PyPy

Our tests are based on the new py.test tool which lets you write unittests without boilerplate. All tests of modules in a directory usually reside in a subdirectory test. There are basically two types of unit tests:

- Interpreter Level tests. They run at the same level as PyPy's interpreter.
- Application Level tests. They run at application level which means that they look like straight python code but they are interpreted by PyPy.

Both types of tests need an objectspace they can run with (the interpreter dispatches operations on objects to an objectspace). If you run a test you can usually give the '-o' switch to select an object space. E.g. '-o thunk' will select the thunk object space. The default is the Standard Object Space which aims to implement unmodified Python semantics.

## 7.1 Interpreter level tests

You can write test functions and methods like this:

    def test_something(space):
        # use space ...

    class TestSomething:
        def test_some(self):
            # use 'self.space' here

Note that the prefix test for test functions and Test for test classes is mandatory. In both cases you can import Python modules at module global level and use plain 'assert' statements thanks to the usage of the py.test tool.

## 7.2 Application Level tests

For testing the conformance and well-behavedness of PyPy it is often sufficient to write "normal" application-level Python code that doesn't need to be aware of any particular coding style or restrictions. If we have a choice we often use application level tests which usually look like this:

    def app_test_something():
        # application level test code

    class AppTestSomething:
        def test_this(self):
            # application level test code

These application level test functions will run on top of PyPy, i.e. they have no access to interpreter details. You cannot use imported modules from global level because they are imported at interpreter-level while you test code runs at application level. If you need to use modules you have to import them within the test function.

Another possibility to pass in data into the AppTest is to use the `setup_class` method of the AppTest. All wrapped objects that are attached to the class there and start with `w_` can be accessed via self (but without the `w_`) in the actual test method. An example:

    from pypy.objspace.std import StdObjSpace

    class AppTestErrno:
        def setup_class(cls):
            cls.space = StdObjSpace()
            cls.w_d = cls.space.wrap({"a": 1, "b", 2})

        def test_dict(self):
            assert self.d["a"] == 1
            assert self.d["b"] == 2

## 7.3 Command line tool test_all

You can run almost all of PyPy's tests by invoking:

    python test_all.py file_or_directory

which is a synonym for the general py.test utility located in the `pypy` directory. For switches to modify test execution pass the `-h` option.

## 7.4 Test conventions

- adding features requires adding appropriate tests. (It often even makes sense to first write the tests so that you are sure that they actually can fail.)
- All over the pypy source code there are test/ directories which contain unittests. Such scripts can usually be executed directly or are collectively run by pypy/test_all.py
- each test directory needs a copy of pypy/tool/autopath.py which upon import will make sure that sys.path contains the directory where 'pypy' is in.

# 8 Changing documentation and website

## 8.1 documentation/website files in your local checkout

Most of the PyPy's documentation and website is kept in pypy/documentation and pypy/documentation/website respectively. You can simply edit or add '.txt' files which contain ReST-markuped files. Here is a ReST quickstart but you can also just look at the existing documentation and see how things work.

## 8.2 Automatically test documentation/website changes

We automatically check referential integrity and ReST-conformance. In order to run the tests you need docutils installed. Then go to the local checkout of the documentation directory and run the tests:

    cd .../pypy/documentation
    python ../test_all.py

If you see no failures chances are high that your modifications at least don't produce ReST-errors or wrong local references. A side effect of running the tests is that you have .html files in the documentation directory which you can point your browser to!

Additionally, if you also want to check for remote references inside the documentation issue:

    python ../test_all.py --checkremote

which will check that remote URLs are reachable.