Using LibMPEG3 to make your own MPEG applications

`Author: Adam Williams broadcast@earthling.net Homepage: heroinewarrior.com`

LibMPEG3 decodes the many many derivatives of MPEG standards into uncompressed data suitable for editing and playback.

libmpeg3 currently decodes:

> MPEG-2 video
> MPEG-1 video
> mp3 audio
> mp2 audio
> ac3 audio (not in Squeak)
> MPEG-2 system streams
> MPEG-1 system streams

The video output can be in many different color models and frame sizes. The audio output can be in twos compliment or floating point.

STEP 1: Verifying file compatibility

Programs using libmpeg3 must `#include "libmpeg3.h"`.

Call `mpeg3_check_sig` to verify if the file can be read by libmpeg3. This returns a 1 if it is compatible and 0 if it isn't.

STEP 2: Open the file

You need an `mpeg3_t*` file descriptor:

`mpeg3_t* file;`

Then you need to open the file:

`file = mpeg3_open(char *path);`

`mpeg3_open` returns a NULL if the file couldn't be opened for some reason. Be sure to check this. Everything you do with libmpeg3 requires passing the `file` pointer.

STEP 3: How many CPUs do you want to use?

Call `mpeg3_set_cpus(mpeg3_t *file, int cpus)` to set how many CPUs should be devoted to video decompression. LibMPEG3 can use any number. If you don't call this right after opening the file, the CPU number defaults to 1.

STEP 4: Get some information about the file.

There are a number of queries for the audio components of the stream:

``

    int mpeg3_has_audio(mpeg3_t *file);
    int mpeg3_total_astreams(mpeg3_t *file);             // Number of multiplexed audio streams
    int mpeg3_audio_channels(mpeg3_t *file, int stream);
    int mpeg3_sample_rate(mpeg3_t *file, int stream);
    long mpeg3_audio_samples(mpeg3_t *file, int stream); // Total length

The audio is presented as a number of streams starting at 0 and including `mpeg3_total_astreams` - 1. Each stream contains a certain number of channels starting at 0 and including `mpeg3_audio_channels` - 1. The methodology is first determine if the file has audio, then get the number of streams in the file, then for each stream get the number of channels, sample rate, and length.

There are also queries for the video components:

``

    int mpeg3_has_video(mpeg3_t *file);
    int mpeg3_total_vstreams(mpeg3_t *file);            // Number of multiplexed video streams
    int mpeg3_video_width(mpeg3_t *file, int stream);
    int mpeg3_video_height(mpeg3_t *file, int stream);
    float mpeg3_frame_rate(mpeg3_t *file, int stream);  // Frames/sec
    long mpeg3_video_frames(mpeg3_t *file, int stream); // Total length

The video behavior is the same as with audio, except that video has no subdivision under streams. Frame rate is a floating point number of frames per second.

STEP 5: Seeking to a point in the file

Each audio stream and each video stream has a position in the file independant of each other stream. A variety of methods are available for specifying the position of a stream: percentage, frame, sample. Which method you use depends on whether you're seeking audio or video and whether you're seeking all tracks to a percentage of the file.

The preferred seeking method if you're writing a player is:

``

    int mpeg3_seek_percentage(mpeg3_t *file, double percentage);
    double mpeg3_tell_percentage(mpeg3_t *file);

This seeks all tracks to a percentage of the file length. The percentage is from 0 to 1.

The alternative is absolute seeking. The audio seeking is handled by:

``

    int mpeg3_set_sample(mpeg3_t *file, long sample, int stream);    // Seek
    long mpeg3_get_sample(mpeg3_t *file, int stream);    // Tell current position

and the video seeking is handled by:

``

    int mpeg3_set_frame(mpeg3_t *file, long frame, int stream); // Seek
    long mpeg3_get_frame(mpeg3_t *file, int stream);            // Tell current position

You can either perform percentage seeking or absolute seeking but not both on the same file handle. Once you perform either method, the file becomes configured for that method.

If you're in percentage seeking mode and you want the current time stamp in the file you can't use mpeg3_tell_percentage because you don't know how many seconds the total length is. The `mpeg3_audio_samples` and `mpeg3_video_frames` commands don't work in percentage seeking. Instead use ``

    double mpeg3_get_time(mpeg3_t *file);

which gives you the last timecode read in seconds. The MPEG standard specifies timecodes being placed in the streams.

STEP 6: Read the data

To read audio data use:

``

    int mpeg3_read_audio(mpeg3_t *file,
    		float *output_f,      // Pointer to pre-allocated buffer of floats
    		short *output_i,      // Pointer to pre-allocated buffer if int16's
    		int channel,          // Channel to decode
    		long samples,         // Number of samples to decode
    		int stream);          // Stream containing the channel

This decodes a buffer of sequential floats or int16's for a single channel, depending on which *output... parameter has a nonzero argument. To get a floating point buffer pass a pre-allocated buffer to `output_f` and NULL to `output_i`. To get an int16 buffer pass NULL to `output_f` and a pre-allocated buffer to `output_i`.

After reading an audio buffer, the current position in the one stream is advanced. How then, do you read more than one channel of audio data? Use ``

    mpeg3_reread_audio(mpeg3_t *file,
    		float *output_f,      /* Pointer to pre-allocated buffer of floats */
    		short *output_i,      /* Pointer to pre-allocated buffer of int16's */
    		int channel,          /* Channel to decode */
    		long samples,         /* Number of samples to decode */
    		int stream);

to read each remaining channel after the first channel.

To read video data there are two methods. RGB frames or YUV frames. To get an RGB frame use:
``

    int mpeg3_read_frame(mpeg3_t *file,
    		unsigned char **output_rows, // Array of pointers to the start of each output row
    		int in_x,                    // Location in input frame to take picture
    		int in_y,
    		int in_w,
    		int in_h,
    		int out_w,                   // Dimensions of output_rows
    		int out_h,
    		int color_model,             // One of the color model #defines given above.
    		int stream);

The video decoding works like a camcorder taking copy of a movie screen. The decoder "sees" a region of the movie screen defined by `in_x, in_y, in_w, in_h` and transfers it to the frame buffer defined by `**output_rows`. The input values must be within the boundaries given by `mpeg3_video_width` and `mpeg3_video_height`. The size of the frame buffer is defined by `out_w, out_h`. Although the input dimensions are constrained, the frame buffer can be any size.

`color_model` defines which RGB color model the picture should be decoded to and the possible values are given in libmpeg3.h. The frame buffer pointed to by `output_rows` must have enough memory allocated to store the color model you select.

You must allocate 4 extra bytes in the last output_row. This is scratch area for the MMX routines.

`mpeg3_read_frame` advances the position in the one stream by 1 frame.

The alternative is YUV frames:
``

    int mpeg3_read_yuvframe(mpeg3_t *file,
    		char *y_output,
    		char *u_output,
    		char *v_output,
    		int in_x,
    		int in_y,
    		int in_w,
    		int in_h,
    		int stream);

The behavior of in_x, in_y, in_w, in_h is identical to mpeg3_read_frame except here you have no control over the output frame size. You must allocate in_w * in_h for the y_output, and in_w * in_h / 4 for the chroma outputs.

STEP 7: Close the file

Be sure to close the file with `mpeg3_close(mpeg3_t *file)` when you're done with it.