FMF (Fly Movie Format) - simple, uncompressed movie storage format

New users are recommended to use MP4 files for video rather than FMF files. Strand Camera supports saving to MP4 files, using several different potential encoders.

The primary design goals of FMF files are:

• Single pass, low CPU overhead writing of lossless movies for realtime streaming applications
• Precise timestamping for correlation with other activities
• Simple format that can be written and read from a variety of languages.

These goals are achieved via using a very simple format. After an initial header containing meta-data such as image size and color coding scheme (e.g. monochromatic 8 bits per pixel, YUV422, etc.), repeated chunks of raw image data and timestamp are saved. Because the raw image data from the native camera driver is saved, no additional processing is performed. Thus, streaming of movies from camera to disk will keep the CPU free for other tasks, but it will require a lot of disk space. Furthermore, the disk bandwidth required is equivalent to the camera bandwidth (unless you save only a region of the images, or if you only save a fraction of the incoming frames).

The FMF file type defines raw image sequences where each image is stored exactly in the raw data bytes as they were acquired from the camera together with with a timestamp. There are two versions implemented, versions 1 and 3 (Version 2 was briefly used internally and is now best forgotten). Version 1 is deprecated and new movies should not be written in this format.

A Rust implementation to read and write .fmf files can be found in the github.com/strawlab/strand-braid repository.

Documentation for the file type and reading/writing .fmf files in Python can be found at the documentation of motmot.FlyMovieFormat.

A MATLAB® implementation can be found in the github.com/motmot/flymovieformat repository.

An R implementation can be found in the github.com/jefferis/fmfio repository.

Converting movies to and from FMF format with the fmf command line program

The fmf command line program from https://github.com/strawlab/strand-braid/tree/main/fmf/fmf-cli can be used for a variety of tasks with .fmf files, especially converting to and from other formats.

Here is the output fmf --help:

strawlab@flycube10:~$
fmf 0.1.0
work with .fmf (fly movie format) files

USAGE:
    fmf <SUBCOMMAND>

FLAGS:
    -h, --help       Prints help information
    -V, --version    Prints version information

SUBCOMMANDS:
    export-fmf       export an fmf file
    export-jpeg      export a sequence of jpeg images
    export-mp4       export to mp4
    export-png       export a sequence of png images
    export-y4m       export to y4m (YUV4MPEG2) format
    help             Prints this message or the help of the given subcommand(s)
    import-images    import a sequence of images, converting it to an FMF file
    info             print information about an fmf file

See https://github.com/strawlab/strand-braid/tree/main/fmf/fmf-cli for more information.

File Structure

The chunk size is specified in the header and is equal to the raw frame pixel data size plus 8 bytes for the timestamp. Thus a 640 pixel wide, 480 pixel high MONO8 format image would have a chunksize of 307208 (equal to 640 * 480 + 8).

Because the chunk size is constant for all frames, any chunk can be accessed by computing its position and seeking to that location. For the same reason, the image data within FMF files can be memory mapped.

Header Version 3

This is the preferred header for all new FMF files.

For the format field, the following pixel formats are known:

This list of pixel formats is not exhaustive and other formats can be added.

Header Version 1

⚠ This version is deprecated and no new files with this format should be written. ⚠

Only supports MONO8 pixel format.

Frame Chunks

Frame chunks have an identical format in FMF v1 and v3 files. From a given camera pixel format and size, they are constant in size and thus the Nth frame can be accessed by seeking to frame0_offset + n*chunksize. The image data is uncompressed raw image data as read directly from the camera.

Types used above

All numbers are little-endian (Intel standard).