9. Array Extensions

The picamera.array module provides a set of classes which aid in constructing n-dimensional numpy arrays from the camera output. In order to avoid adding a hard dependency on numpy to picamera, the module is not automatically imported by the main picamera package and must be explicitly imported.

The following classes are defined in the module:

9.1. PiBaseOutput

class picamera.array.PiBaseOutput(camera, size=None)[source]

Base class for all custom output classes defined in this module.

This class is not intended for direct use, but is a useful base-class for constructing custom outputs. The write() method simply appends data to the buffer attribute until the flush() method is called which in descendent classes is expected to construct a numpy array from the buffered data.

close()[source]

Closes the stream and frees all resources associated with it.

flush()[source]

Does nothing in this implementation but can be overriden in descendent classes to do something after capture/recording is complete.

read(n=-1)[source]

Raises NotImplementedError as this is a write-only stream.

readable()[source]

Returns False, indicating that the stream doesn’t support read().

seek(offset, whence=0)[source]

Raises NotImplementedError as this is a non-seekable stream.

seekable()[source]

Returns False, indicating that the stream doesn’t support seek().

tell()[source]

Raises NotImplementedError as this output has no buffer.

truncate(size=None)[source]

Raises NotImplementedError as this output has no buffer.

writable()[source]

Returns True, indicating that the stream supports write().

write(b)[source]

This base implementation ignores the data and returns the number of bytes, pretending it has written them.

9.2. PiRGBArray

class picamera.array.PiRGBArray(camera, size=None)[source]

Produces a 3-dimensional RGB array from an RGB capture.

This custom output class can be used to easily obtain a 3-dimensional numpy array, organized (rows, columns, colors), from an unencoded RGB capture. The array is accessed via the array attribute. For example:

import picamera
import picamera.array

with picamera.PiCamera() as camera:
    with picamera.array.PiRGBArray(camera) as output:
        camera.capture(output, 'rgb')
        print('Captured %dx%d image' % (
                output.array.shape[1], output.array.shape[0]))

You can re-use the output to produce multiple arrays by emptying it with truncate(0) between captures:

import picamera
import picamera.array

with picamera.PiCamera() as camera:
    with picamera.array.PiRGBArray(camera) as output:
        camera.resolution = (1280, 720)
        camera.capture(output, 'rgb')
        print('Captured %dx%d image' % (
                output.array.shape[1], output.array.shape[0]))
        output.truncate(0)
        camera.resolution = (640, 480)
        camera.capture(output, 'rgb')
        print('Captured %dx%d image' % (
                output.array.shape[1], output.array.shape[0]))

If you are using the GPU resizer when capturing (with the resize parameter of the various capture() methods), specify the resized resolution as the optional size parameter when constructing the array output:

import picamera
import picamera.array

with picamera.PiCamera() as camera:
    camera.resolution = (1280, 720)
    with picamera.array.PiRGBArray(camera, size=(640, 360)) as output:
        camera.capture(output, 'rgb', resize=(640, 360))
        print('Captured %dx%d image' % (
                output.array.shape[1], output.array.shape[0]))

9.3. PiYUVArray

class picamera.array.PiYUVArray(camera, size=None)[source]

Produces 3-dimensional YUV & RGB arrays from a YUV capture.

This custom output class can be used to easily obtain a 3-dimensional numpy array, organized (rows, columns, channel), from an unencoded YUV capture. The array is accessed via the array attribute. For example:

import picamera
import picamera.array

with picamera.PiCamera() as camera:
    with picamera.array.PiYUVArray(camera) as output:
        camera.capture(output, 'yuv')
        print('Captured %dx%d image' % (
                output.array.shape[1], output.array.shape[0]))

The rgb_array attribute can be queried for the equivalent RGB array (conversion is performed using the ITU-R BT.601 matrix):

import picamera
import picamera.array

with picamera.PiCamera() as camera:
    with picamera.array.PiYUVArray(camera) as output:
        camera.resolution = (1280, 720)
        camera.capture(output, 'yuv')
        print(output.array.shape)
        print(output.rgb_array.shape)

If you are using the GPU resizer when capturing (with the resize parameter of the various capture() methods), specify the resized resolution as the optional size parameter when constructing the array output:

import picamera
import picamera.array

with picamera.PiCamera() as camera:
    camera.resolution = (1280, 720)
    with picamera.array.PiYUVArray(camera, size=(640, 360)) as output:
        camera.capture(output, 'yuv', resize=(640, 360))
        print('Captured %dx%d image' % (
                output.array.shape[1], output.array.shape[0]))

9.4. PiBayerArray

class picamera.array.PiBayerArray(camera)[source]

Produces a 3-dimensional RGB array from raw Bayer data.

This custom output class is intended to be used with the capture() method, with the bayer parameter set to True, to include raw Bayer data in the JPEG output. The class strips out the raw data, constructing a 3-dimensional numpy array organized as (rows, columns, colors). The resulting data is accessed via the array attribute:

import picamera
import picamera.array

with picamera.PiCamera() as camera:
    with picamera.array.PiBayerArray(camera) as output:
        camera.capture(output, 'jpeg', bayer=True)
        print(output.array.shape)

Note that Bayer data is always full resolution, so the resulting array always has the shape (1944, 2592, 3); this also implies that the optional size parameter (for specifying a resizer resolution) is not available with this array class. As the sensor records 10-bit values, the array uses the unsigned 16-bit integer data type.

By default, de-mosaicing is not performed; if the resulting array is viewed it will therefore appear dark and too green (due to the green bias in the Bayer pattern). A trivial weighted-average demosaicing algorithm is provided in the demosaic() method:

import picamera
import picamera.array

with picamera.PiCamera() as camera:
    with picamera.array.PiBayerArray(camera) as output:
        camera.capture(output, 'jpeg', bayer=True)
        print(output.demosaic().shape)

Viewing the result of the de-mosaiced data will look more normal but still considerably worse quality than the regular camera output (as none of the other usual post-processing steps like auto-exposure, white-balance, vignette compensation, and smoothing have been performed).

9.5. PiMotionArray

class picamera.array.PiMotionArray(camera, size=None)[source]

Produces a 3-dimensional array of motion vectors from the H.264 encoder.

This custom output class is intended to be used with the motion_output parameter of the start_recording() method. Once recording has finished, the class generates a 3-dimensional numpy array organized as (frames, rows, columns) where rows and columns are the number of rows and columns of macro-blocks (16x16 pixel blocks) in the original frames. There is always one extra column of macro-blocks present in motion vector data.

The data-type of the array is an (x, y, sad) structure where x and y are signed 1-byte values, and sad is an unsigned 2-byte value representing the sum of absolute differences of the block. For example:

import picamera
import picamera.array

with picamera.PiCamera() as camera:
    with picamera.array.PiMotionArray(camera) as output:
        camera.resolution = (640, 480)
        camera.start_recording(
              '/dev/null', format='h264', motion_output=output)
        camera.wait_recording(30)
        camera.stop_recording()
        print('Captured %d frames' % output.array.shape[0])
        print('Frames are %dx%d blocks big' % (
            output.array.shape[2], output.array.shape[1]))

If you are using the GPU resizer with your recording, use the optional size parameter to specify the resizer’s output resolution when constructing the array:

import picamera
import picamera.array

with picamera.PiCamera() as camera:
    camera.resolution = (640, 480)
    with picamera.array.PiMotionArray(camera, size=(320, 240)) as output:
        camera.start_recording(
            '/dev/null', format='h264', motion_output=output,
            resize=(320, 240))
        camera.wait_recording(30)
        camera.stop_recording()
        print('Captured %d frames' % output.array.shape[0])
        print('Frames are %dx%d blocks big' % (
            output.array.shape[2], output.array.shape[1]))

Note

This class is not suitable for real-time analysis of motion vector data. See the PiMotionAnalysis class instead.

9.6. PiMotionAnalysis

class picamera.array.PiMotionAnalysis(camera, size=None)[source]

Provides a basis for real-time motion analysis classes.

This custom output class is intended to be used with the motion_output parameter of the start_recording() method. While recording is in progress, the write method converts incoming motion data into numpy arrays and calls the stub analyse() method with the resulting array (which deliberately raises NotImplementedError in this class).

Warning

Because the analyse() method will be running within the encoder’s callback, it must be fast. Specifically, it needs to return before the next frame is produced. Therefore, if the camera is running at 30fps, analyse cannot take more than 1/30s or 33ms to execute (and should take considerably less given that this doesn’t take into account encoding overhead). You may wish to adjust the framerate of the camera accordingly.

The array passed to analyse() is organized as (rows, columns) where rows and columns are the number of rows and columns of macro-blocks (16x16 pixel blocks) in the original frames. There is always one extra column of macro-blocks present in motion vector data.

The data-type of the array is an (x, y, sad) structure where x and y are signed 1-byte values, and sad is an unsigned 2-byte value representing the sum of absolute differences of the block.

An example of a crude motion detector is given below:

import numpy as np
import picamera
import picamera.array

class DetectMotion(picamera.array.PiMotionAnalysis):
    def analyse(self, a):
        a = np.sqrt(
            np.square(a['x'].astype(np.float)) +
            np.square(a['y'].astype(np.float))
            ).clip(0, 255).astype(np.uint8)
        # If there're more than 10 vectors with a magnitude greater
        # than 60, then say we've detected motion
        if (a > 60).sum() > 10:
            print('Motion detected!')

with picamera.PiCamera() as camera:
    with DetectMotion(camera) as output:
        camera.resolution = (640, 480)
        camera.start_recording(
              '/dev/null', format='h264', motion_output=output)
        camera.wait_recording(30)
        camera.stop_recording()

You can use the optional size parameter to specify the output resolution of the GPU resizer, if you are using the resize parameter of start_recording().