Camera Movement Correction
Background
A common issue to deal with is that after a camera has been spatially calibrated, the camera then moves and needs to be re-calibrated. The calcam.movement module contains tools for dealing with this. The problem is framed as movement of the camera image (a “moved image”) with respect to a “reference image” e.g. an image that has already been calibrated. There are two approaches to deal with this:
Apply transformations to warped images, or pixel coordinates in warped images, to make them conform to a reference calibration.
Update the reference calibration to apply to the moved image.
Which approach is preferable will depend on how you are using calcam results. See the detailed documentation below for details of how these methods are implemented in calcam.movement, or see the Examples page for examples of how to use this functionality.
Determining Camera Movement
Calcam can try to detect the movement between two images automatically, or there is a GUI tool which allows manual feature matching by the user or automatic detection but user validation.
- calcam.movement.detect_movement(ref, moved)
Attempt to auto-detect image movement between two images using sparse optical flow and return a MovementCorrection object representing the movement. If the movement cannot be successfully determined, raises calcam.movement.DetectionFailedError
- Parameters
ref (np.ndarray or calcam.Calibration) – Reference image or calibration to align to. This can be either an array containg a refrence image, or a calcam calibrationn containing an image.
moved (np.ndarray or calcam.Calibration) – Moved image or calibration to align. This can be either an array containg a refrence image, or a calcam calibrationn containing an image.
- Returns
Movement correction object representing the movement correction.
- Return type
- Raises
- calcam.movement.phase_correlation_movement(ref, moved)
Attempt to auto-detect a simple rigid translation of an image relative to a reference image using Phase Correlation and return a MovementCorrection object representing the movement.
- Parameters
ref (np.ndarray or calcam.Calibration) – Reference image or calibration to align to. This can be either an array containg a refrence image, or a calcam calibrationn containing an image.
moved (np.ndarray or calcam.Calibration) – Moved image or calibration to align. This can be either an array containg a refrence image, or a calcam calibrationn containing an image.
- Returns
Movement correction object representing the movement correction.
- Return type
- Raises
- calcam.movement.manual_movement(ref, moved, correction=None, parent_window=None)
Determine camera movement (semi-)manually using a GUI tool. See the Camera Movement Determination GUI doc page for the GUI user guide.
Paremeters:
ref (np.ndarray or calcam.Calibration) : Reference image or calibration to align to. This can be either an array containg a refrence image, or a calcam calibrationn containing an image.
mmoved (np.ndarray or calcam.Calibration) : Moved image or calibration to align. This can be either an array containg a refrence image, or a calcam calibrationn containing an image.
correction (MovementCorrection) : Existing movement correction object to start from.
parent_window (QWidget) : If being called from a QT GUI window class, a reference to the parent window must be passed. If it is not, the parent window might irrecoverably freeze when the movement dialog is closed.
- Returns
Either the image transofmration, or None is the user does not define one.
- Return type
MovementCorrection or NoneType
The MovementCorrection class
The above functions return movement correction objects, which represent the geometrical transform between the reference and moved images. These objects then have various methods for transforming between reference and moved images and coordinates.
- class calcam.movement.MovementCorrection(matrix, im_shape, ref_points, moved_points, src)
Class to represent a geometric transform from a moved image to a reference image. This type of object is returned by image movement correction related functions.
- Parameters
matrix (np.matrix) – 2x3 or 3x3 Affine or projective transform matrix
im_shape (tuple) – Image array dimensions (rows,cols) to which this transform applies
ref_points (np.ndarray) – Nx2 array containing coordinates of points on the reference image
moved_points (np.ndarray) – Nx2 array containing coordinates of corresponding points on the moved image
src (string) – Human-readable description of how the transform was created.
- get_ddscore(ref_im, moved_im, tol=50)
Get the DDScore for this movement correction when applied to the given image pair. DDScore is a score developed by Van-Stroud et.al. which indicates how much better the alignment of two input images after application of the correction. The score ranges from -1 to 1, where negative values indicate the alignment gets worse, 0 is no change or undetermined, and positive values are an improvement.
- Parameters
ref_im (np.ndarray) – Reference aligned image
moved_im (np.ndarray) – Moved image
tol (float) – Tolerance
- Returns
DDscore in the range -1,1
- Return type
float
- classmethod load(filename)
Load a movement correction from a .cmc file on disk.
- Parameters
filename (string) – File name to load from.
- moved_to_ref_coords(x, y)
Given image coordinates in the moved image, return the corresponding coordinates in the reference image.
- Parameters
x (np.ndarray) – Array of x (horizontal) pixel coordinates in the moved image
y (np.ndarray) – Array of y (vertucal) pixel coordinates in the moved image
- Returns
Array of x (horizontal) pixel coordinates in the reference image np.ndarray : Array of y (vertical) pixel coordinates in the reference image
- Return type
np.ndarray
- ref_to_moved_coords(x, y)
Given image coordinates in the reference image, return the corresponding coordinates in the moved image.
- Parameters
x (np.ndarray) – Array of x (horizontal) pixel coordinates in the reference image
y (np.ndarray) – Array of y (vertucal) pixel coordinates in the reference image
- Returns
Array of x (horizontal) pixel coordinates in the moved image np.ndarray : Array of y (vertical) pixel coordinates in the moved image
- Return type
np.ndarray
- property rotation
Image rotation to go from the moved to reference image in degrees clockwise.
- save(filename)
Save the correction to a given filename.
Paremeters:
filaneme (string) : Filename to which to save the correction. The file extension is .cmc; if this is not included in the given filename it is added.
- property scale
Scale factor to go from moved to reference image.
- property translation
x, y translation in pixels to go from the moved to reference image.
- warp_moved_to_ref(image, interp='linear', fill_edges=False)
Warp a moved image to align with the reference one. Note: this can also be used on binned or englarged images, with respect to the one used for the original movement correction determination, provided they are scaled proportionally i.e. have the same aspect ratio as the originals.
- Parameters
image (np.ndarray) – Moved image to warp
interp (string) – Interpolation method to use, can be ‘linear’ or ‘nearest’
fill_edges (bool) – Whether to fill the warped image edges with a repetition of the edge pixels from the image (if True), or leave un-filled images edges as 0 value (if False; this is the default).
- Returns
Two ndarrays: the warped image, and a boolean mask the same shape as the image indicating which pixels contain valid image data (True) and which do not (False).
- Return type
np.ndarray
- warp_ref_to_moved(image, interp='linear', fill_edges=False)
Warp a reference-aligned image to align with the moved one. Note: this can also be used on binned or englarged images, with respect to the one used for the original movement correction determination, provided they are scaled proportionally i.e. have the same aspect ratio as the originals.
- Parameters
image (np.ndarray) – Image to warp
interp (string) – Interpolation method to use, can be ‘linear’ or ‘nearest’
- Returns
Two ndarrays: the warped image, and a boolean mask the same shape as the image indicating which pixels contain valid image data (True) and which do not (False).
- Return type
np.ndarray
Adjusting calibrations for image movement
Having obtained a movement correction object describing the movement, you can update an exicting calcam calibration to apply to the new image:
- calcam.movement.update_calibration(calibration, moved_image, mov_correction, image_src=None, coords='Display')
Update a given calibration to account for image movement. This currently only supports point pair fitting calibrations, not manual alignment calibrations.
- Parameters
calibration (calcam.Calibration) – Calibration to update.
moved_image (np.ndarray) – Moved image
mov_correction (MovementCorrection) – Movement correction object representing the movement between the original calibrated image and the moved image.
image_src (string) – Human-readable description of where the moved image comes from, for data provenance tracking.
coords (string) – ‘Display’ or ‘Original’, whether the movement correction and moved image are in the calibration’s display or original image orientation.
- Returns
Updated calibration object for the moved image
- Return type
Exceptions
- class calcam.movement.DetectionFailedError
Exception raised if the automatic image movement detection fails to determine the image movement with sufficient confidence or quality.