Using Calibration Results

Once you have created calibrations using the Calcam GUI tools, you can import calcam as a Python package and use it to extract information from the calibrations and/or integrate use of the results in to your workflow in Python. This page documents the features of the Calcam API for working with calibration results.

For examples of using these features, see the Examples page.

The Calibration Class

The calibration class is used to represent Calcam calibrations and provides various methods for getting information about the camera viewing geometry.

class calcam.Calibration(load_filename=None, cal_type=None)

Class representing a camera view calibration.

A complete Calibration object contains the camera image which was calibrated (if any), the point pairs used for fitting (if applicable), the camera model parameters, and metadata about each of these.

If instantiated with the name of a .ccc file to load, the resulting object represents the calibration contained in that file. If no file name is given, an empty calibration object of a specified type is created.

Parameters
  • load_filename (str) – File name of the calibration to load. If not given, an “empty” calibration object is created.

  • cal_type (str) – Required only if load_file is not specified i.e. creating an empty calibration object. Must be one of “fit”, “alignment” or “virtual”. If load_file is provided, this is ignored.

get_cam_matrix(subview=None)

Get the camera matrix.

Parameters

subview (int) – For calibrations with multiple sub-views, which sub-view index to return the camera matrix for.

Returns

3x3 camera matrix.

Return type

np.matrix

get_cam_roll(subview=None)

Get the camera roll. This is the angle between the lab +Z axis and the camera’s “view up” direction.

Parameters

subview (int) – For calibrations with multiple sub-views, which sub-view index to return the camera roll for.

Returns

Camera roll in degrees. Positive angles correspond to a clockwise roll of the camera i.e. anti-clockwise roll of the image.

Return type

float

get_cam_to_lab_rotation(subview=None)

Get a 3D rotation matrix which will rotate a point in the camera coordinate system (see Calcam theory documentation) in to the lab coordinate system’s orientation.

Parameters

subview (int) – For calibrations with multiple sub-views, specifies which sub-view to return the rotation matrix for.

Returns

3x3 rotation matrix.

Return type

np.matrix

get_fov(subview=None, fullchip=False)

Get the camera field of view.

Parameters
  • subview (int) – For calibrations with multiple sub-views, which sub-view index to return the field of view for.

  • fullchip (bool) – For calibrations with multiple sub-views, setting this to True will return the field of view defined by the camaera model of the specified sub-view, as if that sub-view covered the whole image.

Returns

2-element tuple containing the full angles of the horizontal and vertical fields of view (h_fov,v_fov) in degrees.

Return type

tuple

get_image(coords='Display')

Get the image which was calibrated.

Parameters

coords (str) – Either Display or Original, what orientation to return the image.

Returns

If the calibration contains an image: returns image data array with shape (h x w x n) where n is the number of colour channels in the image. If the calibration does not contain an image, returns None.

Return type

np.ndarray or NoneType

get_los_direction(x=None, y=None, coords='Display', subview=None)

Get unit vectors representing the directions of the camera’s sight-lines in 3D space.

Can be used together with get_pupilpos() to obtain a full description of the camera’s sight-line geometry.

Parameters
  • x,y (sequence of floats) – Image pixel coordinates at which to get the sight-line directions. x and y must be the same shape. If not specified, the line of sight direction at the centre of every detector pixel is returned.

  • coords (str) – Either Display or Original, specifies which image orientation the provided x and y inputs and/or shape of the returned array correspond to.

  • subview (int) – If specified, forces the use of the camera model from the specified sub-view index. If not given, the correct sub-view(s) will be chosen automatically.

Returns

Array of sight-line vectors. If specifying x_pixels and y_pixels, the output array will be the same shape as the input arrays but with an extra dimension added. The extra dimension contains the [X,Y,Z] components of the sight-line vectors. If not specifying x_pixels and y_pixels, the output array shape will be (h x w x 3) where w and h are the image width and height in pixels.

Return type

np.ndarray

get_pupilpos(x=None, y=None, coords='display', subview=None)

Get the camera pupil position in 3D space.

Can be used together with get_los_direction() to obtain a full description of the camera’s sight line geometry.

Parameters
  • x,y (float or numpy.ndarray) – For calibrations with more than one subview, get the pupil position(s) corresponding to these given image pixel coordinates.

  • coords (str) – Only used if x and y are also given. Either Display or Original, specifies whether the provided x and y are in display or original coordinates.

  • subview (int) – Which sub-view to get the pupil position for. Only required for calibrations with more than 1 sub-view.

Returns

Camera pupil position in 3D space. If not specifying x or y inputs, this will be a 3 element array containing the [X,Y,Z] coordinates of the pupil position in metres. If using x and y inputs, the output array will be the same shape as the x and y input arrays with an additional dimension added; the X, Y and Z components are then given along the new new array dimension.

Return type

np.ndarray

get_raysect_camera(coords='Display', binning=1)

Get a RaySect observer corresponding to the calibrated camera.

Parameters

coords (str) – Either Display or Original specifying the orientation of the raysect camera.

Returns

RaySect camera object.

Return type

raysect.optical.observer.imaging.VectorCamera

get_undistort_coeffs(radial_terms=None, include_tangential=None, subview=None)

Get a set of parameters which can be used to analytically calculate image coordinate un-distortion. This can be useful if you need to calculate point un-distortion faster than the usual numerical method. This fits a model of the form of the perspective distortion model but with coordinate vectors \((x_n, y_n)\) and \((x_d,y_d)\) interchanged. Which coefficients are included can be set by optional input arguments; by default the same terms which were enabled in the calibration fit are also included in this fit. Note: this function does not work with fisheye calibrations.

Parameters
  • radial_terms (int) – Number of terms to include in the radial distortion model, can be in the range 1 - 3. If < 3, higher order coefficients are set to 0. If not provided, uses the same number of terms as the calibration fit.

  • include_tangential (bool) – Whether to include the tangential distortion coefficients p1 and p2. If not given, uses the same option as was used in the calibration.

  • subview (int) – For calibrations with multiple sub-views, what sub-view to get the parameters for.

Returns

A dictionary containing the fitted coeffcients. Radial distortion coefficients are in keys: ‘k1’, ‘k2’ and ‘k3’; tangential coefficients are in keys ‘p1’ and ‘p2’. An additional key ‘rms_error’ gives the RMS fit error, in pixels, which indicates how well these fitted parameters reproduce the full numerical distortion inversion.

Return type

dict

project_points(points_3d, coords='display', check_occlusion_with=None, fill_value=nan, occlusion_tol=0.001)

Get the image coordinates corresponding to given real-world 3D coordinates.

Optionally can also check whether the 3D points are hidden from the camera’s view by part of the CAD model being in the way.

Parameters
  • points_3d – 3D point coordinates, in metres, to project on to the image. Can be EITHER an Nx3 array, where N is the number of 3D points and each row gives [X,Y,Z] for a 3D point, or a sequence of 3 element sequences, where each 3 element array specifies a 3D point.

  • coords (str) – Either Display or Original, specifies which image orientation the returned image coordinates should correspond to.

  • check_occlusion_with (calcam.CADModel or calcam.RayData) – If provided and fill_value is not None, for each 3D point the function will check if the point is hidden from the camera’s view by part of the provided CAD model. If a point is hidden its returned image coordinates are set to fill_value. Note: if using a RayData onject, always use Raydata resulting from a raycast of the complete detector, or else project_points will be incredibly slow.

  • fill_value (float) – For any 3D points not visible to the camera, the returned image coordinates will be set equal to this value. If set to None, image coordinates will be returned for every 3D point even if the point is outside the camera’s field of view or hidden from view.

  • occlusion_tol (float) – Tolerance (in mrtres) to use to check point occlusion. Try increasing this value if having trouble with points being wrongly detected as occluded.

Returns

A list of Nx2 NumPY arrays containing the image coordinates of the given 3D points (N is the number of input 3D points). Each NumPY array corresponds to a single sub-view, so for images without multuiple sub-views this will return a single element list containing an Nx2 array. Each row of the NumPY arrays contains the [X,Y] image coordinates of the corresponding input point. If fill_value is not None, points not visible to the camera have their coordinates set to [fill_value, fill_value].

Return type

list of np.ndarray

set_detector_window(window)

Adjust the calibration to apply to a different detector region for than was used to perform the calibration. Useful for example if a CMOS camera has been calibrated over the full frame, but you now want to use this calibration for data which has been cropped.

Calling this function with None as the single argument sets the calibration back to its “native” state. Otherwise, call with a 4 element tuple specifying the left,top,width and height of the detector window.

Detector window coordinates must always be in original coordinates.

Parameters

window (tuple or list) – A 4-element tuple or list of integers defining the detector window coordinates (Left,Top,Width,Height)

set_extrinsics(campos, upvec=None, camtar=None, view_dir=None, cam_roll=None, src=None)

Manually set the camera extrinsic parameters. Only applicable for synthetic or manual alignment type calibrations.

Parameters
  • campos (sequence) – 3-element sequence specifying the camera position (X,Y,Z) in metres.

  • view_dir (sequence) – 3D vector [X,Y,Z] specifying the camera view direction. Either view_dir or camtar must be given; if both are given then view_dir is used.

  • camtar (sequence) – 3-element sequence specifying a point in 3D space where the camera is pointed to. Either camtar or view_dir must be given; if both are given then view_dir is used.

  • upvec (sequence) – 3-element sequence specifying the camera up vector. Either upvec or cam_roll must be given; if both are given then upvec is used. upvec must be orthogonal to the viewing direction.

  • cam_roll (float) – Camera roll in degrees. This is the angle between the lab +Z axis and the camera’s “view up” direction. Either cam_roll or upvec must be given; if both are given the upvec is used.

  • src (str) – Human-readable string describing where these extrinsics come from, for data provenance. If not given, basic information like current username, hostname and time are used.

undistort_image(image, coords='display')

Correct lens distortion a given image from the calibrated camera, to give an image with a pure perspective projection.

Parameters
  • image (np.ndarray) – (h x w x N) array containing the image to be un-distorted, where N is the number of colour channels.

  • coords (str) – Either Display or Original, specifies which orientation the input image is in.

Returns

Image data array the same shape as the input array containing the corrected image.

Return type

np.ndarray

Rendering images

Calcam can be used to render images of a CAD model from the calibrated camera’s point of view. The render includes all lens distortion in the camera model, so should match exactly with a well calibrated real camera image.

calcam.render_cam_view(cadmodel, calibration, extra_actors=[], filename=None, oversampling=1, aa=1, transparency=False, verbose=True, coords='Display', interpolation='Cubic')

Render an image of a given CAD model from the point of view of a given calibration.

Parameters
  • cadmodel (calcam.CADModel) – CAD model of scene

  • calibration (calcam.Calibration) – Calibration whose point-of-view to render from.

  • extra_actors (list of vtk.vtkActor) – List containing any additional vtkActors to add to the scene in addition to the CAD model.

  • filename (str) – Filename to which to save the resulting image. If not given, no file is saved.

  • oversampling (float) – Used to render the image at higer (if > 1) or lower (if < 1) resolution than the calibrated camera. Must be an integer power of 2.

  • aa (int) – Anti-aliasing factor. 1 = no anti-aliasing.

  • transparency (bool) – If true, empty areas of the image are set transparent. Otherwise they are black.

  • verbose (bool) – Whether to print status updates while rendering.

  • coords (str) – Either Display or Original, the image orientation in which to return the image.

  • interpolation (str) – Either nearest or cubic, inerpolation used if re-sampling of the image during the rendering process. Note: if set to nesrest, will make aa > 1 ineffective.

Returns

Array containing the rendered 8-bit per channel RGB (h x w x 3) or RGBA (h x w x 4) image. Also saves the result to disk if the filename parameter is set.

Return type

np.ndarray

Ray casting

The calibration class can provide line-of-sight directions for given pixel coordinates, however it is often necessary to know where these sight lines terminate at a surface in the CAD model. For example in calculation of geometry matricies for camera data inversion, the length of each camera sight line needs to be known. The function calcam.raycast_sightlines() is provided for this purpose; it determines the 3D coordinates where the given pixels’ sight-lines intersect the CAD model.

calcam.raycast_sightlines(calibration, cadmodel, x=None, y=None, exclusion_radius=0.0, binning=1, coords='Display', verbose=True, intersecting_only=False, force_subview=None, status_callback=None)

Ray cast camera sight-lines to determine where they intersect the given CAD model.

Parameters
  • calibration (calcam.Calibration) – Calibration whose sight-lines to raycast.

  • cadmodel (calcam.CADModel) – CAD model to check intersection with.

  • y (x,) – x and y image pixel coordinates for which to cast sight-lines. If not specified, one ray is cast at the centre of every detector pixel. x and y must be the same shape.

  • exclusion_radius (float) – Distance from camera pupil (in meters) over which to ignore ray intersections with CAD surfaces. This is useful for views involving mirrors and/or prisms where unfolding the optical path results in the virtual pupil location falling behind nearby CAD surfaces. Setting a sufficient exclusion radius will cause the rays to be launched from the other side of these surfaces so they intersect the correct surfaces present in the image.

  • binning (int) – If not explicitly providing x and y image coordinates, pixel binning for ray casting. This specifies NxN binning, i.e. for a value of 2, one ray is cast at the centre of every 2x2 cluster of pixels.

  • coords (str) – Either Display or Original. If specifying x and y coordinates, specifies whether the input x and y are in original or display coords. Otherwise, specifies the orientation of the returned data.

  • intersecting_only (bool) – If set to True, the ray end coordinates and length for sight-lines which do not intersect with the CAD model (i.e. sight lines which “escape” through holes in the model) are set to NaN. This is useful if you are only interested in sight-lines which intersect with CAD model surfaces.

  • force_subview (int) – If specified, forces use of the camera model from this index of sub-view in the calibration. Otherwise, sub-views are chosen according to the sub-view mask in the calibration.

  • verbose (bool) – Whether to print status updates during ray casting (depreciated in favour of status_callback)

  • status_callback (callable) – Callable which takes a single argument to be called with status updates. The argument will either be a string for textual status updates or a float from 0 to 1 specifying the progress of the calculation. If set to None, no status updates are issued. For backwards compatibility, if set to None but verbose is set to True, status_callback will be set such that status updates go to stdout.

Returns

Object containing the results.

Return type

calcam.RayData

class calcam.RayData(filename=None)

Class representing ray casting results.

Objects of this class are returned by calcam.raycast_sightlines(). It can also be used to save and load ray cast results to disk.

Parameters

filename (str) – File name of netCDF file containing saved RayData to load. If not given, an empty RayData object is created.

get_ray_directions(x=None, y=None, im_position_tol=1, coords='Display')

Get unit vectors specifying the sight-line directions. Note that ray casting is not required to get this information: see calcam.Calibration.get_los_direction() for the same functionality, however this can be useful if you have the RayData but not calcam.Calibration object loaded when doing the analysis.

Parameters
  • x,y (array-like) – x and y pixel coordinates at which to get the ray directions. If not specified, the ray directions of every sight-line are returned.

  • im_position_tol (float) – If x and y are specified but no sight-line was cast at exactly the input coordinates, the nearest casted sight-line will be returned instead provided the pixel coordinates wre within this many pixels of the requested coordinates.

  • coords (str) – Either Display or Original, specifies what orientation the input x and y correspond to or orientation of the returned array.

Returns

Array containing the sight-line directions. If the ray cast was for the full detector and x and y are not specified, the array shape will be (h x w x 3) where w nd h are the image width and height. Otherwise it will be the same shape as the input x and y coordinates plus an extra dimension.

Return type

np.ndarray

get_ray_end(x=None, y=None, im_position_tol=1, coords='Display')

Get the 3D x,y,z coordinates where the casted rays / sightlines intersect the CAD model.

Note: it is advised to use this function rather than accessing the ray_end_coords attribute of the object directly because this function will respect any image cropping applied using the set_detector_window() method.

Parameters
  • x,y (array-like) – Image pixel coordinates at which to get the sight-line end coordinates. If not specified, the end coordinates of all casted sight lines will be returned.

  • im_position_tol (float) – If x and y are specified but no sight-line was cast at exactly the input coordinates, the nearest casted sight-line will be returned instead provided the pixel coordinates wre within this many pixels of the requested coordinates.

  • coords (str) – Either Display or Coords, specifies what orientation the input x and y correspond to or orientation of the returned array.

Returns

An array containing the 3D coordinates, in metres, of the points where each sight line intersects the CAD model. If x and y coordinates were given either to this function or to calcam.raycast_sightlines(), the shape of this array is the same as the input x and y arrays with an additional dimension added which contains the [X,Y,Z] 3D coordinates. Otherwise the shape is (h x w x 3) where w and h are the image width and height (in display coords).

Return type

np.ndarray

get_ray_lengths(x=None, y=None, im_position_tol=1, coords='Display')

Get the sight-line lengths either of all casted sight-lines or at the specified image coordinates.

Parameters
  • x,y (array-like) – Image pixel coordinates at which to get the sight-line lengths. If not specified, the lengths of all casted sight lines will be returned.

  • im_position_tol (float) – If x and y are specified but no sight-line was cast at exactly the input coordinates, the nearest casted sight-line will be returned instead provided the pixel coordinates wre within this many pixels of the requested coordinates.

  • coords (str) – Either Display or Coords, specifies what orientation the input x and y correspond to or orientation of the returned array.

Returns

Array containing the sight-line lengths. If the ray cast was for the full detector and x and y are not specified, the array shape will be (h x w) where w nd h are the image width and height. Otherwise it will be the same shape as the input x and y coordinates.

Return type

np.ndarray

get_ray_start(x=None, y=None, im_position_tol=1, coords='Display')

Get the 3D x,y,z coordinates of the “start” of the casted rays / sightlines.

Note: it is advised to use this function rather than accessing the ray_start_coords attribute of the object directly because this function will respect any image cropping applied using the set_detector_window() method.

Parameters
  • x,y (array-like) – Image pixel coordinates at which to get the sight-line start coordinates. If not specified, the start coordinates of all casted sight lines will be returned.

  • im_position_tol (float) – If x and y are specified but no sight-line was cast at exactly the input coordinates, the nearest casted sight-line will be returned instead provided the pixel coordinates wre within this many pixels of the requested coordinates.

  • coords (str) – Either Display or Coords, specifies what orientation the input x and y correspond to or orientation of the returned array.

Returns

An array containing the 3D coordinates, in metres, of the start of each sight line (i.e. camera pupil position). If x and y coordinates were given either to this function or to calcam.raycast_sightlines(), the shape of this array is the same as the input x and y arrays with an additional dimension added which contains the [X,Y,Z] 3D coordinates. Otherwise the shape is (h x w x 3) where w and h are the image width and height (in display coords).

Return type

np.ndarray

history

Human readable description of where the raydata came from.

Type

str

save(filename)

Save the RayData to a netCDF file.

Parameters

filename (str) – File name to save to.

set_detector_window(window)

Adjust the raydata to apply to a different detector region for than was used to perform the original raycast. Useful for example if a CMOS camera has been calibrated over the full frame, but you now want to use this calibration for data which has been cropped.

Calling this function with None as the single argument sets the raydata back to its “native” state. Otherwise, call with a 4 element tuple specifying the left,top,width and height of the detector window.

Detector window coordinates must always be in original coordinates.

Parameters

window (tuple or list) – A 4-element tuple or list of integers defining the detector window coordinates (Left,Top,Width,Height)