I/O

Overview

As a computer vision API it is a requirement that images and videos are loaded into a common structure that can be processed by the rest of the API. In this case, we strive to isolate the I/O functions from the rest of the API. This serves three purposes:

  1. It allows implementations to be agnostic to the method and type of image storage, compression techniques and other factors
  2. It keeps implementations from having to worry about licenses, patents and other factors that can arise from distributing proprietary image formats
  3. It allows implementations to be “future-proof” with regards to future developments of image or video formats

To accomplish this goal the API defines a simple interface of two structures, JaniceImage and JaniceMediaIterator which correspond to a single image or frame and an entire media respectively. These interfaces allow pixel-level access for implementations and can be changed independently to work with new formats.

Structs

JaniceImage

A structure representing a single frame or an image

Fields

Name Type Description
channels uint32_t The number of channels in the image.
rows uint32_t The number of rows in the image.
cols uint32_t The number of columns in the image.
data uint8_t* A contiguous, row-major array containing pixel data.
owner bool True if the image owns its data and should delete it, false otherwise.

JaniceMediaIteratorState

A void pointer to a user-defined structure that contains state required for a JaniceMediaIterator.

JaniceMediaIterator

An interface representing a single image, a sparse selection of video frames or a full video. JaniceMediaIterator implements an iterator interface on media to enable lazy loading via function pointers.

is_video

A function pointer with signature:

JaniceError(JaniceMediaIterator* it, bool* video)

The function sets video to True if it is a video. Otherwise, it sets video to False. it should be considered a video if multiple still images can be retrieved with successive calls to next. This function should return JANICE_SUCCESS if video can be set to True or False.

get_frame_rate

A function pointer with signature:

JaniceError(JaniceMediaIterator* it, float* frame_rate)

The function sets frame_rate to the frame rate of it, if that information is available. If frame_rate can be set to a value this function should return JANICE_SUCCESS, otherwise it should return JANICE_INVALID_MEDIA. In the case of downsampling, this should return the observed frame rate.

get_physical_frame_rate

A function pointer with signature:

JaniceError(JaniceMediaIterator* it, float* physical_frame_rate)

The physical frame rate is the actual frame rate of the video, independent of processing done by media iterator. The function sets physical_frame_rate to the physical frame rate of it, if that information is available. If frame_rate can be set to a value this function should return JANICE_SUCCESS, otherwise it should return JANICE_INVALID_MEDIA.

seek

A function pointer with signature:

JaniceError(JaniceMediaIterator* it, uint32_t frame)

The function sets the internal state of it such that a successive call to next will return the image with index frame. If it is an image, this function should work for frame == 0, in which case it is equivalent to reset, otherwise JANICE_INVALID_MEDIA should be returned. If it is a video, the implementation may optionally do bounds checking on frame. If the seek is successful, JANICE_SUCCESS should be returned.

get

A function pointer with signature:

JaniceError(JaniceMediaIterator* it, JaniceImage* img, uint32_t frame)

This function gets a specific frame from it and stores it in img. It should not modify the internal state of it. If it is an image, this function should work or :code`frame` == 0. If frame != 0 and it is an image, this function should return JANICE_INVALID_MEDIA. If it is a video, the implementation may optionally do bounds checking on frame. If the get is successful, this function should return JANICE_SUCCESS. If the get is not successful, an appropriate error code should be returned and it may be left in an undefined state.

tell

A function pointer with signature:

JaniceError(JaniceMediaIterator* it, uint32_t* frame)

Get the current position of it and store it in frame. If it is an image, this function should return JANICE_INVALID_MEDIA. If it is a video and its position can be successfully queried, this function should return JANICE_SUCCESS. Otherwise, an appropriate error code should be returned.

reset

A function pointer with signature:

JaniceError(JaniceMediaIterator* it)

Reset it to an initial valid state. This function should return JANICE_SUCCESS if it can be reset, otherwise an appropriate error code should be returned.

physical_frame

A function pointer with signature:

JaniceError(JaniceMediaIterator* it, uint32_t frame, uint32_t* physical_frame)

Map an observed frame to a physical frame. If a mapping is possible this function should return JANICE_SUCCESS. Otherwise, an appropriate error code should be returned.

free_image

A function pointer with signature:

JaniceError(JaniceImage* img)

Free any memory associated with img. free_image should be called with the same iterator that allocated img with a call to either next or get. This function should return JANICE_SUCCESS if img is successfully freed, otherwise an appropriate error code should be returned.

free

A function pointer with signature:

JaniceError(JaniceMediaIterator** it)

Free any memory associated with it. This function should return JANICE_SUCCESS if it is freed successfully, otherwise and appropriate error code should be returned.

Fields

Name Type Description
is_video JaniceError(JaniceMediaIterator*, bool*) See is_video.
get_frame_rate JaniceError(JaniceMediaIterator*, float*) See get_frame_rate.
get_physical_frame_rate JaniceError(JaniceMediaIterator*, float*) See get_physical_frame_rate.
next JaniceError(JaniceMediaIterator*, JaniceImage*) See next.
seek JaniceError(JaniceMediaIterator*, uint32_t) See seek.
get JaniceError(JaniceMediaIterator*, JaniceImage*, uint32_t) See get.
tell JaniceError(JaniceMediaIterator*, uint32_t*) See tell.
reset JaniceError(JaniceMediaIterator*) See reset.
physical_frame JaniceError(JaniceMediaIterator*, uint32_t, uint32_t*) See physical_frame.
free_image JaniceError(JaniceImage*) See free_image.
free JaniceError(JaniceMediaIterator**) See free.
_internal JaniceMediaIteratorState A pointer to memory meant for internal use only. The implementation may use this to store persistent state about the iterator.

JaniceMediaIterators

A structure representing a list of JaniceMediaIterator objects.

Fields

Name Type Description
media JaniceMediaIterator* An array of media iterator objects.
length size_t The number of elements in media

JaniceMediaIteratorsGroup

A structure to represent a list of JaniceMediaIterators objects.

Fields

Name Type Description
group JaniceMediaIterators An array of media objects.
length size_t The number of elements in group