The JavaGrinders library provides functions for locating moving objects from video information, either from a live stream or from a recorded movie file. Several object recognition techniques are implemented to follow the movements of individuals over time. The video tracking algorithm analyzes sequential video frames and records the movement of targets in 2D or 3D. With integrated support for multiple HD cameras, the system allows tracking of:

1. •single objects in 2D
   

2. •single objects in 3D using multiple views
   

3. •multiple, color-labeled objects
   

4. •multiple unlabeled objects, each within a given subarea of the video frame
   

5. •multiple unlabeled objects within a shared area
   

Image subtraction removes the digital numeric values of a reference image from the image of interest in order to emphasize changes in information over that which remains constant. Contour tracing algorithms characterize clusters of signal that remain following this step. The tracker draws on routines of the opencv library, which provides highly efficient routines for arithmetic pixel operations. The libuvc library is used to connect to video devices for synchronous access to the camera’s streaming buffers.

Required Hardware includes a computer and a source of digital video frames. Instructions for installing software dependencies are described in a companion blog. Frame rates depend on processing resources, camera interface, lighting conditions, and complexity of object detection. You may either use standard detection algorithms included with the library or create your own for more specialized scenarios.

1. 
   

The application first captures a reference frame showing the empty arena. The introduction of an object of interest results in a signal of characteristic size, shape, and contrast. At defined intervals the program obtains a frame and attempts to locate the object within it. The quality of the tracking increases with the amount of contrast that exists between the animal and background (either dark object on light background, or light object on dark). The application characterizes the contour of the object within a given pixel threshold. The extensible nature of the object library allows for the development of custom search mechanisms. The library provides tracking procedures for one or multiple objects, or colored labels. If an object meeting the criteria is detected within a frame, the coordinate with its time reference and shape characteristics is obtained and reported into the console, an external text file, or an augmented video stream.

During initialization the of a tracker object, the library will attempt to communicate with sources of video information. Click the "reference" button to collect a reference frame. Then click the "analyze" button to start the analysis. If the results are not satisfactory, optimize conditions as outlined below.

The application subtracts the image of an empty arena from the current video frame, then proceeds to characterize whatever differences are left. The quality of the reference frame is a major determinant of how well the tracking will manage to place the object of interest. The application allows you to capture a video frame and use is as a reference. Alternatively the application can export and import reference frames in the form of PNG images. It is critical that this frame accurately portrays the surrounding without the object to be tracked.

To optimize criteria for tracking your object of interest you can use the “Auto TrackerSettings” menu item. The program will identify areas of difference between the current video signal and the recorded reference frame and it will suggest settings that are optimized to detect the object/s that produced this difference.

The library includes support for creating .avi movie files via the AVIProducer. This class is used to assemble Individual frames containing images from a capture stream, rendered spatial and time data elements, or combinations of the two.