Video-tracking Application

This page summarizes the current progress for a free-ware, flexible, java-based, cross-platform video tracker.

Multiple modes are supported:

• Pinpoint a single object in an arena
• ID an dark object on a light background or a light object on a dark background
• Track directionality of a non-round object with characterization of object outline
• Track multiple (light/dark vs. background) objects, each within its own subarea of the frame
• Track multiple, color-labeled objects across the entire video frame.
• Track a single light/dark object in 3D using 3 different views of the arena

Framerates as high as 30fps are possible - depending on hardware and complexity of detection. Either use my pre-canned detection algorithms or use your own homespun ones. A new version of the library (2.7, build 9; released 5/6/08).

Hardware must feature a computer and a source of video frames. Supported are digital video cameras plugged in via a firewire port, USB webcam, analog/digital converter board installed in your computer, or an external A/D box (e.g., dataVideo DAC-200, Dazzle Hollywood DV Bridge). The computer must have a recent version of java (Java 1.4 or higher) and the quicktime for java library (QT6 or higher) installed. Instructions for installing QTJava on Windows is found here. Software is needed that makes individual sequencegrabbers/videodevices available to the system - these are called vDIGs (video digitizer components) and they come automatically with the mac. On windows you may need to download a third-party vdig (e.g. abstract plane or winvdig).

Download

At the heart of the software is a set of java objects (SGTracker, FrameAnalyzer) grouped into the SGTracker library. Its main functionality is based on routines from the Quicktime for java libraries. The present version has been updated to run on Apple Quicktime 7.2.

• SGTrackerSDK: the standard development kit contains the JavaGrinders library (2.7, build 9; released 5/6/08), documentation, compiled programs and demo source code in xCode project format. Download either as
  • SGTrackerSDK.sit (Stuffit archive)
  • SGTrackerSDK.zip (zipped archive)
• SGTrackerLib 2.7b9 API Documentation

Setup

1. The program captures a reference frame of the arena without the object to be tracked
   • filename to store coordinates and time references as text
   • handle how the computer will receive video info from the camera (choose compression: "none" with 256 gray levels, also choose a brightness and contrast seting that will later give you the best contrast between object and background, the framerate is not the one used by the tracking application but the one that feeds it to the computer, it should thus be as fast or greater than what you use for tracking)
2. at defined intervals the program captures a new frame from the video source
3. it subtracts the reference frame, direction of difference depends whether a dark object on light background, or light object on dark is specified. the more the animal sticks out the easier the tracking gets. In a situation where i cant afford a missed or wrong capture i illuminate the arena with diffuse lights from below which makes the object very distinct.
4. it then locates the center of the area that shows the greatest amount of difference given a set of criteria. the search mechanisms is extensible so folks can write their own search mechanisms to look for multiple animals, orientation, etc. the current one used is a simple one that locates the pixel with the greatest difference (i.e., pixel difference), it then checks whether that coordinate is also surrounded by an area of pixels with a particular difference (i.e., row sum difference, column sum difference). it checks whether the size of this area matches a defined size (i.e., object size). if the total difference for the frame is too high then something is considered to have gone wrong (e.g., camera has turned off, changed orientation, lighting conditions are different, etc.) and frame is not used (i.e. frame difference)
5. if criteria are matched then the object was successfully tracked in this frame and the coordinate is written into the text file

Basic Operation

1. start program with empty arena
2. click "reference" button to setup data file and computer/video interface, collect a reference frame
3. add the animal to the arena and click the "setup" button to optimize detection for the object
4. click "analyze" button to start analysis
5. if results are not good, optimize conditions as outlined below

Optimization

1. To choose the best criteria for tracking your object of interest you can use the setup procedure:
2. set reference frame
3. add object into arena
4. click setup button. the program will identify areas of difference and will suggest settings that are optimized to detect them
   • light on dark / dark on light
   • pixel difference
5. these values are hard coded right now to deal with a generalized scenario and i can guide you into fine-tuning it on your end

last updated 7/5/07