How WildTrackAI Works

WildTrackAI uses computer vision models to take images captured in the field, extract any footprints it finds, and then use those footprints to infer key labels for that image. Animal Species and Sex are two examples of the labels the models can infer.

The labels are stored along with other metadata obtained with the image and then used for further analysis that can combine both machine learning models and rules-based algorithms. For example, matching the AI-generated profile of a Jaguar to a database of known individual Jaguars enables Individual Identification.

In order to take on a new prediction task like species classification, models need to be trained on images that are already labeled with the correct species. The more the number and variety of images available to a model for training, the better its prediction accuracy.

How we define success

The success of this project is assessed according to progress against one or more of the following dimensions:

  • Expanded Use Cases: Enabling new analyses (example: identification of species from a footprint image) .
  • Improved accuracy: Meeting or exceeding prediction accuracy for analyses (example: individual identification) performed using the classic FIT technology.
  • Improved tolerance to variability in image quality: Improving resilience of the models to lower quality images (example partial prints, bad lighting, some distortion..) lowers the barrier for collection of images for analysis and model training.
  • Adaptability to a variety of images and other media: Ensuring the system is flexible enough to be able to incorporate multiple types of tracking evidence. As an example, the near term focus is on working with images of all four feet, but in the future the system could be extended to other secondary evidence like scat images and vocalizations, or biometrics such as shell or coat patterns.
  • Time savings: Reducing time and effort involved with collection, submission, analysis and reporting of information on a known species as well as onboarding a new species.
Current Model Performance

Model last refreshed on 2020-10-20 16:40:53

Species Classification
Species Training Test Field
Individual Identification
Species Training Test Field

Legend
  • Train: Images that the models were trained on
  • Test: Images that were used to test model performance.
  • Field: New Images uploaded via Epicollect or submitted directly to WildTrack.

Note that the original WildTrack data set that was used to test and train the models was comprised entirely of images that had the following characteristics:

  • High Quality: rating =5
  • Left Hind Foot: For all species except Otter which was Left Front
  • Closely Cropped: Only the footprint appears in the frame (no ruler or other peripheral objects).
  • Vertical Orientation: Almost entirely North-South orientation

Current Focus

The goal of this phase of the project is to narrow the gap in prediction accuracy between the highy curated initial imageset and that of images typically found in the field (varying quality, orientations, multiple feet, etc). Based on our analysis of results thus far, the next iteration of the system is focused in a few key areas.

  • Augmentation of training image set to account for variation in field image orientation as well which foot (left or right) the image is of.
  • Using labels entered in Epicollect (example: species, foot) in the algorithm
  • Refining algorithm used to make Observation level predictions (i.e. for an entire set of images uploaded)

To get from a raw image captured in the field to a prediction, the system goes through the following steps:

  1. Footprint Detection: The first step is to identify the part of the image that is a footprint and extract it out for downstream analysis. Note that the model can detect and extract multiple footprints in an image. Also, while the example picture shown here is one captured for FIT analysis (using the rulers), the system can work with other images as well , i.e. the rulers are not required .
  2. Species Classification: The footprints extracted fron the previous step are then analyzed to match them to one of a set of known species. As of now, the model has been trained on 11 species: Amur Tiger, Bengal Tiger, Puma, Leopard, Cheetah, African Lion, African Elephant, Bongo, Lowland Tapir, Black Rhino and White Rhino. The model prediction includes a confidence level. If the confidence is below a predefined threshold, the model returns UNKNOWN as the species.
  3. Individual Identification: Finally, the footprint is analyzed for matches against a database of known individuals for the species predicted in the previous step. These models are species-specific and just like the previous step, return the most probable match along with a confidence level. If the most probable match is below a predefined threshold, the model returns UNKNOWN

In order to generate additional data points from the raw image as well as potentially improve the accuracy of Individual Identification, the intent is to add two more model prediction tasks outlined below:

  1. Footprint Detection
  2. Species Classification
  3. Individual Identification
  4. Foot Classification: Currently the individual Identification model only works on images of a single foot (Left Hind for most species, Left Front for Bears and Otters). Being able to accurately determine which foot an image is of would allow for easier capture of more raw images in the field as well as potentially help improve the accuracy of the models
  5. Sex Classification: Determining sex of an individual animal is an important piece of metadata to be generated. Training gender-specific models could also potentially help improve accuracy of Individual Identification.
System Architecture
Species Classification Model Performance

Accuracy statistics for species classification models.

Individual Identification Model Performance

Accuracy statistics for individual identification models.

Count of Images by Species

Total number of images from each species used for model development.

Count of Individuals by Species

Total number of known individuals in each species used for model development.