Getting Started

About AniML

Version 3.3.0

GitHub Repository

The AniML package is available in Python and R for AI-assisted camera trap image processing.

The AniML package provides functions for ingesting raw image and video files and outputs predictions for species using region-specific species classifier models. We provide several species models including for the African Savanna, the Peruvian Amazon, the Andes mountains, and the Western US. AniML provides the results in a number of export formats, including TimeLapse and CamTrapDP. The package also includes AI-based re-indentification tools and custom species model training.

Installation

Install via the command line:

$ pip install animl

Requirements

Required dependencies:

• pytorch
• ultralytics
• onnx-runtime
• pandas

Recommended:

• ExifTool
• CUDA/cuDNN (for GPU)

We recommend using AniML with a GPU. To use with an Nvidia GPU, be sure that to install the CUDA-compatible version of PyTorch

Examples and Usage

Command-line Execution

Once installed, AniML can be run from the command line:

$ python -m animl /path/to/data/folder --detector /path/to/megadetector --classifier /path/to/classifier --classlist /path/to/classlist.txt

The -s flag will sort the images into species folders.
The -v flag will create copies of the images with bounding boxes drawn around the animal detections.

You can use animl in this fashion on any image directory.

If you want more fine-tuned control of certain parameters, you can use the animl.yml config file to specify parameters:

$ python -m animl /path/to/animl.yml

An example configuration .yml file can be found here.

Species Classification Inference

The functionality of animl can be parcelated into its individual functions to suit your data and scripting needs.

1. It is recommended that you use the AniML Working Directory for storing intermediate steps.

    import animl
    workingdir = animl.WorkingDirectory('/path/to/save/data')
   

2. Build the file manifest of your given directory. This will find both images and videos.

    files = animl.build_file_manifest('/path/to/images',
                                      out_file=workingdir.filemanifest,
                                      exif=True,
                                      data_timezone='America/Los_Angeles')
   

   The argument data_timezone indicates the timezone in which the data was collected, so timestamps are correctly interpreted relative to the local timezone.

3. If there are videos, extract individual frames for processing. Select either the number of frames or fps using the argumments. The other option can be set to None or removed.

    allframes = animl.extract_frames(files, frames=3, out_file=workingdir.imageframes, parallel=True)
   

4. Pass all images into MegaDetector. We recommend MDv5a. The function parse_MD will convert the json to a pandas DataFrame and merge detections with the original file manifest, if provided.

    detector = animl.load_detector('/path/to/mdmodel.pt', model_type="mdv5", device='cuda:0')
   
    mdresults = animl.detect(detector,
                             allframes,
                             resize_width=animl.MEGADETECTORv5_SIZE,
                             resize_height=animl.MEGADETECTORv5_SIZE,
                             letterbox=True,
                             file_col="frame",
                             device='cuda:0',
                             checkpoint_path=working_dir.mdraw,
                             quiet=True)
   
    detections = animl.parse_detections(mdresults, manifest=allframes, out_file=workingdir.detections)
   

5. For speed and efficiency, extract the empty/human/vehicle detections before classification.

    animals = animl.get_animals(detections)
    empty = animl.get_empty(detections)
   

6. Classify using the appropriate species model. Merge the output with the rest of the detections if desired.

    classifier, class_list = animl.load_classifier('/path/to/model', '/path/to/classlist.txt', device='cuda:0')
   
    raw_predictions = animl.classify(classifier,
                                     animals,
                                     resize_width=480,
                                     resize_height=480, 
                                     file_col="filepath",
                                     batch_size=4,
                                     out_file=working_dir.predictions)
   

7. Apply labels from class list with or without utilizing timestamp-based sequences.

    manifest = animl.single_classification(animals, empty, raw_predictions, class_list['class'])
   

   or, after defining a station column named “station”,

    manifest = animl.sequence_classification(animals,
                                             empty, 
                                             raw_predictions,
                                             class_list['class'],
                                             station_col='station',
                                             empty_class="",
                                             sort_columns=None,
                                             file_col="filepath",
                                             maxdiff=60)
   

8. (OPTIONAL) Save the Pandas DataFrame’s required columns to csv and then use it to create json for TimeLapse compatibility

    csv_loc = animl.export_timelapse(manifest, imagedir, only_animal = True)
    animl.export_megadetector(manifest, out_file ="final_result.json", detector = 'MegaDetector v5a')
   

9. (OPTIONAL) Create symlinks within a given directory for file browser access.

    manifest = animl.export_folders(manifest, out_dir=working_dir.linkdir, out_file=working_dir.results)
   

Classifer Model Training

1. Assuming a file manifest of training data with species labels, first split the data into training, validation and test splits. This function splits each label proportionally by the given percentages, by default 0.7 training, 0.2 validation, 0.1 Test.

    train, val, test, stats = animl.train_val_test(manifest,
                                                   out_dir='path/to/save/data/', 
                                                   label_col="species",
                                                   val_size: float = 0.2,
                                                   test_size: float = 0.1,
                                                   random_state: int = 42)
   

2. Set up training configuration file. Specify the paths to the data splits from the previous step. See config README.

3. (Optional) Update train.py to include MLOPS connection.

4. Using the config file, begin training

    python -m animl.train --config /path/to/config.yaml
   

   Every 10 epochs (or define custom ‘checkpoint_frequency’), the model will be checkpointed to the ‘experiment_folder’ parameter in the config file, and will contain performance metrics for selection.

5. Testing of a model checkpoint can be done with the “test.py” module. Add an ‘active_model’ parameter to the config file that contains the path of the checkpoint to test. This will produce a confusion matrix of the test dataset as well as a csv containing predicted and ground truth labels for each image.

    python -m animl.test --config /path/to/config.yaml
   

Re-Identification

Exports

API Reference

Full Pipeline

animl.from_paths(image_dir, detector_file, classifier_file, classlist_file, …)

Runs the full detection + classification pipeline on a directory of images or videos.
AniML will add a Animl-Directory folder to the image_dir to store the outputs.

Returns: pandas.DataFrame — results of detection and classification, including file paths, detection categories, and predicted classes.

animl.from_config(config)

Runs the full detection + classification pipeline on a directory of images or videos.
AniML will add a Animl-Directory folder to the working_dir to store the outputs.

An example configuration .yml file can be found here.

Returns: pandas.DataFrame — results of detection and classification, including file paths, detection categories, and predicted classes.

Data Ingestion and Processing

class animl.WorkingDirectory(working_dir)

A WorkingDirectory object creates a folder called “Animl-Directory within the working_dir. Attributes include output file paths to save the outputs of intermediary steps.

• self.filemanifest = “FileManifest.csv”, typically used with build_file_manifest()
• self.imageframes = “ImageFrames.csv”, typically used with extract_frames()
• self.mdraw = “MD_Raw.json”, typically used with detect()
• self.detections = “Detections.csv”, typically used with parse_detections()
• self.predictions = “Predictions.csv”, typically used with classify()
• self.results = “Results.csv”, typically used with the export functions.

If export_folders() or plot_all_bounding_boxes() are used with a WorkingDirectory, it will create a “Sorted” or “Plots” folder respectively within “Animl-Directory”.

animl.build_file_manifest(image_dir, exif=True, out_file=None, …)

Traverse a directory and find image/video files and gather metadata.

To correctly adjust timestamps from exif data, the argument data_timezone should be set to the timezone in which the data was collected. If you are unsure of the timezone, you can list all with zoneinfo.available_timezones() to find the best match, or leave as None to default to the local timezone.

Returns: pandas.DataFrame — object containing file manifest

Output manifest will have the following columns:

• filepath
• filename
• extension
• width
• height
• createdate (if exif = True)
• filemodifydate (if exif = True)
• datetime (if exif = True, contains createdate or filemodifydate as a fallback)
• station (if station_depth is not None)
• camera (if camera_depth is not None)

* For station_depth, if file paths are in the format “image_dir/station/date/file.jpg”, station_depth would be 1 (0 indexed). If None, station column will not be created. Likewise for camera_depth, if file paths are in the format “image_dir/station/camera/date/file.jpg”, camera_depth would be 2 (0 indexed). If None, camera column will not be created.

animl.active_times(manifest, file_col=”filepath”, camera_depth=0, timestamp_col=”datetime”)

Get start and stop dates for each camera folder.

Returns: pandas.DataFrame with a row for each camera and the earliest and latest timestamp of data taken at that camera

animl.sequence_calculation(manifest, station_col=”station”, sort_columns=None, file_col=”filepath”, timestamp_col=”datetime”, maxdiff=60)

Simple sequence calculation based on time differences between consecutive images from the same station.
Unlike sequence_classification(), does not apply any classification or labeling to the sequences.

Returns: pandas.DataFrame — the input DataFrame with an additional ‘sequence’ column indicating sequence membership.

animl.extract_frames(manifest, frames=5, fps=None, out_file=None, out_dir=None, file_col=”filepath”, parallel=True, num_workers=NUM_THREADS)

Extract frames from video files in a given DataFrame. Can sample frames based on a specified number of frames or frames per second (fps).

Returns: pandas.DataFrame — the input dataframe with and additional “frame” column. The value of frame is 0 for images, while videos will now be represented with multiple rows as indicated by frames or fps, with each row containing the sampled frame number.

Detection

animl.load_detector(model_path, model_type, device=None)

Loads a detector model from a file path.

Model types accepted:
[“mdv5”, “mdv6”, “mdv1000-cedar”, “mdv1000-larch”, “mdv1000-sorrel”, “mdv1000-redwood”, “mdv1000-spruce”, “yolov5”, “yolo”, “onnx”]
For yolo models v6+, use “yolo”, for v5, use “yolov5”.

Returns: loaded model object

animl.detect(detector, image_file_names, resize_width, resize_height, …)

Runs a detector model on batches of image files.

Returns: tuple — (detections, failed_files)

• detections: list[dict] of detection results in MegaDetector format, one dict per image
• failed_files: list of files that failed to load during processing (if any)

animl.parse_detections(results, manifest=None, out_file=None, threshold=0.1, file_col=”filepath”)

Converts detector output into a detections DataFrame.

Returns: pandas.DataFrame — one row per detection with columns: filepath, category, category_label, conf, bbox_x, bbox_y, bbox_w, bbox_h, max_detection_conf

animl.get_animals(manifest)

Pulls out MD animal detections for classification

Returns: pandas.DataFrame — subset of manifest containing only animal detections

animl.get_empty(manifest)

Pulls out MD non-animal detections and adds prediction and confidence columns

Returns: pandas.DataFrame — subset of manifest containing empty, vehicle and human detections with added prediction and confidence columns

Classification

animl.load_classifier(model_path, classes, device=None, architecture=”efficientnet_v2_m”, quiet=True)

Creates and loads a classifier model of the given architecture from disk, with the associated class list.

Returns: (model, class_list) — loaded model (of given architecture) and class list or None

animl.load_class_list(classlist_file)

Returns classlist file as DataFrame.

Returns: pandas.DataFrame — the class list file data

animl.class_list_to_dict(class_list, label_col=”class”, id_col=”id”)

Converts a class list DataFrame into a dictionary mapping class IDs to labels.

Returns: dict — mapping of class IDs to labels, e.g. {0: “empty”, 1: “species_a”, 2: “species_b”}

animl.classify(model, detections, resize_width=480, resize_height=480, file_col=”filepath”, …)

Runs prediction for input detections using a preloaded classifier model

Returns: tuple — (predictions, failed_files)

• predictions: np.array of softmaxed logits for each class/image
• failed_files: list of files that failed during processing (if any)

animl.single_classification(animals, empty, predictions_output, class_list, best=False, file_col=”filepath”, failed_files=None)

Assigns predicted class labels and confidences to each row in a detection DataFrame, handling failed files and “empty” detections.

Returns: pandas.DataFrame — DataFrame with columns prediction, confidence, and associated metadata

animl.sequence_classification(animals, empty, predictions_output, class_list, station_col, empty_class=””, …)

Applies class labels to images based on sequential information.

This function applies image classifications at a sequence level by leveraging information from multiple images. A sequence is defined as all images at the same camera and station where the time between consecutive images is <=maxdiff. This can improve classification accuracy, but assumes that only one species is present in each sequence. If you regularly expect multiple species to occur in an image or sequence don’t use this function.

Returns: pandas.DataFrame — sequence-classified results with columns including prediction, confidence, sequence

Re-Identification

animl.load_miew(file_path, device)

Loads a MiewID model from a file path.

Returns: MiewID model object

animl.extract_miew_embeddings(miew_model, manifest, file_col=”filepath”, batch_size=1, num_workers=1, device=None)

Extracts MiewID embeddings for a given set of images.

Returns: numpy.ndarray — array of extracted embeddings

animl.remove_diagonal(A)

Removes the diagonal elements from a square matrix.

Returns: torch.Tensor - Matrix with diagonal elements removed

animl.euclidean_squared_distance(input1, input2)

Computes the Euclidean squared distance between two feature matrices.

Returns: torch.Tensor - Euclidean squared distance matrix

animl.cosine_distance(input1, input2)

Computes the cosine distance between two feature matrices.

Returns: torch.Tensor - Cosine distance matrix

animl.compute_distance_matrix(input1, input2, metric=’euclidean’)

Computes a distance matrix between two feature matrices using the specified metric.

Returns: numpy.ndarray - Distance matrix

animl.compute_batched_distance_matrix(input1, input2, metric=’cosine’, batch_size=10)

Computes a distance matrix between two feature matrices in batches, using the specified metric. This is useful for large datasets that may not fit in memory when computing the full distance matrix at once.

Returns: numpy.ndarray - Computed distance matrix

Model Training

animl.train_classifier(config)

Trains a classifier model based on the provided configuration. For details on the configuration parameters, see the config README.

animl.test_classifier(config)

Tests a classifier model based on the provided configuration, evaluating performance on a test dataset and generating a confusion matrix. For details on the configuration parameters, see the config README.

animl.save_classifier(model, out_dir, epoch, stats, optimizer=None, scheduler=None)

Saves model state weights and optional optimizer/scheduler states to disk.

Returns: None

animl.load_classifier_checkpoint(model_path, model, optimizer, scheduler, device)

Loads the latest checkpoint to resume model training, restoring weights and optimizer/scheduler states.

Returns: int — starting epoch restored from the latest checkpoint

Visualization

animl.get_frame_as_image(video_path, frame=0)

Returns: numpy.ndarray - Matrix representing the cv2 image

animl.plot_box(rows, file_col=”filepath, min_conf=0, classifier_label_col=None, detector_category_col=”category”, show_confidence=False,…)

Plot bounding box(es) for a single image based on the input rows of a DataFrame.

plot_box() is designed for plotting boxes on a single image, while plot_all_bounding_boxes() can handle multiple images and has additional options for saving outputs.

rows must contan the bounding box coordinates (bbox_x, bbox_y, bbox_w, bbox_h), and filepath (filepath) for the image to be plotted.

If classifier_label_col is specified, it will also display the predicted class label on the box. The color of the box(es) can be determined by the detector category column specified by detector_category_col and the colors dictionary.

If show_confidence is True, rows must contain confidence or conf column, and the confidence score will also be displayed on the box.

Returns: None or numpy.ndarray (if return_image is True)

animl.plot_all_bounding_boxes(manifest, out_dir=None, file_col=”filepath”, min_conf=0.1, classifier_label_col=None, detector_category_col=”category”, show_confidence=False,…)

Plot bounding boxes for all rows in a manifest DataFrame, with options to save plotted images.

Returns: None

Export

animl.export_folders(manifest, out_dir, out_file=None, file_col=”filepath”, label_col=”prediction”, timestamp_col=”camera”, …)

Returns: pandas.DataFrame — copy of manifest with additional column link for exported file paths, with images copied to out_file if specified

animl.remove_link(manifest, link_col=”link”)

Deletes symbolic links of images.

Returns: pandas.DataFrame — copy of manifest with column link_col removed

animl.update_labels_from_folders(manifest, export_dir, unique_name_col = “uniquename”, label_col = “prediction”)

Update manifest after human review of symlink directories.

Returns: pandas.DataFrame — copy of manifest with updated labels based on folder names in export_dir after human review

animl.export_train_val_test(manifest, label_col=”class”, file_col=”filepath”, conf_col=”confidence”, out_dir=None, val_size=0.1, test_size=0.1, seed=42)

Returns train_df, val_df, test_df with label_col stratified. test_size and val_size are fractions of the whole dataset (e.g., 0.2 -> 20%).

If there are multiple detections per image, samples are sorted by conf_col confidence score before splitting and only the highest confidence detection per image is used for stratification to ensure that all samples of the same image are in the same split. Otherwise, if there are multiple detections per image and stratification is done on all samples, different samples from the same image could end up in different splits, which can lead to data leakage and overly optimistic performance estimates.

Returns: tuple — (train_df, val_df, test_df) DataFrames for training, validation, and testing, stratified by label_col

animl.export_yolo(train_manifest, val_manifest, test_manifest, class_dict, out_dir, label_col=”class”, file_col=”filepath”, …)

Export a manifest to YOLO format for model training. Saves a .txt file for each image with bounding box coordinates and class labels.

Returns dict — dictionary containing paths to saved YOLO formatted files, number of classes, and class list, e.g.:

{
    "path": "path/to/yolo/",
    "train": "path/to/yolo/images/train",
    "val": "path/to/yolo/images/val",
    "test": "path/to/yolo/images/test",
    "names": ["empty", "species_a", "species_b"],
    "num_classes": 3
}

animl.export_coco(manifest, class_dict, out_file, info=None, licenses=None)

Export a manifest to COCO format.

Returns: dict — COCO format dictionary containing info, licenses, categories, images, and annotations based on the input manifest and class_dict, and saves it to out_file as JSON

animl.export_camtrapdp(manifest, out_dir, file_public=False, classifier_name=None)

Export a manifest to CamtrapDP format. Requires scientific name for the species prediction label and bounding box coordinates for each detection. Assumes MegaDetector category labels and uses category column to determine which rows are “empty” vs “animal” detections.

Returns tuple - media_df, observations_df, and datapackage dict The media_df contains metadata for each media file, the observations_df contains metadata for each observation (detection), and the datapackage dict contains the overall structure and metadata for the CamtrapDP package.

animl.export_camtrapR(manifest, out_dir, out_file=None, label_col=’prediction’, file_col=’filepath’, … )

Export into species-labeled folders organized by station.

Returns pandas.DataFrame — copy of manifest with additional column link for exported file paths

animl.export_timelapse(manifest, out_dir, only_animal=True)

Converts a manifest to a csv file that contains columns needed for TimeLapse conversion

Returns str — file path to the saved TimeLapse formatted CSV file

animl.export_megadetector(manifest, out_file=None, detector=”MegaDetector v5a”, prompt=True)

Converts a manifest DataFrame back into MegaDetector format and saves as a .json file.

If [out_file] is None, ‘.json’ will be appended to the input file.

Author: Dan Morris https://github.com/agentmorris/MegaDetector/tree/main

animl.save_data(data, out_file, prompt=True)

Save data to a given filepath

animl.load_data(file)

Load data from a given filepath.

animl.save_json(data, out_file, prompt=True)

Save a dictionary as a JSON file.

animl.load_json(file)

Load data from a JSON file.

animl.check_file(file, output_type=None)

Check for file existence and prompt user if they want to load.

Returns: bool — True if file exists and user wants to load, False otherwise

Utilities

Troubleshooting