Insect-Foundation (vision backbone)

Intended use. One or two sentences on what this model is for, and the situation it was built for. A reader who is scanning five entries should be able to tell from this paragraph whether this one is theirs.

Architecture and training

The visual backbone is ViT-B/16 and produces 768-dimensional hidden representations.

Training uses Insect-1M, which contains 1,017,036 images covering 34,212 species. Despite its name, the dataset includes other arthropods as well as insects.

Inputs and outputs

Model accepts RGB images. Its default evaluation pipeline resizes the shorter side to 224 pixels using bicubic interpolation, then centre-crops to 224 × 224 pixels. Pixel values are scaled to [0, 1] and normalized using channel means (0.485, 0.456, 0.406) and standard deviations (0.229, 0.224, 0.225).

Performance

The paper reports 75.8% top-1 and 92.1% top-5 accuracy on IP102.

Limitations

IP102 results do not establish performance on every insect taxon, geographic region, life stage, or imaging setup.

The pretrained visual backbone is not a complete species-identification service. Classification requires a additional suitable trained head.

The release does not clearly document which reported pretraining variant the downloadable checkpoint represents.

How to obtain the weights

vit_base_patch16_pretrained.pth from the authors’ GitHub release.

Citation

@InProceedings{Nguyen_2024_CVPR, author = {Nguyen, Hoang-Quan and Truong, Thanh-Dat and Nguyen, Xuan Bac and Dowling, Ashley and Li, Xin and Luu, Khoa}, title = {Insect-Foundation: A Foundation Model and Large-scale 1M Dataset for Visual Insect Understanding}, booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)}, month = {June}, year = {2024}, pages = {21945–21955}, url = {https://openaccess.thecvf.com/content/CVPR2024/html/Nguyen_Insect-Foundation_A_Foundation_Model_and_Large-scale_1M_Dataset_for_Visual_CVPR_2024_paper.html} }