OpenCLIP (ViT-B/16, LAION-2B)

Intended use. Included as a representative general-purpose OpenCLIP image–text model. For insect classification, it can compare images with candidate insect names or descriptions without task-specific training, or provide image embeddings for a classifier trained on labeled insect examples. Its accuracy on the intended taxa and image types requires evaluation.

Architecture and training

This OpenCLIP checkpoint pairs a ViT-B/16 image encoder with a Transformer text encoder. The image encoder divides images into 16 × 16-pixel patches, and both encoders produce 512-dimensional embeddings in a shared image–text space. The model was trained on the approximately two-billion-pair English subset of LAION-5B using contrastive image–text learning.

Inputs and outputs

Accepts RGB images and text, which can be encoded separately. Images are resized and centre-cropped to 224 × 224 pixels using the checkpoint’s OpenCLIP preprocessing transforms. Text is tokenized with the matching tokenizer.

The model produces 512-dimensional image and text embeddings. For zero-shot classification, image embeddings are compared with text embeddings of candidate labels or descriptions, and the highest-scoring candidate is selected.

Performance

The checkpoint’s model card reports 70.2% zero-shot top-1 accuracy on ImageNet-1K, evaluated using the LAION CLIP Benchmark suite.

This measures general image classification using text prompts, not insect identification. Insect-specific performance is not reported in the checkpoint’s model card.

Limitations

Zero-shot classification depends on the candidate labels and prompt wording. Training on general web image–text pairs does not establish reliable recognition of fine-grained insect taxa.

How to obtain the weights

The weights are publicly available on Hugging Face.

Citation

@software{ilharco2021openclip, title = {OpenCLIP}, author = {Ilharco, Gabriel and Wortsman, Mitchell and Wightman, Ross and Gordon, Cade and Carlini, Nicholas and Taori, Rohan and Dave, Achal and Shankar, Vaishaal and Namkoong, Hongseok and Miller, John and Hajishirzi, Hannaneh and Farhadi, Ali and Schmidt, Ludwig}, year = {2021}, publisher = {Zenodo}, doi = {10.5281/zenodo.5143773}, url = {https://doi.org/10.5281/zenodo.5143773} }

@inproceedings{schuhmann2022laion5b, title = {{LAION-5B}: An Open Large-Scale Dataset for Training Next Generation Image-Text Models}, author = {Schuhmann, Christoph and Beaumont, Romain and Vencu, Richard and Gordon, Cade W and Wightman, Ross and Cherti, Mehdi and Coombes, Theo and Katta, Aarush and Mullis, Clayton and Wortsman, Mitchell and Schramowski, Patrick and Kundurthy, Srivatsa R and Crowson, Katherine and Schmidt, Ludwig and Kaczmarczyk, Robert and Jitsev, Jenia}, booktitle = {Thirty-sixth Conference on Neural Information Processing Systems Datasets and Benchmarks Track}, year = {2022}, url = {https://openreview.net/forum?id=M3Y74vmsMcY} }