ALIGN

ALIGN is the method from the paper 'Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision', presented by Google Research at ICML 2021. Using over 1 billion image–alt-text pairs collected without expensive filtering, a simple dual encoder is trained to align image and text representations with a contrastive loss. The authors state that the scale of the data offsets the noise, achieving state-of-the-art results on ImageNet and VTAB classification transfer and on Flickr30K and MSCOCO retrieval. The learned representations enable cross-modal retrieval using text and text+image queries.

• Trained on over 1 billion image–alt-text pairs (noisy data)
• Dual encoder + contrastive loss to align image and text representations
• Classification transfer performance on ImageNet and VTAB
• State-of-the-art results on Flickr30K and MSCOCO image-text retrieval
• Cross-modal retrieval with text and text+image queries

Open source (free, weights/code released)

Released free under an open-source license with no extra cost for self-hosting (usage-based pricing when offered as a cloud API)

Estimate based on AI knowledge - please verify on the official site

• One of 60 AI tools in the Foundation Models field.
• Connected to 10 other tools — #11 in its field.