Salesforce-BLIP-image-captioning-base · azureml v3
`BLIP` (Bootstrapping Language-Image Pre-training) designed for unified vision-language understanding and generation is a new VLP framework that expands the scope of downstream tasks compared to exis…
azureFoundry/azureml/Salesforce-BLIP-image-captioning-base@3
BLIP (Bootstrapping Language-Image Pre-training) designed for unified vision-language understanding and generation is a new VLP framework that expands the scope of downstream tasks compared to existing methods. The framework encompasses two key contributions from both model and data perspectives.
-
BLIP incorporates the Multi-modal Mixture of Encoder-Decoder (MED), an innovative model architecture designed to facilitate effective multi-task pre-training and flexible transfer learning. This model is jointly pre-trained using three vision-language objectives: image-text contrastive learning, image-text matching, and image-conditioned language modeling.
-
BLIP introduces Captioning and Filtering (CapFilt), a distinctive dataset bootstrapping method aimed at learning from noisy image-text pairs. The pre-trained MED is fine-tuned into a captioner that generates synthetic captions from web images, and a filter that removes noisy captions from both the original web texts and synthetic t
Specifications & capabilities
- Input
- image
- Output
- text
- Capabilities
- visionInput
Provider specifications & access
1–1 / 1
- source model id
- azureml/Salesforce-BLIP-image-captioning-base@3
- hosting
- foundry-managed-compute
- license
- bsd-3-clause
Explore more models