2022
5 notes from 2022
-
[Flamingo] Flamingo: a Visual Language Model for Few-Shot Learning
Bridges a frozen vision encoder and a frozen LLM with a Perceiver Resampler + gated cross-attention, unlocking GPT-3-style few-shot in-context learning on image/video tasks.
-
[ATS] Adaptive Token Sampling for Efficient Vision Transformers
Parameter-free, picks a variable number of tokens per image by sampling the attention CDF.
-
[Evo-ViT] Slow-Fast Token Evolution for Dynamic Vision Transformer
Keep all tokens but update informative vs placeholder tokens on different paths.
-
[EViT] Not All Patches Are What You Need: Expediting ViTs via Token Reorganizations
Keep top-k attentive tokens by CLS attention, fuse the rest into one.
-
[BLIP] Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation
A unified vision-language model (MED) that handles both understanding and generation, plus CapFilt — a captioner+filter that bootstraps noisy web captions into cleaner training data.
