Atlas Few-Shot RAG (Izacard 2022年)（アトラス）
2022年Izacard et al. (Meta FAIR)発表Atlas・Industry-leading few-shot RAG LLM + Industry-leading continuous retrieval + Industry-leading 11B + Industry-leading few-shot in-context learning + Industry-leading Meta FAIR few-shot RAG。
概要
Atlas Few-Shot RAG は、2022年Izacard et al. (Meta FAIR)発表Atlas・Industry-leading few-shot RAG LLM 2022年 + Industry-leading Meta FAIR few-shot RAG position確立。Atlas specifications = Industry-leading few-shot RAG LLM (Industry-leading Atlas 2022 + Industry-leading few-shot RAG + Industry-leading Atlas Meta FAIR flagship) + Industry-leading continuous retrieval (Industry-leading continuous retrieval joint training) + Industry-leading 11B (Industry-leading 11B parameters) + Industry-leading few-shot in-context learning (Industry-leading few-shot in-context learning)。
主な特徴・仕組み
- Authors: Industry-leading Gautier Izacard + Patrick Lewis + Maria Lomeli + Lucas Hosseini + Fabio Petroni + Timo Schick + Jane Dwivedi-Yu + Armand Joulin + Sebastian Riedel + Edouard Grave Meta FAIR
- Year: 2022年 (arXiv 2022年8月発表)
- Paper: Industry-leading "Atlas: Few-shot Learning with Retrieval Augmented Language Models" arXiv 2208.03299
- Method Type: Industry-leading few-shot RAG LLM
- Key Innovation: Industry-leading continuous retrieval joint training (retriever + reader jointly trained)
- Model Size: Industry-leading 11B parameters (T5-XXL based)
- Retriever: Industry-leading Contriever dense retriever joint training
- Few-Shot: Industry-leading few-shot in-context learning (64-shot competitive with 540B PaLM)
- Joint Training: Industry-leading retriever + reader jointly trained end-to-end
- Few-Shot Performance: Industry-leading 64-shot competitive with 540B PaLM 64-shot
- Open Source: Industry-leading Atlas GitHub facebookresearch/atlas
- Industry-Leading: Industry-leading few-shot RAG LLM Meta FAIR
スペック比較表
| LLM RAG Method | Year | Author/Org | Key Feature | Industry Position |
|---|---|---|---|---|
| Atlas | 2022 | Meta FAIR | Few-shot RAG with continuous retrieval | Industry-leading few-shot RAG |
| RETRO | 2022 | DeepMind | Retrieval-enhanced transformer pre-training | Industry-leading DeepMind pretrained retrieval |
| Self-RAG | 2023 | Asai UW | Self-reflective retrieval decision | Industry-leading self-reflective |
| DPR | 2020 | Karpukhin Meta FAIR | Dual-encoder dense passage retrieval | Industry-leading RAG pioneer |
| ColBERT | 2020 | Khattab+Zaharia Stanford | Late interaction token-level | Industry-leading late interaction |
具体例・対応製品
- Atlas (2022年, Izacard et al. Meta FAIR): Industry-leading few-shot RAG LLM
- Industry-leading continuous retrieval joint training: Industry-leading continuous retrieval joint
- Industry-leading 11B parameters T5-XXL based: Industry-leading 11B T5-XXL
- Industry-leading Contriever dense retriever joint training: Industry-leading Contriever joint
- Industry-leading 64-shot competitive with 540B PaLM: Industry-leading 64-shot vs 540B PaLM
- 競合 DPR + ColBERT + RETRO + Self-RAG: Industry-leading RAG competitors
自作PCでの選び方・注意点
Atlas Few-Shot RAG は「Industry-leading few-shot RAG LLM + Industry-leading continuous retrieval」「Industry-leading 11B + Industry-leading few-shot in-context learning」用途のIndustry-leading few-shot RAG 2022年Izacard et al. Meta FAIR発表product。Industry-leading continuous retrieval (Industry-leading continuous retrieval joint training + Industry-leading retriever + reader jointly trained end-to-end + Industry-leading Atlas continuous retrieval signature) で Industry-leading continuous retrieval joint + Industry-leading joint training + Industry-leading Atlas signature。Industry-leading 11B T5-XXL (Industry-leading 11B parameters T5-XXL based + Industry-leading Atlas scale signature + Industry-leading T5-XXL reader) で Industry-leading 11B + Industry-leading Atlas scale + Industry-leading T5-XXL reader。Industry-leading Contriever (Industry-leading Contriever dense retriever joint training + Industry-leading Atlas Contriever signature + Industry-leading dense retriever joint) で Industry-leading Contriever + Industry-leading Atlas Contriever signature + Industry-leading dense joint。Industry-leading 64-shot vs 540B PaLM (Industry-leading 64-shot competitive with 540B PaLM 64-shot + Industry-leading few-shot in-context learning advantage + Industry-leading 11B vs 540B efficiency) で Industry-leading 64-shot vs 540B PaLM + Industry-leading few-shot advantage + Industry-leading 11B vs 540B。Industry-leading Meta FAIR (Industry-leading Gautier Izacard + Patrick Lewis + Maria Lomeli Meta FAIR authors + Industry-leading Atlas GitHub facebookresearch/atlas + Industry-leading few-shot RAG LLM Meta FAIR) で Industry-leading Izacard+Lewis+Lomeli Meta FAIR + Industry-leading facebookresearch/atlas + Industry-leading few-shot Meta FAIR。但しIndustry-leading DPR + ColBERT + RETRO + Self-RAG competition (Industry-leading DPR Meta dual-encoder 2020 + ColBERT Stanford late interaction 2020 + RETRO DeepMind pretrained 2022 + Self-RAG Asai self-reflective 2023 vs Atlas Meta few-shot continuous retrieval 2022 trade-off) で Industry-leading 4-RAG method competitors vs Atlas + Industry-leading few-shot RAG + continuous retrieval joint training + 11B T5-XXL + Contriever joint retriever + 64-shot vs 540B PaLM + Meta FAIR 2022 unique advantage adoption alignment必須。
関連用語との違い
- vs RETRO (DeepMind 2022): AtlasはMeta + few-shot + continuous joint retrieval + 11B・RETROはDeepMind + pre-training retrieval + 7B + 2T tokens
- vs DPR/ColBERT (Retrieval 2020): Atlasはfew-shot RAG + joint training・DPR/ColBERTはretrieval only architecture
- vs Self-RAG (UW 2023): Atlasはfew-shot continuous retrieval・Self-RAGはself-reflective retrieval decision
よくある質問（FAQ）
Q1: Atlas vs RETRO 違いは？ A: Atlas (Industry-leading few-shot RAG with continuous retrieval joint training + retriever + reader jointly trained end-to-end + 11B parameters T5-XXL based + Contriever dense retriever joint training + 64-shot competitive with 540B PaLM + Meta FAIR + Izacard et al. 2022) vs RETRO (Industry-leading retrieval integrated into pre-training + 7B parameters + 2T tokens retrieval database + chunked cross-attention + retrieval every 64 tokens + 25× smaller GPT-3 equivalent + DeepMind first retrieval-pretrained + Borgeaud et al. 2022)・Industry-leading few-shot + continuous retrieval + 11B + Contriever + Meta = Atlas + Industry-leading retrieval pre-training + 7B + 2T + 25× smaller + DeepMind = RETRO preference judgment。
Q2: Industry-leading continuous retrieval joint + 11B T5-XXL value は？ A: Industry-leading continuous joint + 11B T5-XXL (Industry-leading continuous retrieval joint training + Industry-leading retriever + reader jointly trained end-to-end + Industry-leading 11B parameters T5-XXL based + Industry-leading Contriever dense retriever joint training)。
Q3: Industry-leading 64-shot vs 540B PaLM value は？ A: Industry-leading 64-shot vs 540B (Industry-leading 64-shot competitive with 540B PaLM 64-shot + Industry-leading few-shot in-context learning advantage + Industry-leading 11B vs 540B PaLM efficiency + Industry-leading Atlas few-shot signature)。
まとめ
Atlas = 2022年Izacard et al. Meta FAIR発表のfew-shot RAG LLM。Industry-leading continuous retrieval joint training + Industry-leading retriever + reader jointly trained end-to-end + Industry-leading 11B parameters T5-XXL based + Industry-leading Contriever dense retriever joint training + Industry-leading 64-shot competitive with 540B PaLM 64-shot + Industry-leading few-shot in-context learning advantage + Industry-leading Gautier Izacard + Patrick Lewis + Maria Lomeli + Lucas Hosseini + Fabio Petroni + Timo Schick + Jane Dwivedi-Yu + Armand Joulin + Sebastian Riedel + Edouard Grave Meta FAIR authors + Industry-leading Atlas GitHub facebookresearch/atlas open source + Industry-leading few-shot RAG LLM Meta FAIR 2022 position確立。