Self-RAG Self-Reflective Retrieval (Asai 2023年)（セルフラグ）
2023年Asai et al. (UW+IBM+CMU+Allen AI)発表Self-RAG・Industry-leading self-reflective RAG LLM + Industry-leading retrieval decision learning + Industry-leading critique tokens + Industry-leading LLaMA-2 7B/13B fine-tuned。
概要
Self-RAG Self-Reflective Retrieval は、2023年Asai et al. (UW+IBM+CMU+Allen AI)発表Self-RAG・Industry-leading self-reflective RAG LLM 2023年 + Industry-leading LLaMA-2 7B/13B fine-tuned position確立。Self-RAG specifications = Industry-leading self-reflective RAG LLM (Industry-leading Self-RAG 2023 + Industry-leading self-reflective RAG + Industry-leading Self-RAG flagship) + Industry-leading retrieval decision learning (Industry-leading retrieval decision learning when to retrieve) + Industry-leading critique tokens (Industry-leading critique tokens self-evaluation) + Industry-leading LLaMA-2 7B/13B fine-tuned (Industry-leading LLaMA-2 7B + 13B fine-tuned)。
主な特徴・仕組み
- Authors: Industry-leading Akari Asai + Zeqiu Wu + Yizhong Wang + Avirup Sil + Hannaneh Hajishirzi UW + IBM Research + CMU + Allen Institute for AI
- Year: 2023年 (arXiv 2023年10月発表 + ICLR 2024 Spotlight)
- Paper: Industry-leading "Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection" arXiv 2310.11511
- Method Type: Industry-leading self-reflective RAG LLM
- Key Innovation: Industry-leading retrieval decision learning + critique tokens self-evaluation
- Critique Tokens: Industry-leading 4 critique tokens (Retrieve / IsRel / IsSup / IsUse)
- Retrieve Token: Industry-leading Retrieve token (yes/no/continue decision)
- IsRel / IsSup / IsUse: Industry-leading relevance + support + usefulness tokens
- Target Models: Industry-leading LLaMA-2 7B + LLaMA-2 13B fine-tuned
- Self-Reflection: Industry-leading self-reflective retrieval + generation + critique
- Adaptive Retrieval: Industry-leading adaptive retrieval (retrieve only when needed)
- Open Source: Industry-leading Self-RAG GitHub AkariAsai/self-rag
- Performance: Industry-leading outperforms ChatGPT + LLaMA-2 70B on multiple benchmarks
- Industry-Leading: Industry-leading self-reflective RAG LLM ICLR 2024 Spotlight
スペック比較表
| LLM RAG Method | Year | Author/Org | Key Feature | Industry Position |
|---|---|---|---|---|
| Self-RAG | 2023 | Asai UW+IBM+CMU+Allen AI | Self-reflective retrieval decision | Industry-leading self-reflective |
| Atlas | 2022 | Meta FAIR | Few-shot RAG with continuous retrieval | Industry-leading few-shot RAG |
| RETRO | 2022 | DeepMind | Retrieval-enhanced transformer pre-training | Industry-leading DeepMind pretrained retrieval |
| DPR | 2020 | Karpukhin Meta FAIR | Dual-encoder dense passage retrieval | Industry-leading RAG pioneer |
| ColBERT | 2020 | Khattab+Zaharia Stanford | Late interaction token-level | Industry-leading late interaction |
具体例・対応製品
- Self-RAG (2023年, Asai et al. UW+IBM+CMU+Allen AI): Industry-leading self-reflective RAG LLM
- Industry-leading 4 critique tokens Retrieve / IsRel / IsSup / IsUse: Industry-leading 4 critique tokens
- Industry-leading adaptive retrieval retrieve only when needed: Industry-leading adaptive retrieval
- Industry-leading LLaMA-2 7B + 13B fine-tuned: Industry-leading LLaMA-2 fine-tuned
- Industry-leading outperforms ChatGPT + LLaMA-2 70B: Industry-leading outperforms ChatGPT+70B
- 競合 DPR + ColBERT + RETRO + Atlas: Industry-leading RAG competitors
自作PCでの選び方・注意点
Self-RAG Self-Reflective Retrieval は「Industry-leading self-reflective RAG LLM + Industry-leading retrieval decision learning」「Industry-leading critique tokens + Industry-leading LLaMA-2 7B/13B fine-tuned」用途のIndustry-leading self-reflective RAG 2023年Asai et al. UW発表product。Industry-leading 4 critique tokens (Industry-leading 4 critique tokens Retrieve / IsRel / IsSup / IsUse + Industry-leading critique tokens self-evaluation + Industry-leading Self-RAG critique signature) で Industry-leading 4 critique tokens + Industry-leading critique self-evaluation + Industry-leading Self-RAG signature。Industry-leading Retrieve token (Industry-leading Retrieve token yes/no/continue decision + Industry-leading retrieval decision learning + Industry-leading adaptive retrieval retrieve only when needed) で Industry-leading Retrieve token decision + Industry-leading retrieval decision learning + Industry-leading adaptive retrieval。Industry-leading IsRel/IsSup/IsUse (Industry-leading IsRel relevance + IsSup support + IsUse usefulness tokens + Industry-leading Self-RAG tokens signature + Industry-leading self-evaluation signature) で Industry-leading IsRel+IsSup+IsUse + Industry-leading Self-RAG tokens + Industry-leading self-evaluation signature。Industry-leading LLaMA-2 fine-tuned (Industry-leading LLaMA-2 7B + LLaMA-2 13B fine-tuned + Industry-leading outperforms ChatGPT + LLaMA-2 70B on multiple benchmarks + Industry-leading Self-RAG LLaMA-2 signature) で Industry-leading LLaMA-2 7B+13B + Industry-leading outperforms ChatGPT+70B + Industry-leading LLaMA-2 signature。Industry-leading ICLR 2024 Spotlight (Industry-leading Akari Asai + Zeqiu Wu + Yizhong Wang + Hannaneh Hajishirzi UW + IBM + CMU + Allen AI authors + Industry-leading Self-RAG GitHub AkariAsai/self-rag + Industry-leading self-reflective RAG LLM ICLR 2024 Spotlight) で Industry-leading Asai+Wu+Wang UW+IBM+CMU+Allen AI + Industry-leading AkariAsai/self-rag + Industry-leading ICLR 2024 Spotlight。但しIndustry-leading DPR + ColBERT + RETRO + Atlas competition (Industry-leading DPR Meta dual-encoder 2020 + ColBERT Stanford late interaction 2020 + RETRO DeepMind pretrained 2022 + Atlas Meta few-shot continuous 2022 vs Self-RAG Asai self-reflective 4 critique tokens LLaMA-2 fine-tuned 2023 trade-off) で Industry-leading 4-RAG method competitors vs Self-RAG + Industry-leading self-reflective + 4 critique tokens Retrieve/IsRel/IsSup/IsUse + adaptive retrieval + LLaMA-2 7B/13B fine-tuned + outperforms ChatGPT+70B + UW+IBM+CMU+Allen AI 2023 + ICLR 2024 Spotlight unique advantage adoption alignment必須。
関連用語との違い
- vs Atlas (Meta Few-shot 2022): Self-RAGはself-reflective + critique tokens + LLaMA-2・AtlasはMeta + few-shot + continuous joint
- vs RETRO (DeepMind Pre-training 2022): Self-RAGはfine-tuned + self-reflective + LLaMA-2・RETROはDeepMind + pre-training + 7B
- vs DPR/ColBERT (Retrieval 2020): Self-RAGはRAG workflow + critique tokens・DPR/ColBERTはretrieval architecture only
よくある質問（FAQ）
Q1: Self-RAG vs Atlas 違いは？ A: Self-RAG (Industry-leading 4 critique tokens Retrieve / IsRel / IsSup / IsUse + retrieval decision learning + adaptive retrieval + critique tokens self-evaluation + LLaMA-2 7B + LLaMA-2 13B fine-tuned + outperforms ChatGPT + LLaMA-2 70B + Akari Asai UW + IBM + CMU + Allen AI + ICLR 2024 Spotlight + 2023) vs Atlas (Industry-leading few-shot RAG with continuous retrieval joint training + retriever + reader jointly trained + 11B parameters T5-XXL + Contriever joint + 64-shot vs 540B PaLM + Izacard et al. Meta FAIR + 2022)・Industry-leading self-reflective + 4 critique tokens + adaptive retrieval + LLaMA-2 fine-tuned + outperforms ChatGPT+70B + ICLR 2024 = Self-RAG + Industry-leading few-shot + continuous joint + 11B T5-XXL + Contriever + 64-shot vs 540B = Atlas preference judgment。
Q2: Industry-leading 4 critique tokens + adaptive retrieval value は？ A: Industry-leading 4 critique tokens + adaptive (Industry-leading 4 critique tokens Retrieve / IsRel / IsSup / IsUse + Industry-leading retrieval decision learning + Industry-leading adaptive retrieval retrieve only when needed + Industry-leading critique tokens self-evaluation + Industry-leading Self-RAG signature)。
Q3: Industry-leading LLaMA-2 fine-tuned + outperforms ChatGPT+70B value は？ A: Industry-leading LLaMA-2 + outperforms (Industry-leading LLaMA-2 7B + LLaMA-2 13B fine-tuned + Industry-leading outperforms ChatGPT + LLaMA-2 70B on multiple benchmarks + Industry-leading Self-RAG LLaMA-2 signature + Industry-leading ICLR 2024 Spotlight)。
まとめ
Self-RAG = 2023年Asai et al. UW+IBM+CMU+Allen AI発表のself-reflective RAG LLM。Industry-leading 4 critique tokens Retrieve / IsRel / IsSup / IsUse + Industry-leading retrieval decision learning when to retrieve + Industry-leading critique tokens self-evaluation + Industry-leading adaptive retrieval retrieve only when needed + Industry-leading LLaMA-2 7B + LLaMA-2 13B fine-tuned + Industry-leading outperforms ChatGPT + LLaMA-2 70B on multiple benchmarks + Industry-leading Akari Asai + Zeqiu Wu + Yizhong Wang + Avirup Sil + Hannaneh Hajishirzi UW + IBM + CMU + Allen AI authors + Industry-leading Self-RAG GitHub AkariAsai/self-rag open source + Industry-leading ICLR 2024 Spotlight + Industry-leading self-reflective RAG LLM 2023 position確立。