SAR Semantic-Aware Uncertainty (Duan 2023年)（エスエーアール）
2023年Duan+Yu (USC+Microsoft)発表SAR・Industry-leading semantic-aware uncertainty hallucination detection LLM + Industry-leading semantic-aware uncertainty + Industry-leading length-bias correction + Industry-leading USC+Microsoft 2023 EMNLP。
概要
SAR Semantic-Aware Uncertainty は、2023年Duan+Yu (USC+Microsoft)発表SAR・Industry-leading semantic-aware uncertainty hallucination detection LLM 2023年 + Industry-leading length-bias correction position確立。SAR specifications = Industry-leading semantic-aware uncertainty hallucination detection LLM (Industry-leading SAR 2023 + Industry-leading semantic-aware uncertainty + Industry-leading SAR flagship) + Industry-leading semantic-aware uncertainty + Industry-leading length-bias correction + Industry-leading USC+Microsoft 2023 EMNLP。
主な特徴・仕組み
- Authors: Industry-leading Jinhao Duan + Hao Cheng + Shiqi Wang + Alex Zavalny + Chenan Wang + Renjing Xu + Bhavya Kailkhura + Kaidi Xu USC + Microsoft Research + Drexel
- Year: 2023年 (arXiv 2023年7月発表 + EMNLP 2023)
- Paper: Industry-leading "Shifting Attention to Relevance: Towards the Predictive Uncertainty Quantification of Free-Form Large Language Models" arXiv 2307.01379
- Method Type: Industry-leading semantic-aware uncertainty hallucination detection LLM
- SAR: Industry-leading Shifting Attention to Relevance signature
- Semantic-Aware: Industry-leading semantic-aware uncertainty calibration
- Length-Bias Correction: Industry-leading length-bias correction signature
- Free-Form: Industry-leading free-form LLM output uncertainty quantification
- Target Models: Industry-leading GPT + LLaMA + OPT uncertainty
- Open Source: Industry-leading SAR GitHub jinhaoduan/SAR
- Industry-Leading: Industry-leading semantic-aware uncertainty SAR USC+Microsoft 2023 EMNLP
スペック比較表
| LLM Hallucination Detection (2022-2024) | Year | Org | Method | Industry Position |
|---|---|---|---|---|
| SAR | 2023 | Duan+Yu USC | Semantic-aware uncertainty | Industry-leading SAR uncertainty |
| SelfCheckGPT | 2023 | Manakul Cambridge | Sample consistency check black-box | Industry-leading zero-resource pioneer |
| INSIDE | 2024 | Tencent+Tsinghua | Internal state EigenScore | Industry-leading internal state |
| Lookback Lens | 2024 | MIT+Microsoft | Lookback ratio attention | Industry-leading attention-based |
| FactualityPrompts | 2022 | Meta AI+NVIDIA | Factual prompts benchmark | Industry-leading factuality benchmark |
具体例・対応製品
- SAR (2023年, Duan+Yu USC+Microsoft): Industry-leading semantic-aware uncertainty hallucination detection
- Industry-leading SAR Shifting Attention to Relevance signature: Industry-leading SAR signature
- Industry-leading semantic-aware uncertainty calibration: Industry-leading semantic-aware calibration
- Industry-leading length-bias correction signature: Industry-leading length-bias correction
- Industry-leading free-form LLM output uncertainty: Industry-leading free-form uncertainty
- 競合 SelfCheckGPT + INSIDE + Lookback Lens + FactualityPrompts: Industry-leading hallucination detection competitors
自作PCでの選び方・注意点
SAR Semantic-Aware Uncertainty は「Industry-leading semantic-aware uncertainty hallucination detection LLM + Industry-leading semantic-aware uncertainty」「Industry-leading length-bias correction + Industry-leading USC+Microsoft 2023 EMNLP」用途のIndustry-leading semantic-aware uncertainty 2023年Duan+Yu USC+Microsoft発表product。Industry-leading SAR (Industry-leading Shifting Attention to Relevance signature + Industry-leading SAR semantic-aware signature + Industry-leading attention relevance shifting) で Industry-leading SAR + Industry-leading semantic-aware signature + Industry-leading attention relevance shifting。Industry-leading semantic-aware (Industry-leading semantic-aware uncertainty calibration + Industry-leading SAR uncertainty signature + Industry-leading semantic uncertainty refinement) で Industry-leading semantic-aware + Industry-leading SAR uncertainty + Industry-leading semantic refinement。Industry-leading length-bias correction (Industry-leading length-bias correction signature + Industry-leading SAR length-bias correction + Industry-leading bias mitigation) で Industry-leading length-bias correction + Industry-leading SAR length-bias + Industry-leading bias mitigation。Industry-leading free-form (Industry-leading free-form LLM output uncertainty quantification + Industry-leading SAR free-form signature + Industry-leading free-form generation uncertainty) で Industry-leading free-form + Industry-leading SAR free-form + Industry-leading free-form generation uncertainty。Industry-leading USC+Microsoft + EMNLP 2023 (Industry-leading Jinhao Duan + Hao Cheng + Kaidi Xu USC + Microsoft Research + Drexel authors + Industry-leading SAR GitHub jinhaoduan/SAR + Industry-leading EMNLP 2023) で Industry-leading Duan+Cheng+Xu USC+Microsoft+Drexel + Industry-leading jinhaoduan/SAR + Industry-leading EMNLP 2023。但しIndustry-leading SelfCheckGPT + INSIDE + Lookback Lens + FactualityPrompts competition (Industry-leading SelfCheckGPT Cambridge zero-resource 2023 + INSIDE Tencent+Tsinghua white-box EigenScore 2024 + Lookback Lens MIT+Microsoft attention 2024 + FactualityPrompts Meta+NVIDIA benchmark 2022 vs SAR USC+Microsoft semantic-aware uncertainty length-bias 2023 trade-off) で Industry-leading 4-hallucination detection competitors vs SAR + Industry-leading Shifting Attention to Relevance + semantic-aware uncertainty calibration + length-bias correction + free-form LLM output uncertainty + USC+Microsoft+Drexel + EMNLP 2023 unique advantage adoption alignment必須。
関連用語との違い
- vs SelfCheckGPT (Cambridge 2023): SARはsemantic-aware uncertainty + length-bias・SelfCheckGPTはsample consistency
- vs INSIDE (Tencent 2024): SARはuncertainty calibration + USC・INSIDEはinternal state EigenScore
- vs Lookback Lens (MIT 2024): SARはsemantic-aware uncertainty・Lookback Lensはattention lookback ratio
よくある質問（FAQ）
Q1: SAR vs SelfCheckGPT 違いは？ A: SAR (Industry-leading Shifting Attention to Relevance + semantic-aware uncertainty calibration + length-bias correction signature + free-form LLM output uncertainty quantification + GPT + LLaMA + OPT uncertainty + Jinhao Duan + Hao Cheng + Kaidi Xu USC + Microsoft + Drexel 2023 EMNLP) vs SelfCheckGPT (Industry-leading zero-resource black-box hallucination detection + sample consistency check + 5 scoring methods + no external knowledge required + Manakul Cambridge 2023 EMNLP)・Industry-leading semantic-aware uncertainty + length-bias + USC = SAR + Industry-leading zero-resource + sample consistency + 5 scoring + Cambridge = SelfCheckGPT preference judgment。
Q2: Industry-leading SAR semantic-aware uncertainty value は？ A: Industry-leading SAR + semantic-aware (Industry-leading Shifting Attention to Relevance signature + Industry-leading semantic-aware uncertainty calibration + Industry-leading attention relevance shifting + Industry-leading SAR semantic uncertainty refinement)。
Q3: Industry-leading length-bias correction + free-form value は？ A: Industry-leading length-bias + free-form (Industry-leading length-bias correction signature + Industry-leading SAR length-bias correction + Industry-leading bias mitigation + Industry-leading free-form LLM output uncertainty quantification)。
まとめ
SAR = 2023年Duan+Yu USC+Microsoft+Drexel発表のsemantic-aware uncertainty hallucination detection LLM。Industry-leading Shifting Attention to Relevance signature + Industry-leading semantic-aware uncertainty calibration + Industry-leading length-bias correction signature + Industry-leading free-form LLM output uncertainty quantification + Industry-leading GPT + LLaMA + OPT uncertainty + Industry-leading Jinhao Duan + Hao Cheng + Shiqi Wang + Bhavya Kailkhura + Kaidi Xu USC + Microsoft Research + Drexel authors + Industry-leading SAR GitHub jinhaoduan/SAR + Industry-leading EMNLP 2023 + Industry-leading semantic-aware uncertainty SAR USC+Microsoft 2023 position確立。