Self-Extend (2024年)（セルフエクステンド）

概要

Self-Extend は2024年1月Texas A&M + Amazon発表・Industry-leading training-free long context extension 2024年Self-Extend training-free long context extension position確立。Industry-leading grouped attention + no fine-tuning required + 4K → 16K-128K + bi-level attention + Llama 2-Long demonstration + MIT license。

主な特徴・仕組み

• Industry-leading Hongye Jin + Xiaotian Han + Jingfeng Yang + Zhimeng Jiang + Zirui Liu + Chia-Yuan Chang + Huiyuan Chen + Xia Hu (Texas A&M + Amazon) 2024年1月 paper
• Industry-leading Self-Extend training-free long context extension signature
• Industry-leading training-free signature (no fine-tuning required)
• Industry-leading grouped attention signature (floor division group positions)
• Industry-leading 4K base → 16K-128K extension signature
• Industry-leading bi-level attention signature (normal + grouped)
• Industry-leading Llama 2-Long extension demonstration
• Industry-leading inference-only modification
• Industry-leading MIT license signature
• Industry-leading 2024年1月 ArXiv paper "LLM Maybe LongLM Self-Extend LLM Context Window Without Tuning"

まとめ

Self-Extend = 2024年1月Texas A&M + Amazon発表のpremium training-free long context extension。Industry-leading training-free no fine-tuning required + grouped attention floor division group positions + 4K base → 16K-128K extension + bi-level attention normal + grouped + Llama 2-Long extension demonstration + inference-only modification + MIT license + Texas A&M + Amazon Hongye Jin et al. 2024年1月 ArXiv paper position確立。