— Simple and Clear: MINIMAL code and architecture changes. Only one up-and-down projection layer introduced, NO recurrent memory caching or sparse attention required. — Train Short, Test Long: NO performance drop on the sequences 4x~8x longer than the training ones (see here). — Continuous Length Extrapolation: Explicitly modeling the continuous dynamics of context window size during length extrapolation. More details about long-text modeling with our CLEX can be found at the git repo. If you find our project useful, hope you can star our repo and cite our paper as follows: «` @article{damonlpsg2023clex, author = {Chen, Guanzheng and Li, Xin and Meng, Zaiqiao and Liang, Shangsong and Bing, Lidong}, title = {CLEX: Continuous Length Extrapolation for Large Language Models}, year = 2023, journal = {arXiv preprint arXiv:2310.16450}, url = {https://arxiv.org/abs/2310.16450} }
Модальности:
Генерация текста
Области применения:
Диалог / чат
Задача: Генерация текста
Автор: DAMO-NLP-SG
Теги: llama, custom_code, en, text-generation-inference, endpoints_compatible
Лайков: 3 | Загрузок: 27
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
