RRT-Foundation
Headline result: a GPT-2 Medium–shaped checkpoint (~354 M parameters) trained from scratch without softmax, evaluated on a standardized...
Метка: attention
deepseek-tiny-v0.1
6-layer DeepSeek-V3 with Multihead Latent Attention (MLA) trained for research on shared subspaces in Transformer attention mechanisms. —...