https://answer.freetools.me/tags/%E8%AE%AD%E7%BB%83%E7%A8%B3%E5%AE%9A%E6%80%A7/ Recent content in 训练稳定性 on Answer Hugo -- 0.152.2 zh-cn Thu, 12 Mar 2026 13:12:12 +0800 https://answer.freetools.me/%E5%81%8F%E7%BD%AE%E9%A1%B9%E7%9A%84%E6%B6%88%E4%BA%A1%E4%B8%BA%E4%BB%80%E4%B9%88%E7%8E%B0%E4%BB%A3%E5%A4%A7%E6%A8%A1%E5%9E%8B%E5%88%A0%E9%99%A4%E4%BA%86%E8%BF%99%E4%B8%AA%E7%9C%8B%E4%BC%BC%E5%BF%85%E4%B8%8D%E5%8F%AF%E5%B0%91%E7%9A%84%E5%8F%82%E6%95%B0/ Thu, 12 Mar 2026 13:12:12 +0800 https://answer.freetools.me/%E5%81%8F%E7%BD%AE%E9%A1%B9%E7%9A%84%E6%B6%88%E4%BA%A1%E4%B8%BA%E4%BB%80%E4%B9%88%E7%8E%B0%E4%BB%A3%E5%A4%A7%E6%A8%A1%E5%9E%8B%E5%88%A0%E9%99%A4%E4%BA%86%E8%BF%99%E4%B8%AA%E7%9C%8B%E4%BC%BC%E5%BF%85%E4%B8%8D%E5%8F%AF%E5%B0%91%E7%9A%84%E5%8F%82%E6%95%B0/ 从PaLM到LLaMA，现代大模型为何纷纷移除偏置项？本文深入分析LayerNorm和残差连接如何使偏置项变得冗余，以及这一设计选择对训练稳定性和参数效率的影响。 https://answer.freetools.me/%E6%A2%AF%E5%BA%A6%E8%A3%81%E5%89%AA%E4%B8%BA%E4%BB%80%E4%B9%88%E8%BF%99%E4%B8%AA%E7%AE%80%E5%8D%95%E6%8A%80%E5%B7%A7%E8%83%BD%E6%8B%AF%E6%95%91%E4%BD%A0%E7%9A%84%E6%B7%B1%E5%BA%A6%E5%AD%A6%E4%B9%A0%E6%A8%A1%E5%9E%8B/ Thu, 12 Mar 2026 03:37:12 +0800 https://answer.freetools.me/%E6%A2%AF%E5%BA%A6%E8%A3%81%E5%89%AA%E4%B8%BA%E4%BB%80%E4%B9%88%E8%BF%99%E4%B8%AA%E7%AE%80%E5%8D%95%E6%8A%80%E5%B7%A7%E8%83%BD%E6%8B%AF%E6%95%91%E4%BD%A0%E7%9A%84%E6%B7%B1%E5%BA%A6%E5%AD%A6%E4%B9%A0%E6%A8%A1%E5%9E%8B/ 深入解析梯度裁剪技术的历史起源、数学原理、实践应用与最新进展。从2012年Pascanu等人的开创性论文，到MIT对梯度裁剪加速训练的理论解释，再到自适应梯度裁剪的最新发展，全面揭示这个看似简单却深刻影响深度学习训练的关键技术。 https://answer.freetools.me/transformer%E4%B8%BA%E4%BD%95%E9%80%89%E6%8B%A9layernorm%E8%80%8C%E4%B8%8D%E6%98%AFbatchnorm%E4%BB%8E%E5%BA%8F%E5%88%97%E6%95%B0%E6%8D%AE%E7%89%B9%E6%80%A7%E5%88%B0%E6%A2%AF%E5%BA%A6%E7%A8%B3%E5%AE%9A%E6%80%A7%E7%9A%84%E6%B7%B1%E5%BA%A6%E8%A7%A3%E6%9E%90/ Mon, 09 Mar 2026 05:42:48 +0800 https://answer.freetools.me/transformer%E4%B8%BA%E4%BD%95%E9%80%89%E6%8B%A9layernorm%E8%80%8C%E4%B8%8D%E6%98%AFbatchnorm%E4%BB%8E%E5%BA%8F%E5%88%97%E6%95%B0%E6%8D%AE%E7%89%B9%E6%80%A7%E5%88%B0%E6%A2%AF%E5%BA%A6%E7%A8%B3%E5%AE%9A%E6%80%A7%E7%9A%84%E6%B7%B1%E5%BA%A6%E8%A7%A3%E6%9E%90/ 深入解析Transformer架构中选择Layer Normalization而非Batch Normalization的技术原因。从NLP序列数据特性、批量统计量波动、训练推理一致性、分布式训练便利性等多维度展开，并结合Pre-LN与Post-LN的梯度稳定性分析，揭示现代大模型归一化选择背后的深层逻辑。