https://answer.freetools.me/categories/%E5%A4%A7%E6%A8%A1%E5%9E%8B%E8%AE%AD%E7%BB%83/ Recent content in 大模型训练 on Answer Hugo -- 0.152.2 zh-cn Thu, 12 Mar 2026 08:26:13 +0800 https://answer.freetools.me/%E6%9D%83%E9%87%8D%E8%A1%B0%E5%87%8F%E4%B8%8El2%E6%AD%A3%E5%88%99%E5%8C%96%E4%B8%BA%E4%BB%80%E4%B9%88%E8%BF%99%E4%B8%AA%E7%9C%8B%E4%BC%BC%E5%BE%AE%E5%B0%8F%E7%9A%84%E5%8C%BA%E5%88%AB%E8%AE%A9adamw%E6%88%90%E4%B8%BA%E5%A4%A7%E6%A8%A1%E5%9E%8B%E8%AE%AD%E7%BB%83%E7%9A%84%E6%A0%87%E9%85%8D/ Thu, 12 Mar 2026 08:26:13 +0800 https://answer.freetools.me/%E6%9D%83%E9%87%8D%E8%A1%B0%E5%87%8F%E4%B8%8El2%E6%AD%A3%E5%88%99%E5%8C%96%E4%B8%BA%E4%BB%80%E4%B9%88%E8%BF%99%E4%B8%AA%E7%9C%8B%E4%BC%BC%E5%BE%AE%E5%B0%8F%E7%9A%84%E5%8C%BA%E5%88%AB%E8%AE%A9adamw%E6%88%90%E4%B8%BA%E5%A4%A7%E6%A8%A1%E5%9E%8B%E8%AE%AD%E7%BB%83%E7%9A%84%E6%A0%87%E9%85%8D/ 深入解析权重衰减与L2正则化在SGD中等价但在Adam中不等价的原因，揭示AdamW如何通过解耦机制实现正确的正则化效果，成为Transformer和大语言模型训练的默认优化器。 https://answer.freetools.me/%E6%A2%AF%E5%BA%A6%E8%A3%81%E5%89%AA%E4%B8%BA%E4%BB%80%E4%B9%88%E8%BF%99%E4%B8%AA%E7%AE%80%E5%8D%95%E6%8A%80%E5%B7%A7%E8%83%BD%E6%8B%AF%E6%95%91%E4%BD%A0%E7%9A%84%E6%B7%B1%E5%BA%A6%E5%AD%A6%E4%B9%A0%E6%A8%A1%E5%9E%8B/ Thu, 12 Mar 2026 03:37:12 +0800 https://answer.freetools.me/%E6%A2%AF%E5%BA%A6%E8%A3%81%E5%89%AA%E4%B8%BA%E4%BB%80%E4%B9%88%E8%BF%99%E4%B8%AA%E7%AE%80%E5%8D%95%E6%8A%80%E5%B7%A7%E8%83%BD%E6%8B%AF%E6%95%91%E4%BD%A0%E7%9A%84%E6%B7%B1%E5%BA%A6%E5%AD%A6%E4%B9%A0%E6%A8%A1%E5%9E%8B/ 深入解析梯度裁剪技术的历史起源、数学原理、实践应用与最新进展。从2012年Pascanu等人的开创性论文，到MIT对梯度裁剪加速训练的理论解释，再到自适应梯度裁剪的最新发展，全面揭示这个看似简单却深刻影响深度学习训练的关键技术。 https://answer.freetools.me/%E6%A2%AF%E5%BA%A6%E7%B4%AF%E7%A7%AF%E7%9C%9F%E7%9A%84%E8%83%BD%E6%A8%A1%E6%8B%9F%E5%A4%A7%E6%89%B9%E9%87%8F%E8%AE%AD%E7%BB%83%E5%90%97%E4%BB%8E%E6%95%B0%E5%AD%A6%E7%AD%89%E4%BB%B7%E6%80%A7%E5%88%B0%E9%9A%90%E6%80%A7%E6%88%90%E6%9C%AC%E7%9A%84%E5%AE%8C%E6%95%B4%E8%A7%A3%E6%9E%90/ Wed, 11 Mar 2026 22:27:41 +0800 https://answer.freetools.me/%E6%A2%AF%E5%BA%A6%E7%B4%AF%E7%A7%AF%E7%9C%9F%E7%9A%84%E8%83%BD%E6%A8%A1%E6%8B%9F%E5%A4%A7%E6%89%B9%E9%87%8F%E8%AE%AD%E7%BB%83%E5%90%97%E4%BB%8E%E6%95%B0%E5%AD%A6%E7%AD%89%E4%BB%B7%E6%80%A7%E5%88%B0%E9%9A%90%E6%80%A7%E6%88%90%E6%9C%AC%E7%9A%84%E5%AE%8C%E6%95%B4%E8%A7%A3%E6%9E%90/ 深入解析梯度累积技术的数学原理、正确实现方式与隐性成本。从GPU显存构成分析到损失归一化的细节，从BatchNorm冲突到分布式训练中的性能陷阱，揭示这个被广泛使用的显存优化技术的完整技术图景。