https://answer.freetools.me/tags/%E4%BC%98%E5%8C%96%E5%99%A8/ Recent content in 优化器 on Answer Hugo -- 0.152.2 zh-cn Thu, 12 Mar 2026 16:06:50 +0800 https://answer.freetools.me/%E7%A5%9E%E7%BB%8F%E7%BD%91%E7%BB%9C%E6%98%AF%E5%A6%82%E4%BD%95%E5%AD%A6%E4%B9%A0%E7%9A%84%E4%BB%8E%E5%89%8D%E5%90%91%E4%BC%A0%E6%92%AD%E5%88%B0%E5%8F%8D%E5%90%91%E4%BC%A0%E6%92%AD%E7%9A%84%E5%AE%8C%E6%95%B4%E8%AE%AD%E7%BB%83%E8%BF%87%E7%A8%8B%E8%A7%A3%E6%9E%90/ Thu, 12 Mar 2026 16:06:50 +0800 https://answer.freetools.me/%E7%A5%9E%E7%BB%8F%E7%BD%91%E7%BB%9C%E6%98%AF%E5%A6%82%E4%BD%95%E5%AD%A6%E4%B9%A0%E7%9A%84%E4%BB%8E%E5%89%8D%E5%90%91%E4%BC%A0%E6%92%AD%E5%88%B0%E5%8F%8D%E5%90%91%E4%BC%A0%E6%92%AD%E7%9A%84%E5%AE%8C%E6%95%B4%E8%AE%AD%E7%BB%83%E8%BF%87%E7%A8%8B%E8%A7%A3%E6%9E%90/ 神经网络是如何学习的：从前向传播到反向传播的完整训练过程解析 https://answer.freetools.me/%E6%9D%83%E9%87%8D%E8%A1%B0%E5%87%8F%E4%B8%8El2%E6%AD%A3%E5%88%99%E5%8C%96%E4%B8%BA%E4%BB%80%E4%B9%88%E8%BF%99%E4%B8%AA%E7%9C%8B%E4%BC%BC%E5%BE%AE%E5%B0%8F%E7%9A%84%E5%8C%BA%E5%88%AB%E8%AE%A9adamw%E6%88%90%E4%B8%BA%E5%A4%A7%E6%A8%A1%E5%9E%8B%E8%AE%AD%E7%BB%83%E7%9A%84%E6%A0%87%E9%85%8D/ Thu, 12 Mar 2026 08:26:13 +0800 https://answer.freetools.me/%E6%9D%83%E9%87%8D%E8%A1%B0%E5%87%8F%E4%B8%8El2%E6%AD%A3%E5%88%99%E5%8C%96%E4%B8%BA%E4%BB%80%E4%B9%88%E8%BF%99%E4%B8%AA%E7%9C%8B%E4%BC%BC%E5%BE%AE%E5%B0%8F%E7%9A%84%E5%8C%BA%E5%88%AB%E8%AE%A9adamw%E6%88%90%E4%B8%BA%E5%A4%A7%E6%A8%A1%E5%9E%8B%E8%AE%AD%E7%BB%83%E7%9A%84%E6%A0%87%E9%85%8D/ 深入解析权重衰减与L2正则化在SGD中等价但在Adam中不等价的原因，揭示AdamW如何通过解耦机制实现正确的正则化效果，成为Transformer和大语言模型训练的默认优化器。 https://answer.freetools.me/%E6%A2%AF%E5%BA%A6%E4%B8%8B%E9%99%8D%E4%BC%98%E5%8C%96%E5%99%A8%E4%BB%8Esgd%E5%88%B0adamw%E4%B8%BA%E4%BB%80%E4%B9%88%E8%BF%99%E4%B8%AA%E9%80%89%E6%8B%A9%E8%83%BD%E5%86%B3%E5%AE%9A%E6%A8%A1%E5%9E%8B%E7%9A%84%E5%91%BD%E8%BF%90/ Thu, 12 Mar 2026 06:04:32 +0800 https://answer.freetools.me/%E6%A2%AF%E5%BA%A6%E4%B8%8B%E9%99%8D%E4%BC%98%E5%8C%96%E5%99%A8%E4%BB%8Esgd%E5%88%B0adamw%E4%B8%BA%E4%BB%80%E4%B9%88%E8%BF%99%E4%B8%AA%E9%80%89%E6%8B%A9%E8%83%BD%E5%86%B3%E5%AE%9A%E6%A8%A1%E5%9E%8B%E7%9A%84%E5%91%BD%E8%BF%90/ 深入解析深度学习中梯度下降优化器的演进历程：从最基础的SGD到现代的AdamW，从Momentum到Nesterov，从AdaGrad到RMSprop。剖析为什么Adam收敛快却泛化不如SGD，为什么权重衰减需要解耦，以及不同场景下如何选择正确的优化器。 https://answer.freetools.me/%E6%A2%AF%E5%BA%A6%E8%A3%81%E5%89%AA%E4%B8%BA%E4%BB%80%E4%B9%88%E8%BF%99%E4%B8%AA%E7%AE%80%E5%8D%95%E6%8A%80%E5%B7%A7%E8%83%BD%E6%8B%AF%E6%95%91%E4%BD%A0%E7%9A%84%E6%B7%B1%E5%BA%A6%E5%AD%A6%E4%B9%A0%E6%A8%A1%E5%9E%8B/ Thu, 12 Mar 2026 03:37:12 +0800 https://answer.freetools.me/%E6%A2%AF%E5%BA%A6%E8%A3%81%E5%89%AA%E4%B8%BA%E4%BB%80%E4%B9%88%E8%BF%99%E4%B8%AA%E7%AE%80%E5%8D%95%E6%8A%80%E5%B7%A7%E8%83%BD%E6%8B%AF%E6%95%91%E4%BD%A0%E7%9A%84%E6%B7%B1%E5%BA%A6%E5%AD%A6%E4%B9%A0%E6%A8%A1%E5%9E%8B/ 深入解析梯度裁剪技术的历史起源、数学原理、实践应用与最新进展。从2012年Pascanu等人的开创性论文，到MIT对梯度裁剪加速训练的理论解释，再到自适应梯度裁剪的最新发展，全面揭示这个看似简单却深刻影响深度学习训练的关键技术。 https://answer.freetools.me/batch-size%E7%9A%84%E9%80%89%E6%8B%A9%E4%B8%BA%E4%BB%80%E4%B9%88%E8%BF%99%E4%B8%AA%E8%B6%85%E5%8F%82%E6%95%B0%E8%83%BD%E5%86%B3%E5%AE%9A%E6%A8%A1%E5%9E%8B%E7%9A%84%E7%94%9F%E6%AD%BB/ Wed, 11 Mar 2026 23:43:13 +0800 https://answer.freetools.me/batch-size%E7%9A%84%E9%80%89%E6%8B%A9%E4%B8%BA%E4%BB%80%E4%B9%88%E8%BF%99%E4%B8%AA%E8%B6%85%E5%8F%82%E6%95%B0%E8%83%BD%E5%86%B3%E5%AE%9A%E6%A8%A1%E5%9E%8B%E7%9A%84%E7%94%9F%E6%AD%BB/ 深度解析神经网络训练中batch size选择背后的理论原理，从泛化差距到尖锐最小值，从梯度噪声到学习率缩放，揭示为什么小batch往往比大batch泛化更好。 https://answer.freetools.me/%E5%A4%A7%E6%A8%A1%E5%9E%8B%E4%B8%BA%E4%BD%95%E6%99%AE%E9%81%8D%E9%80%89%E6%8B%A9adamw%E8%80%8C%E9%9D%9Esgd%E4%BB%8E%E8%87%AA%E9%80%82%E5%BA%94%E5%AD%A6%E4%B9%A0%E7%8E%87%E5%88%B0%E8%A7%A3%E8%80%A6%E6%9D%83%E9%87%8D%E8%A1%B0%E5%87%8F%E7%9A%84%E6%8A%80%E6%9C%AF%E8%A7%A3%E6%9E%90/ Wed, 11 Mar 2026 15:46:26 +0800 https://answer.freetools.me/%E5%A4%A7%E6%A8%A1%E5%9E%8B%E4%B8%BA%E4%BD%95%E6%99%AE%E9%81%8D%E9%80%89%E6%8B%A9adamw%E8%80%8C%E9%9D%9Esgd%E4%BB%8E%E8%87%AA%E9%80%82%E5%BA%94%E5%AD%A6%E4%B9%A0%E7%8E%87%E5%88%B0%E8%A7%A3%E8%80%A6%E6%9D%83%E9%87%8D%E8%A1%B0%E5%87%8F%E7%9A%84%E6%8A%80%E6%9C%AF%E8%A7%A3%E6%9E%90/ 深入解析大语言模型训练中优化器选择的技术原理：从SGD的固定学习率困境，到Adam的自适应机制，再到AdamW的解耦权重衰减。基于Loshchilov-Hutter论文、NeurIPS研究以及GPT-3/LLaMA等模型的训练实践，系统阐述为什么千亿参数模型的训练都选择AdamW，以及这一选择背后的内存代价与泛化权衡。