https://answer.freetools.me/tags/%E9%A2%84%E8%AE%AD%E7%BB%83/ Recent content in 预训练 on Answer Hugo -- 0.152.2 zh-cn Thu, 12 Mar 2026 05:08:13 +0800 https://answer.freetools.me/%E8%AF%AD%E8%A8%80%E6%A8%A1%E5%9E%8B%E7%9A%84%E6%A6%82%E7%8E%87%E6%9C%AC%E8%B4%A8%E4%BB%8E%E6%9D%A1%E4%BB%B6%E6%A6%82%E7%8E%87%E5%88%B0%E4%B8%8B%E4%B8%80%E4%B8%AA%E8%AF%8D%E9%A2%84%E6%B5%8B%E7%9A%84%E6%95%B0%E5%AD%A6%E4%B9%8B%E6%97%85/ Thu, 12 Mar 2026 05:08:13 +0800 https://answer.freetools.me/%E8%AF%AD%E8%A8%80%E6%A8%A1%E5%9E%8B%E7%9A%84%E6%A6%82%E7%8E%87%E6%9C%AC%E8%B4%A8%E4%BB%8E%E6%9D%A1%E4%BB%B6%E6%A6%82%E7%8E%87%E5%88%B0%E4%B8%8B%E4%B8%80%E4%B8%AA%E8%AF%8D%E9%A2%84%E6%B5%8B%E7%9A%84%E6%95%B0%E5%AD%A6%E4%B9%8B%E6%97%85/ 深入解析语言模型的概率本质，从Shannon的信息论到现代大语言模型，揭示为什么"预测下一个词"这个看似简单的目标能够学习到复杂的语言知识和世界知识。 https://answer.freetools.me/eos-token%E4%B8%BA%E4%BB%80%E4%B9%88%E8%BF%99%E4%B8%AA%E7%89%B9%E6%AE%8A%E6%A0%87%E8%AE%B0%E5%86%B3%E5%AE%9A%E4%BA%86%E5%A4%A7%E6%A8%A1%E5%9E%8B%E7%9A%84%E8%AF%B4%E8%AF%9D%E8%BE%B9%E7%95%8C/ Thu, 12 Mar 2026 04:29:51 +0800 https://answer.freetools.me/eos-token%E4%B8%BA%E4%BB%80%E4%B9%88%E8%BF%99%E4%B8%AA%E7%89%B9%E6%AE%8A%E6%A0%87%E8%AE%B0%E5%86%B3%E5%AE%9A%E4%BA%86%E5%A4%A7%E6%A8%A1%E5%9E%8B%E7%9A%84%E8%AF%B4%E8%AF%9D%E8%BE%B9%E7%95%8C/ 深入解析大语言模型中 EOS (End of Sequence) Token 的工作原理、训练机制、跨模型实现差异，以及斯坦福大学关于 EOS 决策与长度外推的前沿研究发现。 https://answer.freetools.me/%E5%9B%A0%E6%9E%9C%E8%AF%AD%E8%A8%80%E6%A8%A1%E5%9E%8B%E4%B8%8E%E6%8E%A9%E7%A0%81%E8%AF%AD%E8%A8%80%E6%A8%A1%E5%9E%8B%E4%B8%A4%E7%A7%8D%E9%A2%84%E8%AE%AD%E7%BB%83%E8%8C%83%E5%BC%8F%E7%9A%84%E6%9C%AC%E8%B4%A8%E5%B7%AE%E5%BC%82/ Wed, 11 Mar 2026 21:12:01 +0800 https://answer.freetools.me/%E5%9B%A0%E6%9E%9C%E8%AF%AD%E8%A8%80%E6%A8%A1%E5%9E%8B%E4%B8%8E%E6%8E%A9%E7%A0%81%E8%AF%AD%E8%A8%80%E6%A8%A1%E5%9E%8B%E4%B8%A4%E7%A7%8D%E9%A2%84%E8%AE%AD%E7%BB%83%E8%8C%83%E5%BC%8F%E7%9A%84%E6%9C%AC%E8%B4%A8%E5%B7%AE%E5%BC%82/ 深度解析Transformer两大预训练范式：因果语言模型(CLM)与掩码语言模型(MLM)的工作原理、注意力机制差异、训练目标、应用场景对比，以及现代大模型为何普遍选择decoder-only架构 https://answer.freetools.me/%E5%A4%A7%E6%A8%A1%E5%9E%8B%E6%98%AF%E5%A6%82%E4%BD%95%E8%A2%AB%E8%AE%AD%E7%BB%83%E5%87%BA%E6%9D%A5%E7%9A%84%E4%BB%8E%E9%A2%84%E8%AE%AD%E7%BB%83%E5%88%B0%E5%AF%B9%E9%BD%90%E7%9A%84%E4%B8%89%E9%98%B6%E6%AE%B5%E6%8A%80%E6%9C%AF%E5%85%A8%E6%99%AF/ Wed, 11 Mar 2026 14:25:15 +0800 https://answer.freetools.me/%E5%A4%A7%E6%A8%A1%E5%9E%8B%E6%98%AF%E5%A6%82%E4%BD%95%E8%A2%AB%E8%AE%AD%E7%BB%83%E5%87%BA%E6%9D%A5%E7%9A%84%E4%BB%8E%E9%A2%84%E8%AE%AD%E7%BB%83%E5%88%B0%E5%AF%B9%E9%BD%90%E7%9A%84%E4%B8%89%E9%98%B6%E6%AE%B5%E6%8A%80%E6%9C%AF%E5%85%A8%E6%99%AF/ 深入解析大语言模型训练的完整技术链路：从海量数据收集与清洗、分词器构建，到预训练阶段的自监督学习与分布式训练，再到监督微调和RLHF/DPO对齐，全面揭示千亿参数模型从零到可用的技术演进过程。