https://answer.freetools.me/tags/%E4%BA%A4%E5%8F%89%E7%86%B5/ Recent content in 交叉熵 on Answer Hugo -- 0.152.2 zh-cn Thu, 12 Mar 2026 05:08:13 +0800 https://answer.freetools.me/%E8%AF%AD%E8%A8%80%E6%A8%A1%E5%9E%8B%E7%9A%84%E6%A6%82%E7%8E%87%E6%9C%AC%E8%B4%A8%E4%BB%8E%E6%9D%A1%E4%BB%B6%E6%A6%82%E7%8E%87%E5%88%B0%E4%B8%8B%E4%B8%80%E4%B8%AA%E8%AF%8D%E9%A2%84%E6%B5%8B%E7%9A%84%E6%95%B0%E5%AD%A6%E4%B9%8B%E6%97%85/ Thu, 12 Mar 2026 05:08:13 +0800 https://answer.freetools.me/%E8%AF%AD%E8%A8%80%E6%A8%A1%E5%9E%8B%E7%9A%84%E6%A6%82%E7%8E%87%E6%9C%AC%E8%B4%A8%E4%BB%8E%E6%9D%A1%E4%BB%B6%E6%A6%82%E7%8E%87%E5%88%B0%E4%B8%8B%E4%B8%80%E4%B8%AA%E8%AF%8D%E9%A2%84%E6%B5%8B%E7%9A%84%E6%95%B0%E5%AD%A6%E4%B9%8B%E6%97%85/ 深入解析语言模型的概率本质，从Shannon的信息论到现代大语言模型，揭示为什么"预测下一个词"这个看似简单的目标能够学习到复杂的语言知识和世界知识。 https://answer.freetools.me/%E4%BA%A4%E5%8F%89%E7%86%B5%E6%8D%9F%E5%A4%B1%E5%87%BD%E6%95%B0%E4%B8%BA%E4%BB%80%E4%B9%88%E8%BF%99%E4%B8%AA%E5%85%AC%E5%BC%8F%E7%BB%9F%E6%B2%BB%E4%BA%86%E6%B7%B1%E5%BA%A6%E5%AD%A6%E4%B9%A0%E7%9A%84%E6%A6%82%E7%8E%87%E9%A2%84%E6%B5%8B/ Wed, 11 Mar 2026 22:38:36 +0800 https://answer.freetools.me/%E4%BA%A4%E5%8F%89%E7%86%B5%E6%8D%9F%E5%A4%B1%E5%87%BD%E6%95%B0%E4%B8%BA%E4%BB%80%E4%B9%88%E8%BF%99%E4%B8%AA%E5%85%AC%E5%BC%8F%E7%BB%9F%E6%B2%BB%E4%BA%86%E6%B7%B1%E5%BA%A6%E5%AD%A6%E4%B9%A0%E7%9A%84%E6%A6%82%E7%8E%87%E9%A2%84%E6%B5%8B/ 从信息论的自信息概念出发，深入解析交叉熵损失函数的数学原理、梯度推导、与最大似然估计的等价性，以及在大语言模型训练中的核心作用。涵盖熵、KL散度、困惑度、数值稳定性、标签平滑等关键技术细节。 https://answer.freetools.me/%E5%9B%B0%E6%83%91%E5%BA%A6%E5%A6%82%E4%BD%95%E6%88%90%E4%B8%BA%E8%AF%AD%E8%A8%80%E6%A8%A1%E5%9E%8B%E8%AF%84%E4%BC%B0%E7%9A%84%E9%BB%84%E9%87%91%E6%A0%87%E5%87%86%E4%BB%8E%E4%BF%A1%E6%81%AF%E8%AE%BA%E5%88%B0%E7%8E%B0%E4%BB%A3%E5%A4%A7%E6%A8%A1%E5%9E%8B%E7%9A%84%E4%BA%94%E5%8D%81%E5%B9%B4%E6%BC%94%E8%BF%9B/ Wed, 11 Mar 2026 20:59:11 +0800 https://answer.freetools.me/%E5%9B%B0%E6%83%91%E5%BA%A6%E5%A6%82%E4%BD%95%E6%88%90%E4%B8%BA%E8%AF%AD%E8%A8%80%E6%A8%A1%E5%9E%8B%E8%AF%84%E4%BC%B0%E7%9A%84%E9%BB%84%E9%87%91%E6%A0%87%E5%87%86%E4%BB%8E%E4%BF%A1%E6%81%AF%E8%AE%BA%E5%88%B0%E7%8E%B0%E4%BB%A3%E5%A4%A7%E6%A8%A1%E5%9E%8B%E7%9A%84%E4%BA%94%E5%8D%81%E5%B9%B4%E6%BC%94%E8%BF%9B/ 深入解析困惑度（Perplexity）作为语言模型评估指标的数学原理、信息论根基、分支因子直观解释、计算方法、以及其在现代大模型评估中的局限性与演进。从1977年IBM的语音识别研究到GPT时代的评估困境，揭示这个看似简单的指标背后的深层逻辑。