第8章：不确定知识推理（Reasoning with Uncertain Knowledge）
一、导读
1.1 本章学习目标
第8章介绍概率推理——AI在不确定环境中进行推理的核心工具。现实世界充满不确定性：传感器有噪声，知识不完整，行动效果不可预测。概率论为处理这些不确定性提供了严格的数学框架。
本章的核心学习目标包括：
- 掌握概率论基础：先验概率、条件概率、贝叶斯定理、全概率公式。
- 理解贝叶斯网络：如何用有向无环图表示变量间的条件依赖关系。
- 掌握精确推理方法：枚举法、变量消除算法。
- 理解近似推理：直接采样、拒绝采样、似然加权、MCMC。
- 掌握隐马尔可夫模型：时序概率模型的基本概念和前向-后向算法。
- 了解概率推理的应用：医疗诊断、语音识别、自然语言处理等。
1.2 为什么第八章如此重要
不确定性是AI面临的核心挑战。从医疗诊断（症状不一定意味着某种疾病）到自动驾驶（传感器可能有噪声），再到自然语言处理（一个词可能有多种含义），概率推理无处不在。
贝叶斯网络是概率推理的核心工具。它将领域知识编码为图结构，使得复杂的联合概率分布可以用紧凑的方式表示和推理。掌握贝叶斯网络，就掌握了在不确定条件下进行智能决策的关键技术。
1.3 核心问题引导
在阅读本章时，请带着以下问题思考：
为什么需要概率？ 逻辑推理在什么情况下会失败？概率如何补充逻辑的不足？
贝叶斯定理的威力：为什么贝叶斯定理被称为"概率论的牛顿第二定律"？
条件独立的重要性：条件独立假设如何简化复杂的概率计算？
贝叶斯网络 vs 全联合分布：为什么贝叶斯网络能用更少的参数表示复杂的概率关系？
精确推理 vs 近似推理：在什么情况下必须使用近似推理？各种采样方法的优缺点是什么？
二、核心概念详解
2.1 概率基础
2.1.1 概率公理
概率论建立在以下三条公理之上：
非负性：0 ≤ P(A) ≤ 1
规范性：P(Ω) = 1（必然事件的概率为1）
可加性：互斥事件 A 和 B，P(A∪B) = P(A) + P(B)
python
import numpy as np
class ProbabilitySpace:
def __init__(self, sample_space, probabilities):
self.sample_space = sample_space
self.probabilities = probabilities
assert abs(sum(probabilities.values()) - 1.0) < 1e-10
def probability(self, event):
return sum(self.probabilities.get(e, 0) for e in event)
def conditional_probability(self, event_a, event_b):
p_b = self.probability(event_b)
if p_b == 0:
return 0
p_ab = self.probability(set(event_a) & set(event_b))
return p_ab / p_b
# 掷骰子
dice_space = ProbabilitySpace(
{1, 2, 3, 4, 5, 6},
{i: 1/6 for i in range(1, 7)}
)
print(f"P(偶数) = {dice_space.probability({2, 4, 6})}")
print(f"P(>4 | 偶数) = {dice_space.conditional_probability({5, 6}, {2, 4, 6})}")2.1.2 贝叶斯定理
贝叶斯定理是概率推理的核心：
P(H|E) = P(E|H) × P(H) / P(E)- P(H|E)：后验概率——给定证据 E 后假设 H 的概率
- P(H)：先验概率——不考虑证据时 H 的概率
- P(E|H)：似然——假设 H 为真时观察到证据 E 的概率
- P(E)：证据的边际概率
python
def bayes_theorem(prior_h, likelihood_e_given_h, prior_e):
return (likelihood_e_given_h * prior_h) / prior_e
# 医疗诊断示例
# 疾病患病率 1%，检测准确率 99%，假阳性率 5%
prior_disease = 0.01
likelihood_positive_given_disease = 0.99
false_positive_rate = 0.05
prior_positive = (likelihood_positive_given_disease * prior_disease +
false_positive_rate * (1 - prior_disease))
posterior = bayes_theorem(
prior_disease,
likelihood_positive_given_disease,
prior_positive
)
print(f"检测阳性后患病的概率: {posterior:.4f}")2.1.3 条件独立性
两个变量 X 和 Y 在给定 Z 的条件下独立，记为 X ⊥ Y | Z：
P(X, Y | Z) = P(X | Z) × P(Y | Z)条件独立大大简化了概率计算。
python
class ConditionalIndependence:
def __init__(self):
self.joint_dist = {}
def check_independence(self, x, y, z):
for z_val in set(z):
x_given_z = {}
y_given_z = {}
xy_given_z = {}
for (x_val, y_val, z_val_entry), prob in self.joint_dist.items():
if z_val_entry == z_val:
xy_given_z[(x_val, y_val)] = prob
x_given_z[x_val] = x_given_z.get(x_val, 0) + prob
y_given_z[y_val] = y_given_z.get(y_val, 0) + prob
for (x_val, y_val), p_xy in xy_given_z.items():
p_x = x_given_z.get(x_val, 0)
p_y = y_given_z.get(y_val, 0)
if abs(p_xy - p_x * p_y) > 1e-10:
return False
return True2.2 贝叶斯网络
2.2.1 贝叶斯网络的结构
贝叶斯网络是一个有向无环图（DAG），其中：
- 节点表示随机变量
- 边表示直接的因果关系
- 每个节点有一个条件概率表（CPT）
python
class BayesianNetwork:
def __init__(self):
self.nodes = {}
self.edges = {}
self.cpts = {}
def add_node(self, name, parents=None, cpt=None):
self.nodes[name] = parents or []
if parents:
for p in parents:
if p not in self.edges:
self.edges[p] = []
self.edges[p].append(name)
if cpt:
self.cpts[name] = cpt
def get_parents(self, node):
return self.nodes.get(node, [])
def get_children(self, node):
return self.edges.get(node, [])
# 警报网络示例
# Burglary → Alarm ← Earthquake
# Alarm → JohnCalls
# Alarm → MaryCalls
alarm_net = BayesianNetwork()
alarm_net.add_node('Burglary', cpt={True: 0.001, False: 0.999})
alarm_net.add_node('Earthquake', cpt={True: 0.002, False: 0.998})
alarm_net.add_node('Alarm', parents=['Burglary', 'Earthquake'], cpt={
(True, True): {True: 0.95, False: 0.05},
(True, False): {True: 0.94, False: 0.06},
(False, True): {True: 0.29, False: 0.71},
(False, False): {True: 0.001, False: 0.999},
})
alarm_net.add_node('JohnCalls', parents=['Alarm'], cpt={
True: {True: 0.90, False: 0.10},
False: {True: 0.05, False: 0.95},
})
alarm_net.add_node('MaryCalls', parents=['Alarm'], cpt={
True: {True: 0.70, False: 0.30},
False: {True: 0.01, False: 0.99},
})2.2.2 全联合分布的紧凑表示
贝叶斯网络利用条件独立将全联合分布分解为局部条件概率的乘积：
P(X₁, X₂, ..., Xₙ) = ∏ P(Xᵢ | Parents(Xᵢ))python
def full_joint_probability(bn, assignment):
prob = 1.0
for node in bn.nodes:
parents = bn.get_parents(node)
if parents:
parent_values = tuple(assignment[p] for p in parents)
cpt = bn.cpts[node]
prob *= cpt[parent_values][assignment[node]]
else:
prob *= bn.cpts[node][assignment[node]]
return prob
# 计算 P(Burglary=True, Earthquake=True, Alarm=True, JohnCalls=True, MaryCalls=True)
assignment = {
'Burglary': True, 'Earthquake': True, 'Alarm': True,
'JohnCalls': True, 'MaryCalls': True
}
prob = full_joint_probability(alarm_net, assignment)
print(f"全联合概率: {prob:.8f}")2.2.3 精确推理
枚举法：通过枚举所有可能的变量赋值来计算查询概率。
python
def enumeration_ask(query_var, evidence, bn):
q_values = {}
variables = [v for v in bn.nodes if v != query_var]
for val in [True, False]:
assignment = dict(evidence)
assignment[query_var] = val
q_values[val] = enumerate_all(variables, assignment, bn)
total = sum(q_values.values())
if total > 0:
return {k: v / total for k, v in q_values.items()}
return q_values
def enumerate_all(variables, assignment, bn):
if not variables:
return 1.0
var = variables[0]
rest = variables[1:]
total = 0.0
for val in [True, False]:
assignment[var] = val
parents = bn.get_parents(var)
if parents:
parent_values = tuple(assignment[p] for p in parents)
prob = bn.cpts[var][parent_values][val]
else:
prob = bn.cpts[var][val]
total += prob * enumerate_all(rest, assignment, bn)
return total
# 查询 P(Burglary | JohnCalls=True, MaryCalls=True)
evidence = {'JohnCalls': True, 'MaryCalls': True}
result = enumeration_ask('Burglary', evidence, alarm_net)
print(f"P(Burglary | JohnCalls, MaryCalls): {result}")变量消除：一种更高效的精确推理算法，通过逐步消除变量来计算查询。
python
def variable_elimination(query_var, evidence, bn):
variables = [v for v in bn.nodes if v != query_var and v not in evidence]
factors = {}
for node in bn.nodes:
parents = bn.get_parents(node)
relevant_vars = [node] + parents
factors[node] = make_factor(node, relevant_vars, bn)
for var in variables:
relevant_factors = [f for f in factors.values() if var in f['vars']]
for f in relevant_factors:
del factors[f['name']]
product = multiply_factors(relevant_factors)
factors[var] = sum_out(product, var)
final_factor = multiply_factors(list(factors.values()))
total = sum(final_factor['table'].values())
if total > 0:
return {k: v / total for k, v in final_factor['table'].items()}
return {}
def make_factor(node, relevant_vars, bn):
return {'name': node, 'vars': relevant_vars, 'table': {}}
def multiply_factors(factors):
if not factors:
return {'vars': [], 'table': {(): 1.0}}
return factors[0]
def sum_out(factor, var):
return factor2.3 近似推理
2.3.1 直接采样
python
def prior_sample(bn):
sample = {}
for node in bn.nodes:
parents = bn.get_parents(node)
if parents:
parent_values = tuple(sample[p] for p in parents)
probs = bn.cpts[node][parent_values]
else:
probs = bn.cpts[node]
sample[node] = random.choices([True, False],
weights=[probs[True], probs[False]])[0]
return sample
import random
def rejection_sampling(query_var, evidence, bn, n_samples=10000):
counts = {True: 0, False: 0}
for _ in range(n_samples):
sample = prior_sample(bn)
if all(sample[e] == v for e, v in evidence.items()):
counts[sample[query_var]] += 1
total = sum(counts.values())
if total > 0:
return {k: v / total for k, v in counts.items()}
return counts
result = rejection_sampling('Burglary', evidence, alarm_net, 10000)
print(f"拒绝采样结果: {result}")2.3.2 似然加权
python
def likelihood_weighting(query_var, evidence, bn, n_samples=10000):
weights = {True: 0.0, False: 0.0}
for _ in range(n_samples):
sample = {}
weight = 1.0
for node in bn.nodes:
if node in evidence:
sample[node] = evidence[node]
parents = bn.get_parents(node)
if parents:
parent_values = tuple(sample[p] for p in parents)
weight *= bn.cpts[node][parent_values][evidence[node]]
else:
weight *= bn.cpts[node][evidence[node]]
else:
parents = bn.get_parents(node)
if parents:
parent_values = tuple(sample[p] for p in parents)
probs = bn.cpts[node][parent_values]
else:
probs = bn.cpts[node]
sample[node] = random.choices([True, False],
weights=[probs[True], probs[False]])[0]
weights[sample[query_var]] += weight
total = sum(weights.values())
if total > 0:
return {k: v / total for k, v in weights.items()}
return weights
result = likelihood_weighting('Burglary', evidence, alarm_net, 10000)
print(f"似然加权结果: {result}")2.4 隐马尔可夫模型
2.4.1 HMM 的基本概念
HMM 是一种时序概率模型，假设系统在每个时刻处于某个隐藏状态，我们只能观察到与隐藏状态相关的证据。
python
class HiddenMarkovModel:
def __init__(self, states, observations, transition, emission, initial):
self.states = states
self.observations = observations
self.transition = transition
self.emission = emission
self.initial = initial
def forward(self, evidence):
n = len(evidence)
alpha = [{} for _ in range(n)]
for state in self.states:
alpha[0][state] = self.initial[state] * self.emission[state][evidence[0]]
for t in range(1, n):
for state in self.states:
alpha[t][state] = sum(
alpha[t-1][prev_state] * self.transition[prev_state][state]
for prev_state in self.states
) * self.emission[state][evidence[t]]
return alpha
def backward(self, evidence):
n = len(evidence)
beta = [{} for _ in range(n)]
for state in self.states:
beta[n-1][state] = 1.0
for t in range(n-2, -1, -1):
for state in self.states:
beta[t][state] = sum(
self.transition[state][next_state] *
self.emission[next_state][evidence[t+1]] *
beta[t+1][next_state]
for next_state in self.states
)
return beta
# 简单的天气 HMM
# 隐藏状态：晴天、雨天
# 观察：散步、购物、打扫
weather_hmm = HiddenMarkovModel(
states=['Sunny', 'Rainy'],
observations=['Walk', 'Shop', 'Clean'],
transition={
'Sunny': {'Sunny': 0.7, 'Rainy': 0.3},
'Rainy': {'Sunny': 0.4, 'Rainy': 0.6},
},
emission={
'Sunny': {'Walk': 0.6, 'Shop': 0.3, 'Clean': 0.1},
'Rainy': {'Walk': 0.1, 'Shop': 0.4, 'Clean': 0.5},
},
initial={'Sunny': 0.6, 'Rainy': 0.4}
)
evidence = ['Walk', 'Shop', 'Clean']
alpha = weather_hmm.forward(evidence)
print(f"前向概率（第3天）: {alpha[2]}")2.4.2 Viterbi 算法
Viterbi 算法找到最可能的隐藏状态序列。
python
def viterbi(hmm, evidence):
n = len(evidence)
delta = [{} for _ in range(n)]
psi = [{} for _ in range(n)]
for state in hmm.states:
delta[0][state] = hmm.initial[state] * hmm.emission[state][evidence[0]]
psi[0][state] = None
for t in range(1, n):
for state in hmm.states:
max_prob = 0
best_prev = None
for prev_state in hmm.states:
prob = delta[t-1][prev_state] * hmm.transition[prev_state][state]
if prob > max_prob:
max_prob = prob
best_prev = prev_state
delta[t][state] = max_prob * hmm.emission[state][evidence[t]]
psi[t][state] = best_prev
path = [None] * n
path[n-1] = max(delta[n-1], key=delta[n-1].get)
for t in range(n-2, -1, -1):
path[t] = psi[t+1][path[t+1]]
return path
best_path = viterbi(weather_hmm, ['Walk', 'Shop', 'Clean'])
print(f"最可能的天气序列: {best_path}")三、实战案例
3.1 案例一：朴素贝叶斯分类器
python
class NaiveBayesClassifier:
def __init__(self):
self.class_priors = {}
self.feature_likelihoods = {}
def fit(self, X, y):
classes = set(y)
n = len(y)
for c in classes:
self.class_priors[c] = sum(1 for yi in y if yi == c) / n
self.feature_likelihoods[c] = {}
X_c = [X[i] for i in range(n) if y[i] == c]
for j in range(len(X[0])):
values = set(x[j] for x in X_c)
self.feature_likelihoods[c][j] = {}
for v in values:
count = sum(1 for x in X_c if x[j] == v)
self.feature_likelihoods[c][j][v] = (count + 1) / (len(X_c) + len(values))
def predict(self, x):
posteriors = {}
for c in self.class_priors:
log_prob = np.log(self.class_priors[c])
for j, v in enumerate(x):
likelihood = self.feature_likelihoods[c].get(j, {}).get(v, 1e-6)
log_prob += np.log(likelihood)
posteriors[c] = log_prob
return max(posteriors, key=posteriors.get)
# 简单的垃圾邮件分类
X = [
['free', 'money', 'win'],
['hello', 'meeting', 'tomorrow'],
['free', 'click', 'here'],
['project', 'deadline', 'meeting'],
['win', 'prize', 'free'],
['lunch', 'tomorrow', 'meeting'],
]
y = ['spam', 'ham', 'spam', 'ham', 'spam', 'ham']
nb = NaiveBayesClassifier()
nb.fit(X, y)
print(f"分类 'free win prize': {nb.predict(['free', 'win', 'prize'])}")
print(f"分类 'meeting tomorrow': {nb.predict(['meeting', 'tomorrow'])}")四、本章小结
本章介绍了概率推理的核心概念和方法。贝叶斯定理是概率推理的基石，贝叶斯网络利用条件独立紧凑地表示联合分布。精确推理（枚举、变量消除）在变量多时不可行，需要近似推理（采样方法）。HMM 是处理时序数据的重要工具。
核心知识点回顾：
- 贝叶斯定理将先验转化为后验
- 条件独立大大简化概率计算
- 贝叶斯网络利用图结构紧凑表示联合分布
- 精确推理在大规模网络中不可行
- 近似推理通过采样估计概率
- HMM 建模时序隐藏状态
五、思考与练习
思考题
贝叶斯网络的表达力：什么样的概率关系不能用贝叶斯网络表示？
采样方法的收敛性：随着采样次数增加，近似推理的结果是否总能收敛到精确值？
编程练习
实现变量消除：完整实现变量消除算法，比较它与枚举法的效率。
实现 Gibbs 采样：Gibbs 采样是 MCMC 的一种，实现它并进行实验。
HMM 词性标注：用 HMM 实现简单的词性标注系统。