CSYS5030 Sample Exam 整卷逐题解析

Posted on In , Word count in article: 6.6k Reading time ≈ 24 mins.
CSYS5030 官方样卷（Sample Exam）整卷逐题解析：20 道选择题（Shannon 信息量、熵、bits/nats、条件熵、冗余与协同、Guess Who 下界、 误导性信息、Venn 图、Gaussian 与 KSG 估计器对比、box-kernel、AIS 嵌入长度、传递熵、自相关与 dynamic correlation exclusion、 条件 TE、有效网络推断）和 3 道简答题（误导性信息的约束、CMI 恒等式证明、用样例库写代码）。样卷没有官方答案，全部为自拟解答， 与 Week 13 复习笔记的答案一致。

CSYS5030 Sample Exam 整卷逐题解析

课程：CSYS5030 — Information Theory and Self-Organisation（Joseph Lizier） 材料：Exam Prep/CSYS5030-Exam-Sample.pdf（20 页）+ CSYS5030-Exam-SampleCoverSheet.pdf ⚠️ 答案来源：样卷没有官方答案，下面全部是自拟、非官方的解答，按课程内容推出。答案与 Week 13 复习笔记 第三节保持一致，这里补充了每题的详细推导、错误选项分析和对应周次。 形式：答题 1.5 小时 + 阅读 10 分钟，闭卷，可带双语词典和非编程计算器；Section A 20 道选择题 × 1 分（Gradescope 答题卡），Section B 3 道简答题共 20 分。试卷附公式表和一个样例信息论库（见 Q23）。

答案速查（自拟）

Q 答 Q 答 Q 答 Q 答
1 B 6 C 11 B 16 B
2 C 7 A 12 E 17 A
3 A 8 B 13 C 18 C
4 D 9 A 14 E 19 C
5 D 10 D 15 C 20 D

Section B：Q21 (a) ，(b) ≈ 0.78 bits；Q22 证明；Q23 代码。

按周分布：Q1–Q10 基本是 W1–W4（信息量、熵、条件熵、MI、冗余/协同），Q11–Q12 是 W3 的 pointwise MI 与 Venn 图，Q13–Q14 是 W6–W7 的估计器与显著性，Q15–Q20 是 W10–W12 的存储、传递与网络推断；Section B 覆盖 W1–W4 的计算与证明，外加用样例库写代码。

Section A：选择题

Q1 掷一个六面骰子并告诉我结果，给了多少信息？ → B 2.58 bits

Original question (English) · (1) You roll a 6-sided die, and then tell me which of the 6 faces was the result. How much information have you given me? (NB: "die" is the singular of "dice")

  • A 0.77 bits
  • B 2.58 bits
  • C 2 bits
  • D There is not enough information to say

按公平骰子： bits。

  • A 0.77 是 （单位用错）；C 2 bits 是 4 面的情况。
  • D 的争议：严格说信息量取决于分布，题目没说「公平」。如果出题意图是考「信息依赖于概率」，可能选 D。我倾向 B（默认均匀假设），考场上建议按 B。
  • 知识点：W01 Shannon 信息量

English answer (Unofficial, self-drafted): B — Assuming a fair die, each face has p = 1/6, so h = −log₂(1/6) = log₂6 ≈ 2.58 bits. 0.77 is log₁₀6 (wrong base) and 2 bits would be a 4-sided die. (D is arguable because fairness is not stated; the default uniform assumption gives B.)

Q2 ____ 之于熵，就像 pointwise MI 之于 MI → C Shannon 信息量

Original question (English) · (2) ______________ is to Entropy as Pointwise Mutual Information is to Mutual Information.

  • A Conditional Entropy
  • B Redundancy
  • C Shannon Information Content
  • D Transfer Entropy

， ：熵是 Shannon 信息量的平均，MI 是逐点 MI 的平均。A 条件熵、B 冗余、D 传递熵都不是「局部 ↔︎ 平均」关系。

知识点：W01、W03 pointwise MI

English answer (Unofficial, self-drafted): C — Entropy is the average of the Shannon information content, H(X) = ⟨h(x)⟩, just as mutual information is the average of the pointwise (local) mutual information, I(X;Y) = ⟨i(x;y)⟩.

Q3 ____ 之于 bits，就像厘米之于英寸 → A nats

Original question (English) · (3) ______________ are to bits as centimetres are to inches.

  • A nats
  • B redundancies
  • C entropies
  • D shannons

同一个量的不同单位：bits 用 ，nats 用 ，1 nat ≈ 1.443 bits。D「shannons」其实是 bit 的别名，不是另一种单位，所以不是最佳答案。

知识点：W01 单位

English answer (Unofficial, self-drafted): A — Nats and bits are two units for the same quantity (natural log vs log base 2; 1 nat ≈ 1.443 bits). "Shannons" is just another name for bits, so it is not a different unit.

Q4 → D bits

Original question (English) · (4) A random variable X takes values 0 and 1, where p(x = 1) = 0.125. Which of the following is correct?

  • A h(x = 1) = 0.125 bits
  • B h(x = 1) = 1.00 bit
  • C h(x = 1) = 0.90 bits
  • D h(x = 1) = 3.00 bits

。A 把概率当信息量；C 0.90 不知从何来；B 是 时的值。

知识点：W01

English answer (Unofficial, self-drafted): D — h(x = 1) = −log₂0.125 = −log₂2⁻³ = 3 bits. A confuses the probability with the information content; 1 bit would need p = 0.5.

Q5 的熵 → D 1.86 bits

Original question (English) · (5) A random variable X has p(0) = 0.1, p(1) = 0.25, p(2) = 0.4, p(3) = 0.25. Which of the following is the correct Shannon entropy?

  • A H(X) = 0.56 bits
  • B H(X) = 0.40 bits
  • C H(X) = 2.00 bits
  • D H(X) = 1.86 bits

C 2.00 是 4 符号均匀时的最大熵——这个分布不均匀，所以一定 < 2。

知识点：W01 熵

English answer (Unofficial, self-drafted): D — H = 0.1×3.322 + 2×0.25×2 + 0.4×1.322 = 0.332 + 1.000 + 0.529 ≈ 1.86 bits. 2.00 bits is the maximum for 4 symbols, reached only by the uniform distribution.

Q6 字母表 {0,1,2,3}，只知 → C 两者都对

Original question (English) · (6) I have a random variable X whose samples are from the alphabet {0,1,2,3}. I know only that the probability of sampling a 1 is p(x = 1) = 0.5. What am I able to conclude:

  • A H(X) ≥ 1 bit
  • B h(x = 1) = 1 bit
  • C Both H(X) ≥ 1 bit and h(x = 1) = 1 bit
  • D I can't determine any of the above without knowing the precise probabilities for the other symbols.
  • bit ✓（只需要这个符号本身的概率）；
  • 剩下 0.5 的概率无论怎么分，熵最小是全部放在一个符号上： bit。所以 bit ✓。
  • D 错在：两个结论都不需要知道其他符号的精确概率。

知识点：W02 熵的界

English answer (Unofficial, self-drafted): C — h(x = 1) = −log₂0.5 = 1 bit needs only p(x = 1). The entropy is smallest when the remaining 0.5 sits on a single symbol, giving H = 2×(−0.5 log₂0.5) = 1 bit, so H(X) ≥ 1 bit. Neither conclusion needs the other probabilities.

Q7 : (0.7, 0.1, 0.1, 0.1)，: (0.7, 0.3) → A

Original question (English) · (7) X has p(0) = 0.7, p(1) = p(2) = p(3) = 0.1; Y has p(0) = 0.7, p(1) = 0.3. Which of the following is correct:

  • A H(X) > H(Y)
  • B h(x = 0) > h(y = 0)
  • C H(X) > H(Y) and h(x = 0) > h(y = 0)
  • D I(X;Y) > 0 bits
  • bits， bits → ✓；
  • ，相等而不是大于 → B、C 错；
  • 只给了边际分布，无法判断 → D 错。

知识点：W01 熵与信息量

English answer (Unofficial, self-drafted): A — H(X) = 0.7×0.515 + 3×0.1×3.322 ≈ 1.36 bits and H(Y) = 0.360 + 0.3×1.737 ≈ 0.88 bits, so H(X) > H(Y). h(x = 0) = h(y = 0) = −log₂0.7 (equal, not greater), and I(X;Y) cannot be determined from the marginals alone.

Q8 时总是成立的是 → B

Original question (English) · (8) For two random variables X, Y if we find that H(X|Y) = 0 bits then which statement is always true:

  • A There is more entropy in Y than X
  • B If you tell me the sample of Y then there is no uncertainty left about X
  • C X and Y have non-zero mutual information
  • D X and Y must provide synergistic information about another variable Z

条件熵为 0 ⟺ 知道 后 没有剩余不确定性（ 是 的函数）。

  • A 不一定： 是 的函数时 ，但可以相等；
  • C 是陷阱：若 （ 是常数）， 。Week 8 讲 Calculation exercise 时专门强调过；
  • D 无从推出。

知识点：W02 条件熵、W08 答疑

English answer (Unofficial, self-drafted): B — H(X|Y) = 0 means X is a function of Y: once Y is known, no uncertainty about X remains. A is not always true (H(X) ≤ H(Y), possibly equal); C fails if X is constant (then I(X;Y) = 0); D does not follow.

Q9 两个传感器 、 都是真实温度 的近乎完美拷贝 → A 大部分信息冗余

Original question (English) · (9) Suppose Y and Z are two sensors measuring the true temperature X, and both sensors are nearly perfect copies of X. Which of the following best describes the information provided by Y and Z about X?

  • A Most of the information is redundant.
  • B Most of the information is synergistic.
  • C The joint variable (Y, Z) provides less information than either Y or Z alone
  • D Y and Z must provide independent information about X

两个传感器提供的是同一份信息。C 错：联合变量的信息不会少于单个变量（ ）；D 错：恰好相反，不是独立的；B 协同是 XOR 型关系。

知识点：W04 冗余与协同

English answer (Unofficial, self-drafted): A — Both sensors carry the same information about X, so it is mostly redundant. Joint information can never be less than either alone (I(X;Y,Z) ≥ I(X;Y)), and synergy is the XOR-type case.

Q10 Guess Who 已缩小到 16 个，最优是/否问题数下界 → D 4

Original question (English) · (10) In the board game Guess Who?, I try to guess my opponent's character out of 24 possibilities. I use Yes or No questions to eliminate possibilities until I can identify their character. I have already narrowed down the opponent's identity to 16 possibilities. If I were able to optimally design my further questions, what does information theory tell me is the lower bound on the number of such further questions I would need on average to identify my opponent?

  • A 2
  • B 6
  • C 3
  • D 4
  • E Information theory is not relevant to this

均匀情况下剩余不确定性 bits，每个是/否问题最多给 1 bit。A 2、C 3 低于下界；B 6 是不优化的问法；E 错，这正是信息论的经典应用。

知识点：W01 熵与最优提问

English answer (Unofficial, self-drafted): D — With 16 equally likely candidates the remaining uncertainty is log₂16 = 4 bits, and each yes/no question gives at most 1 bit, so at least 4 questions are needed on average.

Q11 哪个是误导性信息 → B

Original question (English) · (11) Given p(x=0,y=0) = 0.5, p(x=0,y=1) = 0.1, p(x=1,y=0) = 0.1, p(x=1,y=1) = 0.3, which of the following would be an example of misinformation:

  • A Observing x = 0 is misinformative about y when observing y = 0
  • B Observing x = 0 is misinformative about y when observing y = 1
  • C Observing x = 1 is misinformative about y when observing y = 1
  • D None of these situations

表： ， ， ， ；边际 ， ， ， 。

  • A： ，信息性；
  • B： ，误导性 ✓；
  • C： ，信息性。

知识点：W03 pointwise MI 与误导性信息

English answer (Unofficial, self-drafted): B — Misinformation means negative pointwise MI, i(x;y) = log₂[p(x,y)/(p(x)p(y))] < 0. Marginals are p(x=0) = 0.6, p(y=1) = 0.4, so i(x=0; y=1) = log₂(0.1/0.24) ≈ −1.26 bits < 0. The other cases are positive: A gives log₂(0.5/0.36) ≈ +0.47 bits, C gives log₂(0.3/0.16) ≈ +0.91 bits.

Q12 Venn 图总面积 α+β+γ → E（B 和 C）

Original question (English) · (12) In a Venn diagram of H(X) and H(Y) with regions α, β, γ (β being the overlap), how much entropy is represented by the total area α + β + γ?

  • A I(X;Y)
  • B H(X,Y)
  • C H(X) + H(Y) − I(X;Y)
  • D Both A and B
  • E Both B and C

两个圆的并集 = 联合熵 （重叠部分 β 只算一次）。A 只是中间的 β。

知识点：W03 信息量 Venn 图

English answer (Unofficial, self-drafted): E — The union of the two circles is the joint entropy H(X,Y) = H(X) + H(Y) − I(X;Y) (the overlap β is counted once). I(X;Y) is only the overlap β.

Q13 Gaussian MI ≈ 0（p = 0.28），KSG MI = 0.1347 nats（p = 0） → C 有非线性关系，检测不到线性关系

Original question (English) · (13) A researcher uses JIDT to compute I(X;Y) for 1200 samples of two continuous-valued variables with two estimators. MI_Gaussian = 0.0010 nats (null: 0.0006 ± 0.0008 std dev.; p(surrogate > measured) = 0.28 from 100 surrogates). MI_Kraskov (KSG) alg. 2 = 0.1347 nats (null: 0.0015 ± 0.0211 std dev.; p(surrogate > measured) = 0.00 from 100 surrogates). What conclusion should the researcher draw here?

  • A There is no relationship detectable between X and Y in this data set
  • B There is a linear but no non-linear relationship detectable between X and Y
  • C There is a non-linear but no linear relationship detectable between X and Y
  • D Not enough surrogates were used to draw a satisfactory conclusion
  • E The difference between the estimators suggests an error in the code
  • Gaussian 估计器只能捕获线性关系，结果不显著 → 检测不到线性关系；
  • KSG 能捕获非线性关系，结果高度显著 → 存在关系。
  • A 忽略了 KSG；B 正好反了；D 对 KSG 来说 0/100 已经足够拒绝；E 两个估计器问的是不同的问题，差异正常。
  • 这就是 Week 6–7 心跳-呼吸数据的情形。

知识点：W06 估计器、W07 显著性

English answer (Unofficial, self-drafted): C — The Gaussian estimator only captures linear dependence and is not significant (p = 0.28), while the KSG estimator captures non-linear dependence and is highly significant (p = 0 from 100 surrogates). So a non-linear relationship is present without a detectable linear one; different estimators answering different questions is expected, not a bug.

Q14 box-kernel 相对 KSG 的优势 → E 都不是

Original question (English) · (14) Which of the following is an advantage of using the box-kernel estimator for mutual information over the Kraskov (KSG) estimator:

  • A It can detect non-linear relationships
  • B You can measure the statistical significance of the result
  • C It can handle multi-dimensional variables
  • D All of the above
  • E None of the above

两者都能测非线性、都能做显著性检验（surrogate）、都能处理多维变量，所以 A–D 都不是 box-kernel 独有的优势。KSG 通常还更好：自适应宽度、偏差更小。

知识点：W06 Kernel 与 KSG

English answer (Unofficial, self-drafted): E — Both estimators detect non-linear relationships, both support surrogate significance tests and both handle multi-dimensional variables, so none of A–C is an advantage of the box kernel. KSG is usually preferred (adaptive kernel width, lower bias).

Q15 周期 4 的 0-0-1-1，捕获全部过去影响的最小 k → C 2

Original question (English) · (15) Time-series process Xₜ is discrete-valued with period 4, continually repeating 0-0-1-1. What value should the embedding length k be set to, at minimum, to capture all of the past influence when measuring the active information storage?

  • A 0
  • B 1
  • C 2
  • D 4
  • k = 1：「0 之后」可能是 0 也可能是 1，不确定；
  • k = 2：00→1、01→1、11→0、10→0，完全确定。
  • A、B 不够；D 4 能做到但不是最小。Week 10 合成序列 (b) 就是这个：k = 1 时 AIS ≈ 0，k = 2 时 1 bit。

知识点：W10 嵌入长度 k

English answer (Unofficial, self-drafted): C — With k = 1, a past value of 0 can be followed by 0 or 1, so the next value is uncertain. With k = 2, every pair determines the next value (00→1, 01→1, 11→0, 10→0). k = 4 also works but is not the minimum.

Q16 TE 相对 MI 的关键优势 → B 识别有向的预测性影响（对状态转移）

Original question (English) · (16) Which statement best describes a key advantage of transfer entropy over mutual information when analysing time series?

  • A Transfer entropy can be negative while mutual information cannot.
  • B Transfer entropy can identify directed predictive influence of the past of one process on the state transition of another.
  • C Transfer entropy requires fewer samples to estimate than mutual information.
  • D Transfer entropy can only be applied to continuous-valued variables in addition to discrete-valued variables.
  • A 错：平均 TE 和平均 MI 都非负，局部值都可以为负；
  • C 错：TE 条件于目标过去，维度更高，需要更多样本；
  • D 不成立：离散变量也能用。

知识点：W11 传递熵

English answer (Unofficial, self-drafted): B — TE conditions on the target's past, so it measures directed information from the source's past to the target's next state transition. Average TE and MI are both non-negative (local values of both can be negative), TE needs more samples (higher dimension), and it works for discrete variables too.

Q17 图中 TE（源嵌入 1、目标嵌入 2） → A

Original question (English) · (17) A diagram shows time series Xₜ and Yₜ with labelled variables A, B and C (A = x_{n+1}, B = {x_{n−1}, x_n}, C = y_n). Which of the following is the transfer entropy from Y to X, with source and target embedding lengths 1 and 2 respectively?

  • A I(C;A|B)
  • B I(A;B)
  • C H(A|B,C)
  • D I(B;A|C)
  • E I(B;A)

图（页面图像）： 列中框住 ，标 C； 列中框住 ，标 B； 标 A。

  • D 把源和目标过去的位置弄反了（那是条件于源的「目标过去」MI）；
  • E 是 AIS（），B 同理；C 是条件熵，不是信息量。

知识点：W11 TE 定义

English answer (Unofficial, self-drafted): A — T_{Y→X}(k = 2, l = 1) = I(y_n; x_{n+1} | x_{n−1}, x_n) = I(C; A | B): source past C, target next value A, conditioned on the target past B. I(B;A) would be the active information storage; H(A|B,C) is a conditional entropy, not an information transfer.

Q18 自相关图 → C 两者都对

Original question (English) · (18) A researcher intends to measure the Active Information Storage (AIS) for a time series using the Kraskov (KSG) estimator. The autocorrelation plot (lags up to 20) shows a very strong lag-1 autocorrelation (≈ 0.95) decaying smoothly and staying above the significance threshold until about lag 15. Which statement is most correct about what the researcher can conclude about measuring the AIS:

  • A Dynamic correlation exclusion should be used
  • B The researcher should expect to measure a statistically significant AIS
  • C Both A and B
  • D Neither A nor B

图（页面图像）：滞后 1 处约 0.95，平滑单调衰减，直到约滞后 15 都超出显著性带，之后才落入带内（滞后 20 附近略负）。

  • 强而长的自相关 ⟹ KSG 的近邻会被时间上相邻的样本污染，应使用 dynamic correlation exclusion（Theiler window ≈ 15）→ A ✓；
  • 强自相关说明过去对下一个值有很强的（至少线性的）预测力 ⟹ 应预期显著的 AIS → B ✓。

知识点：W07 自相关与 DYN_CORR_EXCL、W10 AIS

English answer (Unofficial, self-drafted): C — Strong, long-lasting autocorrelation means KSG nearest neighbours would be contaminated by temporally adjacent samples, so dynamic correlation exclusion (a Theiler window of about 15) should be used. It also shows the past strongly predicts the next value, so a significant AIS is expected.

Q19 测 还是 → C 两者都有用

Original question (English) · (19) A researcher has time-series samples for three processes X, Y and Z and wishes to understand the statistical dependence of X on Y. Should they measure the pairwise transfer entropy T_{Y→X} or the conditional transfer entropy T_{Y→X|Z}? Which of the following is the most correct advice:

  • A T_{Y→X|Z} is more correct since it removes redundancies in comparison to the pairwise T_{Y→X}
  • B T_{Y→X} is more correct since it will not include higher order synergies with Z in comparison to T_{Y→X|Z}
  • C Both measures are useful to build a complete understanding of the statistical dependence of X on Y.
  • D Neither measure is useful since they do not directly assess causal effect

条件化既去掉冗余，又引入协同。在「信息回归」中二者按不同顺序出现，相加都等于集体 TE。

  • A、B 各说对一半；D 错：统计依赖本身就是要研究的对象，不必是因果。

知识点：W11 条件 TE

English answer (Unofficial, self-drafted): C — Conditioning on Z both removes redundancy and adds synergy, so each measure shows a different part of the picture; together they give a complete view. Statistical dependence is worth studying even though it is not a direct causal effect.

Q20 有效网络推断的定义 → D

Original question (English) · (20) Define the concept of effective network inference from multivariate time-series (with each variable represented as a node in the network):

  • A Constructing undirected networks to represent pairwise statistical relationships between nodes
  • B Constructing directed networks to represent the physical, directed (causal) connections between nodes
  • C Constructing directed networks to represent pairwise statistical relationships between nodes
  • D Constructing directed networks to provide a minimal, multivariate model which can replicate and explain the time-series updates of each node

构建有向网络，作为能复现并解释每个节点时间序列更新的最小多变量模型。

  • A 是功能网络（无向）；B 是结构网络（物理/因果连接）；C 是有向功能网络（成对统计关系）。

知识点：W12 有效网络推断

English answer (Unofficial, self-drafted): D — Effective networks are directed, minimal multivariate models that can replicate and explain each node's dynamics. A is a functional (undirected) network, B a structural network, C a directed functional network.

Section B：简答题

Q21（8 分）Guess Who 不完整概率表

Original question (English) · (21) (8 marks) We have the following incomplete probability tables for two character features X and Y in a "Guess who?" game with 24 characters: p(x = 0) = 19/24, p(x = 1) = 5/24; p(y) unknown; joint table p(x = 0, y = 1) = α/24, p(x = 1, y = 1) = 1/8, the y = 0 column unknown.

  1. What are the constraints on the number of characters α who have both features X and Y (i.e. p(x = 0, y = 1) = α/24) such that observing y = 1 is misinformative about the observation x = 1? (6 marks)
  2. If 5 characters have feature Y but do not have feature X (i.e. x = 0, y = 1 for these 5 characters), how much information have you gained jointly about a character's identity after finding out this character has features x = 0, y = 0? (2 marks)

24 个角色。已知 ， ；联合表中 ， ，其余为「?」。

(a)（6 分）使得「观测 对 是误导性的」的 α 的约束

误导性 ⟺ ⟺ 。

α 是人数（整数）⟹ 。另一侧由概率表合法性给出： ⟹ 。

得分点：写出误导性的定义（pointwise MI < 0）、求出 、解不等式、取整、别漏了上界。这是 Calculation exercise Q2c 换了个外壳。

(b)（2 分）若 α = 5，得知角色 ，共获得多少信息？

，联合事件的 Shannon 信息量：

别忘了写单位。

知识点：W03 pointwise MI、W01 Shannon 信息量

English answer (Unofficial, self-drafted) — model answer

  1. Observing y = 1 is misinformative about x = 1 iff the pointwise MI is negative: i(x=1; y=1) = log₂[p(x=1|y=1)/p(x=1)] < 0, i.e. p(x=1|y=1) < p(x=1). p(y=1) = α/24 + 3/24 = (3 + α)/24, so p(x=1|y=1) = (3/24)/((3+α)/24) = 3/(3 + α). Condition: 3/(3 + α) < 5/24 ⟺ 72 < 15 + 5α ⟺ α > 11.4. Since α counts characters it is an integer, so α ≥ 12. Upper bound from a valid table: p(x=0, y=0) = (19 − α)/24 ≥ 0 ⟹ α ≤ 19. Answer: 12 ≤ α ≤ 19.

  2. With α = 5: p(x=0, y=0) = (19 − 5)/24 = 14/24. The joint information content is h(x=0, y=0) = −log₂(14/24) = log₂(12/7) ≈ 0.78 bits.

Q22（5 分）证明

Original question (English) · (22) (5 marks) Beginning with the conditional mutual information expression I(X;Y|Z) = H(X|Z) + H(Y|Z) − H(X,Y|Z), use the form of the conditional entropy H(X|Y) = H(X,Y) − H(Y), to prove that I(X;Y|Z) = H(X|Z) − H(X|Y,Z).

从题给的 出发，用 展开每个条件熵：

另一边，把 当作一个联合变量：

两式相同，得证。∎

关键一步： ，即条件熵公式对联合条件变量同样成立。

知识点：W04 条件互信息、W02 条件熵

English answer (Unofficial, self-drafted) — model answer

Expand each conditional entropy with H(A|B) = H(A,B) − H(B): I(X;Y|Z) = [H(X,Z) − H(Z)] + [H(Y,Z) − H(Z)] − [H(X,Y,Z) − H(Z)] = H(X,Z) − H(Z) + H(Y,Z) − H(X,Y,Z). Now treat (Y,Z) as one joint variable on the right-hand side: H(X|Z) − H(X|Y,Z) = [H(X,Z) − H(Z)] − [H(X,Y,Z) − H(Y,Z)] = H(X,Z) − H(Z) + H(Y,Z) − H(X,Y,Z). The two expressions are identical, so I(X;Y|Z) = H(X|Z) − H(X|Y,Z). ∎ (Key step: the conditional-entropy identity also holds when conditioning on a joint variable, H(X|Y,Z) = H(X,Y,Z) − H(Y,Z).)

Q23（7 分）多人游戏数据的代码

Original question (English) · (23) (7 marks) A researcher wants to analyse the information properties in a multivariate time series. The samples are supplied in the variable data, a two-dimensional array. Each column corresponds to samples of a random variable representing the moves of one player in a sequence of multiplayer games ({V₁, V₂, V₃, …}); moves take discrete integer values; rows represent the moves of all players in one game. Write code (Python or Matlab) using the "Sample information-theoretic software library" from the formula sheet; if you do not know the precise syntax, describe it in comments for partial marks.

  1. For each individual player, compute and print the uncertainty about their moves. (2 marks)
  2. How much uncertainty do we have jointly about the moves of players 1, 2 and 3 (V₁, V₂, V₃)? (2 marks)
  3. Identify which other player's next move that player 1's move (V₁) in the previous game gives the most information about (on average)? (3 marks)

data 是二维数组：行 = 一局游戏，列 = 玩家 的出招（离散整数）。只能用公式表上的样例库：entropy_empirical、conditionalentropy_empirical、jointentropy_empirical、mutualinformation_empirical、conditionalmutualinformation_empirical。

# a) 每个玩家出招的不确定性（熵）
numPlayers = data.shape[1]
for i in range(numPlayers):
    print("H(V%d) = %.4f bits" % (i + 1, entropy_empirical(data[:, i])))

# b) 玩家 1、2、3 的联合不确定性：把三列作为一个多维变量
H123 = entropy_empirical(data[:, 0:3])        # 二维数组 = 多维变量
print("H(V1,V2,V3) = %.4f bits" % H123)
# 也可以用 jointentropy_empirical(data[:, 0], data[:, 1:3])

# c) V1 在上一局的出招，对哪个其他玩家下一局出招的信息最多
bestPlayer, bestMI = -1, -1.0
for j in range(1, numPlayers):                # 排除玩家 1 自己
    # 源去掉最后一行，目标去掉第一行 → 对齐第 n 局与第 n+1 局
    mi = mutualinformation_empirical(data[:-1, 0], data[1:, j])
    if mi > bestMI:
        bestPlayer, bestMI = j, mi
print("Player V%d: I = %.4f bits" % (bestPlayer + 1, bestMI))

得分点：

  • a)（2 分）循环每一列 + entropy_empirical + 打印；
  • b)（2 分）联合熵用多维变量（把三列作为一个二维数组传入），而不是三个熵相加（只有独立时才相等）；
  • c)（3 分）时滞对齐（[:-1] 对 [1:]）+ 对每个其他玩家算 MI + 取最大 + 排除自己。

老师在 Week 8 说过：代码题看逻辑流程，语法小错不扣分；不会写某个语法可以用注释描述拿部分分。

知识点：W05 软件与编程、W03 互信息

English answer (Unofficial, self-drafted) — model answer

The code above is the answer. In words, for the paper: - (a) Loop over every column of data and print entropy_empirical(data[:, i]) (the entropy of each player's moves, in bits). - (b) Pass the three columns together as one multi-dimensional variable: entropy_empirical(data[:, 0:3]) (equivalently jointentropy_empirical(data[:, 0], data[:, 1:3])). Do not add three separate entropies; that only equals the joint entropy if the players are independent. - (c) Align time by one game: source data[:-1, 0] (V₁ in game n) and target data[1:, j] (player j in game n + 1). Compute mutualinformation_empirical for every other player j ≠ 1, keep the maximum, and report that player.

答题提醒

  1. 单位必须写（bits / nats）；JIDT 中 KSG / Gaussian 默认 nats，Discrete 默认 bits。
  2. 看清题目问的是变量（平均量 、）还是取值（局部量 、）。
  3. 约束题要同时给出两侧的界（另一侧通常来自概率非负）。
  4. 时滞量（滞后 MI、TE、AIS）在代码里一定要对齐样本。
  5. 更完整的考点清单与常见陷阱见 Week 13 复习笔记。