什麼是辛普森悖論？

{"type":"doc","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"以一種方式查看數據可以講述一個故事，但是有時以另一種方式查看數據可以講述相反的故事。瞭解這一悖論及其發生的原因至關重要，並且可以使用新工具來自動檢測數據集中的這一棘手問題。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"當我們想研究數據中的關係時，我們可以繪製，交叉製表或對該數據建模。當我們這樣做時，我們可能會遇到這樣的情況：從單個數據集的兩個不同視圖中看到的關係導致我們得出相反的結論。這些都是辛普森悖論的案例。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"找到這些案例可以幫助我們更好地理解我們的數據並發現有趣的關係。本文提供了這些情況發生的位置的一些示例，討論了它們如何發生以及爲什麼發生，並提出了在您自己的數據中自動檢測這些情況的方法。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"什麼是辛普森悖論？"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"辛普森悖論是指一種情況，您認爲您瞭解兩個變量之間的關係方向，但是當您考慮其他變量時，該方向似乎相反。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"爲什麼發生辛普森悖論？"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"之所以發生Simpson悖論，是因爲數據的分解（例如，將其拆分爲子組）可能導致某些子組與其他子組相比具有不平衡的表示形式。這可能是由於變量之間的關係，或者僅僅是由於數據已被劃分爲子組的方式。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"範例1：入場"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"辛普森悖論的一個著名的例子出現在加州大學伯克利分校錄取數據。在此示例中，從"},{"type":"text","marks":[{"type":"strong"}],"text":"總體"},{"type":"text","text":"上看研究生錄取數據時"},{"type":"text","marks":[{"type":"strong"}],"text":"，"},{"type":"text","text":"看來男人比女人更容易被錄取（性別歧視！），但是當"},{"type":"text","marks":[{"type":"strong"}],"text":"單獨查看每個部門"},{"type":"text","text":"的數據時"},{"type":"text","marks":[{"type":"strong"}],"text":"，"},{"type":"text","text":"男人比女人更容易被錄取。大多數部門中的女性。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/e8/e82786a38d28bf1d4aaf742fcae0566e.jpeg","alt":null,"title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"這是爲什麼發生這種情況的說明："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"numberedlist","attrs":{"start":null,"normalizeStart":1},"content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":1,"align":null,"origin":null},"content":[{"type":"text","text":"不同部門的接受率非常不同（有些部門比其他部門“難”得多）"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":2,"align":null,"origin":null},"content":[{"type":"text","text":"更多女性申請“更難”的部門"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":3,"align":null,"origin":null},"content":[{"type":"text","text":"因此，女性的總體接受率較低"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"這導致我們問：哪個視圖是正確的視圖？男人或女人的接受率更高？這所大學的招生是否存在性別偏見？"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在這種情況下，似乎最合理的結論是，按部門查看招生率更爲合理，而分類的觀點是正確的。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"範例2：棒球"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"辛普森悖論的另一個例子可以在兩個著名棒球運動員，杰特和大衛。1995年和1996年，David Justice分別擁有較高的打擊率，但兩年來，Derek Jeter的打擊率均較高。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/d3/d3574aac2d544fa5b4bb80bc0254b1c6.jpeg","alt":null,"title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"numberedlist","attrs":{"start":null,"normalizeStart":1},"content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":1,"align":null,"origin":null},"content":[{"type":"text","text":"兩位選手的平均命中率均比1995年高"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":2,"align":null,"origin":null},"content":[{"type":"text","text":"1996年，德里克·杰特（Derek Jeter）的蝙蝠數量明顯增加。1995年，戴維·正義案（David Justice）的人數更多"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":3,"align":null,"origin":null},"content":[{"type":"text","text":"因此，Derek Jeter的平均擊球率更高"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/dd/dd3956f3849f127cfad2b642a0bf9d28.jpeg","alt":null,"title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"Knowledge Studio決策樹顯示了1995年和1996年每個球員的擊球次數不平衡"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"辛普森悖論怎麼辦"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"沒有足夠的領域知識，就很難知道對兩個變量之間的關係的哪個觀點更有意義-一個變量是否包含第三個變量。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"但是在考慮如何處理辛普森悖論之前，我們需要找到一種在數據集中有效檢測它的方法。如前所述，只需分解列聯表或數據點圖並研究結果，便可以找到辛普森悖論（“辛普森對”）的實例。但是，還有其他方法可以使用模型找到辛普森對，例如："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"numberedlist","attrs":{"start":null,"normalizeStart":1},"content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":1,"align":null,"origin":null},"content":[{"type":"text","text":"通過建立決策樹並比較分佈，或者"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":2,"align":null,"origin":null},"content":[{"type":"text","text":"通過建立迴歸模型並比較係數的符號"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"兩者都有好處，但是，這會很快變得困難，尤其是在處理大型數據集時。很難知道數據集中哪些變量可能會顛倒另外兩個變量之間的關係，並且很難手動檢查所有可能的變量對。假設我們有一個只有20個變量的數據集：我們需要檢查將近400對，以確保找到所有Simpson's Paradox的情況。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"即使我們已經搜索（並找到）所有可能的辛普森配對，也要考慮其他挑戰。這些挑戰與解釋有關，例如："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"趨勢是否需要在每個子組中反轉才能將某物視爲“辛普森配對”？還是大多數亞組足夠？"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"子組的大小重要嗎？如果趨勢在許多小的子組中發生了逆轉，而在最大的子組中卻沒有，該怎麼辦？"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"當試圖自動檢測辛普森悖論時，這些最後的挑戰不會消失，但是通過被迫提前做出決定，我們至少可以以系統且一致的方式處理它們。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"結論"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"辛普森悖論是一個棘手的問題，但是出色的分析師或數據科學家可以使用正確的工具和知識來處理它。我希望我的文章可以幫助其他人以更輕鬆地瞭解此問題。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}}]}