（可視化模型表現）Visualizing Model Performance

• Stakeholders outside of the data science team may have little patience for details, and will often want a higher-level, more intuitive view of model performance.

   

•  It is important for the data scientist to realize that clear communication with key stakeholders is not only a primary goal of her job, but also is essential for doing the right modeling (in addition to doing the modeling right).

   

  ————————————————————————————————————————————————

   

  主要內容：

  1. Profit curves
  2. ROC  graph （receiver operating characteristic）
     • AUC graph （The area under the ROC Curve）
  3. Cumulative response and lift curve

   

  ————————————————————————————————————————————————

   

  （一）Profit curve

   

   

  The disadvantage of a profit graph is that it requires that operating conditions be known and specified exactly. With many real-world problems, the operating conditions are imprecise or change over time, and the data scientist must contend with uncertainty.

   

   

  （二）ROC graph

   

  在先驗分佈和損益不清楚的情況下，我們可以依據不同情況依次建立模型繪製Profit graph，但這顯然不能讓人滿意

   

  爲解決這個問題，我們可以將各種情況的模型表現都繪製在一張圖中，其中一種方法便是 Receiver Operating Characteristics (ROC) graph

   

   

   

  ROC graph 的詳細產生過程見下圖

   

   

  ROC數據點的計算方式

   

  Confusion Matrix

  	P	N
  Y	a	b
  N	c	d

   

  ROC  Point

   

   

  An advantage of ROC graphs is that they decouple classifier performance from the conditions under which the classifiers will be used. Specifically, they are independent of the class proportions as well as the costs and benefits.

   

   

  The area under the RCO  curve （AUC）

  • Though a ROC curve provides more information than its area, the AUC is useful when a single number is needed to summarize performance, or when nothing is known about the operating conditions.

   

   

  （三）Cumulative response and lift curve