什么是机器学习？从3个视角谈起：学习任务、学习范式、学习模型

Three Perspectives of Machine Learning 机器学习的三个视角

Perspectives	Description 描述	典型
Learning Tasks 学习任务	Denoting the general problems that can be solved by machine learning. 表示可以用机器学习解决的基本问题。	分类、聚类、回归、排名、密度估计、降维、优化
Learning Paradigms 学习范式	Denoting the typical scenarios that are happened in machine learning. 表示机器学习中发生的典型场景。	有监督学习、无监督学习、强化学习
Learning Models 学习模型	Denoting the approaches that can handle to fulfil a learning task. 表示可以处理完成一个学习任务的方法。	几何、逻辑、网络、概率

1 What is Machine Learning 什么是机器学习

• Machine learning is a branch of artificial intelligence, is the key to intelligence. 机器学习是人工智能的一个分支，是实现智能化的关键。深度学习是机器学习的一个研究领域。
  Its goal is to construct the systems that can learn from data and make predictions on data. 其目标是要构建可以从数据中学习、并对数据进行预测的系统。
• Wikipedia:
  Machine learning is the study of algorithms and mathematical models that computer systems use to progressively improve their performance on a specific task. 机器学习是计算机系统用来逐步提高其在特定任务上的性能的算法和数学模型的研究。
  Machine learning algorithms build a mathematical model of sample data, known as “training data”, in order to make predictions or decisions without being explicitly programmed to perform the task.机器学习算法建立样本数据的数学模型，称为“训练数据”，以便进行预测或决策而无需明确地编程以执行任务。
• Baidu encyclopedia
  Machine learning is a multi-domain interdisciplinary subject, involving probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and so on. 机器学习是一门多领域交叉学科，涉及概率论、统计学、逼近论、凸分析、算法复杂度理论等多门学科
  It focuses on how computers simulate or implement human learning behaviors in order to acquire new knowledge or skills and reorganize existing knowledge structures to improve their performance. 专门研究计算机怎样模拟或实现人类的学习行为，以获取新知识或技能，重新组织已有的知识结构使之不断改善自身的性能。

Relations to Other Disciplines 与其他学科的关系

Relationship between ML and other Fields 机器学习与各领域之间的关系

Feild	Relationship
Pattern Recognition 模式识别	PR ≈ ML. PR from industry, ML from CS. 模式识别≈机器学习。一个源自工业界，一个源自计算机学科。
Computer Vision 计算机视觉	CV=ML+IP. Image Processing provides input for the ML model, ML provide learning algorithms and output the vision result. 计算机视觉=机器学习+图像处理。图像处理负责给机器学习模型提供输入，机器学习负责学习并给出视觉结果。
Data Mining 数据挖掘	DM=ML+DB. Most DM algorithm is the optimization of ML algorithms in DB. 数据挖掘=机器学习+数据库。大部分数据挖掘算法，是机器学习算法在数据库中的优化。
Natural Language Processing (NLP) 自然语言处理	NLP=Text Processing+ML. 自然语言处理=文本处理+机器学习。
Statistic Learning 统计学习	SL is a part of ML. SL focus on mathematical theory research，ML focus on practice. 统计学习是机器学习的一种方法。统计学习偏数学理论研究 ，机器学习偏实践。
Speech Recognition 语音识别	SR=Speech Processing+ML. SR provides input for the ML model. SR and NLP are usually combined to be used. 语音识别=语音处理+机器学习。语音识别负责给机器学习模型提供输入，一般语音识别和自然语言处理技术结合使用。

数据挖掘、机器学习和统计学习的关系

统计学主要是通过机器学习来对数据挖掘发挥影响，而机器学习和数据库则是数据挖掘的两大支撑技术。

• 统计学习：是其它两门技术的基础，更偏重于理论上的完善；
• 机器学习：是统计学习对实践技术的延伸，更偏重于解决小数据量的问题提供算法技术支撑；
• 数据挖掘：更偏重于大数据的实际问题，更注重实际问题的解决，包括真实数据的数据清洗，建模，预测，等操作。
• 机器学习可以分为以支持向量机为代表的统计学习和人工神经网络为代表的联结主义学习。
• 统计学习模型参数往往是可解释的，而人工神经网络就是一个黑箱。

Artificial Intelligence vs. Machine Learning 人工智能与机器学习

• Human Learning 人类学习
  Human being acquire skill with experience accumulated from observations.人类是从观察中积累经验来获取技能。
• Machine Learning 机器学习
  Machine acquire skill with experience accumulated / computed from **data.**机器是从数据中积累或者计算的经验中获取技能。
  

What is Skill in Machine Learning 什么是机器学习的技能

• Skill 技能
  Skill is used to improve some performance measure. (e.g. prediction accuracy)
  技能用于改善某些性能指标。(如:预测精度)
• Why Use Machine Learning to acquire skill 为什么使用机器学习
  Machine learning can improve some performance measure with experience computed from data. 机器学习可以通过从数据中学到的经验来改善某些性能指标。

Three Key Elements in the Formal Definition 形式化定义的三要素

To have a well-defined learning problem, we must identity those three features:要得到一个明确定义的学习问题，我们必须识别如下三个特性：

(1) Determine the task and collect training data.
(2) Obtain the experience from the data
(3) Give the result according to the experience and evaluate the performance of result.

• Example1: A handwriting recognition problem 手写识别问题
  
• Example2: A robot driving problem 机器人驾驶问题
  Task (T):
  driving on public four-lane highways using vision sensors
  使用视觉传感器在公共四车道高速公路上驾驶
  Experience (E):
  a sequence of images and steering commands recorded while observing a human driver 观察人类驾驶员时所记录的一系列图像和操纵命令。
  Performance §:
  average distance traveled before an error (as judged by human overseer). 出错之前行驶的平均距离（由人类督察评判）

2 History of Machine Learning

Timeline of Machine Learning 机器学习的大事年表

Three Parties of Machine Learning 机器学习的三个学派

3 Why Different Perspectives

Difficulty in Understanding Machine Learning 理解机器学习的难点

• How many learning algorithms 有多少种算法
  There are many algorithms for machine learning.有很多机器学习的算法。
  Literally thousands available, and hundreds more published each year.
  大概有数千种，每年又会发表数百种。
• Which algorithm should we choose 应该选择哪种算法
  Suppose we have an application that machine learning might be good for, so we need an appropriate algorithm for learning from data.
  假设我们有一个应用，使用机器学习算法会有帮助，因此需要一个合适的算法用以从数据中进行学习。
  The problem we faced is how to choose one of machine learning
  algorithms.我们面临的问题是如何选择一个机器学习算法。
• What is the difficulty 难点是什么
  Without a category of machine learning, how to determine which
  algorithm could be used?
  若没有机器学习算法的分类法，如何确定应该使用哪种学习算法？
  The categorization relates our perspective on choosing machine leaning.
  这种学习算法的类别关系到我们选择机器学习算法的视角。
• Is one perspective enough 一个视角够吗
  To outlook on most of machine learning algorithms, one perspective is so hard.
  要了解大多数机器学习算法，仅有一个视角是不够的。
  We should look from multiple perspectives to have a full view of machine learning.
  我们应该从多个视角来观察，使之对机器学习有一个完整的理解。

4 Three Perspectives on Machine Learning

（1）What are Learning Tasks 什么是学习任务

• What are Learning Tasks 什么是学习任务
  The learning tasks are used to denote the general problems that can be solved by learning with desired output.
  学习任务用于表示可以用机器学习解决的基本问题。

• Why do we need to Study Learning Tasks 为什么要研究学习任务
  Various types of problems arising in applications:应用中会产生各种类型的问题：
   computer vision, 计算机视觉，
   pattern recognition, 模式识别，
   natural language processing, 自然语言处理，
   ………

• Typical Typical Tasks in Machine Learning 机器学习中的典型任务
  

• Case study: Credit scoring 信用评分
  Two classes: Low-risk and high-risk customers. 二分类：低风险和高风险客户。
  A customer information makes up the input to one of the two classes. 客户信息使该输入构成二分类中的一个。
  After training with past data, a classification rule learned may be: 用过去的数据训练之后，可以学习得到如下分类规则:
  
  

(2) Learning Paradigms 学习范式

• What are Learning Paradigms 什么是学习范式
  The Learning Paradigms are used to denote the typical scenarios that are happened in machine learning.
  学习范式用于表示机器学习中发生的典型场景。
• How to Distinguish Learning Paradigms 怎样区分学习范式
  by the scenarios or styles in machine learning about 根据机器学习的典型场景或样式：
   how it learns from data, 它怎样从数据中学习，
   how it interactives with environment.它如何同环境互动。
• Learning Paradigms in Machine Learning 机器学习中的学习范式
  

(3) Learning Models 学习模型

• What are Learning Models 什么是学习模型
  The learning models are used to denote the approaches that can fulfil a learning task.学习模型用于表示可以完成一个学习任务的方法。
• Why Study Learning Models 为什么要研究学习模型
  The result of machine learning is heavily dependent on the choice of an approach for solving the learning task. 机器学习的效果在很大程度上取决于解决该学习任务时所选用的方法.
• Typical Models for Machine Learning 机器学习的代表性模型
  The Three Perspectives 三个视角
  

5 Applications and Terminologies

(1) Application Fields of Machine Learning 机器学习的应用领域

(2) Some Terminologies in Machine Learning 机器学习中的一些术语

• Samples 样本
  Items or instances of data used for learning or evaluation.用于学习或评估的数据项或实例。

• Features 特征
  The set of attributes, often represented as a vector associated to a sample:属性集，通常表示为与样本相关的向量：
   Handcrafted features: 手工式特征
  e.g., SIFT, HOG, SURF, LBP, GLOH, LESH, CENTRIST.
   Learned features: e.g., by convolutional neural network.学习式特征，例如：通过卷积神经网络。

• Handcrafted Features 手工式特征
   HOG(Histogram of Oriented Gradients，定向梯度直方图)
   Similar to SIFT(Scale-Invariant Feature Transform), but improved accuracy. 与SIFT (尺度不变特征变换) 类似，但改善了精度。
   By distribution of intensity gradients or edge directions. 按照强度梯度或边缘方向分布。
   64×128 detection window. （检测窗口）

• Learned Features 学习式特征
   Humans can learn to see efficiently. Because brains are deep, with many layers of processing. 人类可以有效地学会观察。因为大脑是深度的，具有许多处理层次。
   Some algorithms for such deep architectures, can produce features from raw data for visual recognition. 具有这种深度架构的算法，能从原始数据中生成视觉认知的特征。
   Feature learning also be called representation learning.特征学习也被称为表示学习。
   Understanding deep learning will enable us to build more intelligent machines for visual recognition.
  理解深度学习将使我们能够构建更智能的视觉认知机器。

• Labels 标记
   Values or categories assigned to samples. 在样本上指定的值或类别。
   In classification problems, samples are assigned specific categories.
  分类问题中，样本被指定特定的类别。
   In regression problems, items are assigned real-valued labels.
  回归问题中，项被指定为实值的标记。

• Training sample 训练样本
   Samples used for training learning algorithm. 用于训练学习算法的样本。
   In spam problem, the training sample consist of a set of email samples along with their associated labels.
  对于垃圾邮件问题，训练样本由一组邮件样本以及相关标签组成。

• Validation sample 验证样本
   Validation samples are the labeled data used to tune the parameters of a learning algorithm.
  验证样本是用于调整学习算法参数的、已标注的数据。
   Learning algorithms typically have one or more free parameters, and validation sample is used to select appropriate values for these model parameters.
  学习算法通常具有一个或多个自由参数，因而验证样本用于为这些模型参数选择适当的值。

• Test sample 测试样本
   Samples used to evaluate the performance of a learning algorithm.
  用于评估学习算法性能的样本。
   These predictions are then compared with the labels of the test sample to measure the performance of the algorithm.
  然后将这些预测与测试样本的标签进行比较，以衡量算法的性能。

• Loss function 损失函数
   To measure the difference, or loss, between a predicted label and a true label. 用于度量预测标签和真实标签之间差异或损失。
   Denote the set of all labels as Y and the set of possible predictions as Y’, a loss function L is a mapping: L: Y×Y’→ℝ+
  将所有的标签集表示为Y、并且可能的预测集为Y’，则损失函数L为映射：

• Hypothesis set 假设集（即函数集）
   A set of functions mapping features to the set of labels Y. 假设集是将特征映射为标签Y的函数集。
   For example, the following are a set of functions mapping email features to Y: 例如，映射电子邮件特征的函数集如下：
  Y={spam, non−spam}.

• Abstraction 抽象
  It involves the translation of data into broader representations. 其含义是将数据转化为更广泛的表示。

• Generalization 泛化
  It describes the process of turning abstracted knowledge into a form that can be utilized for action. It is also the ability of a learning algorithm to perform accurately on unseen samples after having experienced a learning data set.
  它形容将抽象知识转化为可用于动作形式的过程。它也是学习算法具有学习数据集
  的经验后，可以对未知样本正确地进行处理的能力。
  

(3) Some Notations

Summary

• Machine learning is to study some algorithms that can learn from and make predictions on data. 机器学习是研究一些可以从数据中学习、并对数据进行预测的算法。
• The different perspectives are aimed to try to have a taxonomy on the algorithms of machine learning, for being easy to understand machine learning. 几个不同视角旨在尝试对机器学习的算法进行分类，以便于理解机器学习。
• Three perspectives on machine learning are proposed in this chapter, those are learning tasks, Learning Paradigms and learning models. 本章提出了机器学习的三个视角，它们是：学习任务、学习范式以及学习模型。