主题：Semi-Supervised Learning

 

Semi-Supervised Learning edited by Olivier Chapelle, Bernhard Scholkopf and Alexander Zien. This book was published by MIT. It is the first book which tells of the felid of Semi-Supervised Learning.

First, I want to say something about the authors. Olivier Chapelle is with Yahoo! Research. He is editor of Semi-Supervised Learning (MIT Press, 2006). His personal page is http://www.kyb.mpg.de/~chapelle. Bernhard Sch?lkopf is Managing Director of the Max Planck Institute for Biological Cybernetics in Tübingen, Germany. He is coauthor of Learning with Kernels (MIT Press, 2002) and is a coeditor of Advances in Kernel Methods: Support Vector Learning (1998), Advances in Large-Margin Classifiers (2000), and Kernel Methods in Computational Biology (2004), all published by The MIT Press. His personal page is http://www.kyb.mpg.de/~bs. Alexander Zien is a Research Scientist at the Max Planck Institute for Biological Cybernetics in Tübingen. His personal page is http://www.kyb.mpg.de/~zien. 
 

In the field of machine learning, semi-supervised learning (SSL) occupies the middle ground, between supervised learning (in which all training examples are labeled) and unsupervised learning (in which no label data are given). As unlabeled data is cheap and labeled data can be hard to get (labels may require experts and may require special devices), Interest in SSL has increased in recent years, such as images, text, and bioinformatics.

This book first comprehensive overview of SSL presents state-of-the-art algorithm, gives taxonomy for SSL methods, analyses of Benchmarks, and puts forward perspectives on future research.

Semi-Supervised Learning first presents the key assumptions and ideas underlying the field: smoothness, cluster or low-density separation, manifold structure, and transduction. The core of the book is the presentation of SSL methods. 

I’m so sorry that I haven’t read this book completeness. About this book, I only offer a little information in all.

 

下面我结合自己看过的一些东西，对这个领域的一些小细节再做进一步介绍（英语我怕自己说不清楚，还是改用汉语）．

半监督学习的算法分为如下几大类：

1.Self-training algorithm(自训练算法)

这个是最早提出的一种研究半监督学习的算法，也是一种最简单的半监督学习算法．

2.Multiview algorithm(多视角算法)

一般多用于可以进行自然特征分裂的数据集中．考虑特殊情况（每个数据点表征两个特征）：每一个数据点看成是两个特征的集合，然后利用协同训练(Co-training algorithm)进行处理．

3.Generative Models(生成模型)

具体算法这里不说，我这里只说一下生成模型（Generative Models）和判别模型（Discriminative Models）的区别．

对于分类和聚类问题而言．①判别模型只关心类的决定边界在哪里；生成模型关心的是类本身而非决定边界．②判别模型只能判定数据点属于哪个类别，无法将过程描述出来；生成模型可以将过程描述．③生成模型可以得到判别模型；判别模型推不出生成模型．④判别模型估计的是条件概率分布（Conditional distribution）；生成模型估计的是联合概率分布（Joint probability distribution）．

常见的判别模型有：Linear discriminate analysis, Support vector machines, Boosting, Conditional random fields, Logistic regression.

常见的生成模型有：Gaussian distribution, Gaussian mixture model, Multinomial distribution, Hidden Markov model, Na?ve Bayes, Latent Dirichlet allocation.

4.Transductive SVM(转导SVM)

强调一点，根据Vapnik的Statistical Learning Theory中提出统计学习中考虑两种不同类型的推理：归纳推理（Inductive inference）和转导推理(Transductive inference)．转导推理的目的是估计某一未知预测函数在给定兴趣点上的值（而不是在该函数的全部定义域上的值）．关键是，通过求解要求较低的问题，可以得到更精确的解．

传统的推理方法是归纳－演绎方法，人们首先根据用已有的信息定义一个一般规则，然后用这个规则来推断所需要的答案．也就是说，首先从特殊到一般，然后从一般到特殊．但是在转导模式中，我们进行直接的从特殊到特殊的推理，避免了推理中的不适定部分．

5.Graph-Based Algorithms(基于图的算法)

这个算法，我给出一个简单的实验，希望大家能看的懂．

此主题相关图片如下：moon1.jpg


此主题相关图片如下：moon2.jpg

此主题相关图片如下：moon3.jpg

    以上也只是我的一些很肤浅的介绍．

美国威斯康辛大学麦迪逊分校的Zhu Xiaojin（Jerry）教授写了一篇综述（实时更新）囊括了半监督学习的所有算法以及他人研究的最新进展，并且在2007年国际机器学习会议（ICML）上做了半监督学习的报告．国内在这个领域做的出色的当属清华大学自动化系张长水教授．

最后非常感谢张兴友老师的热心帮助！谢谢！