Using neural nets to recognize handwritten digits 使用神經網絡識別手寫數字
The human visual system is one of the wonders of the world. Consider the following sequence of handwritten digits:
人類的視覺系統是世界上的一個奇蹟。看看下面的手寫數字序列：
Most people effortlessly recognize those digits as 504192. That ease is deceptive. In each hemisphere of our brain, humans have a primary visual cortex, also known as V1, containing 140 million neurons, with tens of billions of connections between them. And
yet human vision involves not just V1, but an entire series of visual cortices - V2, V3, V4, and V5 - doing progressively more complex image processing. We carry in our heads a supercomputer, tuned by evolution over hundreds of millions of years, and superbly
adapted to understand the visual world. Recognizing handwritten digits isn’t easy. Rather, we humans are stupendously, astoundingly good at making sense of what our eyes show us. But nearly all that work is done unconsciously. And so we don’t usually appreciate
how tough a problem our visual systems solve.
大多數人毫不費力的識別出這些數字是504192。這種容易是虛假的。在我們大腦的每個半球，人類擁有一個主要大腦皮層V1，它包含1.4億個神經元和數百億的神經纖維連接它們。然而人類的視覺系統不僅僅包括V1，還包括一系列完整的視覺皮層——V2、V3、V4和V5——從事更加複雜的圖像處理。我們把我們的大腦看作一臺超級電腦，它被數億年的進化所優化，極度適應於理解視覺世界。識別手寫數字並不容易。更準確的說，我們人類驚人的擅長理解我們眼睛告訴我們的事情。不過這些事情是在不知不覺中完成的。並且我們不能經常意識到我們視覺系統解決的問題多麼的難。
The difficulty of visual pattern recognition becomes apparent if you attempt to write a computer program to recognize digits like those above. What seems easy when we do it ourselves suddenly becomes extremely difficult. Simple intuitions about how we recognize
shapes - “a 9 has a loop at the top, and a vertical stroke in the bottom right” - turn out to be not so simple to express algorithmically. When you try to make such rules precise, you quickly get lost in a morass of exceptions and caveats and special cases.
It seems hopeless.
如果你嘗試寫一個計算機程序來識別上面那樣的數字，你就會意識到視覺模式識別有多難。我們自己做起來似乎十分簡單的事情變得非常難。一個簡單的直覺是我們可以識別邊沿——“數字9在頂部有一個環，在環的右下方有一豎”——結果是算法表達上並沒有這麼簡單。當你嘗試定義這樣精確的條件時，你很快就迷失在例外、告警和特殊事件的陷阱中。這似乎毫無希望。
Neural networks approach the problem in a different way. The idea is to take a large number of handwritten digits, known as training examples
and then develop a system which can learn from those training examples. In other words, the neural network uses the examples to automatically infer rules for recognizing handwritten digits. Furthermore, by increasing the number of training examples, the network
can learn more about handwriting, and so improve its accuracy. So while I’ve shown just 100 training digits above, perhaps we could build a better handwriting recognizer by using thousands or even millions or billions of training examples.
神經網絡用其他方法來解決這個問題。這個方法是取得大量的手寫數字，也就是訓練樣本，並且開發一個系統，它可以從這些訓練樣本中學習識別手寫數字。換句話說，神經網絡利用樣本數據自動推導出識別手寫數字的規則。更進一步，隨着訓練樣本的增多，神經網絡可以從手寫體中學到更多從而提高它的準確性。所以當我們使用數千、數萬或者數億訓練樣本而不僅僅是上面100個訓練數字時，我們可以建立一個更好的手寫數字識別器。
In this chapter we’ll write a computer program implementing a neural network that learns to recognize handwritten digits. The program is just 74 lines long, and uses no special neural network libraries. But this short program can recognize digits with an accuracy
over 96 percent, without human intervention. Furthermore, in later chapters we’ll develop ideas which can improve accuracy to over 99 percent. In fact, the best commercial neural networks are now so good that they are used by banks to process cheques, and
by post offices to recognize addresses.
本章中我們將編寫一個計算機程序實現一個神經網絡來識別手寫數字。這個程序只有僅僅74行，並且沒有用到特殊的神經網絡庫。不過在沒有人工干預的情況下這個小程序的識別準確率超過96%。更進一步，在最後一章我們將進一步將準確率提升到超過99%。事實上，最好的商業神經網絡已經好到被銀行用來處理支票，或是郵局識別地址。
We’re focusing on handwriting recognition because it’s an excellent prototype problem for learning about neural networks in general. As a prototype it hits a sweet spot: it’s challenging - it’s no small feat to recognize handwritten digits - but it’s not so
difficult as to require an extremely complicated solution, or tremendous computational power. Furthermore, it’s a great way to develop more advanced techniques, such as deep learning. And so throughout the book we’ll return repeatedly to the problem of handwriting
recognition. Later in the book, we’ll discuss how these ideas may be applied to other problems in computer vision, and also in speech, natural language processing, and other domains.
我們聚焦於筆跡識別是因爲通常來說對於學習神經網絡它是一個非常好的範例。作爲一個範例，它擊中了一個最有效的點：具有挑戰性——識別手寫數字並不是簡單的事——它也不會難到需要特別複雜難懂的解決方案，或者驚人的計算資源。此外，它是發展更先進技術的絕佳路徑，例如深度學習。在本書中我們將不斷的回顧這個手寫數字識別的問題。在後面的章節，我們將討論如何將這些方法應用在其他問題上，例如計算機視覺、語音識別、自然語言處理和其他領域。
Of course, if the point of the chapter was only to write a computer program to recognize handwritten digits, then the chapter would be much shorter! But along the way we’ll develop many key ideas about neural networks, including two important types of artificial
neuron (the perceptron and the sigmoid neuron), and the standard learning algorithm for neural networks, known as stochastic gradient descent. Throughout, I focus on explaining why things are done the way they are, and on building your neural networks intuition.
That requires a lengthier discussion than if I just presented the basic mechanics of what’s going on, but it’s worth it for the deeper understanding you’ll attain. Amongst the payoffs, by the end of the chapter we’ll be in position to understand what deep
learning is, and why it matters.
當然，如果本章只是編寫一個識別手寫數字的程序，那麼本章將短小很多！不過沿着這個思路我們將延伸出許多關於神經網絡的關鍵概念，包括兩種重要的人工神經元（perceptron和sigmoid神經元），神經網絡的標準學習算法，隨機梯度下降。由始至終，我聚焦於爲什麼這麼做和如何建立你自己的神經網絡。更深入的理解你想實現的內容是非常值得的，但是需要比我僅僅呈現的基本過程更漫長的討論。本章的最後我們將瞭解深度學習是什麼和爲什麼它可以工作。
轉自：http://blog.csdn.net/forrestyanyu/article/details/54668781