文章目錄
Dynamic Routing
Conventional net: 層之間完全連接
- 任何兩個神經元之間的加權都是可以訓練的
- 訓練之後，這些權重是固定的，與輸入無關
CapsNet 也有這些權重 - 但是additional routing coefficients 用於訓練權重（coefficient value 由輸入決定，因此是動態的）
- 沒有動態路由的CapsNet和常規網絡一樣
CNN中類似動態路由的成分是max pooling
Max pooling是從前一層到後一層選擇路由信息的粗糙方法。
Coefficients operate on capsule level
- The contribution of a capsule to another capsule in the next layer is determined by the weight matrix and a single scalar coefficient
- The first capsule u∈R3 contributes to the capsule s∈R2 in the next layer by 0.1⋅W1,2⋅u where
0.1 is the routing coefficient and W1,2 is a 2x3 matrix whose subscript denotes the indices of capsules in the two layers. - The total contribution of a capsule is exactly 1.Capsule u route to capsule v with routing coefficient 0.9 and to capsule s with 0.1.
- This is a voting scheme where each capsule in layer before vote for each capsule in layer after.
(前面對後面投票)
Dynamic Routing: Routing by Agreement
Updating the coefficients.
Idea: a capsule votes more if its transformation agrees with the majority of other capsules
Shows routing coefficients to the first capsule in second layer denoted by ci1 for capsule ui
In the beginning all ci1 initialized to 1⁄2 since each capsule routes to all two capsules in the next layer The activation of the first capsule s1 in the next layer is the sum of all transformed and scaled vectors ci1Wi1ui from the left layer
The amount of change in coefficient (i.e. vote) is determined by the dot product between s_i and each summand ci1Wi1ui
Comparison to fully connected neural network
全連接網絡
膠囊網絡
