CTR之FM模型及keras實現

1. 理論

FM模型：
$y(\mathbf{x})=w_{0}+\sum_{i=1}^{n} w_{i} x_{i}+\sum_{i=1}^{n} \sum_{j=i+1}^{n} w_{i j} x_{i} x_{j}$

n 代表樣本的特徵數量， $x_i$ 是第$i$個特徵的值，$w_0, w_i, w_{ij}$ 是模型的參數。

經過矩陣分解：

$y(\mathbf{x})=w_{0}+\sum_{i=1}^{n} w_{i} x_{i}+\sum_{i=1}^{n} \sum_{j=i+1}^{n}\left\langle\mathbf{v}_{i}, \mathbf{v}_{j}\right\rangle x_{i} x_{j}$

採樣MSE損失函數求解迴歸問題，採樣Hinge或者Cross-entropy損失求解分類問題。二元分類，FM的輸出經過sigmoid變換。

二次項的簡化技巧：
$\sum_{i=1}^{n} \sum_{j=i+1}^{n}\left\langle\mathbf{v}_{i}, \mathbf{v}_{j}\right\rangle x_{i} x_{j}=\frac{1}{2} \sum_{f=1}^{k}\left(\left(\sum_{i=1}^{n} v_{i, f} x_{i}\right)^{2}-\sum_{i=1}^{n} v_{i, f}^{2} x_{i}^{2}\right)$

2. python實現

import pandas as pd
import numpy as np
from sklearn.metrics import log_loss, roc_auc_score
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import LabelEncoder, MinMaxScaler
from sklearn.datasets import load_breast_cancer

import keras.backend as K
from keras.layers import Input,Dense,Add,Activation
from keras import optimizers
from keras.models import Model
from keras.engine.topology import Layer

class CrossLayer(Layer):
    def __init__(self, input_dim, factor_order=10, **kwargs):
        self.input_dim = input_dim
        self.factor_order = factor_order
        super(CrossLayer, self).__init__(**kwargs)

    def build(self, input_shape):
        self.kernel = self.add_weight(name='kernel', 
                                      shape=(self.input_dim, self.factor_order),
                                      initializer='glorot_uniform',
                                      trainable=True)
        super(CrossLayer, self).build(input_shape)

    def call(self, X):
        square_of_sum = K.pow(K.dot(X,self.kernel), 2)
        sum_of_square = K.dot(K.pow(X,2), K.pow(self.kernel,2))
        return 0.5*K.sum(square_of_sum - sum_of_square, axis=1, keepdims=True)

    def compute_output_shape(self, input_shape):
        return (input_shape[0], 1)
        
class FM():
    def __init__(self, input_dim=None, factor_order=2, output_dim=1, epochs=10, batch_size=256):
        self.input_dim = input_dim
        self.output_dim = output_dim
        self.factor_order = factor_order
        self.epochs = epochs
        self.batch_size = batch_size

    def build_model(self,):
        X = Input(shape=(self.input_dim,))
        linear_term = Dense(units=1)(X)
        cross_term = CrossLayer(self.input_dim, self.factor_order)(X)
        logits = Add()([linear_term, cross_term])
        predictions = Activation('sigmoid')(logits)
        self.model = Model(inputs=X, outputs=predictions)
        self.model.compile(loss='binary_crossentropy', optimizer='adam')
        print(self.model.summary())
        return self.model
    
    def fit(self,X_train, y_train):
        self.model.fit(X_train, y_train, epochs=self.epochs, batch_size=self.batch_size)
        return self.model
        
    def predict(self, X_test,batch_size=None):
        if batch_size is None:
            batch_size = self.batch_size
        return self.model.predict(X_test, batch_size=batch_size)
        

if __name__ == "__main__":
    data = load_breast_cancer()
    X_train, X_test, y_train, y_test = train_test_split(data.data, data.target, test_size=0.2,
                                                        random_state=27, stratify=data.target)
    fm = FM(30)
    fm.build_model()
    
    fm.fit(X_train, y_train)
    pred_ans = fm.predict(X_test)
    print('pred_ans',pred_ans)
    print("test AUC", round(roc_auc_score(y_test, pred_ans), 4))

運行結果：

test AUC 0.5

跑出來的結果有點怪，可能數據集不太適合。

---

參考：

1. github fm;
2. FM paper;
3. 推薦系統常用數據集；
4. 淺夢deepFM；
5. keras FM github;
6. (推薦)從FM推演各深度學習CTR預估模型（附代碼）；