序列感知推薦系統

序列感知推薦系統

Sequence-Aware Recommender Systems

在前面的章節中，我們將推薦任務抽象爲一個矩陣完成問題，而不考慮用戶的短期行爲。在本節中，我們將介紹一個推薦模型，該模型考慮按順序排列的用戶交互日誌。它是一種序列感知的推薦程序[Quadrana et al.，2018]，其中的輸入是過去用戶操作的有序且通常帶有時間戳的列表。最近的一些文獻已經證明了在建模用戶的時間行爲模式和發現他們的興趣漂移時結合這些信息的有用性。

我們將介紹的模型，Caser[Tang&Wang，2018]是卷積序列嵌入推薦模型的簡稱，它採用卷積神經網絡捕捉用戶最近活動的動態模式影響。Caser的主要組成部分包括一個水平卷積網絡和一個垂直卷積網絡，旨在分別揭示聯合層和點級序列模式。點級模式表示歷史序列中的單個項對目標項的影響，而聯合級模式表示前幾項操作對後續目標的影響。例如，同時購買牛奶和黃油會導致購買麪粉的概率比只購買其中一種更高。此外，用戶的一般興趣或長期偏好也在最後一個完全連接的層中建模，從而使用戶興趣的建模更加全面。模型的細節描述如下。

1. Model Architectures
   
   
   
   Fig. 1 Illustration of the Caser Model

首先導入所需的庫。

from d2l
import mxnet as d2l

from mxnet
import gluon, np, npx

from mxnet.gluon
import nn

import mxnet
as mx

import random

import sys

npx.set_np()

2. Model Implementation

下面的代碼實現了Caser模型。它由垂直卷積層、水平卷積層和全連通層組成。

class Caser(nn.Block):

def

init(self,num_factors, num_users, num_items, L=5, d=16,

d_prime=4, drop_ratio=0.05, **kwargs):

    super(Caser, self).__init__(**kwargs)

    self.P = nn.Embedding(num_users, num_factors)

    self.Q = nn.Embedding(num_items, num_factors)

    self.d_prime, self.d = d_prime, d

    # Vertical convolution layer

    self.conv_v = nn.Conv2D(d_prime, (L, 1), in_channels=1)

    # Horizontal convolution layer

    h = [i + 1 for i in range(L)]

    self.conv_h, self.max_pool = nn.Sequential(), nn.Sequential()

    for i in h:

self.conv_h.add(nn.Conv2D(d, (i, num_factors), in_channels=1))

        self.max_pool.add(nn.MaxPool1D(L- i + 1))

    # Fully-connected layer

    self.fc1_dim_v, self.fc1_dim_h = d_prime * num_factors, d * len(h)

    self.fc = nn.Dense(in_units=d_prime * num_factors + d * L,

                       activation='relu', units=num_factors)

    self.Q_prime = nn.Embedding(num_items, num_factors * 2)

    self.b = nn.Embedding(num_items, 1)

    self.dropout = nn.Dropout(drop_ratio)

def forward(self, user_id, seq, item_id):

    item_embs = np.expand_dims(self.Q(seq), 1)

    user_emb = self.P(user_id)

    out, out_h, out_v, out_hs = None, None, None, []

    if self.d_prime:

        out_v = self.conv_v(item_embs)

        out_v = out_v.reshape(out_v.shape[0], self.fc1_dim_v)

    if self.d:

        for conv, maxp in zip(self.conv_h, self.max_pool):

            conv_out = np.squeeze(npx.relu(conv(item_embs)), axis=3)

            t = maxp(conv_out)

            pool_out = np.squeeze(t, axis=2)

            out_hs.append(pool_out)

        out_h = np.concatenate(out_hs, axis=1)

    out = np.concatenate([out_v, out_h], axis=1)

    z = self.fc(self.dropout(out))

    x = np.concatenate([z, user_emb], axis=1)

    q_prime_i = np.squeeze(self.Q_prime(item_id))

    b = np.squeeze(self.b(item_id))

    res = (x * q_prime_i).sum(1) + b

    return

res

3. Sequential Dataset with Negative Sampling

爲了處理順序交互數據，我們需要重新實現Dataset類。下面的代碼創建一個名爲SeqDataset的新數據集類。在每個示例中，它輸出用戶標識，即L交互的項目作爲一個序列，下一個項目她交互作爲目標。下面的圖演示了用戶加載數據的過程。假設這個用戶喜歡8部電影，我們按照時間順序組織這8部電影。最新的電影被排除在外作爲測試項目。對於剩下的七部電影，我們可以得到三個訓練樣本，每個樣本包含五個序列(L=5) 電影及其後續項目（its subsequent item）作爲目標項。負樣本也包括在自定義數據集中。

Fig. 2 Illustration of the data generation process

class SeqDataset(gluon.data.Dataset):

def __init__(self, user_ids, item_ids, L, num_users, num_items,

             candidates):

    user_ids, item_ids = np.array(user_ids), np.array(item_ids)

    sort_idx = np.array(sorted(range(len(user_ids)),

                               key=lambda k: user_ids[k]))

    u_ids, i_ids = user_ids[sort_idx], item_ids[sort_idx]

    temp, u_ids, self.cand = {}, u_ids.asnumpy(), candidates

    self.all_items = set([i for i in range(num_items)])

    [temp.setdefault(u_ids[i], []).append(i) for i, _ in enumerate(u_ids)]

    temp = sorted(temp.items(), key=lambda x: x[0])

    u_ids = np.array([i[0] for i in temp])

    idx = np.array([i[1][0] for i in temp])

    self.ns = ns = int(sum([c - L if c >= L + 1 else 1 for c

                            in np.array([len(i[1]) for i in temp])]))

    self.seq_items = np.zeros((ns, L))

    self.seq_users = np.zeros(ns, dtype='int32')

    self.seq_tgt = np.zeros((ns, 1))

    self.test_seq = np.zeros((num_users, L))

    test_users, _uid = np.empty(num_users), None

    for i, (uid, i_seq) in enumerate(self._seq(u_ids, i_ids, idx, L + 1)):

        if uid != _uid:

            self.test_seq[uid][:] = i_seq[-L:]

            test_users[uid], _uid = uid, uid

        self.seq_tgt[i][:] = i_seq[-1:]

        self.seq_items[i][:], self.seq_users[i] = i_seq[:L], uid

def _win(self, tensor, window_size, step_size=1):

    if len(tensor) - window_size >= 0:

        for i in range(len(tensor), 0, - step_size):

            if i - window_size >= 0:

                yield tensor[i - window_size:i]

            else:

                break

    else:

        yield tensor

def _seq(self, u_ids, i_ids, idx, max_len):

    for i in range(len(idx)):

        stop_idx = None if i >= len(idx) - 1 else int(idx[i + 1])

        for s in self._win(i_ids[int(idx[i]):stop_idx], max_len):

            yield (int(u_ids[i]), s)

def __len__(self):

    return self.ns

def __getitem__(self, idx):

    neg = list(self.all_items - set(self.cand[int(self.seq_users[idx])]))

    i = random.randint(0, len(neg) - 1)

    return (self.seq_users[idx], self.seq_items[idx], self.seq_tgt[idx], neg[i])

4. Load the MovieLens 100K dataset

然後，我們以序列感知模式讀取並分割MovieLens 100K數據集，並使用上面實現的序列數據加載器加載訓練數據。

TARGET_NUM, L, batch_size = 1, 3, 4096

df, num_users, num_items = d2l.read_data_ml100k()

train_data, test_data = d2l.split_data_ml100k(df, num_users, num_items,

                                          'seq-aware')

users_train, items_train, ratings_train, candidates = d2l.load_data_ml100k(

train_data, num_users, num_items, feedback="implicit")

users_test, items_test, ratings_test, test_iter = d2l.load_data_ml100k(

test_data, num_users, num_items, feedback="implicit")

train_seq_data = SeqDataset(users_train, items_train, L, num_users,

                        num_items, candidates)

num_workers = 0 if sys.platform.startswith(“win”) else 4

train_iter = gluon.data.DataLoader(train_seq_data, batch_size, True,

                               last_batch="rollover",

                               num_workers=num_workers)

test_seq_iter = train_seq_data.test_seq

train_seq_data[0]

(array(0, dtype=int32), array([110., 255., 4.]), array([101.]), 1645)

培訓數據結構如上圖所示。第一個元素是用戶標識，下一個列表指示該用戶喜歡的最後五個項目，最後一個元素是該用戶在這五個項目之後喜歡的項目。

5. Train the Model

現在，讓我們訓練模型。我們使用與NeuMF相同的設置，包括學習率、優化器和k在最後一節中，使結果具有可比性。

ctx = d2l.try_all_gpus()

net = Caser(10, num_users, num_items, L)

net.initialize(ctx=ctx, force_reinit=True, init=mx.init.Normal(0.01))

lr, num_epochs, wd, optimizer = 0.04, 8, 1e-5, ‘adam’

loss = d2l.BPRLoss()

trainer = gluon.Trainer(net.collect_params(), optimizer,

                    {"learning_rate": lr, 'wd': wd})

d2l.train_ranking(net, train_iter, test_iter, loss, trainer, test_seq_iter,

              num_users, num_items, num_epochs, ctx, d2l.evaluate_ranking,

              candidates, eval_step=1)

train loss 0.866, test hit rate 0.382, test AUC 0.748

29.3 examples/sec on [gpu(0), gpu(1)]

6. Summary
Inferring a user’s short-term and long-term interests can make prediction of the next item that she preferred more effectively.
Convolutional neural networks can be utilized to capture users’ short-term interests from sequential interactions.