隐式反馈

It’s more prevalent to see the companies use the recommended system algorithm to produce users’ favorite items based on the previous shopping experience. Online customers would get recommended items while shopping online from stores like eBay, Amazon, Walmart, etc. The article is focused on item recommendations. The item recommendation system method provides a user-specific ranking for a set of items learned from users’ past datasets such as buying history, viewing history, etc. Nowadays, the recommendation system varies a lot from the input of explicit dataset like ratings to an implicit dataset such as monitoring clicks, view times, purchases, etc. This information is easier to collect, but hard to recommend the users’ favorite items.

看到公司使用推荐的系统算法根据以前的购物经验来生产用户喜欢的商品的情况更为普遍。 在线顾客在从eBay,Amazon,Walmart等商店在线购物时将获得推荐商品。本文重点介绍商品推荐。 物品推荐系统方法为从用户过去的数据集(例如购买历史记录,观看历史记录等)中学习到的一组物品提供了特定于用户的排名。如今,推荐系统的变化范围很大,从像评级这样的显式数据集到隐式数据集,例如监视点击次数,查看时间,购买次数等。此信息更易于收集,但难以推荐用户喜欢的项目。

In this article, you will learn the singular value decomposition and truncated SVD of the recommender system:

在本文中,您将学习推荐系统的奇异值分解和截断SVD:

(1) Personalized Ranking System

(1)个性化排名系统

(2) Problem Statement

(2)问题陈述

(3) Bayesian Personalized Ranking (BPR)

(3)贝叶斯个性化排名(BPR)

(4) BPR Optimization Criterion

(4)BPR优化准则

(5) BPR Learning Algorithm

(5)BPR学习算法

(6) Matrix Factorization

(6)矩阵分解

(7) Adaptive K-nearest-neighbor

(7)自适应K近邻

个性化排名系统 (Personalized Ranking System)

The personalized ranking provides customers with item recommendations of a ranked list of items. The article would focus on recommending customers with a personalized ranked list of items from users’ implicit behavior derived from the past purchase data. Observed from the purchase data, it is available to get the positive observations like users’ bought history, whereas non-observed user-item pairs data is difficult for the model input like the unbought item, non-interested items, or the interesting items for their future purchase.

个性化排名为客户提供了项目排名列表的项目推荐。 本文将重点介绍从过去购买数据得出的用户隐式行为中,推荐具有个性化商品排名列表的客户。 从购买数据中观察,可以得到诸如用户的购买历史之类的正面观察,而对于模型输入(如未购买的商品,不感兴趣的商品或感兴趣的商品),很难观察到用户-商品对数据。他们未来的购买。

问题陈述 (Problem Statement)

For the implicit feedback systems, it is able to detect the positive dataset like bought history. For the remaining data, it is a mixture of actually negative and missing values. Nevertheless, machine learning models are unable to learn the missing data. Typically, the item recommenders output the personalized score X_ui based on the preference of the user for the item, and items are sorted form the predicted score. The machine learning model of item recommenders provides the training data with giving pairs (u, i) ∈ S as a positive class label and all other combinations in (U × I) \ S a negative one.

对于隐式反馈系统,它能够检测到像购买历史记录这样的正数据集。 对于其余数据,它实际上是负值和缺失值的混合。 然而,机器学习模型无法学习丢失的数据。 通常,项目推荐者基于用户对项目的偏好来输出个性化得分X_ui,并且将项目从预测得分中分类。 项目推荐者的机器学习模型为训练数据提供(u,i)∈S对作为正类别标签,而(U×I)\ S中的所有其他组合为负数对。

The model is fit to predict the positive class with value 1 and 0 for the rest. The problem would occur when the model is unable to rank the items ((U × I) \ S), which have been given as negative feedback during the training. An alternative would be to add regularization to the model to prevent overfitting. Another method is to create item pairs as training data and optimize for correctly ranking item pairs instead of scoring single item while the missing values are taken care of. From the photo below, it’s hard for the model to only learn from the observed data. Therefore, all the negative data is replaced with 0.

该模型适合于预测阳性类别,其余类别的值为1和0。 当模型无法对在训练过程中作为负面反馈给出的项目((U×I)\ S)进行排名时,将出现问题。 另一种选择是向模型添加正则化以防止过度拟合。 另一种方法是将物料对创建为训练数据,并针对物料对正确排序进行优化,而不是在照顾缺失值的同时对单个物料评分。 从下面的照片中,模型很难仅从观察到的数据中学习。 因此,所有负数据都将替换为0。

Image for post
Photo0: The observed 0 value and unobserved 1 value.
Photo0:观察到的0值和未观察到的1值。

贝叶斯个性化排名(BPR) (Bayesian Personalized Ranking (BPR))

To overcome the personalized ranking task, the Bayesian personalized ranking incorporates the Bayesian analysis of the problem using the likelihood function for p(i >u j|Θ) and the prior probability for the model parameter p(Θ).

为了克服个性化排名任务,贝叶斯个性化排名使用针对p(i> uj |Θ)的似然函数和模型参数p(Θ)的先验概率,对问题进行了贝叶斯分析。

In this section, we derive a generic method for solving the personalized ranking task. It consists of the general optimization criterion for personalized ranking, BPR-Opt, which will be derived by a Bayesian analysis of the problem using the likelihood function for p(i >u j|Θ) and the prior probability for the model parameter p(Θ)

在本节中,我们导出用于解决个性化排名任务的通用方法。 它由用于个性化排名的一般优化准则BPR-Opt组成,该准则将通过对问题的贝叶斯分析使用p(i> uj |Θ)的似然函数和模型参数p(Θ的先验概率)得出)

BPR优化准则 (BPR Optimization Criterion)

The Bayesian approach is to produce the rankings for all items i ∈ I to maximize the following posterior probability where Θ represents the parameter vector of an arbitrary model class

贝叶斯方法是产生所有项i∈I的排名,以最大化以下后验概率,其中Θ表示任意模型类的参数向量

p(Θ| >u) ∝ p(>u |Θ) p(Θ)

p(Θ|> u)∝ p(> u |Θ)p(Θ)

All user factors are independent with each other, and the ordering of each pair of items (i, j) for a specific user is unique.

所有用户因素彼此独立,并且特定用户的每对项目(i,j)的顺序是唯一的。

Hence, the above user-specific likelihood function can be reproduced with the formula below.

因此,可以使用以下公式来再现上述用户特定的似然函数。

Image for post
User’s likelihood function
用户似然函数

We define the individual probability that a user really prefers item i to item j as:

我们将用户真正喜欢项目i而不是项目j的个体概率定义为:

Image for post
User’s probability on the preference over item i to item j
用户对项目i到项目j的偏好的概率

For the Bayesian modeling approach of the personalized ranking task, a general prior density p(Θ) is introduced, which is a normal distribution with zero mean and variance-covariance matrix ΣΘ.

对于个性化排序任务的贝叶斯建模方法,引入了一般先验密度p(Θ),它是具有零均值和方差-协方差矩阵ΣΘ的正态分布。

p(Θ) ∼ N(0, ΣΘ)

p(Θ)〜N(0,ΣΘ)

By setting ΣΘ = λΘI, we reduce the number of unknown hyper-parameters. The maximum posterior estimator is set to derive the optimization criterion for personalized ranking BPR-Opt.

通过设置ΣΘ=λΘI,我们减少了未知的超参数的数量。 设置最大后验估计量,以得出个性化排名BPR-Opt的优化标准。

Image for post
BPR-Opt mathematic formula
BPR-Opt数学公式

BPR学习算法 (BPR Learning Algorithm)

From the section above, the criterion is derived from personalized ranking, and standard gradient descent is not proper to cope with the problem. Then, LearnBPR is introduced as a stochastic gradient-descent algorithm to optimize model performance.

从上面的部分中,标准是从个性化排名中得出的,并且标准梯度下降不适用于解决该问题。 然后,将LearnBPR作为随机梯度下降算法引入,以优化模型性能。

The gradient of BPR-Opt with respect to the model parameters is:

BPR-Opt相对于模型参数的梯度为:

Image for post
Gradient BPR-Opt mathematic formula
梯度BPR-Opt数学公式

The parameters in the optimized model are specified as the learning rate α and regularization λΘ. The model is a bootstrapping based stochastic gradient descent.

优化模型中的参数指定为学习率α和正则化λΘ。 该模型是基于自举的随机梯度下降。

Image for post
Photo2: bootstrapping based stochastic gradient descent algorithm
图2:基于自举的随机梯度下降算法

We will introduce a popular stochastic gradient descent approach. From the equation below, an update is performed for each triple (u, i, j) ∈ D_s.

我们将介绍一种流行的随机梯度下降方法。 根据以下等式,对每个三元组(u,i,j)∈D_s进行更新。

Image for post
stochastic gradient descent mathematic formula
随机梯度下降数学公式

The order for the approach over the training pairs is crucial. Traversing over the data item-wise or user-wise results in poor convergence while the same user-item pairs would have many updates. The stochastic gradient descent algorithm is introduced to choose the triples randomly (uniformly distributed). Such an approach can reduce the chances to choose the same user-item combination in consecutive update steps. Bootstrap sampling approach with replacement is able to stop the random choice of the user-item pair. The number of a single step is selected linearly in our evaluation based on the number of observed positive feedback S.

训练对中方法的顺序至关重要。 在数据项或用户项上遍历会导致收敛性较差,而相同的用户项对将具有许多更新。 引入随机梯度下降算法随机选择三元组(均匀分布)。 这样的方法可以减少在连续更新步骤中选择相同用户项组合的机会。 具有替换的自举采样方法能够停止用户对项的随机选择。 在我们的评估中,基于观察到的正反馈S的数量,线性选择单个步骤的数量。

使用BPR学习模型 (Learning models with BPR)

Both two diverse model classes of matrix factorization and learned k-nearest-neighbor would model the hidden preferences of a user on an item. First, we decompose the estimator ˆx_uij and define it as:

矩阵分解和学习的k最近邻这两个不同的模型类别都将对用户在某项上的隐藏偏好进行建模。 首先,我们分解估算器ˆx_uij并将其定义为:

xˆuij := ˆxui − xˆuj

xˆuij:= ˆxui-xˆuj

Next, the standard collaborative filtering model is applied to predict ˆx_ul. Such an approach is better than the original one since it classifies the difference of two predictions ˆx_ui − xˆ_uj rather than regress a single predictor ˆx_ul to a single number.

接下来,将标准协作过滤模型应用于预测x_ul。 这种方法比原始方法好,因为它对两个预测differencex_ui-xˆ_uj的差进行分类,而不是将单个预测值ˆx_ul回归为单个数。

矩阵分解 (Matrix Factorization)

The estimate of the matrix X : U × I is used to coping with the problem of predicting ˆx_ui. The target matrix is produced with a matrix product of two low-rank matrices W : |U| × k and H : |I| × k from the matrix factorization.

矩阵X:U×I的估计用于解决预测ˆx_ui的问题。 目标矩阵由两个低秩矩阵W:| U |的矩阵乘积生成。 ×k和H:| I | ×k来自矩阵分解。

Xˆ := W* H^t

Xˆ:= W * H ^ t

From the equation below, there are parameters specified.

在下面的公式中,指定了参数。

K: the dimensionality/rank of the approximationW_u: a feature vector in W, indicating a user u and similarly each row hi of H describes an item i.

K:近似值的维数/等级W_u:W中的特征向量,指示用户u,类似地,H的每一行hi都描述了项i。

Image for post
prediction formula
预测公式

Θ = (W, H): the model parameters for matrix factorization. The parameters are latent variables to model unobserved user and item pairs. Singular value decomposition (SVD) is achieved to approximate Xˆ to X for the least-square. Usually, the SVD approach would overfit the data. Then, there are other recommended methods like regularized least-square optimization, non-negative factorization, maximum margin factorization.

Θ=(W,H):用于矩阵分解的模型参数。 参数是潜在变量,用于对未观察到的用户和物料对建模。 对于最小二乘,实现奇异值分解(SVD)以使X ^近似于X。 通常,SVD方法会过度拟合数据。 然后,还有其他推荐方法,例如正则化最小二乘优化,非负因式分解,最大余量因式分解。

For the ranking task, we use LearnBPR to estimate whether a user prefers one item over another. Through the LearnBPR approach, we need to know the gradient of ˆx_uij with respect to every model parameter θ. Below, it shows the derivatives from the matrix factorization model.

对于排名任务,我们使用LearnBPR来估计用户是否更喜欢一项。 通过LearnBPR方法,我们需要知道ˆx_uij相对于每个模型参数θ的梯度。 下面显示了矩阵分解模型的导数。

For the task of ranking, i.e. estimating whether a user prefers one item over another, a better approach is to optimize against the BPR-Opt criterion. This can be achieved by using our proposed algorithm LearnBPR. As stated before for optimizing with LearnBPR, only the gradient of ˆx_uij with respect to every model parameter θ has to be known. For the matrix factorization model the derivatives are:

对于排序任务,即估计用户是否偏爱一个项目,更好的方法是针对BPR-Opt标准进行优化。 这可以通过使用我们提出的算法LearnBPR来实现。 如前所述,为利用LearnBPR进行优化,仅需知道ˆx_uij相对于每个模型参数θ的梯度。 对于矩阵分解模型,导数为:

Image for post
the derivatives of matrix factorization model
矩阵分解模型的导数

Besides, there are three other regularization terms added.

此外,还添加了其他三个正则化术语。

λW: user features WλH+: positive updates on hif for the item features HλH−: negative updates on hjf for the item features H

λW:用户特征WλH+:对商品特征HIF的正向更新λH-:对商品特征H的hjf负向更新

自适应K近邻 (Adaptive K-nearest-neighbor)

There’s another popular method in collaborative filtering called K-nearest-neighbor method, derived from a similarity measure between either item (item-based) or users (user-based). For the K-nearest-neighbor, we generate the prediction of the recommend item i from the user u based on the past similarity of i to all other items the user has seen in the past of history data — i.e. I + u. The k-nearest neighbors are produced from the k most similar items of I + u. The model of item-based K-nearest-neighbor is listed below for the item prediction. C is the symmetric item-correlation/ item-similarity matrix.

协作过滤中还有另一种流行的方法,称为K最近邻方法,它是根据项目(基于项目)或用户(基于用户)之间的相似性度量得出的。 对于K近邻,我们根据i与用户过去在历史数据中看到的所有其他项目(即I + u)的过去相似性,根据用户u生成推荐项目i的预测。 k个最近邻居是由I + u的k个最相似项产生的。 下面列出了基于项目的K最近邻模型用于项目预测。 C是对称项目相关性/项目相似性矩阵。

Image for post
Item prediction of K-nearest-neighbor
K最近邻的项目预测

Model parameters in kNN are Θ = C. C is determined by applying a heuristic similarity measure, e.g. cosine vector similarity:i.

kNN中的模型参数为Θ=C。通过应用启发式相似性度量(例如余弦矢量相似性:i)确定C。

Image for post
cosine vector similarity of K-nearest-neighbor
K近邻的余弦向量相似度

Similarity measure C would be updated when the model is learning. Parameter C can be applied directly. On the other hand, when the parameter C is too large, the model produces C from the factorization HHt with H : I × k. Later on, we choose to adapt to C without the factorization. To optimize the KNN model for ranking, the LearnBPR algorithm is used to update the gradient of ˆx_uij with respect to the model parameters C.

在学习模型时,将更新相似性度量C。 参数C可以直接应用。 另一方面,当参数C太大时,模型从因式分解HHt生成C,其中H:I×k。 稍后,我们选择不进行因式分解就适应C。 为了优化用于排序的KNN模型,LearnBPR算法用于相对于模型参数C更新ˆx_uij的梯度。

Image for post
KNN gradient of LearnBPR algorithm mathematic formula
LearnBPR算法的KNN梯度数学公式

There are two regularization constants, λ+ for updates on cil, and λ− for updates on cjl.:

有两个正则化常数,λ+用于cil更新,λ-用于cjl更新:

python代码的动手经验 (Hands-on experience of python code)

资料说明: (Data Description:)

There are ratings, users, and movies dataset extracted from grouplens website. These files contain around 1 million anonymous ratings of approximately 4,000 movies from 6,000 MovieLens users. The rating data includes UserID, MovieID, and ratings. The user file contains demographic information from users like gender, age, occupation, and zip-code. The movie dataset has basic information like MovieID, title, and genres.

从grouplens网站提取了评级,用户和电影数据集。 这些文件包含来自6,000个MovieLens用户的大约一百万个匿名分级,涉及大约4,000部电影。 分级数据包括UserID,MovieID和分级。 用户文件包含来自用户的人口统计信息,例如性别,年龄,职业和邮政编码。 电影数据集具有基本信息,例如MovieID,标题和流派。

隐式反馈的贝叶斯个性化排名 (Bayesian Personalized Ranking from Implicit Feedback)

For the modeling approach, the personalized ranking system, maximum posterior estimator derived from a Bayesian analysis, is implemented in Pytorch model. Besides, the optimization of the model is done through a generic learning algorithm based on stochastic gradient descent with bootstrap sampling.

对于建模方法,在Pytorch模型中实现了个性化排名系统,即从贝叶斯分析得出的最大后验估计量。 此外,该模型的优化是通过基于随机梯度下降和自举抽样的通用学习算法完成的。

数据预处理 (Data Preprocessing)

From the users and rating dataset, we create the unique userid mapping with the movies’ ratings given by the unique users. Also, the unique item mapping from the rating dataset. There are two approaches applied when creating train_user_matrix and test_user_matrix. One is randomly selected of the users, and another is considered with the time order factor. With the train and test setlist, we take them to create the user and item pair for the matrix input.

根据用户和评分数据集,我们使用唯一用户提供的电影评分创建唯一的用户ID映射。 同样,来自评级数据集的唯一项目映射。 创建train_user_matrix和test_user_matrix时应用了两种方法。 一个是从用户中随机选择的,另一个是按时间顺序考虑的因素。 通过训练和测试设置列表,我们将它们带到矩阵输入中创建用户和项目对。

def convert_unique_idx(df, column_name):
    column_dict = {x: i for i, x in enumerate(df[column_name].unique())}
    df[column_name] = df[column_name].apply(column_dict.get)
    df[column_name] = df[column_name].astype('int')
    assert df[column_name].min() == 0
    assert df[column_name].max() == len(column_dict) - 1
    return df, column_dict


def split_train_test(df, user_size, test_size=0.2, time_order=False):
    """Split a dataset into `train_user_list` and `test_user_list`.
    Because it needs `user_list` for splitting dataset as `time_order` is set,
    Returning `user_list` data structure will be a good choice."""
    # TODO: Handle duplicated items
    if not time_order:
        test_idx = np.random.choice(len(df), size=int(len(df)*test_size))
        train_idx = list(set(range(len(df))) - set(test_idx))
        test_df = df.loc[test_idx].reset_index(drop=True)
        train_df = df.loc[train_idx].reset_index(drop=True)
        test_user_list = create_user_list(test_df, user_size)
        train_user_list = create_user_list(train_df, user_size)
    else:
        total_user_list = create_user_list(df, user_size)
        print(len(total_user_list))
        train_user_list = [None] * len(total_user_list)
        test_user_list = [None] * len(total_user_list)
        for user, item_list in enumerate(total_user_list):
            # Choose latest item
            item_list = sorted(item_list, key=lambda x: x[0])
            # Split item
            test_item = item_list[math.ceil(len(item_list)*(1-test_size)):]
            train_item = item_list[:math.ceil(len(item_list)*(1-test_size))]
            # Register to each user list
            test_user_list[user] = test_item
            train_user_list[user] = train_item
    # Remove time
    test_user_list = [list(map(lambda x: x[1], l)) for l in test_user_list]
    train_user_list = [list(map(lambda x: x[1], l)) for l in train_user_list]
    return train_user_list, test_user_list
  
def create_pair(user_list):
    pair = []
    for user, item_list in enumerate(user_list):
        pair.extend([(user, item) for item in item_list])
    return pair
  
df, user_mapping = convert_unique_idx(df, 'user')
df, item_mapping = convert_unique_idx(df, 'item')
train_user_list, test_user_list = split_train_test(df,
                                                     user_size,
                                                     test_size=args.test_size,
                                                     time_order=args.time_order)
train_pair = create_pair(train_user_list)

For the model input data, we use the DataLoader object from Pytorch library. DataLoader iterates through the dataset from the combination of a dataset and a sampler. DataLoader enables data to create in batches via arguments batch_size. The parameter determines how many samples per batch to load. There’s another num_worker parameter in the DataLoader class that assigns how many subprocesses to use for data loading.

对于模型输入数据,我们使用Pytorch库中的DataLoader对象。 DataLoader从数据集和采样器的组合遍历数据集。 DataLoader允许通过参数batch_size.批量创建数据batch_size. 该参数确定每批加载多少样品。 DataLoader类中还有另一个num_worker参数,用于分配要用于数据加载的子num_worker数。

造型 (Modelling)

class BPR(nn.Module):
    def __init__(self, user_size, item_size, dim, weight_decay):
        super().__init__()
        self.W = nn.Parameter(torch.empty(user_size, dim))
        self.H = nn.Parameter(torch.empty(item_size, dim))
        nn.init.xavier_normal_(self.W.data)
        nn.init.xavier_normal_(self.H.data)
        self.weight_decay = weight_decay


    def forward(self, u, i, j):
        """Return loss value.

        Args:
            u(torch.LongTensor): tensor stored user indexes. [batch_size,]
            i(torch.LongTensor): tensor stored item indexes which is prefered by user. [batch_size,]
            j(torch.LongTensor): tensor stored item indexes which is not prefered by user. [batch_size,]

        Returns:
            torch.FloatTensor
        """
        u = self.W[u, :]
        i = self.H[i, :]
        j = self.H[j, :]
        x_ui = torch.mul(u, i).sum(dim=1)
        x_uj = torch.mul(u, j).sum(dim=1)
        x_uij = x_ui - x_uj
        log_prob = F.logsigmoid(x_uij).sum()
        regularization = self.weight_decay * (u.norm(dim=1).pow(2).sum() + i.norm(dim=1).pow(2).sum() + j.norm(dim=1).pow(2).sum())
        return -log_prob + regularization


    def recommend(self, u):
        """Return recommended item list given users.

        Args:
            u(torch.LongTensor): tensor stored user indexes. [batch_size,]

        Returns:
            pred(torch.LongTensor): recommended item list sorted by preference. [batch_size, item_size]
        """
        u = self.W[u, :]
        x_ui = torch.mm(u, self.H.t())
        pred = torch.argsort(x_ui, dim=1)
        return pred

First, we create a user matrix (u) with user index, the item index (i) with all the items preferred by users, and item index (j) not preferred by users. Then, the matrix x_ui is generated with the multiplication of u and i, and sums up with each row. And, the matrix x_uj is generated with the multiplication of u and j, and sums up with each row. Matrix x_uij: matrix x_ui — matrix x_uj. The probability is rendered by the sigmoid function of the matrix x_uij and sums up the value. Regularization is added with the weight_decay parameter multiplied by the normalization of the user and item matrix and sums up the square function. The mathematical formula is produced below.

首先,我们创建一个具有用户索引的用户矩阵(u),具有用户喜欢的所有项目的项目索引(i)和用户不喜欢的项目索引(j)。 然后,将矩阵x_ui与u和i相乘,并与每一行求和。 并且,矩阵x_uj是通过u和j的乘积生成的,并与每一行求和。 矩阵x_uij:矩阵x_ui-矩阵x_uj。 概率由矩阵x_uij的S型函数表示,并对值求和。 使用weight_decay参数乘以用户和项目矩阵的归一化来添加正则化,然后对平方函数求和。 数学公式如下。

Image for post

Later on, the prediction outputs the highest probability of the recommended items and is stored in the list.

稍后,预测输出推荐项目的最高概率,并将其存储在列表中。

DataLoader和丢失的功能 (DataLoader and lost function)

# Create dataset, model, optimizer
dataset = TripletUniformPair(item_size, train_user_list, train_pair, True, args.n_epochs)
loader = DataLoader(dataset, batch_size=args.batch_size, num_workers=16)
model = BPR(user_size, item_size, args.dim, args.weight_decay)
optimizer = optim.Adam(model.parameters(), lr=args.lr)
writer = SummaryWriter()


# Training
smooth_loss = 0
idx = 0
f= open("loss.txt","w+")
g = open("precision.txt","w+")
k = open("recall.txt","w+")
for u, i, j in loader:
  optimizer.zero_grad()
  loss = model(u, i, j)
  loss.backward()
  optimizer.step()
  writer.add_scalar('train/loss', loss, idx)
  smooth_loss = smooth_loss*0.99 + loss*0.01
  if idx % args.print_every == (args.print_every - 1):
      print('loss: %.4f' % smooth_loss)
      f.write("%.4f\n" % (smooth_loss))
  if idx % args.eval_every == (args.eval_every - 1):
      plist, rlist = precision_and_recall_k(model.W.detach(),
                                              model.H.detach(),
                                              train_user_list,
                                              test_user_list,
                                              klist=[1, 5, 10])
      print('P@1: %.4f, P@5: %.4f P@10: %.4f, R@1: %.4f, R@5: %.4f, R@10: %.4f' % (plist[0], plist[1], plist[2], rlist[0], rlist[1], rlist[2]))
      writer.add_scalars('eval', {'P@1': plist[0],
                                              'P@5': plist[1],
                                              'P@10': plist[2]}, idx)
      writer.add_scalars('eval', {'R@1': rlist[0],
                                          'R@5': rlist[1],
                                          'R@10': rlist[2]}, idx)
      g.write("%.4f\n"%(plist[0]))
      g.write("%.4f\n"%(plist[1]))
      g.write("%.4f\n"%(plist[2]))
      k.write("%.4f\n"%(rlist[0]))
      k.write("%.4f\n"%(rlist[1]))
      k.write("%.4f\n"%(rlist[2]))
  if idx % args.save_every == (args.save_every - 1):
      dirname = os.path.dirname(os.path.abspath(args.model))
      os.makedirs(dirname, exist_ok=True)
      torch.save(model.state_dict(), args.model)
  idx += 1

We generate a DataLoader Class with a batch size of 4096 and assigns 16 numbers of workers. Since the dataset is quite large, we assign 16 subprocesses to parallel the data processing period. Adam optimizer is set with a learning rate of 0.0001. There’s stochastic gradient descent for the backward propagation to optimize the model. For each batch size, we record the loss, precision rate, and recall.

我们生成一个批量大小为4096的DataLoader类,并分配16个工作线程。 由于数据集非常大,因此我们分配了16个子流程来与数据处理周期平行。 Adam优化器的学习速率设置为0.0001。 反向传播存在随机梯度下降,以优化模型。 对于每个批次大小,我们记录损失,准确率和召回率。

结果 (Result)

There are two different datasets for model input. One is the random data with userid and itemid, and the other is the time-order dataset.

有两个用于模型输入的不同数据集。 一个是具有用户标识和项目标识的随机数据,另一个是时间顺序数据集。

The precision is produced below. The left-hand side is random data. From the left plot, the 1 item recommendation produces a better result than the 5 and 10 movies recommendation. From the right plot, the time order input dataset has a slightly worse result than the random data since the model is hard to catch the time sequence factor into the model.

精度在下面产生。 左侧是随机数据。 从左图可以看出,一项推荐比5和10部电影推荐产生更好的结果。 从右图可以看出,时间顺序输入数据集的结果比随机数据稍差,因为该模型很难将时间序列因子捕获到模型中。

Image for post
Image for post

未来的工作 : (Future work :)

  • Cope with the sparse matrix. From the data preprocessing method, filter the movies, not from the user’s favorite category.

    应对稀疏矩阵。 从数据预处理方法中,而不是从用户的收藏夹类别中筛选影片。
  • Add more features of users matrix-like users’ friend relationships, membership, click rate, etc.

    添加更多的用户功能,例如矩阵用户的朋友关系,成员资格,点击率等。
  • Incorporate the time factor into the model structure

    将时间因素纳入模型结构

结论 (In Conclusion)

  • Bayesian Personalized Ranking uses the likelihood function for p(i >u j|Θ) and the prior probability for the model parameter p(Θ). The Bayesian approach is to produce the rankings for all items i ∈ I to maximize the following posterior probability

    贝叶斯个性化排名将似然函数用于p(i> uj |Θ),并将先验概率用于模型参数p(Θ)。 贝叶斯方法是产生所有项i∈I的排名,以最大化以下后验概率
  • For the ranking task, we use LearnBPR to estimate whether a user prefers one item over another from matrix factorization. The optimized of the model would be achieved by calculating the gradient of ˆx_uij with respect to every model parameter θ.

    对于排名任务,我们使用LearnBPR来估计用户是否从矩阵分解中选择一项而不是另一项。 通过计算相对于每个模型参数θ的ˆx_uij的梯度,可以实现模型的优化。
  • K-nearest-neighbor method, derived from a similarity measure between either item (item-based) or users (user-based), generates the prediction of the recommend item i from the user u based on the past similarity of i to all other items the user has seen in the past of history data.

    从项目(基于项目)或用户(基于用户)之间的相似性度量得出的K最近邻方法,基于i与其他所有对象的过去相似性,从用户u生成推荐项目i的预测用户过去的历史数据中看到的项目。

翻译自: https://towardsdatascience.com/recommender-system-bayesian-personalized-ranking-from-implicit-feedback-78684bfcddf6

隐式反馈

更多推荐