AAAI 2021论文：利用深度元学习对城市销量进行预测

{"type":"doc","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"对于线上和线下的零售行业，销量预测都是一项至关重要的任务，它可以帮助企业更好的预备库存以及在各个仓库之间分配商品。特别是在大型购物节期间，强劲的促销活动将极大地促进消费。然而，可供参考的历史数据却非常稀缺。如何同时对城市的不同区域和不同时间段的销量进行预测，是一个非常具有挑战的问题。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在2020年12月收录的AAAI 2021（CCF-A类）上，京东城市被收录了一篇名为《Robust Spatio-Temporal Purchase Prediction via Deep Meta Learning》的论文。该论文研究了如何通过深度元学习，结合城市中的各项信息以及历史的销量数据，对未来，特别是大型购物节期间，城市中各个区域不同时间段的销量进行预测。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.infoq.cn\/resource\/image\/f8\/c6\/f8c606234021207ecc469b69911328c6.png","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"一、背景"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"购物中心和商店的空间分布、人口的分布属性以及不同的日期类型（例如，工作日，周末或购物节）等因素，将会导致不同的购物方式，如图1所示。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"由于来自某个特定区域以及特定时间范围内的数据过于稀缺，我们无法仅使用某个范围的数据去训练一个高精度的预测模型。同时，城市中不同区域的人口密度和分布特征高度异质，并且随着时间的推移区域特征也会发生变化，对这样复杂的时空购买模式进行建模是一个非常困难的任务。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"为了准确地捕获区域的空间特性，它需要考虑一组综合的空间特征，例如兴趣点（POI）分布，区域的人口统计特征等。如何通过跨时空建模学习提升单个区域在特定时间段的预测精度是我们的主要任务。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.infoq.cn\/resource\/image\/94\/88\/94d569a2ca8587395a7c3b9d6a6af088.png","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","text":"图1 空间分布"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"作者将该预测任务分为主要三个部分："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"1）利用时空元学习推理，对不同的时空区域生成对应的时空表示；"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"2）学习不同时空区域间的共享信息，同时结合区域特定的时空表示，得到更加可靠的预测销量；"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"3）通过时空交替训练使得模型在空间以及时间层面都能够得到更加完善的学习。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.infoq.cn\/resource\/image\/1e\/7e\/1eb9ac970bb8ec198d7aa56d25ecc37e.png","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","text":"图2 摊销网络"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"二、时空表示生成"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"我们开发了一个摊销网络（如图2 Amortization Network 所示），以对目标区域和日期类型的时空表示的近似后验分布进行建模，该模型需要同时考虑销量特征和相关空间特征的影响。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"从图3（a）中我们可以看到，以北京为例，不同城区的销量分布有着明显的区别。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"要完全捕获不同区域预测任务的购买数据的时间特性，还需要考虑历史销量数据之间的线性和非线性时间相关性，例如采购订单和购物车记录中所隐藏的周期模式。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"图3（b）中我们能够看到明显的以周为周期的销量模式变化。为了对这种复杂而动态的时间序列模式进行建模，我们使用了线性变换模块来获得销量的大致基数，并使用一个特殊的跳跃LSTM层来捕获销量变化的非线性时间模式。接着，通过特征融合模块将提取的时间特征与静态特征相结合，以进一步支持构建更具有代表性的销量时空表示。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.infoq.cn\/resource\/image\/83\/d0\/8339eb050c5ab72e8c8a8875a1d367d0.png","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","text":"图3 销量模式变化"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"三、时空销量预测"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"通过上一步，我们已经得到了特定时空区域的销量表示。接下来我们将介绍一种利用共享统计结构的生成模型，来学习不同时空预测任务间所共享信息。对于任务特定的预测，它大大提高了学习效率和预测准确性。该生成模型的结构可以参考上节所属的摊销网络结构的解码器（图2 Generative Model 所示）。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"它使用两个不同的输入来生成最终的销量。第一个输入侧重于通过对最近一段时间的输入销量，利用特征提取和融合技术来挖掘当前的购买表示，该特征描述了近期的销量模式的变化。第二个输入是从摊销网络中采样的特定于某个时空预测任务的时空特征表示。它捕获了目标任务的宏观时空模式。最后，生成模型使用完全连接的网络将两个输入合并，对目标时刻的销量订单进行预测。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"四、时空交替训练"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"不同的时空任务仅与来自特定区域和时间范围的有限数据相关联，无法完全描述特定区域的购买行为。受多视图学习的启发，我们构建了两种视图（空间视图和时间视图）来描述数据的不同观点。两个视图相互补充，以增强学习到的信息并获得更准确的时空数据表示。空间观点倾向于学习不同地区的购买模式，而时间观点倾向于挖掘不同时间段类型的时间变化模式。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"训练期间在不同视图之间进行切换可以在不同区域或日期类型之间学习到不同的时空信息。同时，在交替训练期间还能够整合不同时空任务共享的生成模型，并提高了预测准确性。完整的训练过程如算法1所示。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.infoq.cn\/resource\/image\/06\/8d\/0622cede5d86203573e9c495bb09828d.png","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","text":"算法1"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"五、实验"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"我们使用来自京东的海量高质量在线销量数据集来评估我们的模型。JD数据集包含时间跨度从2015年至2019年的采购订单，购物车数据（作为动态时间序列特征）和包括北京18个行政区块的区域特征（例如POI和人口统计资料数据）作为静态特征。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"为了验证模型的有效性以及鲁棒性，作者对不同时间区间（周内，周末，购物节等）的销量进行预测评估。同时还对模型中的不同组件进行消融分析，对不同组件的有效性进行评估，结果如表1所示。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.infoq.cn\/resource\/image\/67\/44\/6792a1363c0bd584e2b1yy60957b0544.png","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","text":"表1 不同组件的有效性"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"表1中我们可以看到，对于平日较为平缓和购物节的爆发性预测设置中，STMP均优于所有基线方法。在MSE方面，STMP均比最佳基准提高了至少30％。在爆发性销量预测场景中，STMP仍比基准降低约30％至60％。这些结果表明，当实际购买模式与平时截然不同时，我们的方法是有效且可靠的。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"为了进一步研究STMP各个模型组成部分的影响，我们还在表1中比较了STMP及其变体的性能。可以观察到，STMP-VI在总体和周末情景中表现良好，这些情景具有相对稳定的购买模式，但在购物节上的表现要比STMP差。这是因为STMP-VI使用点估计而不是特定于任务的时空表示的分布描述，当数据有限或模式高度不确定时，这会出现问题。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"此外，与STMP或其他变体相比，STMP-SKIP的准确性较差，但仍优于其他基准。这证明需要考虑购买时间序列中的多个周期性模式。这也反映在图4（b）中，我们可以在购买时间序列数据中看到清晰的周期性模式。这些证明对数据中的多个周期性属性进行建模在提高模型性能方面起着重要作用。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.infoq.cn\/resource\/image\/bf\/87\/bf3b3528e7318dbac1589ecb50356587.png","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","text":"图4"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"为了探究模型在不同区域以及时间段的效果，作者挑选了四个具有代表性的区域（覆盖不同销量范围，详细地理位置请参见图4（a））将STMP与基线方法在整个2019年的销量数据上做可视化，如图4所示。在大多数地区，基线无法很好地预测购物节期间购买时间序列的突发性，例如中旬促销和11.11的购买高峰。他们倾向于给出保守的结果，并且只专注于提高整体准确性。但是，STMP利用不同区域和日期类型的时空表示来支持多任务少拍学习，以促进对特定区域和日期类型的更好预测。可以观察到，只有STMP能够准确地预测爆发式的销量（所有图6中的峰值）。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.infoq.cn\/resource\/image\/52\/b3\/5283c16a0b3a91de6334ee4429f2c0b3.png","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","text":"图5"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"同时作者对于购物节历史数据的可参考情况进行了进一步的探究，在购物节销量数据非常稀缺的情况下，每增加一次购物节可见数据，精度会有一定的提升。然而即使在购物节可参考数据只有一次（持续大约一周）的情况下，STMP也具有较高的准确率，这表明了模型的鲁棒性。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.infoq.cn\/resource\/image\/20\/1c\/20a331f36bff9dce1a26543b35a1571c.png","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","text":"表2"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"最后，作者对时空交替训练方式在三个大型购物节上也做了比较。在时间训练中，合并操作将来自同一天类型的实例成批集成，而不考虑空间知识。同时，在空间训练中，模型将来自同一区域的一批实例集成在一起，而无需考虑时间信息。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"从表2中我们可以看到，即使不完整的T训练和S训练也比表1列出的基线提供更高的准确性。但是，与完整的时空交替训练相比，缺乏时空上的联合元知识仍然会导致较低的准确性。这表明不同的训练模式会跨地区和时间类型学习不同的转换。从时空方面的补充训练可以有效地提高预测的准确性。（点击阅读原文可下载完整论文）"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"文章转载自：京东数科技术说（ID：JDDTechTalk）"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"原文链接："},{"type":"link","attrs":{"href":"https:\/\/mp.weixin.qq.com\/s\/9XkWxcoMsQVoHXRL0Dj-Tg","title":"xxx","type":null},"content":[{"type":"text","text":"AAAI 2021论文：利用深度元学习对城市销量进行预测"}]}]}]}