ELECTRA模型跟BERT的区别

原創 勿在浮沙筑高台LS 2020-02-22 00:47

综合分析ELECTRA跟BERT模型的对比

思考：

1.类似GAN模型，但是由于梯度不能从判别模型到生成模型，所以只能综合两者的损失值对生成模型进行损失传递。

2.生成损失  对生成模型进行传导

   生成损失+判别损失 对判别模型进行传导

勿在浮沙筑高台LS
发布了358 篇原创文章 · 获赞 113 · 访问量 82万+
他的留言板 关注
發表評論
所有評論
還沒有人評論，想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章

Twitch表情中的情绪分析

{"type":"doc","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"typ

Martin Anderson 2021-12-07 16:00:03

达摩院AliceMind上新！首个中文表格预训练模型发布，已向业界开源

{"type":"doc","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"typ

刘燕 2021-12-02 18:18:58

在元宇宙里怎么交朋友？Meta发布跨语种交流语音模型，支持128种语言无障碍对话

{"type":"doc","content":[{"type":"blockquote","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null

凌敏 2021-11-23 14:03:53

人工智能时代，如何硬核玩音乐？| InfoQ《大咖说》

直播內容： 在人工智能技術迅速發展的當下，越來越多的領域被這項技術注入新的活力。作爲多媒體領域中不可缺少的組成部分，音樂對於人類的重要性不言而喻。值得一提的是，人工智能在音樂領域的研究早在多年前就已經開始了，並且也落地了很多成熟應用。 當前

InfoQ 中文站 2021-11-12 14:23:49

不是只有数字化水平高，才可以落地知识图谱

{"type":"doc","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"blockq

罗燕珊 2021-11-11 15:23:53

腾讯发布超大预训练系统派大星，聚焦解决BERT等超大模型训练时的“GPU内存墙”问题

{"type":"doc","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"typ

InfoQ编辑部 2021-11-02 13:38:53

微软和英伟达推出训练语言模型MT-NLG：5300亿参数量，是GPT-3的3倍

{"type":"doc","content":[{"type":"blockquote","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null

核子可乐 2021-10-12 14:13:53

谷歌推出Translatotron 2，一种没有深度伪造潜力的语音到语音直接翻译神经模型

{"type":"doc","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"typ

Amreen Bawa 2021-09-10 14:09:01

放心，GPT-3不会“杀死”编程

{"type":"doc","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragr

Alberto Romero 2021-09-03 17:58:55

为什么神经网络不适合理解自然语言 ？

{"type":"doc","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"typ

Ben Dickson 2021-08-04 16:13:54

易聊科技宣布在线客服系统IM永久免费，透视智能客服的商业化潜力

{"type":"doc","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"blockq

刘燕 2021-07-27 17:33:49

5个流行的自然语言处理库及入门用法

{"type":"doc","content":[{"type":"blockquote","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null

Matthew Mayo 2021-07-26 10:43:50

AI虚拟人多模态交互落地难题如何破解？我们在乐享A.I.技术沙龙成都站找到了答案

{"type":"doc","content":[{"type":"blockquote","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null

蔡芳芳 2021-06-24 16:18:54

官宣！达摩院开源秘藏深度语言模型体系AliceMind，NLP正在走向大工业时代

{"type":"doc","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"typ

刘燕 2021-06-22 14:48:49

让普通人秒会编程？微软在Power平台上集成GPT-3，将自然语言直接变成现成代码

{"type":"doc","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"typ

核子可乐 2021-05-28 17:48:57