Source Map在前端监控中的应用和实践

{"type":"doc","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Web前端开发中，对于生产环境的代码通常会进行压缩和混淆处理，以减小代码体积并提高源代码安全性。然而当生产环境有JS错误发生时，压缩和混淆的代码也让定位错误变得更困难。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"我们先来看一个爱奇艺智能前端异常监控平台（鹰眼Hawkeye）检测到的错误日志：​","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/bd/bd35746d0278e8fd2d81e3671f67811b.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","text":"图一 Hawkeye 收集的JS错误示例","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"可以看到，压缩后的错误行号和列号已经丢失，函数名也经过了变换，出错的真实文件名很难确定。如此，根据压缩代码的报错信息是很难定位错误的，但是，线上代码又必须压缩上传处理。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如何平衡？SourceMap正是解决这个矛盾的利器。本文将从Source Map的基本概念及其在前端异常监控中的运用、设计及实践等方面进行介绍。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"一、Source Map介绍","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Source Map是一种存储了源代码和编译后代码映射关系的信息文件，包含代码转换前后的位置信息。现在各种主流的打包工具都支持生成Source Map,例如: Uglify JS、Grunt、Gulp、Webpack等。生成Source Map一般在项目打包的编译阶段进行，打包工具先将源代码解析出AST（Abstract Syntax Tree），在生成编译后的代码时，即图二generate过程中，利用AST中节点的来源信息等生成Source Map。","attrs":{}}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/51/51190662571d81aceb8fdde9b0de54b7.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","text":"图二 编译并生成Source Map","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"二、JS错误采集和利用Source Map解析错误","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"JS错误一般可以通过window.addEventListener监听error事件收集，框架错误一般利用框架暴露的钩子函数收集，比如Vue.config.errorHandler。收集到的错误中，从Error.prototype.stack可解析出错误堆栈信息。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"获取到错误堆栈信息、Source Map文件内容后则可以通过Mozilla的source-map库进行错误定位。该库中的SourceMapConsumer 实例表示一个已解析的源映射的相关信息，我们可以通过为这个实例提供文件位置和文件内容来在生成的源中查询有关原始文件位置的信息。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"目前，各大监控平台均有针对错误监控的Source Map的解析功能。例如开源监控平台Sentry支持Webpack插件和自身的CLI工具上传map文件后进行解析，付费平台Fundebug支持通过前端UI、自身CLI及接口上传map文件后进行解析。但是这些平台也有一些缺点，比如有的不支持使用者选择文件进行关联匹配，有的在使用者选择map匹配不成功时场景交互体验不友好。我们受之启发，开发了基于鹰眼监控系统的Source Map功能，主要有以下优势：","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"支持更简单的配置来通过Webpack的Plugin进行Source Map文件的上传，并且上传过程出现异常不影响项目整体编译流程，代码侵入性低。","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"支持更多的上传和选择Source Map文件的方式，以适用于更多的业务错误监控的场景，且在使用过程中有更明确的提示和更友好的交互。","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"支持map文件管理功能，通过项目名称和版本号选择关联，新版本可以选择历史版本的map文件进行匹配。","attrs":{}}]}]}],"attrs":{}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"下面对Source Map在监控系统中的运用、鹰眼Source Map功能设计及实践详细阐述。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"三、Source Map在监控系统的运用","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"将Source Map运用于前端异常监控系统，主要需要解决的问题便是使用者如何上传Source Map以及监控系统如何将Source Map和错误信息关联。考虑到易用性,鹰眼SourceMap功能支持三种使用方式：针对单个JS报错堆栈从手动上传Source Map、集成插件自动上传关联JS错误堆栈信息、针对单个JS报错堆栈从SourceMap列表中选择关联。如图三所示。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/ce/ce818623a36d542b91e1da9c03db45ab.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","text":"图三 Hawkeye Source Map三大功能","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":"br"}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"相应的，整体架构如下：","attrs":{}}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/e3/e35da58ff28aa5c9169619b6dee2dc5b.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","text":"图四 整体架构","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"以下是对各个模块的介绍：","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Source Map自动上传插件(支持Webpack3+)：使用者在项目打包流程中集成此插件，便可将生成的Source Map自动上传。此插件能够在不影响项目编译流程的前提下，在Webpack输出 asset 到 output目录之后，获取打包生成的map文件并执行上传逻辑。并且可以通过参数配置可以选择在Webpack编译完成执行后对map文件进行删除，不影响上线流程。鉴于历史项目使用Webpack老版本较多，鹰眼针对Webpack不同的版本进行了兼容优化。","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"手动上传Source Map模块：使用者可针对单条JS错误，上传单个Source Map进行匹配，并且进行了同名检测的提示，在能够自助上传map外也能通过提示减少错误匹配的情况。","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Source Map自助选择模块：提供了可以通过Source Map管理中心进行选择map进行关联的功能，对于不能每次每个版本都能上传Source Map文件的项目，能够进行列表和搜索选择需要匹配的map文件。","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Source Map定位展示模块：展示JS报错对应的真实源代码及位置。","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Source Map管理中心：集中管理Source Map文件的添加删除。","attrs":{}}]}]}],"attrs":{}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"上面提到的三种使用方式是互补的关系，其中集成插件自动上传关联JS错误堆栈信息的方式是非常建议使用者接入的，其优势在于可方便地集成到项目的自动化构建发布流程。接入一次之后，收到JS报错时，便可查看此报错在源代码的位置。实现流程如图五示:","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"SourceMap自动上传插件拉取项目打包生成的Source Map文件。","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"调用文件上传中心接口上传文件。文件上传中心有独立的鉴权机制、对接了网络安全服务。","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"获取SourceMap文件存储信息后连同项目、版本提交至Source Map管理中心。提交完成后可以删除构建过程中产生的Source Map文件。","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"使用者在前端异常监控系统查看JS报错时，监控系统根据JS报错的项目、版本号从SourceMap管理中心获取Source Map文件。","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"调用Mozillasource-map库进行解析关联，对源代码信息进行可视化展示。","attrs":{}}]}]}],"attrs":{}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/59/59503b6cadd2607df198807f2e2752e4.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","text":"图五 Hawkeye Source Map自动上传插件流程设计","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"四、项目中接入","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在项目中接入Source Map自动上传插件的方法及使用技巧如下：","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"初始化监控平台接入代码时需要携带versionNum字段，该字段为当前项目的版本号，以便于后期错误堆栈和项目map文件进行版本管理处理。","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在项目打包时生成map文件（需要独立的map文件）。Webpack对于Source Map的生成配置，建议模式使用hidden-source-map，其能够生成完整独立的map文件且不会泄露sourceMappingURL，但缺点是会造成编译耗时增加。cheap-source-map和cheap-module-source-map因省略一些内容，并不会准确完整的还原错误位置。","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"打包部署项目时集成map上传插件, 在devDependencies中安装插件，在项目中的Webpack的plugin配置中增加该插件配置。","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"若接入监控平台时正确传入版本号和项目名称，并且在项目打包时通过map上传插件上传正确版本及项目名称的map文件。鹰眼在发生错误上报堆栈信息时会自动将JS名称和map文件进行匹配，显示其错误正确的源码地址。","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"若无法自动匹配时，也可以从Source Map管理中心选择该项目和版本对应的map文件进行匹配，或者手动上传map文件自行匹配。","attrs":{}}]}]}],"attrs":{}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"在鹰眼监控后台的使用方式如图六标注示意：","attrs":{}}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/5c/5c5172d338b70631e1340fc984991369.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","text":"图六 Hawkeye Source Map功能接入示例","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"五、总结与展望","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"我们再看开篇提到的错误，在鹰眼管理系统中Source Map视图下，可以很方便地看到真实文件名和代码行号了，如图七。由于可以直接定位到JS出错的源代码位置，接入了此功能的项目，JS错误定位排查效率得到了极大的提升。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/1d/1d6faa1f8c535c926adc36a11758c386.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","text":"图七 鹰眼JS错误SourceMap视图","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":"br"}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在一些项目生成Source Map以及集成上传Source Map插件的实践中，我们也遇到过一些难点：","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"1、在一些采用自定义Webpack配置打包的项目中，按照官方常规的devtool:hidden-source-map配置总会使得一些打包生成的JS文件体积增大，除了单独生成的Source Map文件，在一些编译后的JS中混入了一些SourceMap的内容。我们通过详细查看这些体积增大的JS文件的特点和内容，最终对vue-loader、css-loader分别指定CSS的Source Map选项关闭解决问题。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"2、微前端项目的Source Map关联问题。在一些微前端架构项目中，微前端独立上线，有独立的版本控制，而采集报错的Hawkeye JSSDK只需集成在基座项目，微前端打包上线的流程中无法获取基座的版本，会造成SourceMap文件无法自动关联匹配。我们采取一种比较简洁的方式，在采集JSSDK中支持传入微前端版本的函数，在基座项目中根据微前端的标识等计算获取微前端的版本，当采集到报错投递时，将微前端的版本一同投递。这样后台关联时，优先匹配微前端版本号，其次匹配基座版本号，即可成功自动关联。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"3、因为准确的定位错误需要上传完整的独立map文件，在个别较大的项目中，打包生成Source Map文件时，会影响编译速度，比较坏的情况下会增加80%的编译时间。所以后续考虑根据这些项目自身情况，独立封装和优化SourceMap生成模块，而不是通过指定打包工具的选项执行其内置生成流程，尽量减少对项目整体打包耗时的影响。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"另外，根据接入项目的情况，鹰眼Source Map自动上传插件后续也会支持更多打包构建工具，比如rollup、gulp。","attrs":{}}]}]}