NPM 私庫從搭建到數據遷移最後容災備份的一些解決方案

{"type":"doc","content":[{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"前言"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"按照國際慣例，正文開始之前，我們先簡單介紹下目前市面上的 "},{"type":"text","marks":[{"type":"strong"}],"text":"NPM 私庫開源框架"},{"type":"text","text":"。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Verdaccio"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Verdaccio 是 Sinopia 開源框架的一個分支。它提供了自己的小數據庫，以及代理其他註冊中心的能力（例如：npmjs.org 網站)，配置以及部署相對簡單，一步到\"胃\"。如果公司的私包比較少的話或者你想偷懶，可以考慮一下。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Cnpmjs.org"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"大名鼎鼎的 CNPM，想必各位早就感受到了它的速度之“快”，沒錯，它的 Register 服務就是"},{"type":"text","marks":[{"type":"strong"}],"text":"淘寶鏡像"},{"type":"text","text":" (https:\/\/registry.npm.taobao.org\/)。主要是基於 Koa、MySQL 和簡單存儲服務的企業專用 NPM 註冊和 WEB 服務，其中最強大的功能就是它的同步模塊機制（定時同步所有源 Registry 的模塊、只同步已經存在於數據庫的模塊、只同步 Popular 模塊）。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Nexus"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"後端開發的小夥伴應該比較熟悉。Nexus2 主要是用於 Maven\/Gralde 倉庫的統一管理，而 Nexus3 則添加了 NPM 插件，可以對 NPM 提供支持，其中 NPM 倉庫有三種類型，分別是 Hosted（私有倉庫）、Proxy（代理倉庫）、Group（組合倉庫）。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"總體來講，拋開 Nexus，雖然 "},{"type":"text","marks":[{"type":"strong"}],"text":"Cnpmjs.org"},{"type":"text","text":" 在部署過程以及總體設計方案上相對於 Verdaccio 複雜的多，但是它提供更高的拓展性，定製性，可以支持多種業務使用場景。接下來，我們分別從 "},{"type":"text","marks":[{"type":"strong"}],"text":"Cnpmjs.org 容器化部署"},{"type":"text","text":"、"},{"type":"text","marks":[{"type":"strong"}],"text":"數據遷移"},{"type":"text","text":"、"},{"type":"text","marks":[{"type":"strong"}],"text":"OSS 容災備份"},{"type":"text","text":"等內容，層層展開。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"Cnpmjs.org 容器化部署"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"目前，公司的應用部署都是容器化部署，內部搭建了 "},{"type":"text","marks":[{"type":"strong"}],"text":"Ipaas"},{"type":"text","text":" 平臺，應用流程化部署以及一鍵發佈。而 "},{"type":"text","marks":[{"type":"strong"}],"text":"Cnpmjs.org"},{"type":"text","text":" 也附帶了 "},{"type":"text","marks":[{"type":"strong"}],"text":"Dockerfile"},{"type":"text","text":" 以及 "},{"type":"text","marks":[{"type":"strong"}],"text":"docker-compose.yml"},{"type":"text","text":" 文件，所以，這裏大致講解下怎麼用 "},{"type":"text","marks":[{"type":"strong"}],"text":"Docker"},{"type":"text","text":" 部署吧。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"首先讓我們看看 "},{"type":"text","marks":[{"type":"strong"}],"text":"Dockerfile"},{"type":"text","text":" 文件"}]}]}]},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":"FROM node:12\nMAINTAINER zian [email protected]\n\n# Working enviroment\nENV \\\n    CNPM_DIR=\"\/var\/app\/cnpmjs.org\" \\\n    CNPM_DATA_DIR=\"\/var\/data\/cnpm_data\" \n\n# Shell 格式\n# 在 Docker Build 時運行\nRUN mkdir -p ${CNPM_DIR}\n\n# 指定工作目錄：用 WORKDIR 指定的工作目錄，會在構建鏡像的每一層中都存在\nWORKDIR ${CNPM_DIR}\n\n# 複製指令：從上下文目錄中複製目錄或文件到容器裏指定的路徑\nCOPY package.json ${CNPM_DIR}\n\nRUN npm set registry https:\/\/registry.npm.taobao.org\n\nRUN npm install --production\n\nCOPY .  ${CNPM_DIR}\nCOPY docs\/dockerize\/config.js  ${CNPM_DIR}\/config\/\n\n# 聲明端口（7001 爲 Register 服務、7002 爲 web 服務）\nEXPOSE 7001\/tcp 7002\/tcp\n\n# 匿名數據卷：在啓動容器時忘記掛載數據卷，會自動掛載到匿名卷。\nVOLUME [\"\/var\/data\/cnpm_data\"]\n\nRUN chmod +x ${CNPM_DIR}\/docker-entrypoint_prod.sh\n\n# Entrypoint \n# Exec 格式\n# 在 Docker Run 時運行\n# Dockerfile 存在多個 CMD 命令，僅最後一個生效\n# CMD [\"node\", \"dispatch.js\"]\nCMD [\"npm\", \"run\", \"prod\"]\n"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"這裏把 "},{"type":"text","marks":[{"type":"strong"}],"text":"CMD"},{"type":"text","text":" 命令修改爲 "},{"type":"codeinline","content":[{"type":"text","text":"[\"npm\", \"run\", \"prod\"]"}]},{"type":"text","text":"，因爲增加了一層不同環境的 "},{"type":"text","marks":[{"type":"strong"}],"text":"shell"},{"type":"text","text":" 腳本，目前全局變量全都存放在這裏。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"示例：docker-entrypoint_env.sh"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":"export DB='db_cnpmjs'\nexport DB_USRNAME='root'\nexport DB_PASSWORD='123456'\nexport DB_HOST='127.0.0.1'\n\nexport BINDING_HOST='0.0.0.0'\n\nDEBUG=cnpm* node dispatch.js \n"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"再修改下 "},{"type":"text","marks":[{"type":"strong"}],"text":"docker-compose.yml"},{"type":"text","text":" 文件，這裏把 "},{"type":"text","marks":[{"type":"strong"}],"text":"mysql-db"},{"type":"text","text":" 這個服務刪掉了，原因是可通過 "},{"type":"text","marks":[{"type":"strong"}],"text":"\/docs\/dockerize\/config.js"},{"type":"text","text":" 下的配置文件去連接公司測試環境的 "},{"type":"text","marks":[{"type":"strong"}],"text":"MySQL"},{"type":"text","text":" 數據庫，則不需要構建生成 "},{"type":"text","marks":[{"type":"strong"}],"text":"mysql-db 鏡像"},{"type":"text","text":"。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":"version: '3' # docker版本\nservices: # 配置的容器列表\n  web: # 自定義，服務名稱\n    build: # 基於 Dockerfile 構建鏡像(可增加 args )\n      context: .\n      dockerfile: Dockerfile ## 依賴的 Dockerfile 文件\n    image: cnpmjs.org # 鏡像名稱或 id\n    volumes:\n      - cnpm-files-volume:\/var\/data\/cnpm_data\n    ports:\n      - \"7001:7001\"\n      - \"7002:7002\" \n"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"注意點：1、全局配置文件路徑： \/docs\/dockerize\/config.js；2、bindingHost 爲 0.0.0.0。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"最後，在控制檯敲下 "},{"type":"codeinline","content":[{"type":"text","text":"docker-compose up -d"}]},{"type":"text","text":"，即以守護進程模式形式啓動應用，然後打開瀏覽器入 "},{"type":"codeinline","content":[{"type":"text","text":"http:\/\/127.0.0.1:7002"}]},{"type":"text","text":"，就會看到 WEB 頁面。執行 "},{"type":"codeinline","content":[{"type":"text","text":"npm config set registry http:\/\/127.0.0.1:7001"}]},{"type":"text","text":" 可設置爲搭建的私庫的鏡像源地址，這裏推薦使用 "},{"type":"codeinline","content":[{"type":"text","text":"nrm"}]},{"type":"text","text":"，可自由切換 NPM 源。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"展示站點如下圖："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/43\/43b4bb20d2dbb7fc86514d5e238d2c5e.webp","alt":"Image","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"注意點：1、當你改變本地代碼之後，先執行 docker-compose build 構建新的鏡像，然後執行 docker-compose up -d 取代運行中的容器。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"數據遷移"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"由於公司之前用的 "},{"type":"text","marks":[{"type":"strong"}],"text":"Verdaccio"},{"type":"text","text":" 搭建的私庫，要切換使用新的 "},{"type":"text","marks":[{"type":"strong"}],"text":"NPM 私庫"},{"type":"text","text":"，意味着要把之前發佈過的私包全部遷移過來。大概統計了下，有 400 多個 "},{"type":"text","marks":[{"type":"strong"}],"text":"Package"},{"type":"text","text":"，總共有  7000 多個版本，按照正常邏輯，做數據遷移首先會從數據庫下手，但是 "},{"type":"text","marks":[{"type":"strong"}],"text":"Verdaccio"},{"type":"text","text":" 並不依賴數據庫。剛開始沒有一點頭緒，大概看了下 "},{"type":"text","marks":[{"type":"strong"}],"text":"Cnpmjs.org"},{"type":"text","text":" 的源碼，分析了當我們 "},{"type":"text","marks":[{"type":"strong"}],"text":"publish"},{"type":"text","text":" 模塊時，它是怎麼把 "},{"type":"text","marks":[{"type":"strong"}],"text":"NPM 模塊"},{"type":"text","text":" 的元數據存儲到數據庫。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"通過路由文件（"},{"type":"codeinline","content":[{"type":"text","text":"\/routes\/registry.js"}]},{"type":"text","text":"）我們很容易找到 "},{"type":"codeinline","content":[{"type":"text","text":"\/controllers\/registry\/package\/save.js"}]},{"type":"text","text":"，這個文件便是我們想要的。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"核心代碼："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"javascript"},"content":[{"type":"text","text":"\nvar pkg = this.request.body; \/\/ 這裏拿到 npm 模塊元數據，即 package.json 文件經過 libnpmpublish模塊處理過的 Json 數據\nvar username = this.user.name; \/\/ 當前用戶名\nvar name = this.params.name || this.params[0]; \/\/ NPM 模塊名\nvar filename = Object.keys(pkg._attachments || {})[0]; \/\/ NPM 模塊的壓縮後的文件名\nvar version = Object.keys(pkg.versions || {})[0]; \/\/ NPM 模塊的最新版本\n\n"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"javascript"},"content":[{"type":"text","text":"\/\/ Upload Attachment\n\n\/\/ Base64 解碼，獲取模塊文件二進制數據。從 libnpmpublish 模塊瞭解到 tardata.toString('base64')，即NPM 模塊文件流轉 Base64 字符串\nvar tarballBuffer = Buffer.from(attachment.data, 'base64'); \n\/\/ 默認使用 fs-cnpm，將 NPM 模塊文件保存到本地，默認保存路徑：path.join(process.env.HOME, '.cnpmjs.org', 'nfs')\nvar uploadResult = yield nfs.uploadBuffer(tarballBuffer, options);\n\nvar versionPackage = pkg.versions[version];\nvar dist = {\n  shasum: shasum,\n  size: attachment.length\n};\n\n\/\/ If nfs upload return a key, record it\nif (uploadResult.url) {\n  dist.tarball = uploadResult.url;\n} else if (uploadResult.key) {\n  dist.key = uploadResult.key;\n  dist.tarball = uploadResult.key;\n}\nvar mod = {\n  name: name,\n  version: version,\n  author: username,\n  package: versionPackage\n};\n\nmod.package.dist = dist;\n\n\/\/ 模塊數據保存到數據庫\nvar addResult = yield packageService.saveModule(mod);\n"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"即只要我們能夠拿到 "},{"type":"text","marks":[{"type":"strong"}],"text":"NPM"},{"type":"text","text":" 模塊的元數據（即 package.json 被處理過的 JSON 數據），就能把模塊文件上傳到文件系統或者 "},{"type":"text","marks":[{"type":"strong"}],"text":"OSS"},{"type":"text","text":" 服務，同時數據落庫。"},{"type":"text","marks":[{"type":"strong"}],"text":"Verdaccio"},{"type":"text","text":" 有兩個 API 可以拿到其私庫 "},{"type":"text","marks":[{"type":"strong"}],"text":"NPM"},{"type":"text","text":" 模塊全量數據和當前 "},{"type":"text","marks":[{"type":"strong"}],"text":"NPM"},{"type":"text","text":" 模塊的 "},{"type":"text","marks":[{"type":"strong"}],"text":"JSON"},{"type":"text","text":" 數據，路徑分別是 "},{"type":"codeinline","content":[{"type":"text","text":"\/-\/verdaccio\/packages"}]},{"type":"text","text":"，"},{"type":"codeinline","content":[{"type":"text","text":"\/-\/verdaccio\/sidebar\/$PKG$"}]},{"type":"text","text":"，其中有 "},{"type":"text","marks":[{"type":"strong"}],"text":"scope"},{"type":"text","text":" 的模塊的請求路徑是 "},{"type":"codeinline","content":[{"type":"text","text":"\/-\/verdaccio\/sidebar\/$SCOPE$\/$PKG$"}]},{"type":"text","text":"。思路已經很明確了，開始動起來吧！新增 save_zcy.js 文件，基於原來的 "},{"type":"codeinline","content":[{"type":"text","text":"\/controllers\/registry\/package\/save.js"}]},{"type":"text","text":" 稍加改造下。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"核心代碼："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"javascript"},"content":[{"type":"text","text":"\/\/ 請求遠程文件，並返回二進制流\nconst handleFiles = function (url) {\n  return new Promise((resolve, reject) => {\n    try {\n      http.get(url, res => {\n        res.setEncoding('binary') \/\/ 二進制\n        let files = ''\n        res.on('data', chunk => { \/\/ 加載到內存\n          files += chunk\n        }).on('end', () => { \/\/ 加載完\n          resolve(files)\n        })\n      }) \n    } catch (error) {\n      reject(error)\n    }\n  })\n};\n\n\/\/ 獲取遠程模塊文件的二進制數據\nyield handleFiles(dist.tarball).then(res => {\n  \/\/ 利用 Buffer 轉爲對象\n  const tardata = Buffer.from(res, 'binary')\n  pkg._attachments = {};\n  pkg._attachments[filename] = {\n    'content_type': 'application\/octet-stream',\n    'data': tardata.toString('base64'), \/\/ 從緩衝區讀取數據，使用 base64 編碼並轉換成字符串\n    'length': tardata.length,\n  };\n}, error => {\n  this.status = 400;\n  this.body = {\n    error,\n    reason: error,\n  };\n  return;\n});\n"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"接下來我們把控制器 save_zcy.js 接入到 Registry 服務的 APP 路由上。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":"\/\/ 新增 fetchPackageZcy、savePackageZcy 控制器\napp.get('\/:name\/:version', syncByInstall, fetchPackageZcy, savePackageZcy, getOneVersion);\napp.get('\/:name', syncByInstall, fetchPackageZcy, savePackageZcy, listAllVersions);\n"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"控制器 fetchPackageZcy 作用是請求上面的 API（\/-\/verdaccio\/sidebar\/\/ 或 \/-\/verdaccio\/sidebar\/）來拉取對應模塊的 JSON 數據。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/8d\/8d99790bd63ec7bb72357a1cb7335b45.webp","alt":"Image","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"OK，接下來我們寫一個定時任務，每隔一段時間執行 "},{"type":"codeinline","content":[{"type":"text","text":"npm install [name]"}]},{"type":"text","text":"，這樣原來私庫的 "},{"type":"text","marks":[{"type":"strong"}],"text":"NPM"},{"type":"text","text":" 包都能夠 "},{"type":"text","marks":[{"type":"strong"}],"text":"install"},{"type":"text","text":" 並進入到上面的控制器邏輯，大功告成！"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"OSS 容災備份"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"首先，簡單說明下爲什麼要做 "},{"type":"text","marks":[{"type":"strong"}],"text":"OSS 容災備份"},{"type":"text","text":"，有以下幾點。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如果服務器上磁盤損壞，易丟失文件，有一定的風險"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"若服務器磁盤爆滿，可自動降級上傳模塊文件到 "},{"type":"text","marks":[{"type":"strong"}],"text":"OSS"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"基於以上幾點，我們整理了下容災備份方案："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"package publish"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/30\/30d4735575fb9cfadd536f856d8b6c3e.png","alt":"Image","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"即發佈模塊文件時本地存儲，同時上傳到 "},{"type":"text","marks":[{"type":"strong"}],"text":"OSS"},{"type":"text","text":" 作爲備份，用到的插件分別是 "},{"type":"text","marks":[{"type":"strong"}],"text":"fs-cnpm"},{"type":"text","text":"、"},{"type":"text","marks":[{"type":"strong"}],"text":"oss-cnpm"},{"type":"text","text":"。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"package install"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/75\/7565ba2ebec66c0b62f065c6dd49e7b8.webp","alt":"Image","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"即下載模塊文件時，先判斷是否是私包（即包名是否有帶  "},{"type":"text","marks":[{"type":"strong"}],"text":"scope"},{"type":"text","text":" ），如果不是私包代理到上游 "},{"type":"text","marks":[{"type":"strong"}],"text":"Registry"},{"type":"text","text":"，若是私包先判斷服務器本地是否有該私包文件，如果不存在先去 "},{"type":"text","marks":[{"type":"strong"}],"text":"OSS"},{"type":"text","text":" 下載到本地 "},{"type":"text","marks":[{"type":"strong"}],"text":"nfs"},{"type":"text","text":" 目錄下，如果存在則直接從 "},{"type":"text","marks":[{"type":"strong"}],"text":"nfs"},{"type":"text","text":" 目錄找到模塊文件，然後讀取並寫到 "},{"type":"text","marks":[{"type":"strong"}],"text":"downloads"},{"type":"text","text":" 目錄下，最後調用 "},{"type":"text","marks":[{"type":"strong"}],"text":"fs.createReadStream"},{"type":"text","text":" 方法流讀取該文件。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"isEnsureFileExists"},{"type":"text","text":" 即判斷模塊文件本地是否存在，代碼如下："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"typescript"},"content":[{"type":"text","text":"const mkdirp = require('mkdirp');\nconst fs = require('fs');\n\nfunction ensureFileExists(filepath) {\n  return function (callback) {\n    fs.access(filepath, fs.constants.F_OK, callback);\n  };\n}\n"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"注意，在 "},{"type":"text","marks":[{"type":"strong"}],"text":"OSS"},{"type":"text","text":" 下載模塊文件到 "},{"type":"text","marks":[{"type":"strong"}],"text":"nfs"},{"type":"text","text":" 之前，一定要先創建模塊文件目錄，方法如下："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"javascript"},"content":[{"type":"text","text":"const mkdirp = require('mkdirp');\n\nfunction ensureDirExists(filepath) {\n  return function (callback) {\n    mkdirp(path.dirname(filepath), callback);\n  };\n}"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"郵件通知"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"Cnpmjs.org"},{"type":"text","text":" 本來就帶有郵件通知的功能，但只應用錯誤日誌上報。由於我們的私包大部分都是業務組件、工具等，有時候發佈正式版本的業務組件需要通知到業務組件的使用方。目前，我們採用 "},{"type":"text","marks":[{"type":"strong"}],"text":"Maintainers"},{"type":"text","text":" 來維護，包含模塊的維護者及使用者。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"示例："}]},{"type":"codeblock","attrs":{"lang":"javascript"},"content":[{"type":"text","text":"\"maintainers\": [\n  {\n    \"name\": \"yuanzhian\",\n    \"email\": \"[email protected]\"\n  }\n]\n"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"郵箱配置如下："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"javascript"},"content":[{"type":"text","text":"mail: {\n  enable: true,\n  appname: 'cnpmjs.org',\n  from: process.env.EMAIL_HOST,\n  host: 'smtp.mxhichina.com',\n  service: 'qiye.aliyun', \/\/ 使用了內置傳輸發送郵件,查看支持列表：https:\/\/nodemailer.com\/smtp\/well-known\/\n  port: 465, \/\/ SMTP 端口\n  secureConnection: true, \/\/ 使用了 SSL\n  auth: {\n     user: process.env.EMAIL_HOST,\n     pass: process.env.EMAIL_PSD, \/\/ \n   }\n }\n"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"寫在文末"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"未來，我們還可以在 "},{"type":"text","marks":[{"type":"strong"}],"text":"Cnpmjs.org"},{"type":"text","text":" 上做很多定製化開發，比如"},{"type":"text","marks":[{"type":"strong"}],"text":"接入公司內部權限系統"},{"type":"text","text":"、"},{"type":"text","marks":[{"type":"strong"}],"text":"WEB 頁面重構"},{"type":"text","text":"、"},{"type":"text","marks":[{"type":"strong"}],"text":"對接業務組件在線文檔"},{"type":"text","text":"等等。如果你正好也需要搭建 "},{"type":"text","marks":[{"type":"strong"}],"text":"NPM"},{"type":"text","text":" 私有庫，希望這篇文章對你有所幫助。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"horizontalrule"},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"頭圖：Unsplash"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"作者：梓安"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"原文：https:\/\/mp.weixin.qq.com\/s\/NUYooqqTklsSs77VDRLwKA"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"原文：NPM 私庫從搭建到數據遷移最後容災備份的一些解決方案"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"來源：政採雲前端團隊 - 微信公衆號 [ID：Zoo-Team]"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"轉載：著作權歸作者所有。商業轉載請聯繫作者獲得授權，非商業轉載請註明出處。"}]}]}