基於Apache APISIX，新浪微博API網關的定製化開發之路

{"type":"doc","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"微博之前的 HTTP API 網關基於 NGINX 搭建，所有路由規則存放在 NGINX conf 配置文件中，帶來一系列問題：升級步驟長，對服務增、刪、改或跟蹤問題時，不夠靈活且難以排查問題。經過一番調研之後，我們選擇了最接近預期、基於雲原生的微服務 API 網關：Apache APISIX，藉助其動態、高效、穩定等特性以滿足業務的快速響應要求。"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"1 背景說明"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在微博，運維同學要創建一個 API 服務，他需要先在 nginx conf 配置文件裏面寫好，提交到 git 代碼倉庫裏面去，等其它負責上線的運維同學 CheckOut 之後，確認審覈成功，才能它們推送部署到線上去，繼而粗暴通知 NGINX 重新加載，這纔算服務變更成功。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"整個處理流程較長，效率較低，無法滿足低代碼化的 DevOps 運維趨勢。因此，我們期待有一個管理後臺入口，運維同學在 UI 界面上就可以操作所有的 http api 路由等配置。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/wechat\/images\/44\/4441456040626fee1cfc7db125032d75.png","alt":null,"title":null,"style":null,"href":null,"fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"經過一番調研之後，我們選擇了最接近期盼的基於雲原生的微服務 API 網關：Apache APISIX。比較看重的點，有這麼幾個："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"numberedlist","attrs":{"start":1,"normalizeStart":1},"content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":1,"align":null,"origin":null},"content":[{"type":"text","text":"基於 NGINX ，技術棧前後統一，後期灰度升級、安全、穩定性等有保障；"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":2,"align":null,"origin":null},"content":[{"type":"text","text":"內置統一控制面，多臺代理服務統一管理；"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":3,"align":null,"origin":null},"content":[{"type":"text","text":"動態 API 調用，即可完成常見資源的修改實時生效，相比傳統 NGINX 配置 + reload 方式進步明顯；"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":4,"align":null,"origin":null},"content":[{"type":"text","text":"路由選項豐富，滿足微博路由需求；"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":5,"align":null,"origin":null},"content":[{"type":"text","text":"較好擴展性，支持 Consul kv 難度不大；"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":6,"align":null,"origin":null},"content":[{"type":"text","text":"性能表現也不錯。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/wechat\/images\/db\/db5f50e9430ac51c7d9efcb650e3a45a.png","alt":null,"title":null,"style":null,"href":null,"fromPaste":false,"pastePass":false}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"2 爲什麼選擇定製化開發？"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"實際業務情況下，我們是沒辦法直接使用 Apache APISIX 的，原因有以下幾點："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"numberedlist","attrs":{"start":1,"normalizeStart":1},"content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":1,"align":null,"origin":null},"content":[{"type":"text","text":"Apache APISIX 不支持 SaaS 多租戶，實際需要運維的業務線上層應用有很多，每個業務線的開發或運維同學只需要管理維護自己的各種 rules、upstreams 等規則，彼此之間不相關聯；"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":2,"align":null,"origin":null},"content":[{"type":"text","text":"當把路由規則發佈到線上後，如果出現問題則需要快速的回滾支持；"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":3,"align":null,"origin":null},"content":[{"type":"text","text":"當新建或者編輯現有的路由規則時，我們不太放心直接發佈到線上，這時就需要它能夠支持灰度發佈到指定網關實例上，用於仿真或局部測試；"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":4,"align":null,"origin":null},"content":[{"type":"text","text":"需要 API 網關能夠支持 Consul KV 方式的服務註冊和發現機制；"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"上述這些需求目前 Apache APISIX 都沒有內置支持，所以只能通過定製開發才能讓 Apache APISIX 真正在微博內部使用起來。"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"3 在 Apache APISIX 的控制面，我們改了些什麼？"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"我們定製開發時，使用的 Apache APISIX 1.5 版本，Dashboard 也是和 1.5 相匹配的。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"定製開發的目標簡單清晰，即完全零代碼、UI 化，所有七層 HTTP API 服務的創建、編輯、更新、上下線等所有行爲都必須在 Dashboard 上面完成。因此實際環境下，我們禁止開發和運維同學直接調用 APISIX Admin API，假如略過 Dashboard 直接調用 APISIX Admin API，就會導致網關操作沒辦法在 UI 層面上審計，無法走工作流，自然也就沒有多少安全性可言。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"有一種情況稍微特殊，運維需要調用 API 完成服務的批量導入等，可以調用 H5 Dashboard 的 API 來完成，從而遵守統一的工作流。"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"3.1 支持 SaaS 化服務"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"企業層面有完整的產品線、業務線數據庫，每一個具體產品線、業務線可使用一個 saas_id 值表示。然後在創建網關配置數據插入 ETCD 之前，塞入一個 saas_id 值，在邏輯屬性上所有的數據便有了 SaaS 歸屬。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"用戶、角色和實際操作的產品線就有了如下對應關聯："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/wechat\/images\/3a\/3a019e4c544900874b35ec9ff91d2cc8.png","alt":null,"title":null,"style":null,"href":null,"fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"一個用戶可以被分派承擔不同運維角色去管理維護不同的產品線服務。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"管理員角色很好理解，操作維護服務核心角色，針對服務增 \/ 刪 \/ 改 \/ 查等；除此之外，我們還有隻讀用戶的概念，只讀用戶一般是用於查看服務配置、查看工作流、調試等等。"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"3.2 新增路由發佈審覈工作流"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/wechat\/images\/7d\/7d06a7e8840874e4818b340832db638e.png","alt":null,"title":null,"style":null,"href":null,"fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在開源版本中，創建或修改完一個路由之後就可以直接發佈。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"而在我們的定製版本中，路由創建或修改之後，還需要經過審覈工作流處理之後才能發佈，處理流程雖然有所拉長，但我們認爲在企業層面審覈授權後的發佈行爲才更可信。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/wechat\/images\/c7\/c7221dc5dc0de0866582eb77c5ef66c0.png","alt":null,"title":null,"style":null,"href":null,"fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在創建路由規則時，默認情況下必須要經過審覈。爲了兼顧效率，新服務錄入的時候，可選擇免審、快速發佈通道，直接點擊發布按鈕。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/wechat\/images\/4f\/4faf645f3d6694db32baaafac0b82203.png","alt":null,"title":null,"style":null,"href":null,"fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"當一個重要 API 路由某次調整規則發佈上線後出現問題時，可以選擇該路由規則上一個版本進行快速回滾，粒度爲單個路由的回滾，不會影響到其它路由規則。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"單條路由回滾內部處理流程如下圖示。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/wechat\/images\/65\/65f8c40f7e00b1f1fa5c90bbfb7a580c.png","alt":null,"title":null,"style":null,"href":null,"fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"我們需要爲單個路由的每次發佈建立版本數據庫存儲。這樣我們在審覈之後進行全量發佈，每發佈一次會就會產生一個版本號，以及對應的完整配置數據；然後版本列表越積越多。當我們需要回滾的時候去版本列表裏面選擇一個對應版本回滾即可；某種意義上來講，回滾其實是一個特殊形式的全量發佈。"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"3.3 支持灰度發佈"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"我們定製開發的灰度發佈功能和一般社區理解的灰度發佈有所不同，相比全量部署的風險有所降低。當某一個路由規則的變更較大時，我們可以選擇只在特定有限數量的網關實例上發佈並生效，而不是在所有網關實例上發佈生效，從而縮小發布範圍，降低風險，快速試錯。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"雖然灰度發佈是一個低頻的行爲，但和全量發佈之間依然存在狀態的轉換。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/wechat\/images\/4e\/4e84f2e6cc079abe045ec960fcefd582.png","alt":null,"title":null,"style":null,"href":null,"fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"當灰度發佈的佔比減少到 0% 的時候，就是全量發佈的狀態；灰度發佈上升到 100% 的情況下，就是下一次的全量發佈，這就是它的狀態轉換。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/wechat\/images\/ef\/ef9ab94899616371ed0deb61faf862e2.png","alt":null,"title":null,"style":null,"href":null,"fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"上圖爲在操作灰度發佈選擇具體的網關實例時的截圖。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"灰度發佈完整功能除了管理後臺支持，還需要在網關實例上暴露出一些 API 支持。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/wechat\/images\/bc\/bc36b568a8595aafbe5b65e0b1a71530.png","alt":null,"title":null,"style":null,"href":null,"fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"灰度發佈 API 固定 URI, 統一路徑爲 \/admin\/services\/gray\/{SAAS_ID}\/routes。不同 HTTP Method 呈現不同業務含義，POST 表示創建，停止灰度是 DELETE，查看就是 GET。"}]},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"3.3.1 啓動流程"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/wechat\/images\/cc\/cca6ae79dc04508321c9f8af8eba0074.png","alt":null,"title":null,"style":null,"href":null,"fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"從網關層面發佈一個 API，接收數據後 worker 進程校驗發送來的數據的合法性，合法數據會通過事件廣播給所有的 worker 進程。隨後調用灰度發佈 API ，添加完灰度規則，在下一個請求被處理時生效。"}]},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"3.3.2 停用流程"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/wechat\/images\/3c\/3cfb6993ffd068f1f210e04a7fe2d9b5.png","alt":null,"title":null,"style":null,"href":null,"fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"停用流程和灰度分佈流程基本一致，通過 DELETE 的方法調用灰度發佈的 API，廣播給所有的 work 進程，每個 work 接收到需要停用的灰度的 ID 值後在 route 表裏進行檢測。若 route 表裏存在就刪除，然後嘗試從 ETCD 裏面還原出來。如果灰度停用了，要保證原先存在 ETCD 能夠還原出來，不能影響到正常服務。"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"3.4 支持快速導入"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"除了在管理頁面支持創建路由之外，很多運維同學還是比較習慣使用腳本導入。我們有大量的 HTTP API 服務，這些服務要是一個一個手動錄入，會非常耗時。如果通過腳本導入，則能夠降低很多服務遷移阻力。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"通過爲管理後臺暴露出 Go Impport HTTP API，運維同學可以在現成的 Bash Script 腳本文件中填寫分配的 token、SaaS ID 以及相關的 UID 等，從而較爲快速地導入服務到管理後臺中。導入服務後續操作依然還是需要在管理後臺 H5 界面上完成。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/wechat\/images\/3e\/3e77fd2f71e973a6bebe66649450aed1.png","alt":null,"title":null,"style":null,"href":null,"fromPaste":false,"pastePass":false}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"4 在 Apache APISIX 的數據面，我們改了些什麼？"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"基於 Apache APISIX 數據面定製開發需要遵循一些代碼路徑規則。其中，Apache APISIX 網關的代碼和定製代碼分開存放不同路徑，兩者協同工作，各自可獨立迭代。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/wechat\/images\/85\/85889d7cfdc0f5bb69f404178eb12668.png","alt":null,"title":null,"style":null,"href":null,"fromPaste":false,"pastePass":false}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"4.1 安裝包的修改"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"因此打包時，不但有定製代碼，還需要把依賴、配置等全部打包到一起進行分發。至於輸出的格式，要麼選擇 Docker，要麼打到一個 tar 包中，按需選擇。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/wechat\/images\/14\/14475a37e3cb1a4384f14759870e9cb7.png","alt":null,"title":null,"style":null,"href":null,"fromPaste":false,"pastePass":false}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"4.2 代碼的定製開發"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"有些定製模塊需要在被初始化時優先加載，這樣對 Apache APISIX 的代碼入侵就變得很小，只需要修改 NGINX.conf 文件。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/wechat\/images\/05\/05fdcdd9778b938badad17ff4bf44b13.png","alt":null,"title":null,"style":null,"href":null,"fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"比如，需要爲 upstream 對象塞入一個 saas_id 屬性字段，可以在初始化時調用如下方法。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/wechat\/images\/63\/633f2a04edce5b5891ac5c7e24f57243.png","alt":null,"title":null,"style":null,"href":null,"fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"類似修改等，需要在 init_worker_by_lua_* 階段完成調用，完成初始化。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"另外一種情況：如何直接重寫當前已有模塊的實現。比如有一個 debug 模塊，現在需要對它的初始化邏輯進行重構，即對 init_worker 函數進行重寫。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/wechat\/images\/69\/694b02ef3ed38f1dba783faa8b28880b.png","alt":null,"title":null,"style":null,"href":null,"fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"這種方式的好處在於，既能保證 API 原始的物理文件不動，又能加入自定義的 API 具體邏輯的重寫，從而降低了後期代碼管理的成本，也爲後續升級帶來很大的便利。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在生產環境下若有類似需求，可以參考以上做法。"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"4.3 支持 Consul KV 方式服務發現"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"當前微博很多服務採用 Consul KV 的方式作爲服務註冊和發現機制。以前 Apache APISIX 是不支持 Consul KV 方式服務發現機制的，就需要在網關層添加一個 consul_kv.lua 模塊，同時也需要在管理後臺提供 UI 界面支持，如下："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/wechat\/images\/ce\/ce7006930479103f51cd5e73773d7ef0.png","alt":null,"title":null,"style":null,"href":null,"fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在控制檯中的 upstream 列表中，填寫的所有東西一目瞭然，鼠標移動到註冊服務地址上，就會自動呈現所有註冊節點的元數據，極大方便了運維同學日常操作。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/wechat\/images\/50\/50506f2a376bdeb858833a844e8fd649.png","alt":null,"title":null,"style":null,"href":null,"fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"consul_kv.lua 模塊在網關層的配置方式較爲簡單，同時支持多個不同 Consul 集羣連接，當然這也是實際環境要求使然。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/wechat\/images\/04\/049770f96851bc1b573ba08ea55380d6.png","alt":null,"title":null,"style":null,"href":null,"fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"目前這部分代碼已被合併到 APISIX master 分支中，２.４版本已包含。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"該模塊的進程模型採用訂閱發佈模式，每一個網關實例有且只有一個進程去長連接輪詢多個的 Consul 服務集羣，一旦有了新數據就會一一廣播分發到所有業務子進程。"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"5 定製化過程中的一些思考"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"5.1 遷移成本高"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在運維層面其實面臨一個問題，就是遷移成本的問題。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"任何一個新事物出現，用於替換現有的基礎，都不會一馬平川，而是需要經過一段時間慢慢熟悉、增進認知，然後不停試錯，慢慢向前推進，逐漸消除大家心中的各種疑惑。只有在穩定運行一段時間，各種問題都得到解決之後，纔會進入下一步較爲快速的替換階段。毫無疑問，當前 APISIX 在微博的使用還處於逐步推進的階段，我們還在不斷熟悉、學習並深入瞭解，同時解決各種各樣的遷移問題，以期找到最佳實踐路徑。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"舉一個例子，在遷移過程中，需要一一將 nginx.conf 文件中的各種上游以及路由等規則導入到網關係統管理後臺中，這是一個非常枯燥的手動操作過程，因此我們開發了快速導入接口，提供 bash script 腳本一鍵錄入等功能來簡化這個過程。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/wechat\/images\/94\/9405397748d8b67a548426ab5196b9b3.png","alt":null,"title":null,"style":null,"href":null,"fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"同時我們也會遇到 NGINX 各種複雜變量判斷語句，目前主要是發現一個解決一個，不斷積累經驗。"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"5.２定製化程度高，導致後續升級成本較高"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"無論是 Apache APISIX 還是 Apache APISIX Dashboard，我們都做了很多定製化開發。這導致升級最新版 Dashboard 較爲困難。不過，我們定製開發所選擇的 Dashboard 版本基本功能都已具備，滿足日常用途倒也足夠了。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"針對 Apache APISIX 的定製開發，則可以很輕鬆地升級，目前已經完成了若干個小版本的升級。"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"5.３ 反哺社區"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"最後我們聊聊社區，我們也在想怎麼把社區感興趣的功能反哺給 Apache APISIX 社區，讓大家一起使用和修改。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"我們進行定製開發的驅動力主要來自微博內部的實際需求，與 Apache APISIX 社區推動的演進有一些出入，這是客觀存在的事實。但除去一些包含敏感數據的代碼，企業和社區總會在一些比較通用的功能代碼層面存在共同需求，企業和開源社區可以一起推動使之進化得更爲穩定成熟。比如通用的 Consul KV 服務發現模塊、一些高可用配置文件的處理，以及其它問題的修復等。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"這些共同需求一般會在企業內部打磨一段時間，直到完全滿足內部需求之後，再逐步提交到社區開源代碼分支裏，但這也需要一個過程。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"作者"},{"type":"text","marks":[{"type":"strong"},{"type":"strong"},{"type":"strong"}],"text":"介紹"},{"type":"text","marks":[{"type":"strong"}],"text":"："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"聶永，微博基礎架構師，開源愛好者， QCon 大會講師，愛折騰追求 Geek。"}]}]}