基礎軟件創業 8 年，如何精準“踩點”？ ｜ C位面對面

{"type":"doc","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"“2017 年之前，數據大多在本地；2017 年之後，越來越多的人開始在雲上使用大數據。轉年，我們發佈了數據雲的第一個版本。”"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"“2020 年，Snowflake 上市，股價曾一度飆漲到每股 400 多美金，越來越多的人開始注意到這家公司的數據雲平臺。此時，我們的數據雲平臺已經迭代了幾個版本，並均付諸實踐。”"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在此之後，我們看到越來越多的大數據廠商開始在數據雲領域有所嘗試，Snowflake、Databricks 這樣的廠商受到的大衆和資本關注越來越多。此時的國內市場，星環科技已經默默開啓耕耘數據雲平臺的第三個年頭。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"純技術創業這件事兒本身是很難的，這麼多年來，技術風口一直在變，創業者得始終保持着敏銳度纔不至於走向落敗。"},{"type":"text","text":"而風口這東西一旦出現，想追的人可太多了，預判纔有機會殺出重圍。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"在大數據領域創業 12 年，眼見計算引擎經歷了數輪迭代，雲原生興起砸中了一批做容器的，數據雲因爲 Snowflake 上市走入大衆認知，回回都能在風口到來前兩三年做好佈局，這是一種什麼判斷和體驗？"},{"type":"text","text":"本期 C 位面對面，我們邀請到了星環科技創始人兼 CEO 孫元浩，二度做客 InfoQ 的 C 位面對面，這位在基礎軟件領域創業八年，靠着精準踩點扛過大數據領域數次革新的技術人有哪些心得想要分享呢？"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"embedcomp","attrs":{"type":"video","data":{"id":"434477","name":"星環科技C位面對面","poster":"https:\/\/static001.infoq.cn\/resource\/image\/d5\/7e\/d534c7190df8c6yy2250670ebcde7c7e.jpg","url":"https:\/\/media001.geekbang.org\/99bfffd10f35404c8a1385e6135ef46d\/50662ff48dd3444ea5c7aa39b4c7c5c0-b239299ba62ee4d8e85e536753227605-sd.m3u8"}}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"大數據時代的初顛覆，從計算引擎開始"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在大數據的發展歷史上，Hadoop、Spark 絕對是不容忽視的存在。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如果你問開發者：“這些大數據引擎或框架怎麼樣？”，十年前和現在得到的答案可能完全不同。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"十年前，數據大多存儲在本地，選擇雲服務的廠商寥寥無幾；十年後，雲計算的到來讓數據雲開始走進企業。計算引擎在這個過程中經歷了數輪迭代，應用形式也越來越豐富。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"孫元浩整個團隊從 2008 年就開始對大數據市場做調研，2009 年投入研發。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"彼時，國內的雲計算建設剛剛興起，雲廠商開始嘗試在各地建立數據中心，將數據進行集中管理，此時還沒有人會料到十年之後將迎來數據雲的時代，但大家已經發現需要一個高效的數據處理引擎，需求由此誕生。2012 年，國內對大數據需求出現大規模井噴，孫元浩的創業之路由此正式開啓。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"創業之初，整個團隊就已經意識到了現有方案的侷限性，而業界對此達成共識已經是三年後了。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"“從市場機會來看，2013 年其實都有點晚了，運營商、金融機構那時已經對大數據需求明顯增大；從技術本身來看，傳統的集中式計算已經很難在性能上有所提升，必須橫向擴展，我們當時就判斷未來的 10 年將會是分佈式系統的天下，從底層操作系統到上層數據庫、中間件、分析軟件、應用軟件都將據此發生重構。”"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"既然意識到了侷限性，也對即將發生的變化有所預判，孫元浩帶着整個團隊開啓了重構之路。團隊提出的第一個目標是希望可以用一個計算引擎處理不同的數據量，那時主流的廠商還在推混合架構，孫元浩團隊則基於 Spark 的早期版本重新開發了計算引擎。隨後幾年，團隊又摒棄了 HDFS、YARN 等框架，重新構建了分佈式存儲管理系統、資源管理器、編譯器等，實現了大幅重構。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"“我們不是爲了替代而替代，而是爲了實現現有框架無法解決的問題，並附加更多功能，比如數據一致性、兼容 Oracle、DB2 等。”"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"當業內在 2015 年對現有方案的侷限性達成共識並尋找替代方案時，星環科技的 TDH 很自然就進入了公衆視野。隨後幾年，TDH 每年都會進行一次大的版本更新。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/wechat\/images\/23\/2341817903a333432bca0bf8bb053717.png","alt":null,"title":null,"style":null,"href":null,"fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"重構大數據引擎讓星環科技在國內大數據發展初期佔到了先機，但遠未到終點，雲計算的到來爲大數據行業又帶來了新的變化。"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"雲來了，大數據有了新的革新"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"“當所有人希望在雲上使用大數據時忽然發現原有框架設計模式中存儲計算高度耦合，但從雲原生的角度來看，存儲和計算資源完全獨立存儲纔有希望達到極致性價比。”"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"當時，Hadoop面臨諸多爭議，市面上唱衰聲音不斷。很多分析文章將這種“頹勢”與雲計算的發展結合了起來，雲計算帶來了數據處理的新方法，同時讓存儲變得更加廉價，且用戶體驗獲得了成倍提升。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在這個時期，星環科技轉而用容器化實現更好的資源隔離，並推出了大數據雲平臺 TDC，引入了 DevOps 等雲原生最佳實踐技術，可以將數據和業務直接打包成微服務對外提供。基於 Kubernetes 框架，實現了調度層、網絡層和存儲層的統一，可支撐所有大數據計算引擎。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/wechat\/images\/6b\/6b9ff1c72d204ae0ee42a5334d837e3e.png","alt":null,"title":null,"style":null,"href":null,"fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"然而，這些變化在孫元浩看來都不是革命性的，雲廠商基於推廣存儲產品的目的可能會在宣傳時有一定誇大，從技術本身來看，存儲計算的解耦從 Spark 時期就開始了，這些改變只是帶來了使用模式的變化。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"走到這一步，星環科技已經實現了創業初期制定的兩個大目標："},{"type":"text","marks":[{"type":"strong"}],"text":"提供一站式的平臺替代混合架構和處理多種數據模型"},{"type":"text","text":"，並朝着第三個目標繼續前進，也就是讓"},{"type":"text","marks":[{"type":"strong"}],"text":"數據形成生態，用戶在這個巨大的數據商城中實現服務共享。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"因此，數據雲出現了。"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"國外有Snowflake，國內有星環科技"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"2020 年 9 月，Snowflake 正式登陸納斯達克，首日 IPO 籌資高達 33.6 億美元，是有記錄以來金額最大的軟件 IPO。在 Oracle、Teredata、開源軟件 Hadoop、雲廠商 AWS、Google 等巨頭林立的大數據市場，Snowflake 的脫穎而出引發了圈內的重要關注，而市場很快發現 Snowflake 得以成功的關鍵點之一是其打造的企業數據雲平臺。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在數據雲的概念走入大衆視野的兩年前，星環科技已經發布了數據雲的第一個版本，目標就是將數據、應用、分析等生態化，用一個平臺解決所有問題，從技術角度來看，這也是未來的發展方向之一。那麼，“數據雲”到底應該如何理解呢？"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"雖然名字一樣，但孫元浩認爲國內外所構建的“數據雲”不盡相同。"},{"type":"text","text":"從市場趨勢來看，企業通常會把數據進行高度集中，這些數據不僅開放給某個團隊，也會面向整個集團用戶、分公司及供應商，客戶起初可能會爲了開發一個應用買一個大數據平臺，但一段時間之後就會發現這種方式的單點創新不夠，因爲平臺建設越來越多，所以大家逐漸過渡到了集中化建設，這種方式帶來的問題是不同負載要求的計算資源和存儲資源不同，因此需要獨立擴展，擁有云計算彈性擴縮容的特點；軟件棧可以保證層之間的解耦合，每層均獨立擴展；由於使用對象較多需要保證資源之間的良好隔離，互不影響；數據生態化，應用開發需要數據支持，數據集中後，應用也會跟着過來。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"星環數據雲基於雲原生、大數據和 AI 技術構建，通過統一的雲操作系統層支持異構的 CPU 和操作系統，既可運行於私有云、公有云和混合雲等各種 IaaS 之上，也可脫離開 IaaS，直接運行在物理機上，能夠以多租戶和服務的方式提供對數據庫、數據倉庫、數據湖、數據工程、數據科學家和數據應用開發等多種數據工作負載的支撐，同時提供低代碼平臺、DevOps、微服務治理等能力，加速企業數據和應用上雲。"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"怎麼上數據雲？"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在技術架構上，數據雲與過去採用 Oracle 等數據庫的方式還是不同的，原有方式的物力資源是固定的，個別客戶此前將大數據平臺搬到了私有云上面，這會對數據雲的部署帶來一定阻力，因爲已經有了一部分原始投資，這類用戶常見的部署方式是新建數據雲平臺，用來做公司和集團內部的數據共享，定位爲服務平臺的入口，相較於容器技術，虛擬化會造成百分之三十至百分之四十的性能損失。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"當然，也有少數企業會選擇直接基於系統進行建設，將分散在各地的數據進行大集中，這種方式帶來的首要變化就是數據安全得到了加強，原來數據分散在各地，一旦發生數據泄露很難追溯。在《數據安全法》頒佈實施之後，企業對數據安全尤爲重視，數據雲平臺將數據進行集中管理，極大加強數據安全。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"“數據安全可從技術和管理兩個層面進行加強。從技術角度來看，星環科技的數據雲平臺增加了授權、訪問控制、審計、數據追溯等功能，如對外開放數據可事先進行數據脫敏，加強隱私保護，在數據中添加噪音，使用隱私計算和同太加密等技術確保數據在密鑰狀態下被使用，這方面其實誕生了一系列新技術，包括聯邦學習等。從管理角度來看，用戶逐漸意識到需要建設一個專門的團隊對數據進行管理，數據雲平臺是一個龐大的系統，不僅需要建設也需要運營。”"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"星環科技從 TDH 到數據雲，大數據的整體趨勢從Hadoop、Spark等大數據平臺到基於雲原生的數據雲，越來越多的商業軟件在大數據領域發揮着重要作用，開源領域卻再難出現明星級的開源項目，這是爲什麼呢？"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"爲什麼越來越難看到大數據領域出現明星級開源項目？"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"大數據領域再無明星級開源項目出現究其原因是好項目需要的研發投入越來越大。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"大數據領域發展到現在，除非新系統可以達到比現有方案快一個數量級以上纔會吸引大家的注意，這就需要巨大的研發投入，很難靠個人或者單個團隊來完成，這就造成了最近幾年開源軟件很難跟商業軟件進行 PK，雖然也有類似 ClickHouse 這樣引發了一陣關注的計算引擎出現，但其功能非常受限，稍微複雜的 OLAP 分析就會導致性能下降。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"這給商業軟件帶來了機會，也在一定程度上讓大衆對這類基礎設施的要求逐年升高。創業八年，星環科技如何完成自身的逐級突破呢？"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"基礎軟件創業 8 年，如何精準“踩點”？"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"2015 年，業界對 Hadoop 的侷限性達成共識；2013 年，星環科技就已經開始佈局自己的大數據平臺 TDH。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"2017 年，業界開始關注雲原生；2015 年，星環科技就在鼓搗自己的 TDC 大數據雲平臺。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"2017 年，AI 技術變得非常火熱；2013 年，星環科技就推出了自己的數據開發與智能分析平臺。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"2020 年，業內因爲 Snowflake 的上市而開始關注數據雲；2018 年，星環科技推出了自己的數據雲平臺。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/wechat\/images\/c0\/c0b8d188f1ee11ce00c07c66df90537d.png","alt":null,"title":null,"style":null,"href":null,"fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"類似提前佈局的案例在星環科技創業的八年中還有很多，孫元浩似乎並未特別注意這一點。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"“之前經常有人開玩笑說星環科技融了那麼多錢都花到哪裏了，我們大部分都用在了技術研發上，我們的 TDH 平臺八年實現了八個大版本的更新。我們本身是一家開發基礎軟件爲生的公司，我們必須要有自己的獨特性，不能抄襲，並需要在市場中具備競爭優勢，雖然用戶可能不會關心你具體的實現過程，但對我們而言，只有原創纔可以形成差異化競爭優勢，才能夠更優雅得解決用戶問題。”"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如果一定要總結原因，孫元浩認爲就是兩點：市場需求和技術驅動。這兩點大家都懂，做得好卻不容易。技術上，星環科技一直在追求更優美的架構，更高的性能，從而推動了產品的逐年更新；市場上，無非是洞察客戶需求，如果你收集到的客戶需求足夠多，就會很容易分辨出其中的共性需求和個性需求，從而更好地指導產品開發。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"“這兩者可能有聯繫，也可能沒有。過往多年，我看到了很多商業上成功的案例，大多集中在商業模式創業上，這種方式還是可以帶來收益的，但已經接近天花板，下一個階段可能只有靠基礎技術上的硬碰硬才能持續走下去，真正在商業上獲得成功。”"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"“我相信國內的基礎軟件市場在未來 20 年一定會發生變化，國產軟件一定可以異軍突起，佔據一定的市場空間”。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"八年前就是抱着這樣的信念，孫元浩帶着幾個人衝進了這片市場，因爲對技術原創性的追求而不斷革新自己的產品，技術上漸入佳境之後又開始投入基礎軟件人才培養的大軍中。作爲一家純粹的基礎軟件領域的技術創業公司，星環科技的發展無疑是快速的，但面對鏡頭，孫元浩很實在地表示：“我們其實也有嘗試失敗的時候，只是你們沒看到而已。正是經過了這些或失敗或成功的嘗試，我們對客戶和市場纔有了更深入的理解，同時保持對技術的敏感性，在市場和技術的雙重驅動下，最終做出了原創的、創新性的開發”。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}}]}