1. 背景知識
Alogrand團隊Gorbunov等人2020年論文《Pointproofs: Aggregating Proofs for Multiple Vector Commitments》,配套的代碼實現參見:https://github.com/algorand/pointproofs
在該論文中,實現了:
- Pointproofs —— a new vector commitment scheme that supports non-interactive aggregation of proofs across multiple commitments。允許任何第三方 aggregate a collection of proofs with respect to different, independently computed commitments into a single proof represented by an elliptic curve point of 48-bytes。
- 將Pointproofs用於blockchain smart contract。相比於之前的vector commitment方案,Pointproofs可將傳輸一個區塊交易所需的帶寬開銷降低至少60%。
- 以單線程運行時,generate a proof for 8 values with respect to one commitment的時間爲0.08s,aggregate 4000 such proofs across multiple commitments into one proof的時間爲0.25s,verify the aggregated proof的時間爲23s(0.7ms per value proven)。
Vector commitment可用於減少存儲空間:instead of storing a vector of values, one can store only the commitment and receive the values together with their proofs as needed。
Vector commitment可讓application 在storage of all value和 bandwidth taken up by revealed values and proofs 之間進行取捨平衡。
爲了在減少存儲空間的同時儘可能減少帶寬,需要 reduce the proof size。但是,由於需要滿足cryptographically hard to forge的要求,單個proof的size cannot be reduced too far。改進的方式可爲:
- 在單個proof中支持reveal multiple values。最短的單個proof size實現可參見Russell W. F. Lai 和 Giulio Malavolta 在Crypto 2019上發表的論文《Subvector Commitments with Application to Succinct Arguments》中5.2節構建的subvector commitment from Cube Diffie-Hellman Assumption:a proof takes up only 48 bytes (for typical parameter values at conjectured 128-bit security) regardless of how many elements of the vector are being revealed。(參見博客 Subvector Commitments with Application to Succinct Arguments學習筆記)
- 在分佈式應用中,存在大量來源不同的commitments/values/proofs,它們相互 not aware of each other’s data。此時存在以下兩個問題:
1)不存在可produce a single proof for all the values的單一實體;
2)proofs需要於多個不同的commitments關聯。
Pointproofs可有效解決以上問題:a user can independently commit to her variables and provide short proofs for any subset of them; any third party can non-interactively aggregate multiple proofs with respect to different commitments into a single short proof。
Boneh等人2019年論文《Batching techniques for accumulators with applications to IOPs and stateless blockchains》可實現dynamic aggregation for proofs in a single (the same) commitment —— aggregate proofs for elements of a vector into a single proof for a subvector。(參見博客 密碼學累加器cryptographic accumulator)
而在本論文中,Gorbunov等人實現了跨多個commitments的aggregate proofs。
具體的各方案對比爲:
在本論文中,Gorbunov等人的主要貢獻爲:
-
實現了proofs for individual elements of a single vector commitment can be aggregated by any third party into a single proof for the subvector;
-
實現了proofs for subvectors of multiple commitments can be aggregated by any third party into a single proof for the multiple subvectors。
-
在實現aggregation的同時,也提供了hiding屬性。
-
在Libert和Yung 2010年論文《Concise mercurial vector commitments and independent zero-knowledge sets with short proofs》構建的vector commitment基礎上,增加了same-commitment aggregation和cross-commitment aggregation,從而實現了Pointproofs。
1)與此類似,Pointproofs的public parameter size is linear in the size of the committed vector(可將long vector切分爲多個短的vectors,多個短的vectors的proofs可以aggregate,但是commitments不能aggregate,從而縮短了public parameter size,但是增加了total size of the commitments);
2)與此類似,Pointproofs也基於-type assumption。In order to prove security of aggregation, we have to work in the algebraic group model and the random oracle model. We can reduce these assumptions by lowering efficiency and/or security requirements. -
Pointproofs生成的proof爲single point on a pairing-friendly curve (48 bytes at 128-bit security),無論是single value for a single commitment,subvector of values for a single commitment,還是set of subvectors for multiple commitments。
-
Pointproofs中實現了支持aggregation的hiding屬性,僅需增加an additional exponentiation,commitment size和proof size均無需增加。而Dario Catalano 和 Dario Fiore 2013年論文《Vector Commitments and their Applications》中提到的給Vector commitment加hiding屬性的方法爲:add an inner layer of hiding commitments to individual values —— first commit to each message separately using a standard commitment scheme, then apply the VC to the obtained sequence of commitments。但是該方式無法automatically extend to aggregatable vector commitments,因爲proofs for the inner layer are not automatically aggregatable。
-
Pointproofs可用於reduce storage requirements for blockchains。主要針對smart contract智能合約場景。假設一個智能合約有多個變量,所有變量當前值 are committed to a single vector commitment ,每個智能合約有一個commitment。
爲了與智能合約交互,one provides a 480byte proof $ \hat{\pi}$ of the current values of the variables needed for the transaction,transaction成功執行後可能會更新這些變量值。當存在多個智能合約時,cross-commitment aggregation允許compress multiple proofs into a single 48-byte proof 。從而可從根本上消除the bandwidth overhead due to proofs in a proposed block。
將Pointproofs用於智能合約存儲時,針對千萬級accounts,可將validators’ storage requirements降爲4.5GB,assuming one open value per transaction 的情況下,僅需增加31KB per block overhead for 1000 transactions。 -
Pointproofs 代碼實現https://github.com/algorand/pointproofs 中的實際性能表現爲:針對a commitment for 1000 variables of a smart contract at 128-bit security level,生成任意subvector proof的時間爲54~123ms;a block proposer對所有proofs進行cross-commitment aggregate的時間約爲0.07ms per proof;存儲了commitments 的 validator verify the aggregated proofs in a block的時間約爲 0.7~1.9ms per value verified;爲表示變量值的更新(通過交易執行),需要update commitments的時間約爲0.2~0.3ms per variable updated。
cross-commitment aggregation of proofs可用於很多場景,如:
- signature aggregation:compress multiple signatures produced by different users into a short signature。如Jae Hyun Ahn等人2010年論文《Synchronized aggregate signatures: new definitions, constructions and applications》中介紹的sensor networks,KyleBrogle等人2012年論文《Sequential aggregate signatures with lazy verification from trapdoor permutations - (extended abstract)》中介紹的internet routing以及Drijvers等人2020年論文《Pixel: Multi-signatures for consensus》中介紹的POS (Proof-of-Stake) 共識。Aggregating commitment proofs is a natural counterpart to aggregating signatures。
- 多個用戶或實體分別獨立commit to their databases of records(如public keys, healthcare records, transactions等),然後同時produce proofs to reveal several committed values。在這種場景下,cross-commitment aggregation可用於減少帶寬。
vector commitment的相關工作:
- 正式定義了vector commitments:Libert和Yung 2010年論文《Concise mercurial vector commitments and independent zero-knowledge sets with short proofs》,以及Dario Catalano 和 Dario Fiore 2013年論文《Vector Commitments and their Applications》。
- 實現了constant-size proofs for a subvector of values:Kate等人2010年論文《Constant-size commitments to polynomials and their applications》,以及Thakur 2019年論文《Batching non-membership proofs with bilinear accumulators》。
但是Kate等人2010年論文《Constant-size commitments to polynomials and their applications》第3.4節定義的binding notion is not strong enough to preclude openings to two inconsistent subvectors。
而Libert和Yung 2010年論文《Concise mercurial vector commitments and independent zero-knowledge sets with short proofs》, Dario Catalano 和 Dario Fiore 2013年論文《Vector Commitments and their Applications》,Benoˆıt Libert, Somindu C. Ramanna 和 Moti Yung 2016年論文 《Functional Commitment Schemes: From Polynomial Commitments to Pairing-Based Accumulators from Simple Assumptions》,以及Chepurnoy等人2018年論文《Edrax: A cryptocurrency with stateless transaction validation》,這些論文中的vector commitment無法實現constant-size proofs for multiple values。 - pairing-based vector commitments:Dario Catalano 和 Dario Fiore 2013年論文《Vector Commitments and their Applications》,Benoˆıt Libert, Somindu C. Ramanna 和 Moti Yung 2016年論文 《Functional Commitment Schemes: From Polynomial Commitments to Pairing-Based Accumulators from Simple Assumptions》,以及Russell W. F. Lai 和 Giulio Malavolta 在Crypto 2019上發表的論文《Subvector Commitments with Application to Succinct Arguments》。
- polynomial commitments:始於Kate等人2010年論文《Constant-size commitments to polynomials and their applications》,overview信息可參看Benedikt B¨unz等人2019年論文《Proofs for inner pairing products and applications》。
在Boneh等人2020年論文《Efficient polynomial commitment schemes for multiple points and polynomials》中實現了polynomial commitments with batch opening和vector commitments with aggregation,但是其效率要低於本論文實現。
1.1 一些定義
-
Notation:
-
The Algebraic Group Model(AGM) :即adversary輸出的group element值應基於其收到的group element進行有效的group operation計算得出的,而不是隨意創造的。
Suppose adversary is given group elements . Then, for every group element that the adversary outputs, it must also ouput such that . -
security assumption:在bilinear pairing group中求解-wBDHE(weak bilinear Diffie-Hellman exponent problem)很難,即對任意的已知
求解很難。
對於BLS12-381 pairing-friendly curve with ,當前best attack has complexity 。 -
The Random Oracle Model(ROM):本文的security proofs are in the random oracle model。在本文model a cryptographic hash function as a truly random function, accessible to all parties only via oracle queries。本文使用了兩個random oracles 和,both with output space 。
2. vector commitment
採用與Libert和Yung 2010年論文《Concise mercurial vector commitments and independent zero-knowledge sets with short proofs》類似的思路,基於非對稱bilinear pairing group,相應的實現細節爲:
-
Setup: Let be a group of prime order ,along with pairing and generators for respectively. Let be a secret value (known to no one after the initial generation of public parameters). The pulic parameters are given by values in , values in , and one value in (該值易於計算獲得,如):【注意不應包含在public parameters中,否則Prover可僞造證明。】
-
Commit:對vector ,
-
Prove:reveal ,
-
Verify:
2.1 支持aggregation的vector commitment思路集錦
爲了實現reveal multiple values (其中) for a single commitment via a very short proof 。
-
思路一:
直接計算,然後驗證。
該方式不安全,若open ,可commit to 而open爲,違反了binding屬性(只bound to ,而不是中的每一個值。)。
同時,還需要防止inconsistnent reveals for possibly two different sets,如分別open 爲,爲的情況是不允許的。 -
思路二:實現same-commitment aggregation
在verification方程式中引入“隨機”scalars ,
aggregated proof
scalars 的值可通過applying a hash function on some carefully chosen inputs depending on 。類似的思路在Boneh等人2018年論文《Compact multi-signatures for smaller blockchains》的aggregating signatures中有提及。
怎樣選擇來保證binding屬性呢?若爲indeed random,則可保證,即對同一位置open爲兩個不同值的概率可忽略。
可將hash function 看成是a random oracle。同時,還需要restrict the adversary to the so-called algebraic group model,以便可express adversarially generated commitments in terms of public parameters。 -
思路三:實現cross-commitment aggregation
對多個不同的vector進行commit,第個vector 可表示爲 ,對應的commitment爲 ,對set 的open proof爲,則滿足:
若直接將多個vector對應的verification equation都一起相乘,則有:
與思路一類似,上述方式是不安全的,需要再引入額外的random scalars ,相應的aggregated proof爲,對應的verification equation調整爲:
2.2 Same-commitment aggregation
首先考慮的是aggregation of proofs for a single commitment。
基本的算法包括Setup
, Commit
, UpdateCommit
, Aggregate
, Verify
:
- :輸出public parameters,支持的vector 長度爲。
- :輸入爲vector 和randomness ,輸出爲commitment 。
- :輸入爲commitment ,待更新的位置集,將待更新位置集內數據由更新爲後,對應新的commitment 。
- :open位置對應的proof,輸入爲待open位置和,輸出爲proof 。【應該還有一個輸出,對應open位置的具體值】
- :輸入爲commitment ,open位置集,每個位置對應的proof ,輸出爲aggregated proof 。
- :輸入爲commitment ,open位置集,open信息,aggregated proof ,輸出爲,0表示拒絕,1表示接受。
本論文中,爲了具有通用性,定義的Verify
算法總是針對的aggregated proof,哪怕僅僅open了1個位置。同時,上述定義是調用多次Prove
算法生成單個位置的proof然後調用一次Aggregated
算法生成aggregated proof,可能存在其它算法可直接生成aggregated proof從而提升performance,但是並不影響定義。上述定義中,若commitment updated了,需調用Prove
重新生成新的proof,可能存在效率更高的updateproof算法直接update existing proof。(如博客 Vector Commitments and their Applications學習筆記 中提到的ProofUpdate
算法)
整個流程應關注如下屬性:
- Correctness of opening。即保證正確的proof aggregated後可100%驗證通過。
- Correctness of updates。即保證對老的commitment進行update操作的輸出值應與直接多新的vector進行commit的輸出值一致。【有個typo?】
- Binding。即對同一commitment,若open不同的位置集合,應保證集合之間的交集應具有一致性,防止inconsistnent reveals for possibly two different sets。【具體見論文4.3節證明——Proof of binding for same-commitment aggregation。】
具體的實現可爲:
上述具體實現中,可從如下維度優化:
- Setup可通過多方安全計算或private communication等方式來高效安全實現。
- 假設表示the number of non-zero entries in the vector ,則
Commit
運算需要次 exponentiation計算,Prove
比Commit
運算少1次。同時,可通過Pippenger等算法,計算products of exponentiations的效率要高於分別計算exponentiations。 - 實際計算same-commitment aggregated proof 時,若 is known inadvance,則可不用單獨先多次調用
Prove
生成再Aggregate
,可將兩個算法合併直接生成aggregated proof,效率更高:
- Verify時,域內的運算效率>>,計算,將域內的運算轉移到域內,Verify的公式變更爲:
而a product of two pairings can be computed considerably faster than two separate pairings (because the time-consuming final exponentiation needs to be performed only once)。
同時,當,即只open 1個位置時,可設置,這樣可將域內的運算轉移到域內,即上述公式第一項可爲。 - 當,即只open 1個位置時,可設置,相當於不需要執行
Aggregate
運算,直接設置。
具體各算法環節的運算複雜度爲:
2.3 Cross-commitment aggregation
考慮的是aggregation of proofs across commitments。
在Same-commitment aggregation的基礎上,增加了兩組算法AggregateAcross
和VerifyAcross
:
- :輸入爲 組 和相應的same-commitment-aggregated proofs (通過上面的
Aggregate
算法獲得),輸出爲跨commitment的aggregated proof 。 - :輸入爲 組 和cross-commitment-aggregated proof ,用於驗證 is a commitment to a message vector consistent with for all 。
與Same-commitment aggregation類似,也需要滿足Correctness of opening屬性。
Cross-commitment aggregation的binding屬性以實際場景舉例:【具體見論文4.4節證明——Proof of binding for cross-commitment aggregation。】
存在兩組commitments,第一組(第一次)有個不同的commitments,如包含了vectors ;第二組(第二次)有個不同的commitments,如包含了vectors 。
所謂的Cross-commitment aggregation的binding屬性是指,若第一次cross-commitment-aggregated proof 和 第二次的均VerifyAcross
驗證成功,以爲例,第一次對應的commitment爲,open位置集合爲;第二次對用的爲,binding屬性要求兩次open的交集應具有一致性。
若an opening is inconsistent with a same-commitment opening,則將其和其它commitment openings 聚合在一起的cross-commitment aggregated proof應驗證失敗。
?即cross-commitment aggregation的binding屬性要求要強於same-commitment aggregation 中的binding屬性。若cross-commitment中的任一commitment的binding屬性有問題,則相應的cross-commitment aggregation proof應驗證失敗。
特殊地,當時,其實就是same-commitment。
具體的實現可爲:
藉助2.2節中的優化思路,可做如下調整:
2.4 hiding屬性的實現
本文考慮的是simulation-based statistical security——存在efficient randomized simulator ,其中:
- :輸出和trapdoor 。
- ;除了隨機值外無其它輸入,輸出爲a random fake commitment 。【fake commitment】
- :基於trapdoor ,fake , index , value 來生成a fake proof 。【fake proof】
其核心思想爲以上算法生成的fake proofs應statistically indistinguishable from real ones, even given the commitments, and even if the committed messages are chosen adversarially。
fake commitment和fake proof中除了要open的信息外,應leak no information about other messages。
舉例爲:
- 真實的stateful oracle :計算,;
- 仿真的stateful oracle :計算,。【爲實現hiding屬性,很關鍵的一點是不獲取的任何信息,而中僅獲取要open的信息,除待open位置之外其它的信息均不獲取。】
Hiding屬性要求基於真實的stateful oracle 和仿真的stateful oracle 的輸出分佈的差異性應可忽略。
當進行update操作時,由於是確定性的,若使用相同的randomness,通過與之間的關係會reveal 與之間的關係。可以在UpdateCommit
之後再增加rerandomization處理來hide this relationship。本文方案支持rerandomization操作。【commitment can be rerandomized via multiplication by .】
實際hiding屬性的實現採用與Benoît Libert和Moti Yung 2010年論文《Concise Mercurial Vector Commitments and Independent Zero-Knowledge Sets with Short Proofs》中類似的方式——在Commit
時引入隨機值。
加入hiding元素後,具體的算法調整如下:
3. Pointproofs在區塊鏈中的應用
Blockchain:is an append-only public ledger that consists of blocks, with every block containing some number of transactions.
區塊鏈中最根本的問題是對新區塊達成共識,共識的過程包括:
- proposer:propose a block。
- validators:verify the transactions included in the proposed block are valid。【validators的產生可由POS選舉或POW中的self-selected方式產生。】
傳統的方式validator需要維護整個state賬本,存在存儲壓力問題。
在[ST99, Mil12, Whi15, Tod16,But17, Dra17, RMCI17, CPZ18, BBF19]等很多論文中,都提議validator store commitments to vectors of relevant values instead of the values themselves。基於此提議實現的可稱爲stateless client或者stateless blockchain。相應的,transactions中需要包含:
- the values on which they depend;
- the proofs of correctness of these values with respect to the commitments stored by the validators (which the validators would update for successful transactions)。
該模式需要在storage,bandwidth和computation之間做取捨平衡。理想的情況應該是具有small commitments and proof sizes and add little computation and bandwidth overheads to the validators。
簡單的transaction,類似於比特幣賬號之間的轉賬;
複雜的transaction,是基於智能合約的。
現有各方案在解決storage方面的表現對比:【若validator直接存儲明文,當需存儲個account的狀態信息時,需要的存儲空間將近3TB;Pointproofs中若引入central entity,對整個系統維護一個commitment,而不是爲每個account維護一個commitment,但是沒有意義,當個account時,僅需4.5GB已經足夠小了,沒必要再引入中心化的機制。】
當由1000個256-bit message時,各vector commitment方案的參數:
將Pointproofs用於smart-contract-based的transaction:
詳細的實現流程爲:
現有各方案性能對比:
Pointproofs Prove
、Aggregate
、Verify
、AggregateAcross
等基礎算法的性能表現爲: