Proposal: Rust client library

[abo]

Background

Note: at the time writing 2.0 hasn’t been released yet, so I’m making some guesses about the unreleased parts. Please correct these if they’re wrong.

The Quilibrium protocol node implementation and the SDK (part of which is a client library) are in Go. JavaScript support for the client library is provided by compiling the Go code to WASM.

As far as I understand a client library has to provide the following functionality:

• Verify master clock frames
• Verify data proofs
• Create and verify signatures
• Perform secure multi-party computation
• Perform RPC calls
• Platform-specific key management

The original Golang Wesolowski VDF (used for the master clock) and the BLS48581 (used for data proofs) implementations were ported to Rust recently for performance reasons. These Rust implementations are exposed to Go with the Golang plugin for Uniffi-rs. With Uniffi-rs, we can specify a single FFI interface definition that is implemented in Rust and then Uniffi-rs generates ergonomic wrappers for other programming languages (e.g. Swift, Kotlin) for the Rust FFI.

There is an outdated Rust client library called quilibrium-rs that supports generating a Rust RPC client based on protocol buffers files from the Quilibrium repo.

Problem

This post aims to tackle three problems at once:

1. WASM client library size.
2. The need for a Rust client library.
3. The need for client libraries for additional programming languages.

WASM client library size

Having a client library for browsers is a must for Quilibrium, since initial applications are expected to be browser-based. The size of the Go-based WASM client library for JavaScript is currently on the order of tens of megabytes. As a rule of thumb, downloading 10 megabytes takes 10 seconds on weaker internet connections and it can take a minute on bad connections, so a client library of that size is prohibitively large for web applications.

Rust client library

Rust is an increasingly popular language and it’s quite prominent in the crypto-space. Having a Rust client library is important to attract early adopters.

Client libraries for additional programming languages

JavaScript is the most important programming language to support, but Quilibrium will eventually need client libraries for all major programming languages to succeed. While back-end developers can choose to implement services that integrate Quilibrium in an already supported programming language, mobile developers have less room to maneuver, so Swift and Java/Kotlin client libraries are or particular importance.

Proposal

All three problems could be solved by having a full-fledged Rust client library that supports compilation to WASM and eventually implements a Uniffi-rs interface definition. Platform-specific key management is out of scope for this library, but it should support all the other functionality listed at the beginning.

Specifically, this is how the proposed library would solve the problems listed above:

1. WASM client library size: the size of a full-fledged Rust client library compiled to WASM is expected to be on the order of hundreds of kilobytes vs tens of megabytes with Go.
2. The need for a Rust client library: self-explanatory.
3. Client libraries for additional programming languages: Implementing a Uniffi-rs interface in Rust lets us generate ergonomic clients for multiple programming languages. Uniffi-rs supports Kotlin, Swift, Python and Ruby out of the box and there are third-party plugins for C# and Golang.

Proof of Concept

I modified quilibrium-rs to implement a PoC for a Quilibrium RPC client implemented in Rust and compiled to WASM for web browsers. The Rust gRPC client is generated from protocol buffer definitions. Usage instructions are here.

The size of the WASM binary for the PoC is 379kB. I made no effort to optimize this and I expect that it contains a lot of standard library code that only needs to be included once, so I don’t think it’ll grow more than 50% if cryptography-related functionality is added. This means that a 100x decrease in size over the current Go-based WASM client library seems feasible.

gRPC-Web

gRPC assumes HTTP/2 and uses some advanced features of it, but browser APIs are not powerful enough to support this. This is where gRPC-Web comes in which is gRPC over HTTP/1.1 or HTTP/2 for browser compatibility.

To make the PoC work I had to hack gRPC-Web support onto the node with a middleware. There is already a REST middleware for gRPC in the repo, but the Rust library used to generate the RPC client doesn’t support this directly, so making things work with gRPC-Web seemed the better option. We should explore this more though.

Next

For me, the next question to gauge the feasibility of the proposal is if all the necessary cryptographic primitives have reliable Rust implementations.

It’s getting late now, so I’ll wrap up this post here. Please share your thoughts!

[0xOzgur]

As a serial hobbyist developer who is not familiar enough with Go and Rust, especially when developing a web dapp, the size of WASM caught my attention and I realized that this was a problem. That’s why it might be a good idea, but I would prefer other topics to be discussed by experts.

I am sharing the wasm size of the Quil Rocks NFT project to show the comparison of the size of the wasm size.

I would neither want to use nor make such a large sized dapp.

[ful0n]

Hi Abo!

This looks great! Maybe we can start listing the necessary crypto primitives and what we would need for a basic version?
Or create a repo somewhere to open issues for those parts.

[abo]

At this point only @cassie knows for sure, but she’s busy with the launch. After that we can compile a list based on the published Go code.

Note that @cassie released a triple-ratchet Rust implementation today. I forgot to include this in my original list, but it’s probably also needed by the client library.

[cassie]

Indeed, the first iteration of the web sdk utilizes go’s output WASM and is quite large. Moving all components away from go/wasm to rust or at least a smaller go wasm target will be valuable. Will post more about this shortly.