Sep 20, 2019
Introducing waPC
WebAssembly Procedure Calls
When I set out to build Waxosuit a few months ago, my knowledge of how data could flow between the WebAssembly guest⟷host boundary was fairly shallow, and limited to what little I knew of the wasm-bindgen project (a Rust crate that generates JS/Rust bindings for building browser-based apps). If you’re not familiar with the low-level details, WebAssembly on its own is only capable of accepting and passing simple numeric parameters. You can’t pass strings or structs or binary blobs between host and guest as native parameter types. So if you want robust function calling, you’re going to have to build it yourself or use a library. When I started working on Waxosuit, wasm-bindgen was one of the only examples I had for rich Wasm communications.
I knew that if I was going to build a cloud native host runtime for WebAssembly, I was going to have to figure out how to allow data to flow between guest and host and allow bi-directional function invocations.
The interaction model between the browser and wasm-bindgen-produced WebAssembly modules hides the Wasm module behind a JavaScript library face, so as far as the code can tell, it’s just JavaScript turtles all the way down. This means you can instantiate classes in JavaScript that began life as structs in Rust or vise versa. The subtle — but crucially important — detail here is that cross-boundary references must be long-lived. Put another way, each side of the exchange can hold references to data allocated by the other side for an indeterminate amount of time.
By following wasm-bindgen’s example (and annoying its creators with endless questions), I produced an interface that included functions like these to be called by the host runtime:
malloc: Allocate memory within the Wasm modulefree: Release previously allocated memorycall: Execute some function, passing pointers to the allocated memory
This involved learning more about FFI than I had ever wanted to know, but learning new things is always a bonus, so I found the adventure rewarding. The more focus I got on the interaction pattern I wanted to support in Waxosuit, the more I realized that there were some fairly fundamental problems with trying to apply the wasm-bindgen model to running services and functions in the cloud.
The biggest, of course, was that it was stateful. If the host requests memory allocation for data and then crashes, you now have to worry about attempting to reconstitute that Wasm instance with the exact same state it had prior to the crash. I actually built such a recovery mechanism before I realized I was layering unnecessary complexity on top of already unnecessary complexity.
After some discussions with my colleague, Phil Kedy, he mentioned that he’d seen some pretty cool contract patterns in the smart contracts space that didn’t operate this way. Allocating or freeing memory explicitly across the guest/host boundary is not only cloud-antagonistic (requires state management on both sides), but a huge source of potential bugs and memory leaks. What I really needed was a flexible mechanism where neither the host nor the Wasm module would be required to maintain state between calls, or be tightly coupled to the other’s memory management strategies.
Another complicating factor is that I needed the Wasm modules to be able to make host calls during the execution of a guest call. This could be for something as simple as requesting stdout log output or, as in the case of Waxosuit, publishing NATS messages or communicating with a key-value store. So what I really needed was a bi-directional function call mechanism that worked with WebAssembly’s limited native data types. Remember that WebAssembly is also single-threaded, so these calls are synchronous (though your host runtime can provide an asynchronous layer on top like Waxosuit does).
I also set for myself a requirement that I couldn’t tamper with the standard, so this had to work for any Wasm file compiled by any standard-compliant language.
I built this new function calling mechanism directly into Waxosuit and, after iterating on it with Phil a few times, we settled on an interface that was powerful and flexible enough to suit the needs of Waxosuit as well as any other applications that might need WebAssembly procedure calls.
Once I was sure it was working, I extracted the core function invocation code out of the host and guest parts of Waxosuit and we labeled the resulting standard and initial crates waPC (WebAssembly Procedure Calls). Phil has even written a sample guest library in Zig, if you’re interested in writing your Wasm modules in Zig.
So let’s see how it all works. Take a look at the following sequence diagram showing the simplest “hello world” style interaction between the host and the guest (note that the host runtime and guest module are in the same process):
First, the host runtime initiates the call with guest_call passing the length of the operation to be performed (a UTF-8 encoded string) and the length of opaque binary blob to be passed as a parameter.
The guest module then called guest_request on the host, passing a pointer to the operation and the opaque binary parameter with the expectation that the host will populate the data at those locations. Remember that a Wasm host runtime has direct access to a Wasm module’s linear memory blocks, so it’s a quick operation to write that data into the module’s memory.
If the guest call was successful, it invokes guest_response with a pointer+length pair to tell the host how to find the response data (which is also an opaque blob). If it fails, the guest can invoke guest_error to provide a UTF-8 encoded string to the host to help explain the failure.
Finally, the guest returns a 0 to indicate failure and a 1 to indicate success (since Wasm doesn’t have a boolean type).
Next, let’s take a look at a slightly more advanced sequence showing the host making a guest call and the guest making a host call in return.
This interaction starts off like the first one, but in the source of the guest module performing its work, it invokes host_call. This could be to write a log entry or publish a message or send a record to a database — whatever the host runtime is capable of supporting. The host_call function returns 0 or 1 to indicate success or failure. The guest can then call host_response and host_response_len to get the host response’s ptr+size pair for the raw binary data, or host_error and host_error_len to get a ptr+size pair for the host error (as with the previous error function this is a UTF-8 encoded string).
When the guest module returns from guest_call, the host continues on the same way it did in the simple interaction model.
While this pattern explicitly avoids any mention of memory allocation or de-allocation, it is a little chatty. We think the few extra calls are worth the benefit developers get by being able to conform to a far simpler contract and the far smaller surface area for potential points of failure. Thankfully, there’s a Rust crate that simplifies both the host and guest interaction for you!
Here’s how simple it is to create a host in Rust that can perform WebAssembly Procedure Calls
extern crate wapc;
use wapc::prelude::*;pub fn main() -> Result<(), Box<dyn std::error::Error>> {
let module = load_file();
let mut host = WapcHost::new(&module)?; wapc::set_host_callback(host_callback);
let res = host.call("wapc:sample!Hello", b"this is a test")?;
assert_eq!(res, b"hello world!");
Ok(())
}fn host_callback(op: &str, payload: &[u8]) -> Result<Vec<u8>,
Box<dyn std::error::Error>> {
println!("Guest invoked '{}' with payload of {} bytes", op,
payload.len());
Ok(vec![])
}
Here the host is creating a new WapcHost out of the raw bytes from a Wasm file. Then all that’s left is to set up a callback that gets invoked when the guest does a host_call and you can make guest calls with the call function, passing a simple string as the operation name and some raw bytes as the payload (in this sample I’m passing the bytes of a string).
And to build a Wasm module in Rust that supports waPC, you can just use a wapc-guest library in any available language (currently only Zig and Rust, see my previous blog post for why this doesn’t work in Go yet):
extern crate wapc_guest as guest;
use guest::prelude::*;wapc_handler!(handle_wapc);pub fn handle_wapc(operation: &str, msg: &[u8]) -> CallResult {
match operation {
"sample:Guest!Hello" => hello_world(msg),
_ => Err("bad dispatch".into()),
}
}fn hello_world(
_msg: &[u8]) -> CallResult {
let _res = host_call("sample:Host!Call", b"hello")?;
Ok(vec![])
}
Now that we’ve got a foundation like waPC to build on, we can create strongly-typed wrappers around the opaque binary payloads to add domain-specific meaning. This is what Waxosuit does, by allowing these procedure calls to make secure, verified invocations against dynamically bound cloud capabilities.
I am looking forward to people using this pattern and can’t wait to see what other people build on top of waPC!
