Writing Custom Filters

The full source code used in this example can be found in examples/.

Quilkin provides an extensible implementation of Filters that allows us to plug in custom implementations to fit our needs. This document provides an overview of the API and how we can go about writing our own Filters. First we need to create a type and implement two traits for it.

It's not terribly important what the filter in this example does so let's write a Greet filter that appends Hello to every packet in one direction and Goodbye to packets in the opposite direction.

struct Greet;

As a convention within Quilkin: Filter names are singular, they also tend to be a verb, rather than an adjective.

Examples

Greet not "Greets"

Compress not "Compressor".

`Filter`

Represents the actual Filter instance in the pipeline. An implementation provides a read and a write method (both are passthrough by default) that accepts a context object and returns a response.

Both methods are invoked by the proxy when it consults the filter chain read is invoked when a packet is received on the local downstream port and is to be sent to an upstream endpoint while write is invoked in the opposite direction when a packet is received from an upstream endpoint and is to be sent to a downstream client.

struct Greet;
use quilkin::filters::prelude::*;

impl Filter for Greet {
    fn read(&self, ctx: &mut ReadContext) -> Option<()> {
        ctx.contents.extend(b"Hello");
        Some(())
    }
    fn write(&self, ctx: &mut WriteContext) -> Option<()> {
        ctx.contents.extend(b"Goodbye");
        Some(())
    }
}

`StaticFilter`

Represents metadata needed for your [Filter], most of it has to with defining configuration, for now we can use () as we have no configuration currently.

use quilkin::filters::prelude::*;
struct Greet;
impl Filter for Greet {}
impl StaticFilter for Greet {
    const NAME: &'static str = "greet.v1";
    type Configuration = ();
    type BinaryConfiguration = ();

    fn try_from_config(config: Option<Self::Configuration>) -> Result<Self, Error> {
        Ok(Self)
    }
}

Running

We can run the proxy using [Proxy::TryFrom][Proxy::TryFrom] function. Let's add a main function that does that. Quilkin relies on the Tokio async runtime, so we need to import that crate and wrap our main function with it.

We can also register custom filters in quilkin using FilterRegistry::register

Add Tokio as a dependency in Cargo.toml.

[dependencies]
quilkin = "0.2.0"
tokio = { version = "1", features = ["full"]}

Add a main function that starts the proxy.

// src/main.rs
#[tokio::main]
async fn main() -> quilkin::Result<()> {
    quilkin::filters::FilterRegistry::register(vec![Greet::factory()].into_iter());

    let (_shutdown_tx, shutdown_rx) = tokio::sync::watch::channel(());
    let server: quilkin::Proxy = quilkin::Config::builder()
        .port(7001)
        .filters(vec![quilkin::config::Filter {
            name: Greet::NAME.into(),
            config: None,
        }])
        .endpoints(vec![quilkin::endpoint::Endpoint::new(
            (std::net::Ipv4Addr::LOCALHOST, 4321).into(),
        )])
        .build()?
        .try_into()?;

    server.run(shutdown_rx).await
}

Now, let's try out the proxy. The following configuration starts our extended version of the proxy at port 7001 and forwards all packets to an upstream server at port 4321.

# quilkin.yaml
version: v1alpha1
port: 7001
filters:
  - name: greet.v1
clusters:
  default:
    localities:
        - endpoints:
            - address: 127.0.0.1:7001

Next we to setup our network of services, for this example we're going to use the netcat tool to spawn a UDP echo server and interactive client for us to send packets over the wire.

# Start the proxy
cargo run -- &
# Start a UDP listening server on the configured port
nc -lu 127.0.0.1 4321 &
# Start an interactive UDP client that sends packet to the proxy
nc -u 127.0.0.1 7001

Whatever we pass to the client should now show up with our modification on the listening server's standard output. For example typing Quilkin in the client prints Hello Quilkin on the server.

Configuration

Let's extend the Greet filter to have a configuration that contains what greeting to use.

The Serde crate is used to describe static YAML configuration in code while Tonic/Prost is used to describe dynamic configuration as Protobuf messages when talking to a management server.

YAML Configuration

First let's create the type for our configuration:

Add the yaml parsing crates to Cargo.toml:

# [dependencies]
serde = "1.0"
serde_yaml = "0.8"

Define a struct representing the config:

// src/main.rs
#[derive(Serialize, Deserialize, Debug, schemars::JsonSchema)]
struct Config {
    greeting: String,
}

Update the Greet Filter to take in greeting as a parameter:

// src/main.rs
struct Greet {
    config: Config,
}

impl Filter for Greet {
    fn read(&self, ctx: &mut ReadContext) -> Option<()> {
        ctx.contents
            .splice(0..0, format!("{} ", self.config.greeting).into_bytes());
        Some(())
    }
    fn write(&self, ctx: &mut WriteContext) -> Option<()> {
        ctx.contents
            .splice(0..0, format!("{} ", self.config.greeting).into_bytes());
        Some(())
    }
}

Protobuf Configuration

Quilkin comes with out-of-the-box support for xDS management, and as such needs to communicate filter configuration over Protobuf with management servers and clients to synchronise state across the network. So let's add the binary version of our Greet configuration.

Add the proto parsing crates to Cargo.toml:

[dependencies]
# ...
tonic = "0.5.0"
prost = "0.7"
prost-types = "0.7"

Create a Protobuf equivalent of our YAML configuration.

// src/greet.proto
syntax = "proto3";

package greet;

message Greet {
  string greeting = 1;
}

Generate Rust code from the proto file:

There are a few ways to generate Prost code from proto, we will use the prost_build crate in this example.

Add the following required crates to Cargo.toml, and then add a build script to generate the following Rust code during compilation:

# [dependencies]
bytes = "1.0"

# [build-dependencies]
prost-build = "0.7"

// src/build.rs
fn main() {
    prost_build::compile_protos(&["src/greet.proto"], &["src/"]).unwrap();
}

To include the generated code, we'll use [tonic::include_proto], then we just need to implement std::convert::TryFrom for converting the protobuf message to equivalvent configuration.

// src/main.rs
mod proto {
    tonic::include_proto!("greet");
}

impl TryFrom<proto::Greet> for Config {
    type Error = ConvertProtoConfigError;

    fn try_from(p: proto::Greet) -> Result<Self, Self::Error> {
        Ok(Self {
            greeting: p.greeting,
        })
    }
}

impl From<Config> for proto::Greet {
    fn from(config: Config) -> Self {
        Self {
            greeting: config.greeting,
        }
    }
}

Now, let's update Greet's StaticFilter implementation to use the two configurations.

// src/main.rs
use quilkin::filters::StaticFilter;

impl StaticFilter for Greet {
    const NAME: &'static str = "greet.v1";
    type Configuration = Config;
    type BinaryConfiguration = proto::Greet;

    fn try_from_config(
        config: Option<Self::Configuration>,
    ) -> Result<Self, quilkin::filters::Error> {
        Ok(Self {
            config: Self::ensure_config_exists(config)?,
        })
    }
}

That's it! With these changes we have wired up static configuration for our filter. Try it out with the following configuration:

# quilkin.yaml
version: v1alpha1
port: 7001
filters:
- name: greet.v1
  config:
    greeting: Hey
endpoints:
- address: 127.0.0.1:4321

Quilkin Book