Important note: If you are looking for a comprehensive guide into Rust macros, please keep on searching – this one is just a quick glimpse at what sits under the hood of the #[] syntax. One who wrote it has no real experience or knowledge. All he has is his keyboard, google search engine and his faith that one day he will reach the zen state of coding.

The goal

Today’s goal: to create a macro that will reverse the name of any function (yes it is possible!) and inject some extra code into its body. In short: make the following code compile.

#[reverse_name(test)]
fn rust_is_fun() {
    println!("Called by function");
}

fn main() {
    nuf_si_tsur();
}

The solution

The code presented below does exactly what we need. The whole project can be found here.

use syn;
use proc_macro::TokenStream;
use proc_macro2::Span;
use quote::quote;


#[proc_macro_attribute]
pub fn reverse_name(attr: TokenStream, item: TokenStream) -> TokenStream {

    // turn TokenStream into a syntax tree
    let func = syn::parse_macro_input!(item as syn::ItemFn);

    // extract fields out of the item
    let syn::ItemFn {
        attrs,
        vis,
        mut sig,    // mutable as we are going to change the signature
        block,
    } = func;

    let name = (format!("{}", sig.ident)).chars().rev().collect::<String>();
    sig.ident = syn::Ident::new(&name, Span::call_site());

    let item_str = attr.to_string();

    let output = quote! {
        #(#attrs)*
        #vis #sig {
            println!("Injected: {}", #item_str);
            #block
        }
    };

    // See the body of our new function (printed during build)
    println!("New function:\n{}", output.to_string());

    // Convert the output from a `proc_macro2::TokenStream` to a `proc_macro::TokenStream`
    TokenStream::from(output)
}

Only a few copy-paste actions, some glue code here and there, and done. But what exactly have I done?

WTH have I done?

Not knowing why the code does not work is a bad thing, but not knowing why the code does work is even worse. Let’s try to figure out what exactly happened above.

We added some extra print function and while building the project we can see its output:

New function:
fn nuf_si_tsur()
{
    println! ("Injected: {}", "test") ; { println! ("Called by function") ; }
    println! ("Again injected: {}", "test") ;
}

So our compiler took the source code, found part marked with the reverse_name attribute, and fed it into our function replacing the original code with its output. In theory, we can manipulate the code in any crazy way we want (although I guess that black magic macros in Rust are just as bad as in C).

Q&A

Some questions arose when writing the code so it’s time to search for the answers.

1. Why do we need a separate proc-macro crate for macros?

As we saw our macro code was used to manipulate the code while performing the build. It means, that the functions need to be available to the compiler it starts its work. And since functions are written in Rust they must be available as binaries so we need to compile them in a separate module. Also, note that when doing a cross-compilation (eg. for ARM microcontroller) the macro code always needs to compile for your development, and not the target, machine. Another reason to keep it separated.

2. Why proc_macro and proc_macro2?

The proc_macro crate is the library that makes all the macro magic work. Proc_macro2 is “A wrapper around the procedural macro API of the compiler’s proc_macro crate.” This part is confusing but it looks like the proc_macro can’t be used by eg. syn crate and we need yet another crate redefining the same types (like Ident or Span). Something that might change in the future I guess but for now, we need both.

3. What is syn and quote?

Functions inside syn crate translate TokenStream into a syntax tree that represents any code construction present in the Rust language. In our example, the ItemFn structure holds all the parts that can be present in a free-standing function (parameters, name, body, etc.) Quote does the opposite – it translates syntax tree back into a token stream. It has a very interesting feature that allows writing a string that looks very similar to a code. Makes things more readable.

4. Can I debug a macro translated function

No. At least not without some extra effort. In theory, you could print (as we did in our example), copy-paste, and debug any function created by the macro engine. Another option would be to use a tool like cargo-expand that recursively expands all the macros used in the code.

Summary

Rust macros are a very powerful, and yet easy-to-use feature. I was using Python to generate C++ and C code for a long time but Rust sets new standards when it comes to code generation.

I won my first battles with Rust compiler, and now I have a code that is failproof. But having software that does not crash is not enough. I need to make sure, that my application does what it is supposed to do. Even more important, I need a guarantee that its behavior won’t change in the future, after all the refactoring, bug fixing, and adding new features. Also I need a magic force to drive my architecture (so I can add yet another cool abbreviation in my CV). What I need is Unit Testing.

Time for another adventure. Let the google search engine guide us on our journey.

Simple test

Our first goal is to test this piece of code (full source can be found here).

pub fn find_marmot(depth: u32) -> bool {
    depth > 20
}

A very nice surprise is that Rust already comes with support for unit testing so there is no need to install any external crate. Testing is as simple as adding few extra lines of code into the source file…

#[cfg(test)]
mod test {
    use super::*;

    #[test]
    fn when_search_not_deep_then_no_marmot_found() {
        assert_eq!(find_marmot(19), false);
    }
}

…and running the test target.

$cargo test
   Compiling marmot_test v0.1.0
    Finished test
     Running unittests

running 1 test
test marmot_hole::test::when_search_not_deep_then_no_marmot_found ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

Super easy but also super primitive*. It is not something that you could compare with a proper unit test framework like Google Test or Catch2. It looks more like a small addon that allows you to run multiple small programs and count the number of panics. No extra features like test fixtures, mocks or parameterized tests.

Less simple test

This is our following function to test:

pub fn chance_to_find_marmot(day_of_weeek: u8) -> f32 {
    match day_of_weeek {
        1 => 1.0,
        2 | 3 | 4 => 1.0 / 2.0,
        5 | 6 | 7 => 1.0 / 3.0,
        _ => panic!("Not a day of week")
    }
}

While I can test for panic with a should_panic attribute, there is no check dedicated for floating point results that would not suffer from floating point inaccuracy (see EXPECT_FLOAT_EQ from GTest). To add this functionality, we need to install assert_approx_eq crate and use it like this:

#[test]
#[should_panic]
fn when_invalid_week_day_should_panic() {
    chance_to_find_marmot(9);
}

#[test]
fn when_friday_than_0_33_chance_of_fiding_marmot() {
    let res = chance_to_find_marmot(5);
    assert_approx_eq!(res, 0.333, 0.01)
}

Parameterized tests

Searching for a way to do parameterized tests I encounter rstest crate. So it looks that I will have to install yet another external framework. With rstest installed I can write my test like this:

#[rstest]
#[case(1, 1.0)]
#[case(2, 0.5)]
#[case(3, 0.5)]
#[case(4, 0.5)]
#[case(5, 0.33)]
#[case(6, 0.33)]
#[case(7, 0.33)]
fn check_all_days(#[case] input: u8,#[case] expected: f32) {
    assert_approx_eq!(expected, chance_to_find_marmot(input), 0.01)
}

It also supports test fixtures but does not offer anything more than that.

Mocking framework

Now we start moving even more uphill. There are so many different mocking frameworks for Rust that it is really hard to judge which one is the best. Luckily for us, someone has walked this path before and left this useful overview. Mockall has the greatest number of features and quick research reveals that it is the only framework that is still actively developed. So the choice is not so hard after all.

Let’s add a simple trait and see how we can mock it.

pub trait HidingPlace {
    fn has_marmot(&self) -> bool;
}

pub fn find_marmot_in(hiding_place: &dyn HidingPlace) -> bool {
    return hiding_place.has_marmot();
}

Following documentation I just do this:

use mockall::*;
use mockall::predicate::*;

#[automock]
pub trait HidingPlace {
    fn has_marmot(&self) -> bool;
}

And it works. I can now use MockHidingPlace structure in my tests. However, I don’t feel comfortable with polluting my code with test specific statements, so I will try to move it into the dedicated module. More doc reading and I found a way to do this:

#[cfg(test)]
mod test {
    use super::*;
    use mockall::*;
    use mockall::predicate::*;

    mock! {
        pub Hole {}
        impl HidingPlace for Hole{
            fn has_marmot(&self) -> bool;
        }
    }

    #[test]
    fn when_search_deep_then_marmot_found() {
        let mut mock = MockHole::new();
        mock.expect_has_marmot()
            .times(1)
            .returning(||true);
        assert_eq!(find_marmot(21, &mock), true);
    }
}

Summary

Setting up a unit test framework in Rust means adding specialized crates into your project. Each one brings single piece of functionality, like mocking or float asserts, and together they create a full testing environment. Not as perfect as solutions we know from C or C++ but good enough to test our code and move to the next chapter of our journey.

* In this blog post, Mozilla guys suggest using the standard test tool from Rust so maybe I just underestimate its potential