Query groups and query group structs

When you define a query group trait:

#[salsa::query_group(HelloWorldStorage)]
trait HelloWorld {
    // For each query, we give the name, some input keys (here, we
    // have one key, `()`) and the output type `Arc<String>`. We can
    // use attributes to give other configuration:
    //
    // - `salsa::input` indicates that this is an "input" to the system,
    //   which must be explicitly set. The `salsa::query_group` method
    //   will autogenerate a `set_input_string` method that can be
    //   used to set the input.
    #[salsa::input]
    fn input_string(&self, key: ()) -> Arc<String>;

    // This is a *derived query*, meaning its value is specified by
    // a function (see Step 2, below).
    fn length(&self, key: ()) -> usize;
}

the salsa::query_group macro generates a number of things, shown in the sample generated code below (details in the sections to come).

and associated storage struct) that represent things which don't have "public" Note that there are a number of structs and types (e.g., the group descriptor names. We currently generate mangled names with __ afterwards, but those names are not meant to be exposed to the user (ideally we'd use hygiene to enforce this).

// First, a copy of the trait, though with extra supertraits and
// sometimes with some extra methods (e.g., `set_input_string`)
trait HelloWorld: 
    salsa::Database + 
    salsa::plumbing::HasQueryGroup<HelloWorldStorage>
{
    fn input_string(&self, key: ()) -> Arc<String>;
    fn set_input_string(&mut self, key: (), value: Arc<String>);
    fn length(&self, key: ()) -> usize;
}

// Next, the "query group struct", whose name was given by the
// user. This struct implements the `QueryGroup` trait which
// defines a few associated types common to the entire group.
struct HelloWorldStorage { }
impl salsa::plumbing::QueryGroup for HelloWorldStorage {
    type DynDb = dyn HelloWorld;
    type GroupStorage = HelloWorldGroupStorage__;
}

// Next, a blanket impl of the `HelloWorld` trait. This impl
// works for any database `DB` that implements the
// appropriate `HasQueryGroup`.
impl<DB> HelloWorld for DB
where
  DB: salsa::Database,
  DB: salsa::plumbing::HasQueryGroup<HelloWorldStorage>,
{
  ...
}

// Next, for each query, a "query struct" that represents it.
// The query struct has inherent methods like `in_db` and
// implements the `Query` trait, which defines various
// details about the query (e.g., its key, value, etc).
pub struct InputQuery { }
impl InputQuery { /* definition for `in_db`, etc */ }
impl salsa::Query for InputQuery {
    /* associated types */
}

// Same as above, but for the derived query `length`.
// For derived queries, we also implement `QueryFunction`
// which defines how to execute the query.
pub struct LengthQuery { }
impl salsa::Query for LengthQuery {
    ...
}
impl salsa::QueryFunction for LengthQuery {
    ...
}

// Finally, the group storage, which contains the actual
// hashmaps and other data used to implement the queries.
struct HelloWorldGroupStorage__ { .. }

The group struct and QueryGroup trait

The group struct is the only thing we generate whose name is known to the user. For a query group named Foo, it is conventionally called FooStorage, hence the name HelloWorldStorage in our example.

Despite the name "Storage", the struct itself has no fields. It exists only to implement the QueryGroup trait. This trait has a number of associated types that reference various bits of the query group, including the actual "group storage" struct:

struct HelloWorldStorage { }
impl salsa::plumbing::QueryGroup for HelloWorldStorage {
    type DynDb = dyn HelloWorld;
    type GroupStorage = HelloWorldGroupStorage__; // generated struct
}

We'll go into detail on these types below and the role they play, but one that we didn't mention yet is GroupData. That is a kind of hack used to manage send/sync around slots, and it gets covered in the section on slots.

Impl of the hello world trait

Ultimately, every salsa query group is going to be implemented by your final database type, which is not currently known to us (it is created by combining multiple salsa query groups). In fact, this salsa query group could be composed into multiple database types. However, we want to generate the impl of the query-group trait here in this crate, because this is the point where the trait definition is visible and known to us (otherwise, we'd have to duplicate the method definitions).

So what we do is that we define a different trait, called plumbing::HasQueryGroup<G>, that can be implemented by the database type. HasQueryGroup is generic over the query group struct. So then we can provide an impl of HelloWorld for any database type DB where DB: HasQueryGroup<HelloWorldStorage>. This HasQueryGroup defines a few methods that, given a DB, give access to the data for the query group and a few other things.

Thus we can generate an impl that looks like:

impl<DB> HelloWorld for DB
where
    DB: salsa::Database,
    DB: salsa::plumbing::HasQueryGroup<HelloWorld>
{
    ...
    fn length(&self, key: ()) -> Arc<String> {
      <Self as salsa::plumbing::GetQueryTable<HelloWorldLength__>>::get_query_table(self).get(())
    }
}

You can see that the various methods just hook into generic functions in the salsa::plumbing module. These functions are generic over the query types (HelloWorldLength__) that will be described shortly. The details of the "query table" are covered in a future section, but in short this code pulls out the hasmap for storing the length results and invokes the generic salsa logic to check for a valid result, etc.

For each query, a query struct

As we referenced in the previous section, each query in the trait gets a struct that represents it. This struct is named after the query, converted into snake case and with the word Query appended. In typical Salsa workflows, these structs are not meant to be named or used, but in some cases it may be required. For e.g. the length query, this structs might look something like:

struct LengthQuery { }

The struct also implements the plumbing::Query trait, which defines a bunch of metadata about the query (and repeats, for convenience, some of the data about the group that the query is in):

            impl salsa::Query for #qt
            {
                type Key = (#(#keys),*);
                type Value = #value;
                type Storage = #storage;

                const QUERY_INDEX: u16 = #query_index;

                const QUERY_NAME: &'static str = #query_name;

                fn query_storage<'a>(
                    group_storage: &'a <Self as salsa::QueryDb<'_>>::GroupStorage,
                ) -> &'a std::sync::Arc<Self::Storage> {
                    &group_storage.#fn_name
                }

                fn query_storage_mut<'a>(
                    group_storage: &'a <Self as salsa::QueryDb<'_>>::GroupStorage,
                ) -> &'a std::sync::Arc<Self::Storage> {
                    &group_storage.#fn_name
                }
            }

Depending on the kind of query, we may also generate other impls, such as an impl of salsa::plumbing::QueryFunction, which defines the methods for executing the body of a query. This impl would then include a call to the user's actual function.

                impl salsa::plumbing::QueryFunction for #qt
                {
                    fn execute(db: &<Self as salsa::QueryDb<'_>>::DynDb, #key_pattern: <Self as salsa::Query>::Key)
                        -> <Self as salsa::Query>::Value {
                        invoke(db, #(#key_names),*)
                    }

                    recover
                }

Group storage

The "group storage" is the actual struct that contains all the hashtables and so forth for each query. The types of these are ultimately defined by the Storage associated type for each query type. The struct is generic over the final database type:

struct HelloWorldGroupStorage__ {
    input: <InputQuery as Query::Storage,
    length: <LengthQuery as Query>::Storage,
}

We also generate some inherent methods. First, a new method that takes the group index as a parameter and passes it along to each of the query storage new methods:

        impl #group_storage {
            trait_vis fn new(group_index: u16) -> Self {
                group_storage {
                    (
                        queries_with_storage:
                        std::sync::Arc::new(salsa::plumbing::QueryStorageOps::new(group_index)),
                    )*
                }
            }
        }

And then various methods that will dispatch from a DatabaseKeyIndex that corresponds to this query group into the appropriate query within the group. Each has a similar structure of matching on the query index and then delegating to some method defined by the query storage:

        impl #group_storage {
            trait_vis fn fmt_index(
                &self,
                db: &(#dyn_db + '_),
                input: salsa::DatabaseKeyIndex,
                fmt: &mut std::fmt::Formatter<'_>,
            ) -> std::fmt::Result {
                match input.query_index() {
                    fmt_ops
                    i => panic!("salsa: impossible query index {}", i),
                }
            }

            trait_vis fn maybe_changed_after(
                &self,
                db: &(#dyn_db + '_),
                input: salsa::DatabaseKeyIndex,
                revision: salsa::Revision,
            ) -> bool {
                match input.query_index() {
                    maybe_changed_ops
                    i => panic!("salsa: impossible query index {}", i),
                }
            }

            trait_vis fn cycle_recovery_strategy(
                &self,
                db: &(#dyn_db + '_),
                input: salsa::DatabaseKeyIndex,
            ) -> salsa::plumbing::CycleRecoveryStrategy {
                match input.query_index() {
                    cycle_recovery_strategy_ops
                    i => panic!("salsa: impossible query index {}", i),
                }
            }

            trait_vis fn for_each_query(
                &self,
                _runtime: &salsa::Runtime,
                mut op: &mut dyn FnMut(&dyn salsa::plumbing::QueryStorageMassOps),
            ) {
                for_each_ops
            }
        }