explain databases, runtimes, etc

This commit is contained in:
Niko Matsakis 2022-08-18 05:37:22 -04:00
parent a1a8533cef
commit b6aec13b20
7 changed files with 85 additions and 12 deletions

View file

@ -30,14 +30,10 @@
- [Videos](./videos.md)
- [Plumbing](./plumbing.md)
- [Jars and ingredients](./plumbing/jars_and_ingredients.md)
- [Generated code](./plumbing/generated_code.md)
- [Diagram](./plumbing/diagram.md)
- [Query groups](./plumbing/query_groups.md)
- [Database](./plumbing/database.md)
- [The `salsa` crate](./plumbing/salsa_crate.md)
- [Query operations](./plumbing/query_ops.md)
- [maybe changed after](./plumbing/maybe_changed_after.md)
- [Fetch](./plumbing/fetch.md)
- [Databases and runtime](./plumbing/database_and_runtime.md)
- [Query operations](./plumbing/query_ops.md)
- [maybe changed after](./plumbing/maybe_changed_after.md)
- [Fetch](./plumbing/fetch.md)
- [Derived queries flowchart](./plumbing/derived_flowchart.md)
- [Cycle handling](./plumbing/cycles.md)
- [Terminology](./plumbing/terminology.md)
@ -47,11 +43,14 @@
- [Derived query](./plumbing/terminology/derived_query.md)
- [Durability](./plumbing/terminology/durability.md)
- [Input query](./plumbing/terminology/input_query.md)
- [Ingredient](./plumbing/terminology/ingredient.md)
- [LRU](./plumbing/terminology/LRU.md)
- [Memo](./plumbing/terminology/memo.md)
- [Query](./plumbing/terminology/query.md)
- [Query function](./plumbing/terminology/query_function.md)
- [Revision](./plumbing/terminology/revision.md)
- [Salsa item](./plumbing/terminology/salsa_item.md)
- [Salsa struct](./plumbing/terminology/salsa_struct.md)
- [Untracked dependency](./plumbing/terminology/untracked.md)
- [Verified](./plumbing/terminology/verified.md)

View file

@ -9,5 +9,11 @@ We refer to this as the "plumbing".
The plumbing section is broken up into chapters:
* The [jars and ingredients](./plumbing/jars_and_ingredients.md) covers how each salsa item (like a tracked function)
specifies what data it needs and runtime, and how links between items work.
* The [jars and ingredients](./plumbing/jars_and_ingredients.md) covers how each salsa item (like a tracked function) specifies what data it needs and runtime, and how links between items work.
* The [database and runtime](./plumbing/database_and_runtime.md)
* The [query operations](./plumbing/query_ops.md) chapter describes how the major operations on function ingredients work. This text was written for an older version of salsa but the logic is the same:
* The [maybe changed after](./plumbing/maybe_changed_after.md) operation determines when a memoized value for a tracked function is out of date.
* The [fetch](./plumbing/fetch.md) operation computes the most recent value.
* The [derived queries flowchart](./plumbing/derived_flowchart.md) depicts the logic in flowchart form.
* The [cycle handling](./plumbing/cycles.md) handling chapter describes what happens when cycles occur.
* The [terminology](./plumbing/terminology.md) section describes various words that appear throughout.

View file

@ -0,0 +1,44 @@
# Database and runtime
A salsa database struct is declared by the user with the `#[salsa::db]` annotation.
It contains all the data that the program needs to execute:
```rust,ignore
#[salsa::db(jar0...jarn)]
struct MyDatabase {
storage: Storage<Self>,
maybe_other_fields: u32,
}
```
This data is divided into two categories:
* Salsa-governed storage, contained in the `Storage<Self>` field. This data is mandatory.
* Other fields (like `maybe_other_fields`) defined by the user. This can be anything. This allows for you to give access to special resources or whatever.
## Parallel handles
When used across parallel threads, the database type defined by the user must support a "snapshot" operation.
This snapshot should create a clone of the database that can be used by the parallel threads.
The `Storage` operation itself supports `snapshot`.
The `Snapshot` method returns a `Snapshot<DB>` type, which prevents these clones from being accessed via an `&mut` reference.
## The Storage struct
The salsa `Storage` struct contains all the data that salsa itself will use and work with.
There are three key bits of data:
* The `Shared` struct, which contains the data stored across all snapshots. This is primarily the ingredients described in the [jars and ingredients chapter](./jars_and_ingredients.md), but it also contains some synchronization information (a cond var). This is used for cancellation, as described below.
* The data in the `Shared` struct is only shared across threads when other threads are active. Some operations, like mutating an input, require an `&mut` handle to the `Shared` struct. This is obtained by using the `Arc::get_mut` methods; obviously this is only possible when all snapshots and threads have ceased executing, since there must be a single handle to the `Arc`.
* The `Routes` struct, which contains the information to find any particular ingredient -- this is also shared across all handles, and its construction is also described in the [jars and ingredients chapter](./jars_and_ingredients.md). The routes are separated out from the `Shared` struct because they are truly immutable at all times, and we want to be able to hold a handle to them while getting `&mut` access to the `Shared` struct.
* The `Runtime` struct, which is specific to a particular database instance. It contains the data for a single active thread, along with some links to shraed data of its own.
## The Salsa runtime
The salsa runtime offers helper methods that are accessed by the ingredients.
It tracks, for example, the active query stack, and contains methods for adding dependencies between queries (e.g., `report_tracked_read`) or [resolving cycles](./cycles.md).
It also tracks the current revision and information about when values with low or high durability last changed.
Basically, the ingredient structures store the "data at rest" -- like memoized values -- and things that are "per ingredient".
The runtime stores the "active, in-progress" data, such as which queries are on the stack, and/or the dependencies accessed by the currently active query.

View file

@ -0,0 +1,4 @@
# Ingredient
An *ingredient* is an individual piece of storage used to create a [salsa item](./salsa_item.md)
See the [jars and ingredients](../jars_and_ingredients.md) chapter for more details.

View file

@ -0,0 +1,4 @@
# Salsa item
A salsa item is something that is decorated with a `#[salsa::foo]` macro, like a tracked function or struct.
See the [jars and ingredients](../jars_and_ingredients.md) chapter for more details.

View file

@ -0,0 +1,9 @@
# Salsa struct
A salsa struct is a struct decorated with one of the salsa macros:
* `#[salsa::tracked]`
* `#[salsa::input]`
* `#[salsa::interned]`
See the [salsa overview](../../overview.md) for more details.

View file

@ -16,16 +16,23 @@ use super::{ParallelDatabase, Revision};
/// The "storage" struct stores all the data for the jars.
/// It is shared between the main database and any active snapshots.
pub struct Storage<DB: HasJars> {
/// Data shared across all databases.
/// Data shared across all databases. This contains the ingredients needed by each jar.
/// See the ["jars and ingredients" chapter](https://salsa-rs.github.io/salsa/plumbing/jars_and_ingredients.html)
/// for more detailed description.
///
/// Even though this struct is stored in an `Arc`, we sometimes get mutable access to it
/// by using `Arc::get_mut`. This is only possible when all parallel snapshots have been dropped.
shared: Arc<Shared<DB>>,
/// The "ingredients" structure stores the information about how to find each ingredient in the database.
/// It allows us to take the [`IngredientIndex`] assigned to a particular ingredient
/// and get back a [`dyn Ingredient`][`Ingredient`] for the struct that stores its data.
///
/// This is kept separate from `shared` so that we can clone it and retain `&`-access even when we have `&mut` access to `shared`.
routes: Arc<Routes<DB>>,
/// The runtime for this particular salsa database handle.
/// Each handle gets its own runtime, but the runtimes have shared state between them.s
/// Each handle gets its own runtime, but the runtimes have shared state between them.
runtime: Runtime,
}