mirrors/salsa
1
0
Fork 0
mirror of https://github.com/salsa-rs/salsa.git synced 2024-11-28 17:42:00 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

add RFC 4, 5

Browse source
This commit is contained in:
Niko Matsakis 2020-06-27 10:25:20 +00:00
parent baee86bc25
commit b953732981
3 changed files with 380 additions and 1 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
book/src/SUMMARY.md
View file

@ -15,3 +15,5 @@
- [RFC 0001: Query group traits](./rfcs/RFC0001-Query-Group-Traits.md)
- [RFC 0002: Intern queries](./rfcs/RFC0002-Intern-Queries.md)
- [RFC 0003: Query dependencies](./rfcs/RFC0003-Query-Dependencies.md)
- [RFC 0004: LRU](./rfcs/RFC0004-LRU.md)
- [RFC 0005: Durability](./rfcs/RFC0005-Durability.md)

73
book/src/rfcs/RFC0004-LRU.md Normal file
View file

@ -0,0 +1,73 @@
# Summary
Add Least Recently Used values eviction as a supplement to garbage collection.
# Motivation
Currently, the single mechanism for controlling memory usage in salsa is garbage
collection. Experience with rust-analyzer shown that it is insufficient for two
reasons:
* It's hard to determine which values should be collected. Current
implementation in rust-analyzer just periodically clears all values of
specific queries.
* GC is in generally run in-between revision. However, especially after just
opening the project, the number of values *within a single revision* can be
high. In other words, GC doesn't really help with keeping peak memory usage
under control. While it is possible to run GC concurrently with calculations
(and this is in fact what rust-analyzer is doing right now to try to keep high
water mark of memory lower), this is highly unreliable an inefficient.
The mechanism of LRU targets both of these weaknesses:
* LRU tracks which values are accessed, and uses this information to determine
which values are actually unused.
* LRU has a fixed cap on the maximal number of entries, thus bounding the memory
usage.
# User's guide
It is possible to call `set_lru_capacity(n)` method on any non-input query. The
effect of this is that the table for the query stores at most `n` *values* in
the database. If a new value is computed, and there are already `n` existing
ones in the database, the least recently used one is evicted. Note that
information about query dependencies is **not** evicted. It is possible to
change lru capacity at runtime at any time. `n == 0` is a special case, which
completely disables LRU logic. LRU is not enabled by default.
# Reference guide
Implementation wise, we store a linked hash map of keys, in the recently-used
order. Because reads of the queries are considered uses, we now need to
write-lock the query map even if the query is fresh. However we don't do this
bookkeeping if LRU is disabled, so you don't have to pay for it unless you use
it.
A slight complication arises with volatile queries (and, in general, with any
query with an untracked input). Similarly to GC, evicting such a query could
lead to an inconsistent database. For this reason, volatile queries are never
evicted.
# Alternatives and future work
LRU is a compromise, as it is prone to both accidentally evicting useful queries
and needlessly holding onto useless ones. In particular, in the steady state and
without additional GC, memory usage will be proportional to the lru capacity: it
is not only an upper bound, but a lower bound as well!
In theory, some deterministic way of evicting values when you for sure don't
need them anymore maybe more efficient. However, it is unclear how exactly that
would work! Experiments in rust-analyzer show that it's not easy to tame a
dynamic crate graph, and that simplistic phase-based strategies fall down.
It's also worth noting that, unlike GC, LRU can in theory be *more* memory
efficient than deterministic memory management. Unlike a traditional GC, we can
safely evict "live" objects and recalculate them later. That makes possible to
use LRU for problems whose working set of "live" queries is larger than the
available memory, at the cost of guaranteed recomputations.
Currently, eviction is strictly LRU base. It should be possible to be smarter
and to take size of values and time that is required to recompute them into
account when making decisions about eviction.

304
book/src/rfcs/RFC0005-Durability.md Normal file
View file

@ -0,0 +1,304 @@
# Summary
- Introduce a user-visibile concept of `Durability`
- Adjusting the "durability" of an input can allow salsa to skip a lot of validation work
- Garbage collection -- particularly of interned values -- however becomes more complex
- Possible future expansion: automatic detection of more "durable" input values
# Motivation
## Making validation faster by optimizing for "durability"
Presently, salsa's validation logic requires traversing all
dependencies to check that they have not changed. This can sometimes
be quite costly in practice: rust-analyzer for example sometimes
spends as much as 90ms revalidating the results from a no-op
change. One option to improve this is simply optimization --
[salsa#176] for example reduces validation times significantly, and
there remains opportunity to do better still. However, even if we are
able to traverse the dependency graph more efficiently, it will still
be an O(n) process. It would be nice if we could do better.
[salsa#176]: https://github.com/salsa-rs/salsa/pull/176
One observation is that, in practice, there are often input values
that are known to change quite infrequently. For example, in
rust-analyzer, the standard library and crates downloaded from
crates.io are unlikely to change (though changes are possible; see
below). Similarly, the `Cargo.toml` file for a project changes
relatively infrequently compared to the sources. We say then that
these inputs are more **durable** -- that is, they change less frequently.
This RFC proposes a mechanism to take advantage of durability for
optimization purposes. Imagine that we have some query Q that depends
solely on the standard library. The idea is that we can track the last
revision R when the standard library was changed. Then, when
traversing dependencies, we can skip traversing the dependencies of Q
if it was last validated after the revision R. Put another way, we
only need to traverse the dependencies of Q when the standard library
changes -- which is unusual. If the standard library *does* change,
for example by user's tinkering with the internal sources, then yes we
walk the dependencies of Q to see if it is affected.
# User's guide
## The durability type
We add a new type `salsa::Durability` which has there associated constants:
```rust
#[derive(Copy, Clone, Debug, Ord)]
pub struct Durability(..);
impl Durability {
// Values that change regularly, like the source to the current crate.
pub const LOW: Durability;
// Values that change infrequently, like Cargo.toml.
pub const MEDIUM: Durability;
// Values that are not expected to change, like sources from crates.io or the stdlib.
pub const HIGH: Durability;
}
```
h## Specifying the durability of an input
When setting an input `foo`, one can now invoke a method
`set_foo_with_durability`, which takes a `Durability` as the final
argument:
```rust
// db.set_foo(key, value) is equivalent to:
db.set_foo_with_durability(key, value, Durability::LOW);
// This would indicate that `foo` is not expected to change:
db.set_foo_with_durability(key, value, Durability::HIGH);
```
## Durability of interned values
Interned values are always considered `Durability::HIGH`. This makes
sense as many queries that only use high durability inputs will also
make use of interning internally. A consequence of this is that they
will not be garbage collected unless you use the specific patterns
recommended below.
## Synthetic writes
Finally, we add one new method, `synthetic_write(durability)`,
available on the salsa runtime:
```rust
db.salsa_runtime().synthetic_write(Durability::HIGH)
```
As the name suggests, `synthetic_write` causes salsa to act *as
though* a write to an input of the given durability had taken
place. This can be used for benchmarking, but it's also important to
controlling what values get garbaged collected, as described below.
## Tracing and garbage collection
Durability affects garbage collection. The `SweepStrategy` struct is
modified as follows:
```rust
/// Sweeps values which may be outdated, but which have not
/// been verified since the start of the current collection.
/// These are typically memoized values from previous computations
/// that are no longer relevant.
pub fn sweep_outdated(self) -> SweepStrategy;
/// Sweeps values which have not been verified since the start
/// of the current collection, even if they are known to be
/// up to date. This can be used to collect "high durability" values
/// that are not *directly* used by the main query.
///
/// So, for example, imagine a main query `result` which relies
/// on another query `threshold` and (indirectly) on a `threshold_inner`:
///
/// ```
/// result(10) [durability: Low]
/// |
/// v
/// threshold(10) [durability: High]
/// |
/// v
/// threshold_inner(10) [durability: High]
/// ```
///
/// If you modify a low durability input and then access `result`,
/// then `result(10)` and its *immediate* dependencies will
/// be considered "verified". However, because `threshold(10)`
/// has high durability and no high durability input was modified,
/// we will not verify *its* dependencies, so `threshold_inner` is not
/// verified (but it is also not outdated).
///
/// Collecting unverified things would therefore collect `threshold_inner(10)`.
/// Collecting only *outdated* things (i.e., with `sweep_outdated`)
/// would collect nothing -- but this does mean that some high durability
/// queries that are no longer relevant to your main query may stick around.
///
/// To get the most precise garbage collection, do a synthetic write with
/// high durability -- this will force us to verify *all* values. You can then
/// sweep unverified values.
pub fn sweep_unverified(self) -> SweepStrategy;
```
# Reference guide
## Review: The need for GC to collect outdated values
In general, salsa's lazy validation scheme can lead to the accumulation
of garbage that is no longer needed. Consider a query like this one:
```rust
fn derived1(db: &impl Database, start: usize) {
let middle = self.input(start);
self.derived2(middle)
}
```
Now imagine that, on some particular run, we compute `derived1(22)`:
- `derived1(22)`
- executes `input(22)`, which returns `44`
- then executes `derived2(44)`
The end result of this execution will be a dependency graph
like:
```
derived1(22) -> derived2(44)
|
v
input(22)
```
Now. imagine that the user modifies `input(22)` to have the value `45`.
The next time `derived1(22)` executes, it will load `input(22)` as before,
but then execute `derived2(45)`. This leaves us with a dependency
graph as follows:
```
derived1(22) -> derived2(45)
|
v
input(22) derived2(44)
```
Notice that we still see `derived2(44)` in the graph. This is because
we memoized the result in last round and then simply had no use for it
in this round. The role of GC is to collect "outdated" values like
this one.
###Review: Tracing and GC before durability
In the absence of durability, when you execute a query Q in some new
revision where Q has not previously executed, salsa must trace back
through all the queries that Q depends on to ensure that they are
still up to date. As each of Q's dependencies is validated, we mark it
to indicate that it has been checked in the current revision (and
thus, within a particular revision, we would never validate or trace a
particular query twice).
So, to continue our example, when we first executed `derived1(22)`
in revision R1, we might have had a graph like:
```
derived1(22) -> derived2(44)
[verified: R1] [verified: R1]
|
v
input(22)
```
Now, after we modify `input(22)` and execute `derived1(22)` again, we
would have a graph like:
```
derived1(22) -> derived2(45)
[verified: R2] [verified: R2]
|
v
input(22) derived2(44)
[verified: R1]
```
Note that `derived2(44)`, the outdated value, never had its "verified"
revision updated, because we never accessed it.
Salsa leverages this validation stamp to serve as the "marking" phase
of a simple mark-sweep garbage collector. The idea is that the sweep
method can collect any values that are "outdated" (whose "verified"
revision is less than the current revision).
The intended model is that one can do a "mark-sweep" style garbage
collection like so:
```rust
// Modify some input, triggering a new revision.
db.set_input(22, 45);
// The **mark** phase: execute the "main query", with the intention
// that we wish to retain all the memoized values needed to compute
// this main query, but discard anything else. For example, in an IDE
// context, this might be a "compute all errors" query.
db.derived1(22);
// The **sweep** phase: discard anything that was not traced during
// the mark phase.
db.sweep_all(...);
```
In the case of our example, when we execute `sweep_all`, it would
collect `derived2(44)`.
## Challenge: Durability lets us avoid tracing
This tracing model is affected by the move to durability. Now, if some
derived value has a high durability, we may skip tracing its
descendants altogether. This means that they would never be "verified"
-- that is, their "verified date" would never be updated.
This is why we modify the definition of "outdated" as follows:
- For a query value `Q` with durability `D`, let `R_lc` be the revision when
values of durability `D` last changed. Let `R_v` be the revision when
`Q` was last verified.
- `Q` is outdated if `R_v < R_lc`.
- In other words, if `Q` may have changed since it was last verified.
## Collecting interned and untracked values
Most values can be collected whenever we like without influencing
correctness. However, interned values and those with untracked
dependencies are an exception -- **they can only be collected when
outdated**. This is because their values may not be reproducible --
in other words, re-executing an interning query (or one with untracked
dependencies, which can read arbitrary program state) twice in a row
may produce a different value. In the case of an interning query, for
example, we may wind up using a different integer than we did before.
If the query is outdated, this is not a problem: anything that
dependend on its result must also be outdated, and hence would be
re-executed and would observe the new value. But if the query is *not*
outdated, then we could get inconsistent result.s
# Alternatives and future work
## Rejected: Arbitrary durabilities
We considered permitting arbitrary "levels" of durability -- for
example, allowing the user to specify a number -- rather than offering
just three. Ultimately it seemed like that level of control wasn't
really necessary and that having just three levels would be sufficient
and simpler.
## Rejected: Durability lattices
We also considered permitting a "lattice" of durabilities -- e.g., to
mirror the crate DAG in rust-analyzer -- but this is tricky because
the lattice itself would be dependent on other inputs.