From b953732981f5bea664efe0884fb82a514e214af0 Mon Sep 17 00:00:00 2001 From: Niko Matsakis Date: Sat, 27 Jun 2020 10:25:20 +0000 Subject: [PATCH] add RFC 4, 5 --- book/src/SUMMARY.md | 4 +- book/src/rfcs/RFC0004-LRU.md | 73 +++++++ book/src/rfcs/RFC0005-Durability.md | 304 ++++++++++++++++++++++++++++ 3 files changed, 380 insertions(+), 1 deletion(-) create mode 100644 book/src/rfcs/RFC0004-LRU.md create mode 100644 book/src/rfcs/RFC0005-Durability.md diff --git a/book/src/SUMMARY.md b/book/src/SUMMARY.md index a1b56c4..b350829 100644 --- a/book/src/SUMMARY.md +++ b/book/src/SUMMARY.md @@ -14,4 +14,6 @@ - [Template](./rfcs/template.md) - [RFC 0001: Query group traits](./rfcs/RFC0001-Query-Group-Traits.md) - [RFC 0002: Intern queries](./rfcs/RFC0002-Intern-Queries.md) - - [RFC 0003: Query dependencies](./rfcs/RFC0003-Query-Dependencies.md) \ No newline at end of file + - [RFC 0003: Query dependencies](./rfcs/RFC0003-Query-Dependencies.md) + - [RFC 0004: LRU](./rfcs/RFC0004-LRU.md) + - [RFC 0005: Durability](./rfcs/RFC0005-Durability.md) \ No newline at end of file diff --git a/book/src/rfcs/RFC0004-LRU.md b/book/src/rfcs/RFC0004-LRU.md new file mode 100644 index 0000000..6885353 --- /dev/null +++ b/book/src/rfcs/RFC0004-LRU.md @@ -0,0 +1,73 @@ +# Summary + +Add Least Recently Used values eviction as a supplement to garbage collection. + +# Motivation + +Currently, the single mechanism for controlling memory usage in salsa is garbage +collection. Experience with rust-analyzer shown that it is insufficient for two +reasons: + +* It's hard to determine which values should be collected. Current + implementation in rust-analyzer just periodically clears all values of + specific queries. + +* GC is in generally run in-between revision. However, especially after just + opening the project, the number of values *within a single revision* can be + high. In other words, GC doesn't really help with keeping peak memory usage + under control. While it is possible to run GC concurrently with calculations + (and this is in fact what rust-analyzer is doing right now to try to keep high + water mark of memory lower), this is highly unreliable an inefficient. + +The mechanism of LRU targets both of these weaknesses: + +* LRU tracks which values are accessed, and uses this information to determine + which values are actually unused. + +* LRU has a fixed cap on the maximal number of entries, thus bounding the memory + usage. + +# User's guide + +It is possible to call `set_lru_capacity(n)` method on any non-input query. The +effect of this is that the table for the query stores at most `n` *values* in +the database. If a new value is computed, and there are already `n` existing +ones in the database, the least recently used one is evicted. Note that +information about query dependencies is **not** evicted. It is possible to +change lru capacity at runtime at any time. `n == 0` is a special case, which +completely disables LRU logic. LRU is not enabled by default. + +# Reference guide + +Implementation wise, we store a linked hash map of keys, in the recently-used +order. Because reads of the queries are considered uses, we now need to +write-lock the query map even if the query is fresh. However we don't do this +bookkeeping if LRU is disabled, so you don't have to pay for it unless you use +it. + +A slight complication arises with volatile queries (and, in general, with any +query with an untracked input). Similarly to GC, evicting such a query could +lead to an inconsistent database. For this reason, volatile queries are never +evicted. + +# Alternatives and future work + +LRU is a compromise, as it is prone to both accidentally evicting useful queries +and needlessly holding onto useless ones. In particular, in the steady state and +without additional GC, memory usage will be proportional to the lru capacity: it +is not only an upper bound, but a lower bound as well! + +In theory, some deterministic way of evicting values when you for sure don't +need them anymore maybe more efficient. However, it is unclear how exactly that +would work! Experiments in rust-analyzer show that it's not easy to tame a +dynamic crate graph, and that simplistic phase-based strategies fall down. + +It's also worth noting that, unlike GC, LRU can in theory be *more* memory +efficient than deterministic memory management. Unlike a traditional GC, we can +safely evict "live" objects and recalculate them later. That makes possible to +use LRU for problems whose working set of "live" queries is larger than the +available memory, at the cost of guaranteed recomputations. + +Currently, eviction is strictly LRU base. It should be possible to be smarter +and to take size of values and time that is required to recompute them into +account when making decisions about eviction. diff --git a/book/src/rfcs/RFC0005-Durability.md b/book/src/rfcs/RFC0005-Durability.md new file mode 100644 index 0000000..e3bf012 --- /dev/null +++ b/book/src/rfcs/RFC0005-Durability.md @@ -0,0 +1,304 @@ +# Summary + +- Introduce a user-visibile concept of `Durability` +- Adjusting the "durability" of an input can allow salsa to skip a lot of validation work +- Garbage collection -- particularly of interned values -- however becomes more complex +- Possible future expansion: automatic detection of more "durable" input values + +# Motivation + +## Making validation faster by optimizing for "durability" + +Presently, salsa's validation logic requires traversing all +dependencies to check that they have not changed. This can sometimes +be quite costly in practice: rust-analyzer for example sometimes +spends as much as 90ms revalidating the results from a no-op +change. One option to improve this is simply optimization -- +[salsa#176] for example reduces validation times significantly, and +there remains opportunity to do better still. However, even if we are +able to traverse the dependency graph more efficiently, it will still +be an O(n) process. It would be nice if we could do better. + +[salsa#176]: https://github.com/salsa-rs/salsa/pull/176 + +One observation is that, in practice, there are often input values +that are known to change quite infrequently. For example, in +rust-analyzer, the standard library and crates downloaded from +crates.io are unlikely to change (though changes are possible; see +below). Similarly, the `Cargo.toml` file for a project changes +relatively infrequently compared to the sources. We say then that +these inputs are more **durable** -- that is, they change less frequently. + +This RFC proposes a mechanism to take advantage of durability for +optimization purposes. Imagine that we have some query Q that depends +solely on the standard library. The idea is that we can track the last +revision R when the standard library was changed. Then, when +traversing dependencies, we can skip traversing the dependencies of Q +if it was last validated after the revision R. Put another way, we +only need to traverse the dependencies of Q when the standard library +changes -- which is unusual. If the standard library *does* change, +for example by user's tinkering with the internal sources, then yes we +walk the dependencies of Q to see if it is affected. + +# User's guide + +## The durability type + +We add a new type `salsa::Durability` which has there associated constants: + +```rust +#[derive(Copy, Clone, Debug, Ord)] +pub struct Durability(..); + +impl Durability { + // Values that change regularly, like the source to the current crate. + pub const LOW: Durability; + + // Values that change infrequently, like Cargo.toml. + pub const MEDIUM: Durability; + + // Values that are not expected to change, like sources from crates.io or the stdlib. + pub const HIGH: Durability; +} +``` + +h## Specifying the durability of an input + +When setting an input `foo`, one can now invoke a method +`set_foo_with_durability`, which takes a `Durability` as the final +argument: + +```rust +// db.set_foo(key, value) is equivalent to: +db.set_foo_with_durability(key, value, Durability::LOW); + +// This would indicate that `foo` is not expected to change: +db.set_foo_with_durability(key, value, Durability::HIGH); +``` + +## Durability of interned values + +Interned values are always considered `Durability::HIGH`. This makes +sense as many queries that only use high durability inputs will also +make use of interning internally. A consequence of this is that they +will not be garbage collected unless you use the specific patterns +recommended below. + +## Synthetic writes + +Finally, we add one new method, `synthetic_write(durability)`, +available on the salsa runtime: + +```rust +db.salsa_runtime().synthetic_write(Durability::HIGH) +``` + +As the name suggests, `synthetic_write` causes salsa to act *as +though* a write to an input of the given durability had taken +place. This can be used for benchmarking, but it's also important to +controlling what values get garbaged collected, as described below. + +## Tracing and garbage collection + +Durability affects garbage collection. The `SweepStrategy` struct is +modified as follows: + +```rust +/// Sweeps values which may be outdated, but which have not +/// been verified since the start of the current collection. +/// These are typically memoized values from previous computations +/// that are no longer relevant. +pub fn sweep_outdated(self) -> SweepStrategy; + +/// Sweeps values which have not been verified since the start +/// of the current collection, even if they are known to be +/// up to date. This can be used to collect "high durability" values +/// that are not *directly* used by the main query. +/// +/// So, for example, imagine a main query `result` which relies +/// on another query `threshold` and (indirectly) on a `threshold_inner`: +/// +/// ``` +/// result(10) [durability: Low] +/// | +/// v +/// threshold(10) [durability: High] +/// | +/// v +/// threshold_inner(10) [durability: High] +/// ``` +/// +/// If you modify a low durability input and then access `result`, +/// then `result(10)` and its *immediate* dependencies will +/// be considered "verified". However, because `threshold(10)` +/// has high durability and no high durability input was modified, +/// we will not verify *its* dependencies, so `threshold_inner` is not +/// verified (but it is also not outdated). +/// +/// Collecting unverified things would therefore collect `threshold_inner(10)`. +/// Collecting only *outdated* things (i.e., with `sweep_outdated`) +/// would collect nothing -- but this does mean that some high durability +/// queries that are no longer relevant to your main query may stick around. +/// +/// To get the most precise garbage collection, do a synthetic write with +/// high durability -- this will force us to verify *all* values. You can then +/// sweep unverified values. +pub fn sweep_unverified(self) -> SweepStrategy; +``` + +# Reference guide + +## Review: The need for GC to collect outdated values + +In general, salsa's lazy validation scheme can lead to the accumulation +of garbage that is no longer needed. Consider a query like this one: + +```rust +fn derived1(db: &impl Database, start: usize) { + let middle = self.input(start); + self.derived2(middle) +} +``` + +Now imagine that, on some particular run, we compute `derived1(22)`: + +- `derived1(22)` + - executes `input(22)`, which returns `44` + - then executes `derived2(44)` + +The end result of this execution will be a dependency graph +like: + +``` +derived1(22) -> derived2(44) + | + v +input(22) +``` + +Now. imagine that the user modifies `input(22)` to have the value `45`. +The next time `derived1(22)` executes, it will load `input(22)` as before, +but then execute `derived2(45)`. This leaves us with a dependency +graph as follows: + +``` +derived1(22) -> derived2(45) + | + v +input(22) derived2(44) +``` + +Notice that we still see `derived2(44)` in the graph. This is because +we memoized the result in last round and then simply had no use for it +in this round. The role of GC is to collect "outdated" values like +this one. + +###Review: Tracing and GC before durability + +In the absence of durability, when you execute a query Q in some new +revision where Q has not previously executed, salsa must trace back +through all the queries that Q depends on to ensure that they are +still up to date. As each of Q's dependencies is validated, we mark it +to indicate that it has been checked in the current revision (and +thus, within a particular revision, we would never validate or trace a +particular query twice). + +So, to continue our example, when we first executed `derived1(22)` +in revision R1, we might have had a graph like: + + +``` +derived1(22) -> derived2(44) +[verified: R1] [verified: R1] + | + v +input(22) +``` + +Now, after we modify `input(22)` and execute `derived1(22)` again, we +would have a graph like: + +``` +derived1(22) -> derived2(45) +[verified: R2] [verified: R2] + | + v +input(22) derived2(44) + [verified: R1] +``` + +Note that `derived2(44)`, the outdated value, never had its "verified" +revision updated, because we never accessed it. + +Salsa leverages this validation stamp to serve as the "marking" phase +of a simple mark-sweep garbage collector. The idea is that the sweep +method can collect any values that are "outdated" (whose "verified" +revision is less than the current revision). + +The intended model is that one can do a "mark-sweep" style garbage +collection like so: + +```rust +// Modify some input, triggering a new revision. +db.set_input(22, 45); + +// The **mark** phase: execute the "main query", with the intention +// that we wish to retain all the memoized values needed to compute +// this main query, but discard anything else. For example, in an IDE +// context, this might be a "compute all errors" query. +db.derived1(22); + +// The **sweep** phase: discard anything that was not traced during +// the mark phase. +db.sweep_all(...); +``` + +In the case of our example, when we execute `sweep_all`, it would +collect `derived2(44)`. + +## Challenge: Durability lets us avoid tracing + +This tracing model is affected by the move to durability. Now, if some +derived value has a high durability, we may skip tracing its +descendants altogether. This means that they would never be "verified" +-- that is, their "verified date" would never be updated. + +This is why we modify the definition of "outdated" as follows: + +- For a query value `Q` with durability `D`, let `R_lc` be the revision when + values of durability `D` last changed. Let `R_v` be the revision when + `Q` was last verified. +- `Q` is outdated if `R_v < R_lc`. + - In other words, if `Q` may have changed since it was last verified. + +## Collecting interned and untracked values + +Most values can be collected whenever we like without influencing +correctness. However, interned values and those with untracked +dependencies are an exception -- **they can only be collected when +outdated**. This is because their values may not be reproducible -- +in other words, re-executing an interning query (or one with untracked +dependencies, which can read arbitrary program state) twice in a row +may produce a different value. In the case of an interning query, for +example, we may wind up using a different integer than we did before. +If the query is outdated, this is not a problem: anything that +dependend on its result must also be outdated, and hence would be +re-executed and would observe the new value. But if the query is *not* +outdated, then we could get inconsistent result.s + +# Alternatives and future work + +## Rejected: Arbitrary durabilities + +We considered permitting arbitrary "levels" of durability -- for +example, allowing the user to specify a number -- rather than offering +just three. Ultimately it seemed like that level of control wasn't +really necessary and that having just three levels would be sufficient +and simpler. + +## Rejected: Durability lattices + +We also considered permitting a "lattice" of durabilities -- e.g., to +mirror the crate DAG in rust-analyzer -- but this is tricky because +the lattice itself would be dependent on other inputs. +