resident page count is independent from page handler. Showing resident
pages in the guest memory while vmm-swap is disabled is useful for
tracking vmm-swap usage.
This also include typo fix regident -> resident.
BUG=b:291193201
TEST=manually tested
Change-Id: Ib1d0fbd3958705446c73481abf6261bbffe134a0
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4705404
Reviewed-by: David Stevens <stevensd@chromium.org>
Commit-Queue: Shin Kawamura <kawasin@google.com>
Existing compute_resident_pages() has been invalid since it does not
take removed pages by MADV_REMOVE by virtio-balloon into account.
The overhead from lseek(2) syscall is acceptable because crosvm swap
status command is not called frequently and it only takes less than 100
milliseconds.
Removed `Region::swap_active` as well since it is no longer used.
Alternatives:
* Reduce the counter on `UFFD_EVENT_REMOVE`
* This is incorrect because the pages designated by the address range
of `UFFD_EVENT_REMOVE` may have been empty originally.
* Track the active pages in the guest memory using a list/map data
structure.
* We can reduce the syscall overhead from `lseek(2)`. However the data
structure only used for the resident memory counting is a waste of
memory.
BUG=b:291193201
TEST=manually tested
Change-Id: I7254c93a73b777e16d8863a67cf5639b4efb4aad
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4686451
Reviewed-by: David Stevens <stevensd@chromium.org>
Commit-Queue: Shin Kawamura <kawasin@google.com>
Truncating a large file can result in a significant amount of IO for
updating the filesystem metadata. When swap is disabled because an app
is being launched, this IO ends up competing with the IO needed to
launch the app, resulting in significant latency.
When disabling swap because of an app launch, cleaning up the swap file
is not high priority. However, when disabling swap because of low disk
space, the swap file needs to be cleaned up immediately. As such, make
the slow cleanup of the swap file an extra parameter to disabling swap.
BUG=b:288989607
TEST=tast run -buildbundle=crosint DUT arc.SwapAppLatency
TEST=cargo test -p swap --features enable
Change-Id: Icab2a55cdb9350cbb9e380583a31b546bd8e23cf
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4653111
Reviewed-by: Junichi Uekawa <uekawa@chromium.org>
Commit-Queue: David Stevens <stevensd@chromium.org>
Reviewed-by: Shin Kawamura <kawasin@google.com>
Reviewed-by: Keiichi Watanabe <keiichiw@chromium.org>
Reduce logging verbosity of swap status messages. Whatever made the
status request can log the message if it is important.
BUG=None
TEST=CQ
Change-Id: I88a7293ec156fe1d51d7115cfb08802f4f03b02b
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4630876
Reviewed-by: Shin Kawamura <kawasin@google.com>
Commit-Queue: David Stevens <stevensd@chromium.org>
userfaultfd is not available in unit test environment but only in
integration tests. Use FakeDeadUffdChecker for unit testing.
BUG=b:266898615
TEST=./tools/run_tests2
Change-Id: I146ee5b798304c5a7f898fa4c39a466055fe3ed8
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4562369
Commit-Queue: Shin Kawamura <kawasin@google.com>
Reviewed-by: Ningyuan Wang <ningyuan@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
We will add unit tests for UffdList. Having a single file for UffdList
is easier to write unit tests.
This adds a minor refactoring to register the main uffd to WaitContext
within UffdList logic. Also removed the comment for the main uffd
registeration due to blocking on fork because it is obsolete since we
don't use the fork feature of userfaultfd(2).
BUG=b:266898615
TEST=./tools/run_tests2
Change-Id: I53f6d9944d282f7a7528dd6882cd7527031af1b8
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4562368
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Reviewed-by: Ningyuan Wang <ningyuan@google.com>
Commit-Queue: Shin Kawamura <kawasin@google.com>
Hotplug devices are forked from JailWarden process due to minijail
workaround. It is possible to pass `&mut SwapController` to the
JailWarden process on forking, but the ownership of the reference is not
clear. The actual ownership of the object is cloned when forking at the
higher call stack, however it is not clear for readers.
Introduce swap::SwapDeviceHelper to make the ownership clear. The
JailWarden process will hold the ownership of swap::SwapDeviceHelper.
BUG=b:266898615
TEST=./tools/run_tests2
Change-Id: I64edf9e41a70edfd156bcca060656b5641d785fb
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4560804
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Shin Kawamura <kawasin@google.com>
Reviewed-by: Ningyuan Wang <ningyuan@google.com>
When a hot plug device is detached, its uffd becomes dead. This commit
adds a garbage collector to remove dead uffds and prevent the uffd list
and obsolete opened file descriptors from growing indefinitely.
BUG=b:267124393
BUG=b:281791015
BUG=b:266898615
TEST=https://crrev.com/c/4562369
Change-Id: I11d3298b8e4838bbb843e4dc10f29f32a02b0646
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4525480
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Shin Kawamura <kawasin@google.com>
The monitor process blocks the new device process until the new
userfaultfd is set up to guarantee the guest memory is tracked by the
new userfaultfd.
This is required to support hotplug devices.
BUG=b:266898615
TEST=Confirm crosvm boots with vmm-swap.
Change-Id: I4be007b2c7d8f4daf18e95d606938de401fbe1bc
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4520799
Reviewed-by: David Stevens <stevensd@chromium.org>
Commit-Queue: Shin Kawamura <kawasin@google.com>
"enable" feature in swap crate was set as default to run
unit/integration tests of the crate. https://crrev.com/c/4486546
However missing `default-features = false` at non-root Cargo.toml cause
other crates (e.g. x86_64) installing userfaultfd always.
This makes the swap/enable non-default. CQ still runs unit/integration tests of the swap crate on aarch64 and x86_64.
BUG=b:281935498
TEST=cargo test -p swap --features=swap/enable
Change-Id: I57436c3cb8cbdfac04c3145cc599261b89cf4e0e
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4518090
Commit-Queue: Shin Kawamura <kawasin@google.com>
Reviewed-by: Dennis Kempin <denniskempin@google.com>
SwapStatus, SwapState, SwapStateTransition, SwapMetrics are directly
contained into crosvm_control.h
SwapState is translated as C++ enum.
SwapStateTransition::time_ms is now u64 because u128 does not support
repr(C).
Changed usize in the structs to u64 because usize is translated as
uintptr_t.
BUG=b:265386761
TEST=cargo build -p crosvm_control
Change-Id: Ia9df56d40c1884067a712620e0fc797e07af77a2
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4486549
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Reviewed-by: David Stevens <stevensd@chromium.org>
Commit-Queue: Shin Kawamura <kawasin@google.com>
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Status does not depend on any swap related feature. Exposing Status to
all variants is easy to implement swap status FFI at crosvm_control.
The new "swap/enable" feature switches the actual vmm-swap functionality
to be compiled. The feature is enabled by default on "swap" crate but
disabled on the root package for test dependency.
Vmm-swap feature is enabled by `--features=swap` flag on cargo build as
before.
swap/src/controller.rs is copied from swap/src/lib.rs.
BUG=b:265386761
TEST=tools/dev_container tools/run_tests2
Change-Id: Ifc2539a62d0f594fd5bbb41623c735ea2621f7b6
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4486546
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Commit-Queue: Shin Kawamura <kawasin@google.com>
Reviewed-by: David Stevens <stevensd@chromium.org>
The deadlock could happen on this scenario:
1. User enables vmm-swap. The main process sends Command::Enable to the
monitor process.
2. User request the current status at the same time. The main process
sends Command::Status to the monitor process and wait for the
response from the monitor process.
3. The monitor process start enabling vmm-swap. Sends
VmSwapCommand::Suspend to the main process and wait for
VmSwapResponse::SuspendCompleted.
4. The main process is blocked by the step 2 and the request from step 3
is never consumed.
The root issue issue is that Command::Status can be inserted between
Command::Enable and VmSwapCommand::Suspend. This CL simplifies the
communication between the main and monitor processes on enabling
vmm-swap and resolve the deadlock.
VmSwapCommand and corresponding Tube was added by
https://crrev.com/c/4293656, but is now removed by this CL.
BUG=b:275671628
TEST=manual test
Change-Id: Ia838b1feddb4a3f41bd729e9147adc5e2df866aa
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4387662
Reviewed-by: David Stevens <stevensd@chromium.org>
Commit-Queue: Shin Kawamura <kawasin@google.com>
"crosvm swap trim <socket_path>" command for trimming 2 types of pages
in the staging memory before swapping out to reduce the disk I/O.
* clean pages
* zero pages
The doc comment of TrimContext explains the page types.
BUG=b:265592787
TEST=cargo test -p swap
Change-Id: I5c33149f7d0bfd712f07fd11eb9aa07c1a8b0e7a
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4357224
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Shin Kawamura <kawasin@google.com>
Reviewed-by: David Stevens <stevensd@chromium.org>
Another background thread will execute trimming in the next CL
(https://crrev.com/c/4357224). BackgroundJobControl will make the logic
reusable for trim.
BUG=b:265592787
TEST=cargo test -p swap
Change-Id: Id4761c6680e8d503f5f375ab410e253a42b1edf9
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4364564
Commit-Queue: Shin Kawamura <kawasin@google.com>
Reviewed-by: David Stevens <stevensd@chromium.org>
Introduce a struct argument, MemoryRegionInformation, for use with
with_regions.
No functional change intended.
BUG=b:232360323
Change-Id: Icd40630c32878b3bd185aeeedcf9e1e9de20e20b
Suggested-by: Frederick Mayle <fmayle@google.com>
Signed-off-by: Elliot Berman <quic_eberman@quicinc.com>
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4379522
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
ARCVM tracks its memory usage by monitoring the process group shared
memory usage `Platform.Memory.ARC.Shmem`. The staging memory should be
a shared memory not a private anonymous memory.
The staging memory in Region originally does not need to be `Option`
because leaked memory on the staging memory is removed when vmm-swap is
disabled. Rather it has introduced useless complexity to the codebase.
BUG=b:269682102
TEST=cargo test -p swap
Change-Id: I3f032340753b14519dc7dcd76690a7b4b248c1f4
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4332624
Reviewed-by: David Stevens <stevensd@chromium.org>
Commit-Queue: Shin Kawamura <kawasin@google.com>
This is for optimizing swap in performance. This improves the app launch
latency during vmm-swap being disabled from 8s to 2s (while it was
originally 900ms if vmm-swap is disabled).
Each handler method is guarded by mutex.
There would be futher optimization to protect region index calculation
only and unblock UFFD_COPY on both main and swap-in threads. However we
must experiment the benefit and the complexity introduced to the
codebase from it first.
BUG=b:265606668
TEST=cargo test -p swap
Change-Id: I556d7a616da1843fbff8d02d80c85550c7eed93a
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4317028
Reviewed-by: David Stevens <stevensd@chromium.org>
Commit-Queue: Shin Kawamura <kawasin@google.com>
Separate the main loop in monitor_process() into the cases when vmm-swap
is enabled and disabled. This is required for swap in on a background
thread support (https://crrev.com/c/4317028) because `PageHandler`
should not be wrapped with `Option` to use the reference of it from a
background thread spawned in `std:🧵:scope()`.
This also refactor several things related to the main loop:
* Remove obsolete `UffdEvent::Fork` related code.
* Swap all in before exit.
* Continue working even if swap_out fails.
* move_guest_to_staging() to be more robust for failures.
BUG=b:271062908
BUG=b:265606668
TEST=manual test
Change-Id: Ie36efc789905483a85ba735260ef1b49da0bd779
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4323037
Reviewed-by: David Stevens <stevensd@chromium.org>
Commit-Queue: Shin Kawamura <kawasin@google.com>
Pre-populating page cache improves the latency of swap-in which I/O wait
mostly have taken up. This was because page cache populations were
triggered on page fault basis. Since swap-in operation moves pages
sequentially, pre-populating has a benefit on the latency.
MADV_WILLNEED causes the kernel to asynchronously start populating the
pages, and MLOCK_ONFAULT ensures that the data remains in the page cache
until it is UFFD_COPYed. mlock(2) without MLOCK_ONFAULT is not
applicable because it blocks the calling thread until populated.
MADV_DONTNEED is also important in terms of the swap-in latency. It
frees the page cache which is already UFFD_COPYed to be reused for later
readaheading swap file. Otherwise the memory pressure from the swap file
page cache population triggers system-wide page frame collection which
is heavy.
Pre-populating page cache alone is actually not enough to improve the
swap-in latency without swapping in on a background thread
(https://crrev.com/c/4317028).
BUG=b:265606668
TEST=cargo test -p swap
Change-Id: I2069c1260d0cf45499298999a71621e563f28f30
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4314186
Reviewed-by: David Stevens <stevensd@chromium.org>
Commit-Queue: Shin Kawamura <kawasin@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
A page fault handling may be executed concurrently during swapping in.
This is the first step of vmm-swap in optimization. At the last step
(https://crrev.com/c/4317028), the background thread executes swapping
in.
BUG=b:265606668
TEST=cargo test -p swap
Change-Id: I42ec6b75485c31c9db26c22aa115b50288951b06
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4314185
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Reviewed-by: David Stevens <stevensd@chromium.org>
Commit-Queue: Shin Kawamura <kawasin@google.com>
As you can see in the doc comment of `Lazy::force()`, `Lazy::force()`
"is equivalent to the `Deref` impl, but is explicit".
BUG=none
TEST=manual test
Change-Id: I77b5e6ab1985e7efe1bdb86eefb5271438c451fa
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4333208
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Shin Kawamura <kawasin@google.com>
The monitor process uses the `jail` crate which devices use to create
sandbox.
The syscalls listed in the seccomp filter policy file is originally
generated from a profile by strace. Also there are additional syscalls
from common_device.policy:
* another variant of syscalls in the profile
* clone, dup, readlinkat
* the basic set which will be added by minijail compiler anyway.
* restart_syscall, exit, exit_group, rt_sigreturn
* syscalls appears only on DUT (not workstation).
* set_robust_list, sigaltstack, rseq
Used `common_device.policy` as a reference for syscalls which require
detailed conditions (e.g. clone, mmap, openat, etc).
This adds seccomp filter policy only for x86_64. The policy files for
other architectures will be added later.
BUG=b:258351526
TEST=manually tested
Change-Id: I3e584449ed9330a57ae1d2bd6c56a7554b6584ef
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4253073
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Reviewed-by: David Stevens <stevensd@chromium.org>
Commit-Queue: Shin Kawamura <kawasin@google.com>
The design sending SIGSTOP to the main process and the device processes
from the monitor process does not work with sandboxing the monitor
process because user namespace without pid namespace is not supported by
minijail.
This changes the design so that monitor process sends a message to the
main process, which is then responsible for guaranteeing that nothing
except the monitor process will access guest memory when vmm-swap is
being enabled. The main process does this by sending SIGSTOP to device
processes and suspending the vCPU threads. Although there are other
threads in the main process, none of them currently access guest memory.
The vmm-swap feature now does not support `--disable-sandbox`. It would
be possible to support it by sending `SIGSTOP` from the monitor process
to the main process. However there is no clear use case for vmm-swap
with `--disable-sandbox`, so the extra complexity is not worth it.
BUG=b:270248453
TEST=manual test
Change-Id: Ie24d5a5b5f8e6999d133beb9b4c3562e26427838
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4293656
Reviewed-by: David Stevens <stevensd@chromium.org>
Commit-Queue: Shin Kawamura <kawasin@google.com>
forking a process with multiple threads will lead to
undefined behavior. Unfortunately, newer versions of
libtest will spawn multiple threads even when running
a with --test-threads=1.
This change implements a custom, test harness using
libtest-mimic. It mimics the libtest CLI but enforces
a single threaded test process.
BUG=b:270167741
BUG=b:268496046
TEST=tools/run_tests --dut=host
Change-Id: Icef6a1b65bab7f5cd5021c01fbd94487fa0ca5fb
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4295157
Reviewed-by: George Burgess <gbiv@chromium.org>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Frederick Mayle <fmayle@google.com>
If the pages on the guest memory MADV_REMOVEed, the pages are likely not
be accessed for a long time. Keeping those pages is not good use of
resources.
This also fix the minor bug that staging memory is not cleared by
MADV_REMOVE.
BUG=b:269981962
BUG=b:269531558
TEST=cargo test -p swap
Change-Id: Ic43e21104becd8a1dfca32a2cd0733bf40154fa1
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4269719
Reviewed-by: David Stevens <stevensd@chromium.org>
Commit-Queue: Shin Kawamura <kawasin@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Opeing a file on the monitor process is blocking the upcoming minijail
migration. Open the swap file on the main process and pass the file
descriptor to the monitor process.
Using a single swap file and mmap different ranges of it for different
regions is efficient.
BUG=b:269372016
TEST=cargo test -p swap
Change-Id: I4be150419d263ebe76ad00dae2cafce6e8660cbd
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4253294
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Shin Kawamura <kawasin@google.com>
Reviewed-by: David Stevens <stevensd@chromium.org>
page fault logger is only for debugging purpose. It has a security
concern about exposing the trigger to enable the logger. The logger will
make the page fault handling slower and consumes disk resources.
Instead, the page fault logger is only enabled by compiler feature flag
like:
```
cargo build --features=swap,swap/log_page_fault
```
Opening a file on the monitor process is blocking the upcoming minijail
migration. Open the log file on the main process and pass the file
descriptor to the monitor process.
BUG=b:269372016
TEST=manual test
Change-Id: I5df2a32da7fb149680fa24617ffd905b2483108b
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4253287
Reviewed-by: David Stevens <stevensd@chromium.org>
Commit-Queue: Shin Kawamura <kawasin@google.com>
On some Linux hpage_pmd_size returns the size with a line break which
cause failure on parse.
BUG=b:258351526
TEST=manual test
Change-Id: If8b9a43022bd0dee5035069b9aaa68114ebaaa86
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4252438
Reviewed-by: David Stevens <stevensd@chromium.org>
Commit-Queue: Shin Kawamura <kawasin@google.com>
USERFAULTFD_IOC_NEW works with /dev/userfaultfd opened with at least
either of read or write permission. Crosvm opens the file as read-only
because:
* we should keep permission as small as possible and,
* minijail for ARCVM supports either of read-only or read-write
permission.
BUG=b:268148895
TEST=manual test
Change-Id: Ie02915fd232d710f62de7a280a51d56c9f50c577
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4235682
Reviewed-by: David Stevens <stevensd@chromium.org>
Commit-Queue: Shin Kawamura <kawasin@google.com>
userfaultfd(2) syscall requires CAP_SYS_PTRACE of root user namespace.
For better permission control /dev/userfaultfd was introduced from Linux
6.1. Since ARCVM runs inside a sandbox user namespace and enabling
/proc/sys/vm/unprivileged_userfaultfd opens up userfaultfd to the entire
system, crosvm for ARCVM has to use /dev/userfaultfd to create a
userfaultfd.
For systems which do not have /dev/userfaultfd, the Factory falls back
to userfaultfd(2).
BUG=b:268145007
BUG=b:266642532
TEST=manual test
Change-Id: I44b657877093d2a75627432619c1233b7ac2464e
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4196763
Commit-Queue: Shin Kawamura <kawasin@google.com>
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Reviewed-by: David Stevens <stevensd@chromium.org>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Re-format the response of `crosvm swap status` command and make it easy
to parse structure as JSON.
BUG=b:265606668
TEST=cargo test -p swap
Change-Id: Ia5216a6ae4b2083bce3829848687cebf932d0933
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4216453
Commit-Queue: Shin Kawamura <kawasin@google.com>
Reviewed-by: David Stevens <stevensd@chromium.org>
This is a reland of commit d39e5811df
This change mark VolatileRef as deprecated instead of removing it
completely. This change also removed all related methods and functions
using VolatileRef that libcras don't use.
Original change's description:
> data_model: remove VolatileRef
>
> All uses except in test code have been eliminated, so we can remove it
> now.
>
> This was an unsafe abstraction, and we have better alternatives (such as
> the read_obj()/write_obj() functions) that do not create a long-lived
> mutable reference that could easily alias other slices.
>
> BUG=None
> TEST=tools/presubmit --all
>
> Change-Id: I84f1e2487d6211ce24b8fc992fa7675765870132
> Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/3824000
> Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
> Reviewed-by: Keiichi Watanabe <keiichiw@chromium.org>
> Reviewed-by: Alexandre Courbot <acourbot@chromium.org>
TESTED=CQ
BUG=b:204409584
FIXED=b:236759218
Change-Id: I4019870a2321fcd8610669862b5e6ed9bf7c2282
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4215512
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Zihan Chen <zihanchen@google.com>
This is a refactoring to use original Error type instead of UffdError
from userfaultfd crate. This helps /dev/userfaultfd support which
returns std::io::Error on opening userfaultfd.
This contains a minor change to move register/unregister helpers to
userfaultfd.rs
BUG=b:266642532
TEST=cargo test -p swap
Change-Id: I24079999b018c0f1683fd7b4ee0f97153bd8914d
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4196761
Reviewed-by: David Stevens <stevensd@chromium.org>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Shin Kawamura <kawasin@google.com>
This implements the equivalent logic on crosvm as
UFFD_FEATURE_EVENT_FORK. When each device process forks, the ProxyDevice
creates userfaultfd and send it to the monitor process by
SwapController::on_process_forked().
Crosvm does not have any child processes which may access the guest
memory except device processes as of now. Crosvm forks
virgl_render_server, but the mmap is not preserved in the process on
execve(2) since it is a different binary. Also no device process forks
grandchild processes according to the seccomp policy.
We actually can't use UFFD_FEATURE_EVENT_FORK because the feature does
not support non-root user namespace (go/uffd-fork-user-ns) and ARCVM
runs in a non-root user namespace.
This also adds syscalls to seccomp policies for devices to allow the
processes to create and setup a userfaultfd.
BUG=b:266641923
TEST=manually tested
Change-Id: Ide3088e1e95ae3c8259e3f4324124b3376e760b7
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4194228
Reviewed-by: David Stevens <stevensd@chromium.org>
Commit-Queue: Shin Kawamura <kawasin@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Enabling vmm-swap copies all the guest memory to the staging memory
while freezing whole the crosvm. Reducing the latency of it is
important.
The most time consuming part of enabing vmm-swap is pure memory copy
which easily can be executed on the multi-thread.
BUG=b:263830401
TEST=cargo test -p swap
Change-Id: I307ad1c459158113cc654e7ba1541381a38b65bb
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4142083
Reviewed-by: David Stevens <stevensd@chromium.org>
Commit-Queue: Shin Kawamura <kawasin@google.com>
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Instead of configuring which crates to --exclude in
test_config.py, we can use conditional compilation to
exclude code that is not supported on windows.
This allows more fine-grained control and also allows
us to use plain cargo for building without complicated
configuration and exclusions.
BUG=b:265829867
TEST=cargo test --lib --bins --workspace
--target=x86_64-pc-windows-gnu
--features=all-mingw64
Change-Id: I8422c3f08053bc27d9896b220876a56bd25543d6
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4165868
Reviewed-by: Vikram Auradkar <auradkar@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Both StagingMemory and SwapFile have similar requirements for tracking
page state. Create a single PresentList utility with the needed
functionality, and refactor both to use the utility.
This change helps the effort to swap-in concurrently.
BUG=b:265606668
TEST=cargo test -p swap
Change-Id: If21649de918d9b98fdcbf895e5368b46c277b8e3
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4164548
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Reviewed-by: David Stevens <stevensd@chromium.org>
Commit-Queue: Shin Kawamura <kawasin@google.com>
When Transparent Hugepage on the host linux is enabled and crosvm runs
with `--hugepages` flag enabled, MADV_REMOVE can be unreliable if it
isn't aligned to THP size. On kernels without 71725ed10c40 ("mm: huge
tmpfs: try to split_huge_page() when punching hole"), MADV_REMOVE will
simply zero out subpages. Even with that change, if anything else in the
kernel happens to hold a ref to a THP concurrently with MADV_REMOVE,
then the THP cannot be split and will simply be zeroed. When THP aren't
split, subsequent faults will be minor faults instead of major faults
and thus won't be handled by the current vmm-swap implementation.
This patch queries /sys/kernel/mm/transparent_hugepage/hpage_pmd_size
for the THP size. If that file is not available (e.g. due to
sandboxing), then it falls back to a THP size of 2MiB, which is the case
for all common configurations supported by crosvm.
BUG=b:265242249
TEST=cargo test -p swap
Change-Id: I466069d1bc07e7d1e78d57e8830471e0dc3a6c0b
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4160616
Reviewed-by: David Stevens <stevensd@chromium.org>
Commit-Queue: Shin Kawamura <kawasin@google.com>
data_model is probably going away in the future, but
until we can release it as pre-release alpha for internal
efforts.
(context: https://fxbug.dev/114169)
BUG=b:173630595
TEST=compile
Change-Id: Ie13ec3ab9083d315c78a035b532132296a8fb651
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4082333
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Commit-Queue: Gurchetan Singh <gurchetansingh@chromium.org>
We don't expect errors from MADV_REMOVE. But we should check the error
and crash instead of ignoring them.
BUG=b:265356139
TEST=cargo test -p swap
Change-Id: Ia05d88d3478f9a1e23543fafc143c8c8340e3db0
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4160625
Reviewed-by: David Stevens <stevensd@chromium.org>
Commit-Queue: Shin Kawamura <kawasin@google.com>
Added `max_batch_size` parameter to PageHandler::move_to_staging().
Defer the MADV_REMOVE until the size of the batch reaches the max size
to reduce the system calls. If the size of a chunk is too big, it splits
the chunk into different batches to avoid memory usage spike.
BUG=b:263830401
TEST=cargo test -p swap
Change-Id: I8ca7e592a311980271cffd9bba8fbbe2a582ac3e
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4151867
Reviewed-by: David Stevens <stevensd@chromium.org>
Commit-Queue: Shin Kawamura <kawasin@google.com>
On `crosvm swap out` command, the monitor process starts writing the
active pages in the staging memory to the swap file while it handles
page fault (and other) events concurrently.
The swap out operation is now abortable with `crosvm swap disable`
command.
Further optimizations (e.g. moving the memory to staging in
multi-threads, write multiple chunks into the swap file) will be
implemented at later CLs.
BUG=b:263830401
TEST=cargo test -p swap
Change-Id: I0849065e6be29400c823618b53b8c416c224e875
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4123201
Reviewed-by: David Stevens <stevensd@chromium.org>
Commit-Queue: Shin Kawamura <kawasin@google.com>
This splits the `crosvm swap enable` command, which pauses the whole
crosvm processes while it write all the active pages on the guest memory
into the swap file, into 2 commands.
On `crosvm swap enable` command, the monitor process just moves the
guest memroy to the staging memory while blocking the whole crosvm
processes with SIGSTOP.
On `crosvm swap out` command, the monitor process writes the active
pages in the staging memory to the swap file.
The page faults between `swap enable` and `swap out` are swapped in from
the staging memory.
By waiting some time before sending swap out request after enabling
vmm-swap, hot pages on the guest memory, which are frequently accessed,
are swapped in from the staging memory and we can reduce the TBW of the
swap file and reduce the latency of page faults for hot pages.
BUG=b:263830401
TEST=cargo test -p swap -p base
Change-Id: Ia83f5052cc0bd4d25089c6a6680e40a1fa1661c0
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4134149
Reviewed-by: David Stevens <stevensd@chromium.org>
Commit-Queue: Shin Kawamura <kawasin@google.com>
Refactoring to make the codebase simple.
BUG=b:260543132
TEST=cargo test -p swap
Change-Id: If0f28e28f8678ee764867161910ae60c94a89f73
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4125178
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Reviewed-by: David Stevens <stevensd@chromium.org>
Commit-Queue: Shin Kawamura <kawasin@google.com>
Using the cached page size shift has several performance benefits listed
at the comment of the file.
BUG=b:260543132
TEST=cargo test -p swap
Change-Id: Ic7a19135b7a2e29c032c73fa7dca1d853cdb5e48
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4125177
Reviewed-by: David Stevens <stevensd@chromium.org>
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Commit-Queue: Shin Kawamura <kawasin@google.com>
