This CL transitions most structs to RawDescriptor and the
associated traits if possible.
BUG=b:162363783
TEST=./build_test
Change-Id: Iabae6ac212787836d77de2b9ffb5d451421ab0dd
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2530911
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: Michael Hoyle <mikehoyle@google.com>
Commit-Queue: Michael Hoyle <mikehoyle@google.com>
Instead of creating IoSourceExt from AsRawFd implementers, we've
switched to creating from a marker trait `IntoAsync`. This lets us use
other types like RawDescriptor easily with this crate. By using the
marker, we also provide some type safety by requiring consumers of
IoSourceExt to declare that their type is expected to work with async
operations. This way we can provide stronger guarantees that an async
IO trait object will behave in a reasonable way.
This CL also purges the cros_async -> base and io_uring -> base
references, and provides the base types needed to add new async
primitives to base.
BUG=none
TEST=builds
Change-Id: I0b0ce6ca7938b22ae8e8fb4e604439f0292678f2
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2504481
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Noah Gold <nkgold@google.com>
Reviewed-by: Chirantan Ekbote <chirantan@chromium.org>
The qcow disk format implemented the discard and write zeroes requests
by deallocating the underlying cluster when possible, with the
assumption that a deallocated cluster would return all zeroes if it is
read until it is written again.
Add an alternate implementation of write_zeroes that is used when a
backing file is present that uses the straightforward approach of
allocating all clusters and zeroing them out in the raw file rather than
using deallocation.
BUG=b:171230599
TEST=BLKZEROOUT test case
TEST=cargo test -p disk
Change-Id: I745f6dc7aa411ec9b1be0150ba1bc96c011ada9a
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2490605
Reviewed-by: Dylan Reid <dgreid@chromium.org>
Reviewed-by: Cody Schuffelen <schuffelen@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
This CL makes the following fundamental changes to cros_async:
1. Removes PollOrRing and replaces it with IoSourceExt, and the
subtraits ReadAsync & WriteAsync. The blanket implementation of
IoSourceExt has been dropped, and replaced with source specific
implementations of the trait. Those implementations are where
the code from PollOrRing has been moved.
2. Pinning for IoSource has been dropped from UringSource & the uring
futures. This appears to be safe because the IoSource doesn't contain
any self refs, or perform any operations beyond forwarding to the
RegisteredSource. (The FD is duped before being passed to
RingWakerState by RegisteredSource, so there doesn't seem to be any
data which would require pinning.)
3. U64Source was replaced by EventAsync.
It also switches all Error enums to use thiserror, which reduces
boilerplate.
BUG=None
TEST=cargo test -p cros_async
Cq-Depend: chromium:2421742
Change-Id: Ie1dd958da2e1f8dec1ae1fd8c0b4e754223d330d
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2416996
Reviewed-by: Chirantan Ekbote <chirantan@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Noah Gold <nkgold@google.com>
Note the CL size is large entirely due to the rename,
the changes are mostly negligible.
Also making a few small additional changes in sys_util
areas that don't need much attention in base. This includes
typedefing and adding specific imports for areas that don't
require significant interface changes.
BUG=b:162363783
TEST=./build_test
Change-Id: I4a2c9c4cdce7565806ed338e241c6b8c82c855c7
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2415180
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Michael Hoyle <mikehoyle@google.com>
This is very largely boilerplate, but changes some interfaces
to accept different parameters, esp using Descriptor.
BUG=b:162363783
TEST=./build_test
Change-Id: I81b513c0de9448c2de505fe5d3a61a29f4eccb78
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2342591
Reviewed-by: Zach Reizner <zachr@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Michael Hoyle <mikehoyle@google.com>
The previous disk size (0x1_0000_0000 = 4 GiB) would overflow usize on
32-bit platforms, causing a unit test build failure. Shrink it to half
the size, which is still plenty for the purposes of the tests.
Seek offsets are natively 64 bits wide, even on 32-bit platforms,
whereas usize can be only 32 bits wide, so rearrange the casts such that
the final seek offset and intermediate values used to calculate it are
stored in u64 instead of usize.
BUG=chromium:1112839
TEST=cargo test -p disk
TEST=FEATURES=test emerge-kevin crosvm
Change-Id: Ia25762541104124960d9298e6ad7e56f71aa3031
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2347076
Reviewed-by: Dylan Reid <dgreid@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
* Make AsyncDisk as a supertrait of some sys_util's traits
* Rename write_zeroes to write_zeroes_at
BUG=none
TEST=cargo test
Change-Id: I714d82092b3ee2930383541b67b958dc1a7cf441
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2340732
Reviewed-by: Keiichi Watanabe <keiichiw@chromium.org>
Tested-by: Keiichi Watanabe <keiichiw@chromium.org>
Commit-Queue: Keiichi Watanabe <keiichiw@chromium.org>
For now, this crate simply re-exports all of sys_util, but it will
be updated to provide new interfaces when needed. This is the
first step to making crosvm not directly depend on sys_util, so
that we can make the interface changes we need without fear of
negatively affecting (i.e. completely breaking) other usages
within chromeos.
BUG=b:162363783
TEST=./build_test
Change-Id: I7d0aa3d8a1f66af1c7fee8fd649723ef17027150
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2325168
Tested-by: Michael Hoyle <mikehoyle@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Michael Hoyle <mikehoyle@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Reviewed-by: Zach Reizner <zachr@chromium.org>
The AsyncDisk Trait will be used by block to issue asynchronous
operations on kernels that support async IO.
TEST=Added unit tests
Cq-Depend: chromium:2265040
Change-Id: I83adb83df278c4d19ac5dd4e5b91fb5bf96d4e89
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2306785
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: Dylan Reid <dgreid@chromium.org>
Commit-Queue: Dylan Reid <dgreid@chromium.org>
The VolatileSlice API changed, but some callers were not updated to
match. Fix them up so that builds with additional features enabled
(e.g. kokoro) pass again.
BUG=None
TEST=cargo test -p disk --features=composite-disk
TEST=docker/wrapped_smoke_test.sh
Change-Id: I97b3cd04549af30b7dc1515245f025d9439669bc
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2216399
Tested-by: Daniel Verkamp <dverkamp@chromium.org>
Auto-Submit: Daniel Verkamp <dverkamp@chromium.org>
Reviewed-by: Dylan Reid <dgreid@chromium.org>
Reviewed-by: Zach Reizner <zachr@chromium.org>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
Change VolatileSlice so that it is ABI-compatible with iovec. This
allows us to directly pass in a VolatileSlice for a C function that
expects an iovec without having to create temporaries that convert from
one to the other.
Also change all the parameters from u64 to usize. It's not possible to
address more memory than fits into a usize so having u64 here didn't
really provide much benefit and led to a lot of tedious casting back and
forth all over the place.
BUG=none
TEST=unit tests
Cq-Depend: chromium:2206621
Change-Id: I258f9123c603d9a4c6c5e2d4d10eb4aedf74466d
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2203998
Tested-by: Chirantan Ekbote <chirantan@chromium.org>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Chirantan Ekbote <chirantan@chromium.org>
This was adding ~11 minutes to the boot time with sparse images.
BUG=b:151981838
TEST=launch cuttlefish with sparse images.
Change-Id: I707258566aee338f742a3a5fe94d0bad8302c447
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2111117
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Cody Schuffelen <schuffelen@google.com>
When there is an error opening one of the composite disk components now,
it gives the message `failed to open component file: "No such file or
directory (os error 2)"` without specifying the file path it tried to
use. Exposing the file path will make it faster to act on errors, rather
than trying to examine the composite disk file for paths.
TEST=n/a
BUG=b:150150052
Change-Id: I9341b330e7e6dcd517d5bfb5262b1657a2da46fe
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2072738
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Cody Schuffelen <schuffelen@google.com>
Android defines its own "sparse disk" format, which its images are
usually published in. Cuttlefish has special-cased this to build raw
images in the android build system, but it still causes a performance
hit when downloading and extracting the image zip files. Experimentally,
running bsdtar on the zip file of raw images is about 50 seconds slower
than bsdtar on the equivalent zip file of android sparse images.
These disks can only be opened as read-only, as the Android Sparse
format is designed around writing once then interpreting the contents
while flashing a physical device through e.g. fastboot.
TEST=Run with aosp/1184800 on cuttlefish, unit tests
BUG=b:145841395
Change-Id: I13337b042e92841bd3cba88dc8b231fde88c091e
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/1956487
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Cody Schuffelen <schuffelen@google.com>
This preserves any data that the backing file had on a cluster when
doing a write to a subset of that cluster. These writes cause a
performance penalty on creating new clusters if a backing file is
present.
TEST=unit tests
BUG=b:140069322
Change-Id: I724990225617c05e5f2dea39e39ce84c940328fc
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/1982832
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Cody Schuffelen <schuffelen@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Reads to qcow files with backing files will fall through to the backing
file if there is no allocated cluster. As of this change, a write will
still trash the cluster and hide any data already present.
TEST=unit tests
BUG=b:140069322
Change-Id: Iba353fa1e7c25bb6267eb96b30b8f5a6ac61d423
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/1982831
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
The new functionality can be invoked through
"crosvm create_qcow2 --backing_file=backing new_file".
The old behavior of creating a qcow image with a particular size is
still available with its original syntax.
This is relevant to implement as by default something like qemu-img will
create a new image that assumes the backing file is raw or qcow, while
crosvm can use its knowledge of other formats (such as composite disk,
and later android sparse) to determine the true size of the backing
file.
TEST=unit tests
BUG=b:140069322
Change-Id: I22de6a79c6d8566a9fcb0bc8124e2d74fea9ca55
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/1982833
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
This trait provides a generic interface for allocating space on the
filesystem within a given file. It is equivalent to the fallocate(2)
system call with the default mode (mode = 0).
BUG=chromium:858815
TEST=cargo build --features=composite-disk
Change-Id: I2f4e8aceb4878790e8dec2e3d539071915efd205
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2015828
Reviewed-by: Dylan Reid <dgreid@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
This will allow passing in other data sources for the qcow read methods,
without adding extra copies.
TEST=Unit tests
BUG=b:140069322
Change-Id: I2815fa6e416b554968b97959e4b6cd4c93a722f6
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/1982829
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Cody Schuffelen <schuffelen@google.com>
Add a new disk-specific DiskGetLen trait which uses io::Seek instead of
File::metadata() to determine the length so that it works on raw block
devices (e.g. /dev/sda) as well as regular files.
BUG=b:146811529
TEST=`crosvm run --disk /dev/sda` and verify block device length
Change-Id: I6936863490efaa479a3c8745c75c373748c800a1
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/1990855
Reviewed-by: Zach Reizner <zachr@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Tested-by: Trent Begin <tbegin@chromium.org>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
Move qcow from being its own crate into a module of the disk crate,
similar to the composite disk module.
This will allow use of qcow from disk and vice versa without introducing
a circular crate dependency.
BUG=None
TEST=./build_test.py
TEST=USE='asan fuzzer' emerge-nami crosvm
Change-Id: I77512bbe4b94faff1b5460f9796ee56505135580
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/1972477
Reviewed-by: Cody Schuffelen <schuffelen@google.com>
Reviewed-by: Zach Reizner <zachr@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
This new trait allows DiskFile implementors to provide the length of the
file directly rather than using SeekFrom::End with seek().
BUG=None
TEST=./build_test
TEST=Boot Termina in crosvm
Change-Id: I9447ebb43dbd5fbb32a3a6b6d2fc969b9406cdbc
Signed-off-by: Daniel Verkamp <dverkamp@chromium.org>
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/1913961
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Dylan Reid <dgreid@chromium.org>
This eliminates an extra seek per guest write zeroes request.
Additionally, it allows us to stop depending on the file cursor and pass
the offset directly, making multi-queue implementation easier.
BUG=chromium:858815
TEST=Boot Termina in crosvm
Change-Id: I8b15a39752a1b68597a2b1e1fd72382a484a3cb2
Signed-off-by: Daniel Verkamp <dverkamp@chromium.org>
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/1913521
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Dylan Reid <dgreid@chromium.org>
This was unintentionally broken when changing ReadWriteVolatile to
ReadWriteVolatileAt (the composite unit tests aren't built or run by
default since they are behind a feature flag).
BUG=None
TEST=cargo test --features=composite-disk -p disk
Change-Id: I864d7063bd27a401c3d878fcd22b52e2357fde8d
Signed-off-by: Daniel Verkamp <dverkamp@chromium.org>
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/1913960
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Cody Schuffelen <schuffelen@google.com>
Reviewed-by: Dylan Reid <dgreid@chromium.org>
Fixes clippy warning:
warning: this .into_iter() call is equivalent to .iter() and will not
move the slice
BUG=None
TEST=bin/clippy
Change-Id: I5361495f02609dd5e5313b7bc792e663841212ad
Signed-off-by: Daniel Verkamp <dverkamp@chromium.org>
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/1896091
Reviewed-by: Zach Reizner <zachr@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Use the "at" variants of the read/write functions in the block device.
This reduces the number of syscalls on the host per I/O to one
(pread64/pwrite64) rather than two (lseek + read/write).
The CompositeDiskFile implementation is also updated in this commit,
since it's both a producer and consumer of DiskFile, and it isn't
trivial to update it in a separate commit without breaking compilation.
BUG=None
TEST=Start Crostini on kevin, banon, and nami
Change-Id: I031e7e87cd6c99504db8c56b1725ea51c1e27a53
Signed-off-by: Daniel Verkamp <dverkamp@chromium.org>
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/1845948
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Stephen Barber <smbarber@chromium.org>
This adds a new disk file type next to raw files and qcow images that
represent an indirection to further raw disk files. The disk file
itself is a proto file with references to file paths for other disks to
open and their virtual offsets and lengths.
The intention is to make it easy to assemble a single virtual hard disk
out of several distinct partition files. In the particular case of
Cuttlefish running Android in a VM, this is relevant as the Android
build system distributes partitions as separate raw files. While the
simple solution is to pass each partition as a separate raw disk, some
functionality (like the bootloader) assumes there is a partition table
with multiple distinct partitions on a single disk.
Implementing composite disk support in the VMM bridges this gap through
supporting the general-purpose case of a disk built out of multiple
component files.
If desired, this can be extended to support qcow files to support
unusual configurations like a mixed qcow/raw disk.
Enabled with the "composite-disk" feature.
Bug: b/133432409
Change-Id: I2b0c47d92fab13b5dc0ca5a960c7cfd2b7145b87
Signed-off-by: Cody Schuffelen <schuffelen@google.com>
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/1667767
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
This supports virtio disks that depend on multiple file descriptors. All
of the file descriptors are passed to the jail when relevant.
Bug: b/133432409
Change-Id: Idf2e24cd2984c0d12a47a523c13d24c1ba8d173e
Signed-off-by: Cody Schuffelen <schuffelen@google.com>
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/1691761
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
