Commit graph

165 commits

Author SHA1 Message Date
Zihan Chen
13b958b967 disk: Add seekable zstd disk support
A raw disk image can be compressed as a seekable zstd and attached
transaprently to a VM as a block device.

TESTED=can ro mount and read seekable compressed debian rootfs

BUG=b:377945783

Change-Id: Iba1950dbfc0ba99b0581e842964848d5a447b824
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/6024317
Commit-Queue: Zihan Chen <zihanchen@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Auto-Submit: Zihan Chen <zihanchen@google.com>
2024-11-22 21:13:38 +00:00
Frederick Mayle
b32355490c disk: more detailed create_composite_disk unit test
Change-Id: I2ab63ce9b6e01a6593388cdb94d1e9df8f8b259b
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/6036618
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Frederick Mayle <fmayle@google.com>
2024-11-20 19:38:15 +00:00
Daniel Verkamp
4abcdf6b45 disk: qcow: read table clusters in a single I/O
The previous implementation of QcowRawFile::read_pointer_table() called
read_exact() on the underlying File for each individual entry in the
table; for a typical 64KB cluster size, this would mean read() would be
called 8192 times per cluster. Similarly, read_refcount_block() would
issue 32768 individual read() calls per cluster.

Using zerocopy, we can read the whole Vec<u64> or Vec<u16> reinterpreted
as a byte slice in a single read() call, then perform endian translation
and masking in a separate loop, which should be significantly more
efficient.

Change-Id: Ia66f4b9363be63935388adaa185a31b515f47951
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/5977144
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
Reviewed-by: Cody Schuffelen <schuffelen@google.com>
Reviewed-by: Frederick Mayle <fmayle@google.com>
2024-10-30 00:31:03 +00:00
Daniel Verkamp
aa25762cd1 disk: qcow: flush BufWriter before dropping
As mentioned in the docs[1], we should be flushing the BufWriter to
ensure we can catch write errors.

[1]: https://doc.rust-lang.org/std/io/struct.BufWriter.html

Change-Id: I7b9d9f306618692fc5cb1c93f0c7ff7aa87c6e16
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/5977146
Reviewed-by: Cody Schuffelen <schuffelen@google.com>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
Reviewed-by: Frederick Mayle <fmayle@google.com>
2024-10-30 00:15:45 +00:00
Frederick Mayle
511af5a116 disk: windows: restrict file sharing
This makes the behavior closer to Linux (which uses flock).

Change-Id: Iff587b58647eec7378972e5442d4da95f3e2b7a8
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/5841071
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Reviewed-by: Noah Gold <nkgold@google.com>
Commit-Queue: Frederick Mayle <fmayle@google.com>
2024-10-07 21:30:55 +00:00
Frederick Mayle
5921c375cc disk: add option to disable file locking
The recent changes to make crosvm lock nested disk files is running
afoul of Android Virtualization Framework's SELinux policies. The file
locking doesn't add much value in that environement because all the
shared files are read-only, so add an opt-out for locking to buy us time
to re-engineer the policies.

BUG=b:330911976

Change-Id: I0b35732978e946a2331507d6061729d53955a8d3
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/5849284
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Frederick Mayle <fmayle@google.com>
2024-09-10 22:07:48 +00:00
Frederick Mayle
d7b16c03b6 disk: composite: propagate read-only param
In practice this probably only matters for the flock calls because, when
a disk is read-only, the block device should never issue write calls to
it to begin with.

BUG=b:330911976

Change-Id: I9fc4e8e09e9c5a51f53ec342fae1c69c984999c0
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/5838023
Commit-Queue: Frederick Mayle <fmayle@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
2024-09-09 20:43:44 +00:00
Frederick Mayle
6cde4e3b9e disk: consolidate disk file opening logic
The disk crate is now responsible for opening the file and handling
options like OVERLAPPED and O_DIRECT.

O_DIRECT is now applied to all nested disk files. For now, OVERLAPPED
contiues to only apply to the root disk file.

BUG=b:330911976
BUG=b:190435784

Change-Id: Ib46a965c0589bf1c1e8e4ae5f0c81747530eff98
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/5842394
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Frederick Mayle <fmayle@google.com>
2024-09-09 20:39:03 +00:00
Frederick Mayle
6b965beb80 disk: use flock on nested disk files
flock is called on the root disk file to make sure nothing is
concurrently modifying a disk (which would appear to the guest as disk
corruption). This CL extends that functionality to all of the nested
files of composite and qcow2 disks.

BUG=b:330911976

Change-Id: I4489222943c3530255ffc52aba5a11aa4bc82f9a
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/5840920
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Reviewed-by: Cody Schuffelen <schuffelen@google.com>
Commit-Queue: Frederick Mayle <fmayle@google.com>
2024-09-09 20:37:08 +00:00
Frederick Mayle
312ce69076 disk: minor QcowFile::new_from_backing cleanup
new_from_header will open the backing file as part of the normal code
path, we don't need to manually insert it into the resulting QcowFile
object.

Change-Id: I06dfa0954b9ac4b8b18fc43b3047d684088ceef8
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/5840919
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Frederick Mayle <fmayle@google.com>
2024-09-09 20:29:46 +00:00
Frederick Mayle
2b9ea3f859 disk: pass more disk options to nested DiskFiles
No behavior difference intended. This is prep work to treat nested
DiskFiles consistently, e.g. using flock for them.

BUG=b:330911976

Change-Id: Id887ca21a9d5d186ba4dc77280c1ce4bfec8b319
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/5840918
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Frederick Mayle <fmayle@google.com>
2024-09-09 20:29:41 +00:00
Jiyong Park
b5d19dbaf2 Add support for partition GUID
Note that this CL doesn't mandate specifying the partition GUID. If
partition GUID is omitted, it will be automatically created as before.

The commandline interface has changed but in a backwards compatible
manner. The argument synatax for 'composite-disk' command is

LABEL:PATH:[writable]:[uuid]

BUG=352244386
TEST=tools/dev_container tools/presubmit

Change-Id: Ibf5ba0c2a3aa16e8c9393943c2fee0c71c38ed25
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/5690989
Commit-Queue: Jiyong Park <jiyong@google.com>
Auto-Submit: Jiyong Park <jiyong@google.com>
Commit-Queue: Frederick Mayle <fmayle@google.com>
Reviewed-by: Frederick Mayle <fmayle@google.com>
2024-07-11 01:15:29 +00:00
Daniel Verkamp
dc310d7cb6 Replace ::max_value() with ::MAX
The max_value() function is considered to be a "legacy numeric constant"
now, and future clippy versions will warn about it:

<https://rust-lang.github.io/rust-clippy/master/index.html#legacy_numeric_constants>

BUG=None
TEST=tools/clippy # with rust-toolchain = "1.79"

Change-Id: I72a333dc4aa1f48cf71744c848f050097a7e7f55
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/5690374
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
Reviewed-by: Dennis Kempin <denniskempin@google.com>
2024-07-09 23:26:03 +00:00
Daniel Verkamp
4f7b7de714 base: remove get_filesystem_type()
This was used in a single informational log statement when opening a
disk image, which was added for debugging an issue and is no longer
needed. It was also only ever implemented for Linux and not other
platforms.

BUG=None
TEST=tools/dev_container tools/presubmit

Change-Id: Id77244a883b6bb4b90ee1a909fa305ab4193c17e
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/5656451
Reviewed-by: Noah Gold <nkgold@google.com>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
2024-06-25 22:24:10 +00:00
Frederick Mayle
456fe75fad devices: virtio-blk: don't stop the worker thread on sleep/reset
The cros_async Overlapped I/O implementation on Windows does not support
roundtripping from AsyncDisk to DiskFile and then back to AsyncDisk
again with a new executor instance because, once a file is added to an
IOCP instance, it is permanently associated with that instance.
Attempting to add it to new IOCP instance results in an error.

It is possible to workaround this by changing the cros_async API so that
we can keeping using the same IOCP instance when we restart the worker
thread, but we decided that is too complex.

Instead, we'll just keep the worker thread running indefinitely. Since
`AsyncDisk::into_inner` isn't portable, we delete it so that someone in
the future doesn't accidentally design their code around it.

The main new complexity in the code is support for swapping out the
active `Interrupt` in the worker thread. Technically crosvm seems to
always resend the same `Interrupt` object, but it seems best to not make
any assumptions.

BUG=b:301269927

Change-Id: Iee0492d18fe640f45ae08ed946536b9a9dd6f525
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/5640657
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Frederick Mayle <fmayle@google.com>
Reviewed-by: Noah Gold <nkgold@google.com>
2024-06-21 02:02:13 +00:00
Daniel Verkamp
79ad1c82b2 disk: qcow: test CacheMap::get()
This exercises more of the CacheMap API, and it also silences a clippy
warning about the value inside the test's NumCache structure being
unused.

BUG=b:344974550
TEST=cargo test -p disk --features=qcow
TEST=tools/clippy

Change-Id: I4cbee164fdd7c0bc662c804c50482d118ec05df3
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/5626478
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
2024-06-18 18:15:26 +00:00
Daniel Verkamp
7cf3dcacbf Fix clippy::suspicious_open_options warnings
Use .create_new() rather than .create() for cases where we always want
to ensure that the file does not already exist, and add .truncate(false)
for a case where we do want to open an existing file and don't want to
overwrite it.

BUG=b:344974550
TEST=tools/clippy

Change-Id: Ie82a6db306532c600c140efab3d310b6c7cf25a7
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/5604660
Reviewed-by: Alexandre Courbot <acourbot@chromium.org>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
2024-06-10 19:39:47 +00:00
Daniel Verkamp
52b8e42869 Cargo.toml: avoid "*" versions for external crates
Ensure that every Cargo.toml dependency on a third-party crates.io crate
specifies at least a major version, or a minor version for 0.x crates,
to ensure that if a new major version is published, it cannot cause API
breaks.

The versions are selected to match the ones already in Cargo.lock, so
this should have no functional change, but it will help prevent new "*"
versions from being introduced via copy-and-paste.

For rationale, see the Cargo FAQ:
<https://doc.rust-lang.org/cargo/faq.html#can-libraries-use--as-a-version-for-their-dependencies>

`minijail`, `audio_streams`, and `cras` are left as "*" for now, since
they have unusual situations (imported from a submodule and/or replaced
at build time with ebuild magic).

BUG=None
TEST=tools/dev_container tools/presubmit
TEST=verify Cargo.lock is unchanged

Change-Id: Ifa18199f812f01d2d10bfb4146b3353c1a76527c
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/5555656
Reviewed-by: Frederick Mayle <fmayle@google.com>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
2024-05-22 01:01:42 +00:00
Daniel Verkamp
92665a611f Cargo.toml: move tempfile to dev-dependencies
The tempfile crate is only used in tests, so it does not need to be in
the main [dependencies] section of any of our first-party crates.

Also fix a couple of [dev_dependencies] instances to the officially
documented [dev-dependencies] name for consistency.

BUG=None
TEST=cargo tree

Change-Id: I1dcb6be10c302baec4e8ebe6f72480f76e4e3760
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/5521511
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
2024-05-07 20:00:48 +00:00
Frederick Mayle
c736d23241 disk: remove mutex from AsyncDiskFileWrapper
The disk traits no longer require exclusive refs. It might be reasonable
to forgo the blocking thread as well, but I'm leaving it as is for now.

BUG=b:338274203

Change-Id: I286c8983a7e850b3aa96c96d9388c77d1c9b7f04
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/5507758
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Frederick Mayle <fmayle@google.com>
2024-05-02 19:32:31 +00:00
Frederick Mayle
f909e42679 base: don't require &mut for File{Sync,Allocate} methods
Was only needed for QcowFile, which can now use its internal mutex.

BUG=b:338274203

Change-Id: I7db283cb946508daf8fd7bd8cf9397aa7f97a478
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/5507757
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Frederick Mayle <fmayle@google.com>
2024-05-02 19:32:26 +00:00
Frederick Mayle
5381cc2b66 base: don't require &mut for WriteZeroesAt methods
Was only needed for QcowFile, which can now use its internal mutex.

BUG=b:338274203

Change-Id: Iff684f5a25b0f488b006e05eebe913f92f404a2b
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/5507756
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Frederick Mayle <fmayle@google.com>
2024-05-02 19:32:21 +00:00
Frederick Mayle
8515568a51 base: don't require &mut for FileReadWriteAtVolatile methods
Was only needed for QcowFile, which can now use its internal mutex.

BUG=b:338274203

Change-Id: I810b537e971bf007b6b34f0cbc605f2b3943a763
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/5507754
Commit-Queue: Frederick Mayle <fmayle@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
2024-05-02 19:32:17 +00:00
Frederick Mayle
1cbaa32809 base: delete PunchHoleMut trait
Was only needed for QcowFile, which can now use its internal mutex.

BUG=b:338274203

Change-Id: I995cf646a66909b9b146b9a7c0ff7d5a8348acd7
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/5507755
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Frederick Mayle <fmayle@google.com>
2024-05-02 19:32:09 +00:00
Frederick Mayle
4189f738b0 disk: qcow: add internal mutex
A lot of disk traits require exclusive references to file-like objects
for operations that are generally threadsafe, e.g. you ought to be able
to have an `Arc<File>` and perform pread/pwrite-like operations from
multiple threads concurrently.

Qcow files are the only case stopping us from changing these APIs. This
patch adds a big lock inside of the implementation. Follow up patches
will make use of that lock to switch various APIs from `&mut self` to
`&self`.

There is already a big lock around qcow files in AsyncDiskFileWrapper
and that lock will go away after all the APIs are switched over, so it
is unlikely there will be a performance penalty in the end.

BUG=b:338274203

Change-Id: Ifbac628395ab64eb9a327ceba31e03fd6ba2c192
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/5507753
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Frederick Mayle <fmayle@google.com>
2024-05-02 19:32:03 +00:00
Daniel Verkamp
20d3d0080d disk: replace div_round_up with .div_ceil()
The div_ceil() is available since Rust 1.73. Replace our custom version
with the standard library function.

BUG=None
TEST=cargo test -p disk

Change-Id: I6b44d1f5189b1653d8617a51af41cd8a7cc3d162
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/5438907
Reviewed-by: Takaya Saeki <takayas@chromium.org>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
2024-04-10 17:49:15 +00:00
Daniel Verkamp
742791deef tree-wide: replace data_model::zerocopy_from_*()
Use zerocopy functions directly instead. This is a step toward replacing
our data_model crate with standard crates.io functionality where
possible.

BUG=b:312312646
TEST=tools/dev_container tools/presubmit

Change-Id: I9717edce6fe2b4ca53ad9043db61de2e9bc55b78
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/5046345
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
Reviewed-by: Noah Gold <nkgold@google.com>
2024-03-13 18:03:24 +00:00
Kaiyi Li
c28067d1d9 Reformat comments
Test: presubmit
Change-Id: I39c261d9985989873b698213c5d8b653fc13757b
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/5299850
Auto-Submit: Kaiyi Li <kaiyili@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
2024-02-15 23:30:13 +00:00
Frederick Mayle
1cad723426 cros_async: move punch hole special case into disk crate
https://crrev.com/c/5019243 added a `once_cell::unsync::OnceCell` to
`IoSource`, making it `!Sync`, which, by definition, makes `&IoSource`
`!Send` and so (nearly) any future that does IO will be `!Send`.

This commit removes the `OnceCell` from `IoSource`, making it `Sync`
again. Instead, we eagerly check whether a backing file is a block
device when the file is converted to an `AsyncDisk` so that the result
is a plain `bool` and move the branch into `SingleDiskFile::punch_hole`.

The other `AsyncDisk` implementors don't need this logic.
"android_sparse" is read-only (i.e. you can't punch a hole).
"composite_disk" delegates hole punching to the backing disk.
"qcow" might benefit from it, but it doesn't use async yet, so
there is no regression.

BUG=b:271297810

Change-Id: Ifd1d2361adabfd4db1b48e71d3a3a31e57a0cfb1
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/5086673
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
2023-12-15 19:40:13 +00:00
Dennis Kempin
73aed77b49 Run rustfmt on whole codebase with nightly enabled
Nigthly is enabled as part of https://crrev.com/c/4950268
This change contains the formatting changes resulting from the switch.

BUG=b:302055317
TEST=dev_container presubmit format --no-delta

Change-Id: Idaf2b8bae2e09c624b19d9cd3dd5fc8e4d099b3c
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/5067088
Reviewed-by: Zihan Chen <zihanchen@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
2023-11-29 18:41:29 +00:00
Daniel Verkamp
fcc28f5573 data_model: move IoBuf and VolatileMemory to base
IoBuf is defined in a platform-specific way (it is either iovec on unix
or WSABUF for windows), so it fits into the mission statement of the
base crate, which is meant to be *the* platform abstraction layer in
crosvm.

IoBufMut and VolatileMemory/VolatileSlice are defined in terms of IoBuf,
so they are also moved into base for consistency. Every crate that uses
these types already depended on base, so no extra dependencies are
added, and a few can be removed.

BUG=b:312312646
TEST=tools/dev_container tools/presubmit

Change-Id: I4dddc55d46906dfc55b88e8e6a967d7e1c1922dd
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/5046605
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
2023-11-27 21:42:49 +00:00
Takaya Saeki
dd2769b549 base: move PunchHoleMut from disk crate to base crate
`PunchHoleMut` trait is defined in the disk crate, but the base crate is
more suitable since `PunchHole` is defined by the base. This change
moves `PunchHoleMut` to base.

BUG=None
TEST=./tools/presubmit

Change-Id: Iee3eb48c88e58e4f54118bedde62a78fa9ff05dc
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/5019622
Commit-Queue: Takaya Saeki <takayas@chromium.org>
Reviewed-by: Keiichi Watanabe <keiichiw@chromium.org>
2023-11-20 09:56:30 +00:00
Takaya Saeki
85f1221536 cros_async: remove fallocate from IoSource and delegate it to base
Currently, cros_async's async IO source provides async `fallocate`
function, and `punch_hole` and `write_zeroes_at` are delegated to
`fallocate`. However, this design has three rooms for improvements.
1. Actually, no one calls async `fallocate`.
2. We have duplicated `fallocate` and `AllocateMode` implementations in
   `base::sys`.
3. Windows has to pretend to have `fallocate` to provide `punch_hole`
   and `write_zeroes_at`

So, this change removes `fallocate` and `AllocateMode`, which no one
actually calls, from `IoSource` and deletegate it to `base::sys`. With
this, we don't have to maintain `fallocate` implementation in
cros_async.

This will also enable us to call a non-`fallocate` systemcall in
`punch_hole` implementation which will be necessary to correctly support
block files.

BUG=b:302437024
TEST=./tools/presubmit

Change-Id: I8e7fcd94380da980cc52e6f751b134fa325d7b39
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/5019241
Reviewed-by: Keiichi Watanabe <keiichiw@chromium.org>
Reviewed-by: Noah Gold <nkgold@google.com>
Commit-Queue: Takaya Saeki <takayas@chromium.org>
2023-11-14 07:12:11 +00:00
Alyssa Ross
d9bc6e99ff Fix tests with non-4K pages
Change-Id: Ifc242d81fbaa7022554b96a9bb181ae390f231d7
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/5017868
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
2023-11-10 23:26:00 +00:00
Vikram Auradkar
a35c24c3d0 block: introduce a way to use overlapped source
BUG=b:303249046
TEST=tested downstream

Change-Id: If436c5bff6a44d6dbebb6e0aa2bf9e743dd25f59
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4950198
Reviewed-by: Noah Gold <nkgold@google.com>
Commit-Queue: Vikram Auradkar <auradkar@google.com>
2023-10-25 16:29:00 +00:00
A. Cody Schuffelen
4748c54b95 Rename "unix" to "linux" in code and docs
$ for DIR in $(find . -name "unix"); do mv $DIR $(echo $DIR | sed "s/unix/linux/"); done
$ for FILE in $(find . -name "unix.rs"); do mv $FILE $(echo $FILE | sed "s/unix/linux/"); done
$ find . -type f -not -path '*/\.git/*' | xargs -I {} sed -E -i "s/mod unix/mod linux/g" {}
$ find . -type f -not -path '*/\.git/*' -not -path '*/third_party/perfetto/*' | xargs -I {} sed -E -i "s/([^o][^s])::unix/\1::linux/g" {}
$ find . -type f -not -path '*/\.git/*' | xargs -I {} sed -E -i "s/use unix::/use linux::/g" {}
$ find . -type f -not -path '*/\.git/*' -not -path '*/third_party/perfetto/*' | xargs -I {} sed -E -i "s/sys::unix/sys::linux/g" {}
$ find . -type f -not -path '*/\.git/*' | xargs -I {} sed -E -i "s/use unix as platform/use linux as platform/g" {}

Test: ./tools/dev_container ./tools/presubmit
Bug: b/298269162
Change-Id: I2c8acb14d77a5588dab4eae124f4a9afbb9025f5
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4909060
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Reviewed-by: Vikram Auradkar <auradkar@google.com>
Commit-Queue: Cody Schuffelen <schuffelen@google.com>
Reviewed-by: Frederick Mayle <fmayle@google.com>
2023-10-11 01:15:07 +00:00
A. Cody Schuffelen
97dff044f8 Replace #[cfg(unix)] with #[cfg(any(target_os = "android", target_os = "linux"))]
Updates are made to source and documentation.

This more accurately represents the currently supported platforms of
Android/Linux and Windows, without unexpectedly including other
unix-like operating systems.

Command to reproduce:
$ find . -type f -not -path '*/\.git/*' | xargs -I {} sed -i 's/cfg(unix)/cfg(any(target_os = "android", target_os = "linux"))/g' {}
$ cargo fmt

md files manually updated to fix line lengths.

Renaming `unix` modules to `linux` will be done in a later CL.

Test: ./tools/dev_container ./tools/presubmit
Bug: b/298269162
Change-Id: I42c1bf0abf80b9a0df25551613910293217c7295
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4909059
Commit-Queue: Cody Schuffelen <schuffelen@google.com>
Reviewed-by: Frederick Mayle <fmayle@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Reviewed-by: Noah Gold <nkgold@google.com>
2023-10-11 00:43:29 +00:00
Zihan Chen
cbba480ebe crosvm: Uprev zerocopy to 0.7.x
zerocopy 0.7.x has finally released as a stable version, uprev it
to allow some remaining structs to be derivable.

TEST=CQ

BUG=b:300969352
FIXED=b:300969352

Change-Id: I90f0dfb09494f875fef1cd11bfcbd48030846092
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4878761
Auto-Submit: Zihan Chen <zihanchen@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Zihan Chen <zihanchen@google.com>
2023-10-10 17:34:32 +00:00
Daniel Verkamp
03dd1feab7 Cargo.toml: depend on zerocopy 0.6 explicitly
Previously all of the zerocopy imports just used "*", but there are
API changes in zerocopy 0.7, so ensure we get a compatible version by
depending on 0.6 everywhere.

This is a no-op since Cargo.lock already specifies a 0.6.x version, but
it will prevent accidentally upgrading to 0.7.x without updating to the
new API.

BUG=b:301283548
TEST=cargo build

Change-Id: Ifd702d982a09b5083dddd666dc6f3052cba22214
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4878502
Reviewed-by: Zihan Chen <zihanchen@google.com>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
2023-09-20 21:28:58 +00:00
Junichi Uekawa
8d83d0d1c1 crosvm: Rename to open_file_or_duplicate.
open_file does not always open_file. It reuses given fd when it points
to a special path /proc/self/fd/XX.

BUG=b:291832160
TEST=build

Change-Id: If821834e3adbc457616c71e5b8425962286245e7
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4706830
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Junichi Uekawa <uekawa@chromium.org>
2023-07-24 01:30:27 +00:00
Daniel Verkamp
ce5a78e83b clippy fixes for Rust 1.70 and 1.71
Change-Id: If86c6cd531b854293a93208de5254664f5ee6bec
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4637612
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
2023-07-13 20:29:05 +00:00
Frederick Mayle
49794f44da virtio: block: flush disk image as part of sleep
This is mainly for qcow2, which doesn't immediately write all mutations
to file. It is important that all writes be sent to the OS so that
snapshots taken while asleep do not lose information.

BUG=b:266514327
TEST=patched into AOSP and manually ran snapshot-restore flow

Change-Id: I2a9f24120d51fd053ff1440d0e1750e9bbebf379
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4679843
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Frederick Mayle <fmayle@google.com>
Reviewed-by: Elie Kheirallah <khei@google.com>
2023-07-12 22:26:17 +00:00
Daniel Verkamp
024a5e44f2 cros_async: add MemRegion iterator
MemRegionIter allows iteration over a &[MemRegion] list, with additional
helper functions to skip bytes and truncate the list.

In particular, this is useful for the virtio DescriptorChain utilities
(Reader/Writer) that previously needed to call collect() to allocate a
new copy of the regions list when it was necessary to limit the length
or skip part of the beginning of the chain.

This also replaces the functionality of MemRegion::truncate(), which
used Cow<[MemRegion]>, meaning it would sometimes need to allocate and
copy. The new MemRegionIter implementation never needs to copy the list
of regions; it just maintains a couple of internal counters and adjusts
the returned MemRegions on the fly.

BUG=None
TEST=tools/dev_container tools/presubmit
TEST=cargo test -p cros_async

Change-Id: Ie66c7a8c4061a166c986c6d6937e21549ddb205e
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4559367
Reviewed-by: Alexandre Courbot <acourbot@chromium.org>
Reviewed-by: Frederick Mayle <fmayle@google.com>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
2023-06-28 19:45:38 +00:00
Zihan Chen
a62b3f33e4 disk/android_sparse: Remove DataInit usage
TEST=CQ

BUG=b:204409584

Change-Id: I7d0998d3749cd09b45ba00a588158c0f4fb78846
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4618948
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
Auto-Submit: Zihan Chen <zihanchen@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Reviewed-by: Cody Schuffelen <schuffelen@google.com>
2023-06-15 19:56:37 +00:00
Takaya Saeki
7781380f01 devices: virtio: block: use fdatasync for FLUSH
Currently, virtio-blk translates a FLUSH request to fsync system call.
However, fsync always updates the metadata of a file, which causes extra
journaling for metadata update. Since virtio-blk cares only the data of
the given block image file, we should avoid updating extra metadata.

After this change, virtio-blk now calls fdatasync for a FLUSH request.
If a write operation wrote to a data block that is already allocated
before, fdatasync can avoid triggering jdb2 journaling on an ext4 file
system.

Note there are structs which fall back to fsync for fdatasync. In
addition, io_uring executor does not implement fdatasync yet.

We observed statistically significant play_store_shown_time improvement
on trogdor-arc-r by -1.614% with this change. Including non
statistically significant data, we observed the following improvements.

|         Name          |  delta  | Count |
|-----------------------|---------|-------|
| trogdor-arc-r (lazor) | -1.614% |  170  |
| brya-arc-t (primus)   | -1.478% |  200  |
| octopus-arc-t (apel)  | -0.887% |  190  |
| kukui-arc-r (kakadu)  | -0.451% |  170  |

BUG=b:281609112
TEST=run a vm with block and confirmed fdatasync is called by strace

Change-Id: Idc94a3ec169e9a5e04394079967f6d79ff4c32db
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4523029
Commit-Queue: Takaya Saeki <takayas@chromium.org>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
2023-05-12 18:15:54 +00:00
Daniel Verkamp
a360baa819 Cargo.toml: upgrade protobuf 2.x -> 3.x
system_api bindings have been regenerated with protobuf 3.2; this should
be okay to land before the full ChromeOS system_api migration, since
crosvm always uses its own copy of the bindings rather than the ones
provided by the dev-rust/system_api package.

The protoc-rust crate is replaced with protobuf_codegen in 3.x.

BUG=b:277243607
BUG=b:279834784
TEST=tools/dev_container tools/presubmit

Change-Id: I6aad45ded2639d7506a7238800584bebab196455
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4405309
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
Reviewed-by: Noah Gold <nkgold@google.com>
2023-04-28 19:32:01 +00:00
Daniel Verkamp
2c1d17fe41 tree-wide: remove unnecessary casts
Fixes clippy lint in Rust 1.68

BUG=b:276487055
TEST=tools/clippy # with rust 1.68

Change-Id: Ib9d83b8dc15a93e815600391e93b8bb1788f7dc4
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4391107
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
2023-04-17 23:43:59 +00:00
Daniel Verkamp
27d1a90062 disk: use Seek::stream_position() convenience function
Replace seek(SeekFrom::Current(0)) with the stream_position() helper
function, which has been stable since Rust 1.51.

BUG=b:276487055
TEST=tools/clippy # with rust 1.68

Change-Id: I21e9af65f6ad5ad8443dd1d2b95c08d3be91f174
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4390741
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
Reviewed-by: Dennis Kempin <denniskempin@google.com>
2023-04-11 20:44:34 +00:00
Junichi Uekawa
d03c99fc0c crosvm: Uprev uuid > 1
Uprev to version available in Debian.

This seem to end up doing uprev to 1.3 now.

BUG=b:265082456
BUG=b:229895468
TEST=build

Change-Id: I550778acb675c9034b9cfcea77f4ae847e2d2ea1
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4364559
Commit-Queue: Junichi Uekawa <uekawa@chromium.org>
Reviewed-by: Keiichi Watanabe <keiichiw@chromium.org>
Auto-Submit: Junichi Uekawa <uekawa@chromium.org>
Reviewed-by: Frederick Mayle <fmayle@google.com>
2023-04-04 23:56:13 +00:00
Frederick Mayle
6245e630b8 cros_async: introduce concrete type IoSource for async IO
This removes two layers of boxing and trait objects. One for the IO
source objects and another for the futures generated by calling methods
on them.

Benefits:

* IO futures will now be `Send` (if the underlying source is `Send`,
  which is generally the case).
* It is easier to work with a concrete type than a `Box<dyn ...>`.
* More efficient (less boxing and virtual function calls).

Slightly smaller binary size:

  $ cargo build --all-features --profile chromeos && ls -l target/chromeos/crosvm
  before: 10,811,576
  after:  10,692,792 (1.1% smaller)

Change-Id: I7b52cd24945cfb4ff0a6c377739b5d4ab9cacb0b
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4004980
Reviewed-by: Noah Gold <nkgold@google.com>
2023-04-04 20:17:23 +00:00