Commit graph

284 commits

Author SHA1 Message Date
Abhishek Bhardwaj
7a1a6cd58b crosvm: Implement communication logic in virtio-vhost-user PCI device
This change adds the PCI device that will act as the conduit between
vhost vmm and vhost device in a virtio-vhost-user
specification. It only implements the communication logic i.e. rx / tx
from the vmm socket and virtio queues associated with the device.

BUG=b:194136484
TEST=Compile.

Change-Id: Ib47045b7633b77b73ed7bd428ca981caa6645275
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3146213
Auto-Submit: Abhishek Bhardwaj <abhishekbh@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Abhishek Bhardwaj <abhishekbh@chromium.org>
Reviewed-by: Keiichi Watanabe <keiichiw@chromium.org>
Reviewed-by: Chirantan Ekbote <chirantan@chromium.org>
2021-09-17 22:07:55 +00:00
Federico 'Morg' Pareschi
a1184822c7 Add vhost-user-fs device
This CL adds a vhost user FS device to crosvm. The implementation
requires the following arguments:

--socket = a path to a unix socket crosvm is going to connect to
--tag = a virtio fs tag to mount within the VM
--shared-dir = path to the shared directory

Optional arguments are:
--uid-map = UID translation from inside the VM to the outside
--gid-map = GID translation from inside the VM to the outside

Example command:
crosvm device fs --socket $HOME/test --tag fs-tag --shared-dir $DIR
crosvm run --vhost-user-fs=$HOME/test:fs-tag ...

BUG=b:179636297
TEST=launch fs device and verified that mounted directory works

Change-Id: Icab9e4be65092ef817006408b50bb3bf35033c62
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3062161
Tested-by: kokoro <noreply+kokoro@google.com>
Auto-Submit: Morg <morg@chromium.org>
Commit-Queue: Morg <morg@chromium.org>
Reviewed-by: Chirantan Ekbote <chirantan@chromium.org>
2021-09-15 02:06:49 +00:00
Daniel Verkamp
3eeaf6a339 linux: fill pmem alignment area with anon map
Rather than mapping past the end of the file when using a pmem backing
file that is not 2 MiB aligned, use an anonymous mapping to fill the
remaining part of the arena.

This partially reverts https://crrev.com/c/2153103 while keeping the
effective change: the anonymous mapping used to fill the padding is now
added with the same protection as the file mapping.

Also handle images that are not a multiple of the page size (typically
4096 bytes) - the memory mapping on the host will handle zero-filling
reads and discarding writes past the end of the mapped file as long as
we map a size containing the partial last page.

BUG=chromium:1244217
TEST=Boot crosvm with non-2MB aligned pmem disk; read the last few bytes
TEST=./test_all

Change-Id: Ibe8da170175bb9befce924122b912a28a6dc0e7b
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3131444
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Chirantan Ekbote <chirantan@chromium.org>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
2021-09-03 20:00:13 +00:00
Daniel Verkamp
28359e141f sys_util: remove unsafe struct_util functions
Replace the uses of read_struct() and read_struct_slice() with the
safe DataInit::from_reader() implementation.

BUG=b:197263364
TEST=./test_all
TEST=Boot bzImage kernel
TEST=Boot raw ELF kernel extracted with extract_vmlinux

Change-Id: I80f98243bfb58a7ae93e1686bc4d92b0cd485cda
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3108249
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
Reviewed-by: Chirantan Ekbote <chirantan@chromium.org>
2021-08-27 23:48:24 +00:00
Fergus Dall
51200519a2 Don't rely on being able to look up users/groups
Some devices need to have their current uid mapped in their sandbox
for bind mounts to work as expected. Currently crosvm looks up the
uid/gid for "crosvm" and maps that.

This logic is dubious anyway, since crosvm should be using whatever
user/group it was started under rather then trying to switch (which is
a priviliged operation), but putting concierge in a user namespace
breaks it entierly because the crosvm user gets remapped to a
different numeric value.

Replace the current approach with mapping the current euid/egid,
whatever it may be.

BUG=chromium:1240116
TEST=Manually tested

Change-Id: I0e9b95ed04834da1adedb72bee52ac4359f06041
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3105907
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Allen Webb <allenwebb@google.com>
Reviewed-by: Chirantan Ekbote <chirantan@chromium.org>
2021-08-27 17:57:57 +00:00
Andrew Walbran
85d6ef16ca Use macros for trait implementations.
The previous implementations were exactly the same as the macro
expansions, so this should be exactly the same but less verbose.

BUG=None
TEST=cargo test

Change-Id: I9c97c4c9cb71491d5f8f5a8ce52e64abdac54245
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3026666
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Auto-Submit: Andrew Walbran <qwandor@google.com>
Commit-Queue: Andrew Walbran <qwandor@google.com>
2021-08-17 16:07:51 +00:00
Chirantan Ekbote
af9dccf591 Add try_clone() and FromRawDescriptor for Tube and UnixSeqpacket
BUG=b:179755651
TEST=cargo test

Change-Id: Icf77f7427972b940215a2a4d95aa09ae409e4ff0
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2987590
Auto-Submit: Chirantan Ekbote <chirantan@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Chirantan Ekbote <chirantan@chromium.org>
Reviewed-by: Keiichi Watanabe <keiichiw@chromium.org>
2021-08-17 11:48:07 +00:00
Chirantan Ekbote
e4b9f30e24 sys_util: Add UnlinkUnixListener
Like UnlinkUnixDatagram but for UnixListener instead.

BUG=b:179755651
TEST=cargo test

Change-Id: I874ba2c76f525ac6cbc86d16b45dc8ff43d0c7f4
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2987587
Auto-Submit: Chirantan Ekbote <chirantan@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Chirantan Ekbote <chirantan@chromium.org>
Reviewed-by: Noah Gold <nkgold@google.com>
Reviewed-by: Keiichi Watanabe <keiichiw@chromium.org>
2021-08-17 11:48:05 +00:00
Allen Webb
87b96033b7 sys_util: Add wrappers for fchmod and fchown.
This wrappers allow for permissions and ownership changes that are less
sensitive to time-of-check-time-of-use vulnerabilities.

BUG=None
TEST=CQ passes

Change-Id: I6d5e4809a0b0113f3a95b0395d3cfb82431a3fd8
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3049330
Auto-Submit: Allen Webb <allenwebb@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Allen Webb <allenwebb@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
2021-08-04 05:20:21 +00:00
Allen Webb
1919f432c2 sys_util: Add Ord, and PartialOrd so VsockAddr can be used in BTreeMap.
BUG=None
TEST=sirenia builds with the BTreeMap CL.

Change-Id: I8575484574d03e17cd7810f52fef10a31f0bff67
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3068875
Auto-Submit: Allen Webb <allenwebb@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Allen Webb <allenwebb@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
2021-08-03 21:47:14 +00:00
Junichi Uekawa
7bea39fef5 crosvm: O_DIRECT for opening disk image.
Introduce a new disk flag o_direct=true, which uses O_DIRECT for open.
Block access and memory buffer needs to be aligned to 512 bytes.

BUG=b:190435784
BUG=b:184204645
TEST=boot tests (boot_test_suspend_resume, boot_test_vm) keep running with o_direct=true.

Change-Id: I17888f8a6ef2a38baba2c17dbb0d7aa18b70dbfd
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2821112
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Junichi Uekawa <uekawa@chromium.org>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
2021-07-30 06:34:11 +00:00
Andrew Walbran
bc55e30b89 Handle /proc/self/fd/N specially for composite disk components and QCOW2.
Follow-up to https://crrev.com/c/3000986.

BUG=b:192256642
TEST=cargo test

Change-Id: I08c87f9295e3c8d977c6028e5d8d7e8beeafbaec
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3024191
Auto-Submit: Andrew Walbran <qwandor@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Chirantan Ekbote <chirantan@chromium.org>
Reviewed-by: Chirantan Ekbote <chirantan@chromium.org>
2021-07-15 08:33:52 +00:00
Allen Webb
c6fa73d76f Fix clippy warnings and Cargo.lock
This fixes:
* version mismatches in Cargo.lock
* style issues
* implementations of Into that should be From
* deprecated protobuf APIs

It also adds RUST_BACKTRACE=1 to the kokoro tests.

BUG=None
TEST=./bin/preupload-clippy

Change-Id: I8e9157c903f2080a5fdcc4d3e4ed72fbad41c64f
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3024427
Auto-Submit: Allen Webb <allenwebb@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Allen Webb <allenwebb@google.com>
2021-07-15 03:33:17 +00:00
Chirantan Ekbote
701b55adf4 sys_util: Add clone_descriptor
This is like `clone_fd` but takes a `dyn AsRawDescriptor` instead.

BUG=b:179755841
TEST=use it in a later change

Change-Id: I432a57f139f88ca7771e83a3edf09eb4f3d6156f
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2927189
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Zach Reizner <zachr@chromium.org>
Reviewed-by: Keiichi Watanabe <keiichiw@chromium.org>
Commit-Queue: Chirantan Ekbote <chirantan@chromium.org>
2021-06-03 09:57:58 +00:00
Zach Reizner
026f72f9fb enable core scheduling regardless of enabled features
Now that core scheduling has been upstreamed, it can be enabled for all
platforms. This change keeps the Chrome OS version as a fallback if the
chromeos feature is enabled.

This change was patterned after http://crrev.com/c/2896464.

TEST=./test_all
BUG=b:153989878

Change-Id: I22a88c7af30e38f58acc7189cfe27a1af89d271d
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2929044
Auto-Submit: Zach Reizner <zachr@chromium.org>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Reviewed-by: Dylan Reid <dgreid@chromium.org>
Reviewed-by: Joel Fernandes <joelaf@google.com>
Commit-Queue: Zach Reizner <zachr@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
2021-06-03 00:06:11 +00:00
Chirantan Ekbote
a60ffb8c36 sys_util: Switch syslog error to thiserror
BUG=none
TEST=cargo build

Cq-Depend: chromium:2900011
Change-Id: I97d8fcef1b0c381a5adb691813b15dfe534f66e6
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2894326
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Chirantan Ekbote <chirantan@chromium.org>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Reviewed-by: Stephen Barber <smbarber@chromium.org>
2021-05-18 16:42:20 +00:00
Allen Webb
009afce6d4 sys_util: Migrate libchromeos::read_dir and syscall!
BUG=chromium:1193155
TEST=cargo test -- --test-threads=1

Change-Id: Ic5f8d23299d0f217b607231c56dfc6d9866e30df
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2829947
Tested-by: Allen Webb <allenwebb@google.com>
Commit-Queue: Allen Webb <allenwebb@google.com>
Reviewed-by: Chirantan Ekbote <chirantan@chromium.org>
2021-04-22 13:43:57 +00:00
Zach Reizner
6d6efb20c0 remove dependencies on syscall_defines
The syscall_defines crate is redundant with an up to date libc. This
change removes any dependency on syscall_defines. A new libc is required
to bring in some new syscall numbers like the ones for io_uring.

TEST=./test_all
BUG=None

Cq-Depend: chromium:2832000
Change-Id: I6df7fb992bacb5efd54cefca08836d52f4bfcd8c
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2832001
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Allen Webb <allenwebb@google.com>
Reviewed-by: Dylan Reid <dgreid@chromium.org>
Reviewed-by: Chirantan Ekbote <chirantan@chromium.org>
Commit-Queue: Zach Reizner <zachr@chromium.org>
2021-04-21 21:01:38 +00:00
Allen Webb
82561b445b sys_util: Fix 1.51.0 acronym case clippy warnings.
This also fixes a self convention warning for sys_util::sock_ctl_msg and
removes a stray semicolon from ac97_bus_master.

BUG=None
TEST=cargo clippy && cargo test

Change-Id: I7fcf7577e09888836f7664d7127f94c9c24d3cfa
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2840050
Tested-by: Allen Webb <allenwebb@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Stephen Barber <smbarber@chromium.org>
Commit-Queue: Allen Webb <allenwebb@google.com>
2021-04-21 00:14:11 +00:00
Sergey Senozhatsky
0b6065aafa mmap: add use_hugepages() helper method
Add use_hugepages() MemoryMapping method, which will advice the
kernel to use Huge Pages for memory mappings.

BUG=b:174206107
TEST=arc.Boot.vm on hatch-arc-r

Signed-off-by: Sergey Senozhatsky <senozhatsky@chromium.org>
Change-Id: I8cf61d71bb7e135594c1a4bc3c68674a1576b766
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2812545
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Chirantan Ekbote <chirantan@chromium.org>
2021-04-20 13:24:16 +00:00
Allen Webb
c9de52bea5 sys_util: Migrate additional code from libchromeos-rs.
This migrates:
* libchromeos::base::rand_util
* libchromeos::net
* libchromeos::scoped_path
* libchromeos::vsock

BUG=chromium:1193155
TEST=cargo test -- --test-threads=1

Change-Id: I801c7cbf8001dcd386792bb120781dacb9a699c7
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2832311
Tested-by: Allen Webb <allenwebb@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Stephen Barber <smbarber@chromium.org>
Commit-Queue: Allen Webb <allenwebb@google.com>
2021-04-19 22:44:13 +00:00
Dylan Reid
e35d827ec4 sys_util: descriptor_reflection - fix warning in nightly
New rustc(1.53) introduces a warning:
warning: derive helper attribute is used before it is introduced
   --> /home/dgreid/src/chromiumos/src/platform/crosvm/sys_util/src/descriptor_reflection.rs:391:3
    |
391 | #[serde(transparent)]
    |   ^^^^^
392 | #[derive(Serialize, Deserialize)]
    |          --------- the attribute is introduced here

Fix that by reversing the order.

Change-Id: Ifd77322bb9552a3ba6c1c86019e9cb366328b173
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2818897
Reviewed-by: Zach Reizner <zachr@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Dylan Reid <dgreid@chromium.org>
2021-04-19 22:17:09 +00:00
Allen Webb
5e185834fc sys_util: Add scoped_signal_handler.
This adds a scoped signal handler struct that performs some basic checks
to avoid accidental replacement of an existing handler and cleanup when
the handler goes out of scope.

Also, adds scoped_signal_handler::wait_for_interrupt() which blocks
until SIGINT is received (e.g. via Ctrl-C).

BUG=None
TEST=cargo test --workspace -- --test-threads=1

Change-Id: I5e649eac3d3ee0b842b200fc553acac44b2dfe94
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2807306
Reviewed-by: Zach Reizner <zachr@chromium.org>
Reviewed-by: Chirantan Ekbote <chirantan@chromium.org>
Tested-by: Allen Webb <allenwebb@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Allen Webb <allenwebb@google.com>
2021-04-16 18:04:32 +00:00
Allen Webb
a09d09c6dc sys_util: Fix clippy errors for dev-lang/rust-1.51.0
BUG=None
TEST=cargo clippy && cargo test

Change-Id: I7d73176469baedaaf847ddaae73f959c173d67a6
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2827974
Reviewed-by: Stephen Barber <smbarber@chromium.org>
Tested-by: Allen Webb <allenwebb@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Allen Webb <allenwebb@google.com>
2021-04-16 15:14:25 +00:00
Chirantan Ekbote
a00991cd84 Replace dup with fcntl(F_DUPFD_CLOEXEC)
Fds created via dup don't share file descriptor flags with the original
fd, which means that they don't have the FD_CLOEXEC flag set.  Use
fcntl(F_DUPFD_CLOEXEC) so that this flag gets set for the duplicated fds
as well.

BUG=none
TEST=unit tests

Change-Id: Ib471cf40acac1eacf72969ba45247f50b349ed58
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2809687
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Chirantan Ekbote <chirantan@chromium.org>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Reviewed-by: Zach Reizner <zachr@chromium.org>
2021-04-15 10:34:04 +00:00
Noah Gold
9af97d72fa [base] update/clean mmap interfaces.
This CL addresses some minor issues with the existing interface:
1. from_descriptor is too generic for some platforms that require
   special handling for file/File backed mappings.
2. Nearly all call sites pass either File or SharedMemory. Now
   we just have from_ methods for those types to preserve type
   information.
3. Other platforms require additional fields in MemoryMapping, so a
   tuple struct no longer makes sense.
4. The mmap syscall error message was misleading as we use it for more
   than just the mmap syscall.

BUG=None
TEST=builds

Change-Id: I74c41bad52bb81880a11231cd18f47e233548a24
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2815614
Reviewed-by: Udam Saini <udam@google.com>
Reviewed-by: Zach Reizner <zachr@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Noah Gold <nkgold@google.com>
2021-04-15 02:10:35 +00:00
Chirantan Ekbote
4b47aa7105 fs: Fix device socket
validate_raw_fd assumes that the fd passed in was not created by crosvm
and returns EBADF if it sees that the fd has the FD_CLOEXEC flag set.
We can't use it for fds created by the fs device since those do have
that flag set.

We're already taking a `&dyn AsRawFd` as the parameter so just assume
it's valid and clone it directly since there's no safe way to create an
invalid one.

BUG=none
TEST=vm.Fio.virtiofs_dax* tests are no longer failing

Change-Id: I10d9752e0960143fb58a63d2b76f64d34ec464d0
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2809686
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Reviewed-by: Zach Reizner <zachr@chromium.org>
Commit-Queue: Chirantan Ekbote <chirantan@chromium.org>
2021-04-09 08:04:30 +00:00
Allen Webb
53a5d09dd1 sys_util: Migrate code from libchromeos::linux.
BUG=chromium:1193155
TEST=emerge-${BOARD} dev-rust/sys_util

Cq-Depend: chromium:2788637
Change-Id: Ifabac09d2ebf39387cf96ebb1571d70e77eedc13
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2789053
Tested-by: Allen Webb <allenwebb@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Dylan Reid <dgreid@chromium.org>
Commit-Queue: Allen Webb <allenwebb@google.com>
2021-04-05 21:22:49 +00:00
Zach Reizner
48e1e52928 base: add tube module
Tube is the replacement for MsgSocket and related types and traits.

TEST=run_tests
BUG=b:176847362

Change-Id: I290279a714eb04c5cc6f2aef15ba7c61c708ab08
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2726980
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Zach Reizner <zachr@chromium.org>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
2021-04-01 01:59:29 +00:00
Allen Webb
44c728c123 sys_util: Add functionality to signals.
This refactors register_signal_handler to take a callback with the
correct parameter (c_int).

It adds functionality for clearing signal handlers, checking if a signal
handler is defined, and waiting for a signal. As part of this work a
helper function was added to create libc::timespec structs from
std::time::Duration, and get the max Duration that can be represented by
libc::timespec.

BUG=None
TEST=cargo test

Cq-Depend: chromium:2782855, chromium:2782856
Change-Id: Id531d9aaeeeb65f0f6f9b64bed4aba024a2328cb
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2783043
Tested-by: Allen Webb <allenwebb@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Allen Webb <allenwebb@google.com>
Reviewed-by: Stephen Barber <smbarber@chromium.org>
Reviewed-by: Zach Reizner <zachr@chromium.org>
2021-03-29 20:54:20 +00:00
Zach Reizner
de0c63dbcd implement Serialize and Deserialize for base and sys_util types
These types make up parts of larger messages that implemented
MsgOnSocket.

BUG=b:176847362
TEST=run_tests

Change-Id: I1f99e08f494d646ad0566eb556e2c28726d1d217
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2733207
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Zach Reizner <zachr@chromium.org>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
2021-03-29 19:39:52 +00:00
Zach Reizner
daceeb59c6 sys_util: add tool for raw descriptor reflection
This CL enables serde based serialize and deserialize types to interact
with a side-channel stash of descriptors. The idea is that many types
could implement Serialize and Deserialize, but they have descriptors
buried in them, such as a File or kernel object. Although these can be
serialized literally, usually the transciever of the serialized value
needs explicit access to the descriptors to send them over, such as with
Unix domain sockets.

TEST=cargo test -p base
BUG=b:176847362

Cq-Depend: chromium:2757675
Change-Id: I3273889f8d43cca761a54c531a981b7ab1ad3c03
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2576633
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Zach Reizner <zachr@chromium.org>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
2021-03-29 19:39:51 +00:00
Gurchetan Singh
6e8f33aa0a rutabaga_gfx: convert to SafeDescriptor
To be truly OS-agnostic, we need an OS-agnostic Rust wrapper over
the OS-specific handle type.  SafeDescriptor seems to be the best
option, and I hope it on crates.io in the future.

This converts virtio_gpu/rutabaga to use the SafeDescriptor handle
when practical.  minigbm still uses File in some places, since it
needs to SeekFrom(..), but minigbm is a Linux only thing anyways.

BUG=b:173630595
TEST=boot VM in 2D/3D mode

Change-Id: I18d735844d479f52d82d7976bf9b4e383b2e2252
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2779492
Commit-Queue: Gurchetan Singh <gurchetansingh@chromium.org>
Commit-Queue: Zach Reizner <zachr@chromium.org>
Tested-by: Gurchetan Singh <gurchetansingh@chromium.org>
Reviewed-by: Michael Hoyle <mikehoyle@google.com>
Reviewed-by: Zach Reizner <zachr@chromium.org>
2021-03-23 00:44:10 +00:00
Zach Reizner
d09392e37e uprev rust-toolchain and fix clippy warnings
Some judgement calls were made about unnecessary wrapping. Usually they
would get resolved by removing the wrapping or returning a convenient
error, but the ones that returned results for consistency with other
functions were added to the allow list.

The error handling in the usb code had a lot of unit error types which
is now a clippy lint. This was resolved by either removing the result
entirely or returning a convenient error.

The field_reassign_with_default lint is faulty and was added to the list
of supressions. This affected virtio-wayland code.

BUG=b:179277332
TEST=cargo clippy with rustc 1.50+

Change-Id: Ie812cdeaf7c42f4f2b47b1dc87f05a7c87a60f8f
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2757510
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Zach Reizner <zachr@chromium.org>
Reviewed-by: Gurchetan Singh <gurchetansingh@chromium.org>
Auto-Submit: Zach Reizner <zachr@chromium.org>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
2021-03-22 21:41:07 +00:00
Dylan Reid
2320fc7a83 sys_util: sock ctrl - add send/recv flexibility
For historical reasons, crosvm switched to sending `IntoIobuf`, this
isn't implementable for a slice of slices as it would require an
allocation. The rust-vmm fork of sys_util still supports sending
`&[&[u8]]`. Adding that ability back makes it easier to use their
crates.

In addition crosvm allows receiving to slices by not iovecs, again the
opposite of the rust-vmm version. Add that ability as well.

Change-Id: I4566f232adb47a0802e39e3f6c09c03f97190eb3
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2740533
Reviewed-by: Keiichi Watanabe <keiichiw@chromium.org>
Reviewed-by: Chirantan Ekbote <chirantan@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Tested-by: Keiichi Watanabe <keiichiw@chromium.org>
Commit-Queue: Keiichi Watanabe <keiichiw@chromium.org>
2021-03-11 08:06:27 +00:00
Keiichi Watanabe
0074b288ef sys_util: Fix clippy errors and warnings
BUG=chromium:1111728
TEST=cargo clippy --all-features --all-targets in sys_util
TEST=./test_all

Change-Id: Ib544bd1f87fa8a3244b4e6643f54eae265ef55bd
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2717905
Tested-by: Keiichi Watanabe <keiichiw@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Auto-Submit: Keiichi Watanabe <keiichiw@chromium.org>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
2021-03-04 20:23:12 +00:00
Dennis Kempin
57e0d5d4dc Make Kokoro Happy
Fixes a clippy issue and disables the panic_safe test.

This test occasionally deadlocks (Maybe a 5% chance) when running
inside a test VM. I filed a bug to re-enable it.

BUG=b:179924844
TEST=kokoro passes

Change-Id: I6e9b5fc38e7bbab3fd4b2b8ba8cb5532dd14af7e
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2697860
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
Auto-Submit: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
2021-02-17 07:25:38 +00:00
Dylan Reid
503c5abef6 devices: Add an asynchronous block device
This enables the use of basic disk images with async IO. A new
block_async.rs is added which mostly mirrors block, except that all
IO operations are asynchronous allowing for multiple virt queues to be
used.

The old block remains unchanged and is still used for qcow, android
sparse, and composite disks. Those should be converted to as time
allows, but this dual approach will have to do for now so ARCVM disk
performance can be properly evaluated.

fio --ioengine=libaio --randrepeat=1 --direct=1 --gtod_reduce=1
--name=test --filename=test --bs=4k --iodepth=64 --size=4G
--readwrite=randrw --rwmixread=75

desktop with nvme:

before:
READ: bw=36.2MiB/s (37.9MB/s), 36.2MiB/s-36.2MiB/s (37.9MB/s-37.9MB/s),
io=3070MiB (3219MB), run=84871-84871msec
WRITE: bw=12.1MiB/s (12.7MB/s), 12.1MiB/s-12.1MiB/s (12.7MB/s-12.7MB/s),
io=1026MiB (1076MB), run=84871-84871msec
after:
READ: bw=257MiB/s (269MB/s), 257MiB/s-257MiB/s (269MB/s-269MB/s),
io=3070MiB (3219MB), run=11964-11964msec
WRITE: bw=85.8MiB/s (89.9MB/s), 85.8MiB/s-85.8MiB/s (89.9MB/s-89.9MB/s),
io=1026MiB (1076MB), run=11964-11964msec

samus with 5.6 kernel
before:
READ: bw=55.3MiB/s (57.9MB/s), 55.3MiB/s-55.3MiB/s (57.9MB/s-57.9MB/s),
io=768MiB (805MB), run=13890-13890msec
WRITE: bw=18.5MiB/s (19.4MB/s), 18.5MiB/s-18.5MiB/s (19.4MB/s-19.4MB/s),
io=256MiB (269MB), run=13890-13890msec
after:
READ: bw=71.2MiB/s (74.7MB/s), 71.2MiB/s-71.2MiB/s (74.7MB/s-74.7MB/s),
io=3070MiB (3219MB), run=43096-43096msec
WRITE: bw=23.8MiB/s (24.0MB/s), 23.8MiB/s-23.8MiB/s (24.0MB/s-24.0MB/s),
io=1026MiB (1076MB), run=43096-43096msec

kevin with 5.6 kernel
before:
READ: bw=12.9MiB/s (13.5MB/s), 12.9MiB/s-12.9MiB/s (13.5MB/s-13.5MB/s),
io=1534MiB (1609MB), run=118963-118963msec
WRITE: bw=4424KiB/s (4530kB/s), 4424KiB/s-4424KiB/s (4530kB/s-4530kB/s),
io=514MiB (539MB), run=118963-118963msec
after:
READ: bw=12.9MiB/s (13.5MB/s), 12.9MiB/s-12.9MiB/s (13.5MB/s-13.5MB/s),
io=1534MiB (1609MB), run=119364-119364msec
WRITE: bw=4409KiB/s (4515kB/s), 4409KiB/s-4409KiB/s (4515kB/s-4515kB/s),
io=514MiB (539MB), run=119364-119364msec

eve with nvme and 5.7 kernel
before:
READ: bw=49.4MiB/s (51.8MB/s), 49.4MiB/s-49.4MiB/s (51.8MB/s-51.8MB/s),
io=3070MiB
(3219MB), run=62195-62195msec
WRITE: bw=16.5MiB/s (17.3MB/s), 16.5MiB/s-16.5MiB/s (17.3MB/s-17.3MB/s),
io=1026MiB
 (1076MB), run=62195-62195msec
after
READ: bw=125MiB/s (131MB/s), 125MiB/s-125MiB/s (131MB/s-131MB/s),
io=3070MiB (3219MB), run=24593-24593msec
WRITE: bw=41.7MiB/s (43.7MB/s), 41.7MiB/s-41.7MiB/s
(43.7MB/s-43.7MB/s), io=1026MiB (1076MB), run=24593-24593msec

rammus with 5.10 kernel
before:
READ: bw=6927KiB/s (7093kB/s), 6927KiB/s-6927KiB/s (7093kB/s-7093kB/s),
io=3070MiB (3219MB), run=453822-453822msec
WRITE: bw=2315KiB/s (2371kB/s), 2315KiB/s-2315KiB/s (2371kB/s-2371kB/s),
io=1026MiB (1076MB), run=453822-453822msec
after:
Run status group 0 (all jobs):
READ: bw=10.0MiB/s (11.5MB/s), 10.0MiB/s-10.0MiB/s (11.5MB/s-11.5MB/s),
io=3070MiB (3219MB), run=279111-279111msec
WRITE: bw=3764KiB/s (3855kB/s), 3764KiB/s-3764KiB/s (3855kB/s-3855kB/s),
io=1026MiB (1076MB), run=279111-279111msec

BUG=chromium:901139
TEST=unitests
TEST=boot a test image and run fio tests from the guest to measure speed.
TEST=start ARCVM
TEST=tast run $DUT crostini.ResizeOk.dlc_stretch_stable

Change-Id: Idb63628871d0352bd18501a69d9c1c887c37607b
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2306786
Tested-by: Keiichi Watanabe <keiichiw@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Dylan Reid <dgreid@chromium.org>
Reviewed-by: Chirantan Ekbote <chirantan@chromium.org>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Keiichi Watanabe <keiichiw@chromium.org>
2021-02-17 04:11:55 +00:00
Andrew Walbran
9cfdbd9cc0 Fix clippy errors and some warnings.
TEST=cargo test

Change-Id: I91afe78141c717a592eec5ec77acac2a500ba163
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2623941
Tested-by: kokoro <noreply+kokoro@google.com>
Tested-by: Keiichi Watanabe <keiichiw@chromium.org>
Reviewed-by: Andrew Walbran <qwandor@google.com>
Reviewed-by: Keiichi Watanabe <keiichiw@chromium.org>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Andrew Walbran <qwandor@google.com>
2021-01-18 15:50:25 +00:00
Keiichi Watanabe
eefe7fb19e devices: virtio: fs: DAX based shared memory support
Support virtio-fs's DAX (direct memory access) operation which allows the guest
to directly access file pages.
Specifically, FUSE_SETUP_MAPPING and FUSE_REMOVE_MAPPING operations are
supported.

This option can be used by specifing `dax` option when mount a file system in
the guest.

The DAX optoin improved file I/O performance in most cases.
In Fio tests, both of read and write score were improved by 1.3-14x depending on
test cases.
In Blogbench tests, which create many small files, DAX improved the write score
by 1.5x while the read score was reduced to ~25% (20391 -> 4593).

Here is an excerpt of results:
Fio
* seq_read:     10.2x (143528 -> 1464911)
* seq_write:     3.3x (61253 -> 896791)
* rand_read:    11.6x (138753 -> 1612739)
* rand_write:   14.6x (61253 -> 896791)
* surfing_read:  1.3x (98473 -> 127907)
* surfing_write: 1.3x (83309 -> 108089)

Blogbench
* read:  0.23x (20391 -> 4593)
* write: 1.50x (248 -> 373)

BUG=b:147341783
TEST=Run vm.{Blogbench, Fio} with CL:2291856

Change-Id: I4a47c601412ed32d926de6304337e1594252d258
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2108315
Tested-by: kokoro <noreply+kokoro@google.com>
Tested-by: Keiichi Watanabe <keiichiw@chromium.org>
Commit-Queue: Keiichi Watanabe <keiichiw@chromium.org>
Reviewed-by: Chirantan Ekbote <chirantan@chromium.org>
2020-12-16 06:03:54 +00:00
Chirantan Ekbote
6daa9a5a2c data_model: Replace as_iobuf() with AsRef and AsMut
IoSliceMut doesn't currently provide a way to access the fields of the
internal iovec. Additionally, the as_iobuf() method returns a copy of
the iovec rather than a reference, which can cause hard to diagnose
issues.

Replace the as_iobuf() method with implementations of the AsRef and
AsMut traits. This should also make it easier to hide these impls behind
a `#[cfg(unix)]` in the future.

BUG=none
TEST=unit tests

Change-Id: I58f8c5f579f3d03ec6e612a32343e8423eced8ae
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2571147
Reviewed-by: Noah Gold <nkgold@google.com>
Reviewed-by: Dylan Reid <dgreid@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Chirantan Ekbote <chirantan@chromium.org>
2020-12-09 14:31:11 +00:00
Michael Hoyle
a596a07b0c Final major RawDescriptor transition.
This CL transitions most structs to RawDescriptor and the
associated traits if possible.

BUG=b:162363783
TEST=./build_test

Change-Id: Iabae6ac212787836d77de2b9ffb5d451421ab0dd
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2530911
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: Michael Hoyle <mikehoyle@google.com>
Commit-Queue: Michael Hoyle <mikehoyle@google.com>
2020-11-13 02:38:47 +00:00
Noah Gold
b3fca20e24 Refactor cros_async interface to avoid RawFd.
Instead of creating IoSourceExt from AsRawFd implementers, we've
switched to creating from a marker trait `IntoAsync`. This lets us use
other types like RawDescriptor easily with this crate.  By using the
marker, we also provide some type safety by requiring consumers of
IoSourceExt to declare that their type is expected to work with async
operations. This way we can provide stronger guarantees that an async
IO trait object will behave in a reasonable way.

This CL also purges the cros_async -> base and io_uring -> base
references, and provides the base types needed to add new async
primitives to base.

BUG=none
TEST=builds

Change-Id: I0b0ce6ca7938b22ae8e8fb4e604439f0292678f2
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2504481
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Noah Gold <nkgold@google.com>
Reviewed-by: Chirantan Ekbote <chirantan@chromium.org>
2020-11-10 11:40:50 +00:00
Michael Hoyle
6a0960a4a7 base: First steps towards universal RawDescriptor
This involves removing RawFd trait implementations from
Event and Timer, and resolving the echoing dependencies from
there.

Ultimately, this results mostly in changes across ioctl
(new thin layer in base), kvm, msg_on_socket, and a few other
areas. As usual, most changes are negligible.

BUG=b:162363783
TEST=./build_test

Change-Id: I47dec94666bc3430fed627b4767919c9cb4fce6f
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2462330
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Michael Hoyle <mikehoyle@google.com>
2020-10-31 07:12:34 +00:00
Michael Hoyle
e392c46953 base: Add WaitContext
This is a layer above PollContext with a more generic interface.
As PollContext is used so widely, this is quite a large change
in order to accomodate the interface update, especially with the
use of RawDescriptor. In some cases this has caused an echo
of updates to RawDescriptor, which is fine because of our eventual
goal to move the whole codebase to it regardless.

Note there are a few instances of forcing the RawDescriptor update
chain to stop, ex. ioctl. This is to keep the scope of this CL
concentrated and avoid changing entire other areas.

Note that this CL leaves out a few additional pieces of work:
 - The sole usage of EpollContext over PollContext (event_loop),
which poses a bigger challenge for interface changes
 - Full PollToken renaming, which is a tiny change turned difficult
due to the unavailability of type aliases for traits.
 - Renaming certain methods which have been updated to use
RawDescriptor such as keep_fds. Some have enough dependencies that
they are worth avoiding to keep this CL pointed, but will be
addressed in future CLs to make sure the whole codebase is on the
fd->descriptor train

BUG=b:162363783
TEST=./build_test

Change-Id: Iff2cfe8f90dea55f1388f8e91bdc698e121a8e43
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2455726
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Michael Hoyle <mikehoyle@google.com>
2020-10-31 07:12:30 +00:00
Michael Hoyle
3038e40a6b base: Refactor mmap to use builder pattern.
BUG=b:165423256
TEST=./build_test

Change-Id: Ia67c1a7fe29b66c9cab38476eecde8c25a55617b
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2442569
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Michael Hoyle <mikehoyle@google.com>
2020-10-16 07:15:24 +00:00
Zach Reizner
ab04758814 build_test: misc options, improvements, amd bug fixes
New option, --size-only, speeds up using build_test for getting release
binary size by skipping everything else. The lto flag is also added for
release builds to get a more realistic comparison.

The list of crates to test is built up automatically instead of
hard coded. To modify what gets included, empty .build_test_* files are
checked for existance. This is better than hard coding the list of
packages because it was frequently out of date.

For certain crate tests, a dynamic library that only exists in a sysroot
is required. This change includes a fix that adds the sysroot's lib
directory to the LD_LIBRARY_PATH env variable, similar to how
PKG_CONFIG_LIBDIR is modified.

TEST=build_test
BUG=None

Change-Id: I626cbcccf40035a0d29001cef7989a091848e4c9
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2444273
Tested-by: Zach Reizner <zachr@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Auto-Submit: Zach Reizner <zachr@chromium.org>
Reviewed-by: Zach Reizner <zachr@chromium.org>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Zach Reizner <zachr@chromium.org>
2020-10-15 13:40:32 +00:00
Noah Gold
ee71c80b08 Mark MsgSocket as !Sync.
Semantically, MsgSocket is !Sync: multiple threads reading/writing to
the same end of the socket will result in U.B.. This CL brings the
syntactic definition of MsgSocket in line with its semantic meaning by
marking it as !Sync. While earlier work caught one of the other
consumers that was using a socket end in a Sync manner, this CL has
found another: the usb crate.

The fixes proposed for the usb crate are mostly for compilation's sake,
and should have little material effect on the operation of crosvm. Note
that they will assure correctness, since the send/recv for the socket
end involved are in the same critical section.

BUG=None
TEST=builds

Change-Id: I38469d9ed3228a43457b7d43879a6b4fff97b9a4
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2466960
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Noah Gold <nkgold@google.com>
2020-10-14 01:36:09 +00:00
Michael Hoyle
685316f0bd base: EventFd -> Event renaming
Note the CL size is large entirely due to the rename,
the changes are mostly negligible.

Also making a few small additional changes in sys_util
areas that don't need much attention in base. This includes
typedefing and adding specific imports for areas that don't
require significant interface changes.

BUG=b:162363783
TEST=./build_test

Change-Id: I4a2c9c4cdce7565806ed338e241c6b8c82c855c7
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2415180
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Michael Hoyle <mikehoyle@google.com>
2020-10-06 13:50:09 +00:00
Michael Hoyle
08d86a44a0 base: Add timer package
Very little of substance is added here, just the base boilerplate

BUG=b:162363783
TEST=./build_test

Change-Id: I2e3b3b45cf1d7234784d769b4dced31f10a8774d
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2366110
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Michael Hoyle <mikehoyle@google.com>
Reviewed-by: Zach Reizner <zachr@chromium.org>
2020-09-30 19:44:42 +00:00