Commit graph

119 commits

Author SHA1 Message Date
Dennis Kempin
0c09cdbdbc cros_async: Fix windows timer flake
The test is flaky by nature since there is no API guarantee on how
long SetWaitableTimer will sleep. This is flaky on our Luci bots, so
increase the permitted deviation a bit.

BUG=b:238232551
TEST=./tools/run_tests --build-target=mingw64

Change-Id: Ib227be097a0a5559898571cb6b77c37e7fd61cff
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3749936
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Vikram Auradkar <auradkar@google.com>
2022-07-07 22:23:08 +00:00
Alexandre Courbot
a5c1b2ff7f cros_async: add async_from_local()
Add an async_from_local() method, which does the same thing as
async_from() (create an IoSourceExt for asynchronous I/O), but without
the requirement that the passed argument implements Send. Async I/O
sources created that way can only be used in thread-local executors.

BUG=b:229554679
TEST=cargo build

Change-Id: Id4cd0cb177e5fed4b72fab12a6f82f964b9bac28
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3670773
Commit-Queue: Alexandre Courbot <acourbot@chromium.org>
Reviewed-by: Keiichi Watanabe <keiichiw@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
2022-06-08 02:29:19 +00:00
Daniel Verkamp
4c0de58cbf base: unix: rename PollContext to EventContext
We already exported PollContext as EventContext for use in the portable
WaitContext wrapper, so this makes no difference to callers.

A few remaining references to PollContext in documentation and errors
are fixed up as well to avoid confusion.

BUG=None
TEST=cargo build

Change-Id: I89943d4d9e135a0da7dd0fbff470b6a14efd669a
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3687059
Reviewed-by: Noah Gold <nkgold@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
2022-06-06 17:07:35 +00:00
Daniel Verkamp
f0f10c4209 base: remove WatchingEvents in favor of EventType
Migrate the last few users of WatchingEvents to the equivalent EventType
values so we can remove the unix-specific WatchingEvents type.

BUG=b:213153157
TEST=cargo build

Change-Id: I215127309591125c4c64ff2963e7d2b8e1914cf3
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3687057
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
Reviewed-by: Noah Gold <nkgold@google.com>
2022-06-06 17:07:33 +00:00
Daniel Verkamp
3e8054ee7c base: unix: merge EpollContext into PollContext
The main distinction between PollContext and EpollContext was that the
latter was safe to use from multiple threads. This was not true of the
more widely-used PollContext for two reasons:

1. The `events` array was stored inside the `PollContext` structure,
   whereas `EpollContext` required the caller to pass their own storage
   for events.
2. `PollContext` had a hangup detection feature used to debug busy
   looping cases that result from a failure to remove hungup file
   descriptors from the context.

Point 1 is resolved by returning a `SmallVec` of events that avoids
allocation for the normal case (and in fact it should avoid allocation
in all cases on Linux, where the maximum number of events returned from
a single `epoll_wait()` call is limited to the same size as the
preallocated `SmallVec`). This simplifies the API and also means that
there is no need for per-context storage.

Point 2 can't easily be resolved, since it would require the `wait` call
to mutate shared state (this could be done by adding a mutex around the
shared data, but that seems like too much overhead for the value of the
feature). Instead, this patch just removes the hangup detection code.

BUG=b:213153157
TEST=tools/dev_container tools/presubmit --all

Change-Id: Ia48c46de96976da27cb5387e3e5e8fcf92d0e85b
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3633111
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
Reviewed-by: Noah Gold <nkgold@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
2022-06-02 21:44:39 +00:00
Anton Romanov
ace7edca0e crosvm: cros_async: disable unstable test
Test verifies if operation is not executed if executor is dropped before
running. We do not currently guarantee that

BUG=none
TEST=cargo test

Change-Id: I56d98e4af869c26542c7ebacb127ed3bfd980553
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3655277
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Anton Romanov <romanton@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
2022-05-19 23:34:28 +00:00
Daniel Verkamp
5e43f03d80 Fix rustdoc bare URL warnings
BUG=None
TEST=cargo doc

Change-Id: Ib9d14d8aa38eef97c02c60fca3928b70a833b6bb
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3630420
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
2022-05-06 20:46:55 +00:00
Anton Romanov
f94281434b crosvm:cros_async:uring: Implement async operation cancellation
BUG=None
TEST=presubmit

Change-Id: I8d0bd6109e8230f953ee736a805677609f5c434d
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3628241
Auto-Submit: Anton Romanov <romanton@google.com>
Commit-Queue: Anton Romanov <romanton@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
2022-05-06 00:13:38 +00:00
Daniel Verkamp
404c8261e9 cros_async: ignore punch_hole tests on wine
Wine does not support the FSCTL_SET_ZERO_DATA request:

  fixme:ntdll:server_ioctl_file Unsupported ioctl 980c8 (device=9 access=2
  func=32 method=0)

So ignore the tests that require it when using the mingw toolchain,
which we assume means testing in wine for now.

BUG=None
TEST=tools/dev_container ./tools/run_tests --target=host --arch=win64

Change-Id: I351758fac3b3e1eb44545bb1d2340ddec4461120
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3626021
Reviewed-by: Noah Gold <nkgold@google.com>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Anton Romanov <romanton@google.com>
2022-05-05 21:11:58 +00:00
Daniel Verkamp
9ab4f29e7a devices: vhost-user: fs: keep executor fd inside jail
A previous commit moved the creation of the executor for the vhost-user
device processes before the minijail jail_and_fork() call, and the file
descriptor of the executor was not marked for preservation when entering
the jail, so running the vhost-user-fs device would result in errors
like this:

  [WARNING:cros_async/src/sys/unix/uring_executor.rs:343] Failed to
  submit NOP for waking up executor: Failed to enter io uring: 9

This is because the io_uring context file descriptor was no longer
valid, so uring system calls would return 9 (EBADF).

Add AsRawDescriptors implementations to UringExecutor and FdExecutor so
we can preserve the fd across the minijail fork.

Other vhost-user devices were not affected because they don't enter a
jail; the fs device requires this so it can do uid/gid remapping.

BUG=b:231396155
TEST=crosvm device fs --socket /tmp/vhost.sock --tag test --shared-dir /

Fixes: d368c1520a ("devices: vhost-user: fs: remove global executor variable")
Change-Id: I38cf02034bbc28de2582e2ea92bdc13b1d6055d2
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3628292
Reviewed-by: Keiichi Watanabe <keiichiw@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Lepton Wu <lepton@chromium.org>
2022-05-05 20:03:03 +00:00
Anton Romanov
6cc69d676d crosvm: cros_async: Drain uring operations when dropping uring executor
Otherwise we may drop the memory that kernel still references.

BUG=b:230934649
TEST=cargo test

Change-Id: I1ab4fcc721118f744b8975d01fc907c511309585
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3625899
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Anton Romanov <romanton@google.com>
Reviewed-by: Dennis Kempin <denniskempin@google.com>
2022-05-03 22:48:45 +00:00
Vikram Auradkar
68fb967912 base: Export event related symbols from base in windows
BUG=b:213153157
TEST=presubmit

Change-Id: If4b122b70c11ec1b1591296ff2f67c389fa89671
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3625708
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Reviewed-by: Noah Gold <nkgold@google.com>
Commit-Queue: Vikram Auradkar <auradkar@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
2022-05-03 21:04:21 +00:00
Vikram Auradkar
cdb9e125ae base: rename EventFd to Event
Note that
- the file/symbols within `src/sys/unix/eventfd.rs` have not
  changed yet.
- base still exports EventFd to keep common/cros_asyncv2 happy

BUG=b:213153157
TEST=presubmit

Change-Id: Ie0a4308e8501d2e91364b049e6575d656af866cd
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3624568
Reviewed-by: Noah Gold <nkgold@google.com>
Commit-Queue: Vikram Auradkar <auradkar@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
2022-05-03 21:04:19 +00:00
Daniel Verkamp
55a5b54bf2 base: clean up vestigal sys_util references
The sys_util crate has been merged into base, but a few places still
refer to the old name. Fix or remove them as appropriate.

BUG=b:227226222
TEST=tools/presubmit --all

Change-Id: Icf9b57aff672b7c1afec768c9694e059f0f9a43d
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3621205
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
Reviewed-by: Dennis Kempin <denniskempin@google.com>
2022-05-02 20:16:18 +00:00
Noah Gold
dc8d1dbdca cros_async: update Timer & sleep docs for Windows.
base::Timer's sleep may wake early on Windows, which has knockon effects
here where we consume it.

BUG=b:229680949
TEST=n/a

Change-Id: I44a8f78e3dbc01ef562101c2706b89d2a88e5bf7
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3617161
Commit-Queue: Noah Gold <nkgold@google.com>
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
2022-04-29 19:10:41 +00:00
Noah Gold
a81eebfe8e cros_async: move IntoAsync to use AsRawDescriptor
Originally cros_async was not permitted to depend on base, and as such
was precluded from using RawDescriptor/AsRawDescriptor. Now that we can
use base, we are switching over to those types/traits as they allow for
much cleaner cross platform code.

BUG=b:228904576
TEST=builds (also tested applying downstream).

Change-Id: Id5da60b127656568069795707a48beca1a015151
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3584139
Reviewed-by: Keiichi Watanabe <keiichiw@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Noah Gold <nkgold@google.com>
2022-04-19 22:19:06 +00:00
Noah Gold
68bbe92339 cros_async: upstream Windows support.
Major changes:
* Adds Windows implementations
* Refactors cros_async to use the styleguide for cross platform code.
* Adds platform specific Descriptor impls to Timer & Event (short term;
  this will go away in the next CL in the series).

Minor adjustments:
* The doctests for the Executor were passing the wrong type of seek
  parameter to write_from_vec. It was disabling seeking (None) when it
  should have been  seeking to zero (Some(0)).

BUG=b:213147081
TEST=tested by Windows & Linux bots.

Change-Id: Id7e025ceb9f1be4a165de1e9ba824cf60dd076ff
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3579735
Reviewed-by: Alexandre Courbot <acourbot@chromium.org>
Reviewed-by: Keiichi Watanabe <keiichiw@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Noah Gold <nkgold@google.com>
2022-04-19 22:19:05 +00:00
Dennis Kempin
1def2e61d9 reland: Remove temporarily duplicated code from codebase
This removes sys_util(_core), which moved into base::unix/common, as
well as common/(cros_async,io_uring), which moved into the root
directory.

The only reason the code was still around is that they were still
used in the ChromeOS codebase.
ChromeOS has pinned the version of crosvm it uses for these libraries
so we can go ahead and remove the code.

A few remaining references to sys_util have been updated to base.

BUG=b:227226222,b:229016539
TEST=presubmit

Change-Id: I7a711044de7e067b217287f2bac822d6ac7d3964
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3593852
Reviewed-by: Allen Webb <allenwebb@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
2022-04-19 21:48:29 +00:00
Dennis Kempin
66be807115 Revert "Remove temporarily duplicated code from codebase"
This reverts commit aac461e25e.

Reason for revert: Breaks audio_streams portage build

Original change's description:
> Remove temporarily duplicated code from codebase
>
> This removes sys_util(_core), which moved into base::unix/common, as
> well as common/(cros_async,io_uring), which moved into the root
> directory.
>
> The only reason the code was still around is that they were still
> used in the ChromeOS codebase.
> ChromeOS has pinned the version of crosvm it uses for these libraries
> so we can go ahead and remove the code.
>
> A few remaining references to sys_util have been updated to base.
>
> BUG=b:227226222,b:229016539
> TEST=presubmit
>
> Change-Id: I35a3d1f0ea28182b77abf9b423fcab4cad525981
> Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3580118
> Reviewed-by: Allen Webb <allenwebb@google.com>
> Tested-by: kokoro <noreply+kokoro@google.com>
> Commit-Queue: Dennis Kempin <denniskempin@google.com>

Bug: b:227226222,b:229016539
Change-Id: I907279ab47718355cd57915830580929dc157f84
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3593846
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Maciek Swiech <drmasquatch@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
2022-04-19 18:24:13 +00:00
Noah Gold
9da4030ab0 cros_async: use portable enum for fallocate modes
fallocate modes were previously unix specific constants, which won't work after
we add support for Winodws. Here, we've put a generic abstraction in place so
that each platform can implement the details as appropriate.

Thanks to acourbot@ for suggesting the splits in this series.

BUG=b:213147081
TEST=see final CL in series.

Change-Id: I822ad6c4a26eea716482029e8a6c0489aa72c595
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3583613
Reviewed-by: Alexandre Courbot <acourbot@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Noah Gold <nkgold@google.com>
2022-04-19 05:37:42 +00:00
Noah Gold
78ef113617 cros_async: make AsyncTimer use base::Timer
TimerFd is the sys_util era Unix-specific timer, which doesn't support Windows.
This CL moves us to base::Timer, which is the cross platform equivalent.

Thanks to acourbot@ for suggesting the splits in this series.

BUG=b:213147081
TEST=see final CL in series.

Change-Id: I41c8ad88da77c48397ed466ff11aecd703c470a8
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3583612
Reviewed-by: Alexandre Courbot <acourbot@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Noah Gold <nkgold@google.com>
2022-04-19 05:37:41 +00:00
Noah Gold
084aa95af1 cros_async: make AsyncEvent use base::Event
EventFd is the original sys_util era Unix-specific Event. We need to use
base::Event, which is the cross platform event that works on Windows.

Thanks to acourbot@ for suggesting the splits in this series.

BUG=b:213147081
TEST=see final CL in series.

Change-Id: I174313cb544c5fc3768810365a42e6eaac1ca91a
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3583611
Reviewed-by: Alexandre Courbot <acourbot@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Noah Gold <nkgold@google.com>
2022-04-19 05:37:40 +00:00
Dennis Kempin
aac461e25e Remove temporarily duplicated code from codebase
This removes sys_util(_core), which moved into base::unix/common, as
well as common/(cros_async,io_uring), which moved into the root
directory.

The only reason the code was still around is that they were still
used in the ChromeOS codebase.
ChromeOS has pinned the version of crosvm it uses for these libraries
so we can go ahead and remove the code.

A few remaining references to sys_util have been updated to base.

BUG=b:227226222,b:229016539
TEST=presubmit

Change-Id: I35a3d1f0ea28182b77abf9b423fcab4cad525981
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3580118
Reviewed-by: Allen Webb <allenwebb@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
2022-04-18 18:35:07 +00:00
Junichi Uekawa
33af75be29 crosvm: Bump version number.
There are two copies of io_uring and cros_async that are slightly
different. Bump the version number of new one.

BUG=None
TEST=cargo build

Change-Id: I7f82d3c5f01633bef7d0ce14ab777bbb50d4fbd5
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3565625
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Commit-Queue: Junichi Uekawa <uekawa@chromium.org>
2022-04-08 01:04:50 +00:00
Daniel Verkamp
2d01c1df92 base, sys_util: remove Timer::is_armed function
This is currently only used in tests, and removing it makes the
cross-platform Timer API simpler to implement. The Windows version of
Timer did not implement this API, so it was already unusable in portable
code.

BUG=b:215618361
TEST=tools/presubmit
TEST=cargo test -p base timer

Change-Id: I57ab15e8b652d0df3664d95bc7759b9c84fe5e10
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3570178
Reviewed-by: Noah Gold <nkgold@google.com>
Reviewed-by: Alexandre Courbot <acourbot@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
2022-04-05 20:11:58 +00:00
Noah Gold
c286772db8 base: upstream Tube cross platform support.
Upstreams support for Tubes on Windows, splitting Tube into platform
specific files. This contains several critical enhancements:

* POSIX Tubes support multi producer multi consumer configurations, but
  Windows has remained strictly SPSC for each direction. Windows cannot
  support MPMC, and that configuration is not really something we want
  either. To address that, this CL introduces directional Tubes. A
  SendTube is clonable, and a RecvTube is not, which gives us MPSC.

* This CL also fixes multiple interface conflicts that have developed
  between Linux & Windows:
    + send wasn't async on the Linux AsyncTube.
    + send data wasn't passed as owned on the Linux AsyncTube.
    + Adds the 'static constraint for AsyncTube::send on POSIX. This is an
      requirement on Windows.
    + Event::read_timeout doesn't need to take &mut self, and it wasn't
      downstream. This CL switches to &self.

* Adds the missing notifier.rs file in base.

Note that this CL does not attempt to remove balloon's usage of
Tube::try_clone. That's a somewhat involved issue that should be tackled in
its own CL.

Test: tested downstream on Windows & Linux bots, upstream on Linux bots.

Bug: b:221484449

Change-Id: I288dbc1d1e42f8ce08258cdaaf85100ca93721ef
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3536897
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Noah Gold <nkgold@google.com>
2022-03-22 19:00:54 +00:00
Dennis Kempin
0928f91f98 Refactoring: Copy common/io_uring to io_uring
This applies the script from https://crrev.com/c/3534501

BUG=b:22320646
TEST=presubmit

Change-Id: Ib0c5bdb55df2098930fb1c65e9bd59288c44f357
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3534502
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: Dennis Kempin <denniskempin@google.com>
2022-03-21 19:52:56 +00:00
Dennis Kempin
4193d87a97 cros_async: Depend on base, not sys_util
base was previously providing some async types which now would
cause a circular dependency. Those have been moved into cros_async.

BUG=b:22320646
TEST=presubmit

Change-Id: I1f526ccfc5882f3a64404f714b13ac92ebfddcd6
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3533614
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
2022-03-21 19:52:56 +00:00
Dennis Kempin
55c6a3b5cd Refactoring: Move common/cros_async to cros_async
This runs the script added in https://crrev.com/c/3533607

BUG=b:22320646
TEST=presubmit

Change-Id: I2e7efdb35508d45281f046e64c24aa43e27f2000
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3533608
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
2022-03-21 19:52:56 +00:00
Dennis Kempin
8a1c50d5cd Refactoring: Move various general purpose crates to common/
This change contains the results of running

./tools/contib/cargo_refactor.py

This will break the next uprev, and needs to be synchronizized
with the corresponding ebuild changes in https://crrev.com/c/3248925

BUG=b:195126527
TEST=./tools/run_tests

Change-Id: Ied15a1841887bb8f59fba65b912b81acf69beb73
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3248129
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
2021-10-29 22:31:43 +00:00
Daniel Verkamp
7fd815ed22 crosvm: switch to upstream tempfile crate
Use the crates.io implementation of tempfile instead of our own version.

Our reimplementation is kept in the tree for now in case of dependencies
outside of the crosvm tree; it can be removed later once those are fully
switched over to the crates.io implementation.

BUG=b:199204746
TEST=emerge-hatch crosvm

Change-Id: I07d3404239302ab9a17f4ddc82a9479b256e4eb4
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3209839
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Chirantan Ekbote <chirantan@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
2021-10-11 18:35:55 +00:00
Chirantan Ekbote
2590740b08 cros_async: Fix underflow in BlockingPool worker threads
The `num_idle` field of the shared state between BlockingPool worker
threads can underflow in the following case:

* state.num_idle == 2.
* We spawn 2 new tasks into the BlockingPool.
* Both idle worker threads are woken up.  `state.num_idle` goes to 0.
* The first worker thread wakes up and pulls a task from the queue.
  That task finishes very quickly so the worker thread pulls the second
  task from the queue before the second worker thread is scheduled.
* The second worker thread is scheduled.  It sees that
  `s.tasks.is_empty() == true` so it goes back to waiting on the
  Condvar.
* The second worker thread's wait times out and it tries to decrement
  `state.num_idle` leading to underflow.

Fix this by adding a `num_notified` field to the shared worker state.
This field acts like a counter for the number of idle worker threads
that have been woken up.

When an idle thread is waiting on a Condvar, rather than checking if the
task queue is empty, it will instead check if num_notified > 0.  When an
idle worker thread observes that num_notified > 0 it decrements it by 1
and then goes back to processing tasks from the queue.  num_idle is only
decremented when num_notified is 0.

Change the num_idle decrement to a checked_sub so that we can catch it
even when -Coverflow_checks=off.  Also add a test for this case.  This
test consistently panics without the num_notified changes.

BUG=none
TEST=unit tests

Change-Id: Ia1b348605e0d02415635cdd023db1c10201ab661
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3139159
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Richard Zhang <rizhang@google.com>
Reviewed-by: Noah Gold <nkgold@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Chirantan Ekbote <chirantan@chromium.org>
2021-09-03 12:58:17 +00:00
Daniel Verkamp
ddcf7bd2ab cros_async: convert to ThisError and sort
BUG=b:197143586
TEST=cargo check

Cq-Depend: chromium:3105313
Change-Id: Ic9757b7e1947970910245fe954e47e4a2b7aa28e
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3105074
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
Reviewed-by: Chirantan Ekbote <chirantan@chromium.org>
2021-08-26 22:29:41 +00:00
Andrew Walbran
bd9b1bc45b Add newlines to end of Cargo.toml files.
Omitting them causes issues with cargo2android.py.

TEST=cargo test

Change-Id: Ib7d62b4218527e436d6b3fc13142e05ed67d2d6c
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3097680
Auto-Submit: Andrew Walbran <qwandor@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Chirantan Ekbote <chirantan@chromium.org>
Commit-Queue: Andrew Walbran <qwandor@google.com>
2021-08-17 20:20:41 +00:00
Chirantan Ekbote
5590a24b1b cros_async: Add a deadline to BlockingPool::shutdown
Add an optional deadline to BlockingPool::shutdown.  Any worker threads
that have not yet exited once the deadline expires will be detached.
This ensures that we don't end up blocking indefinitely while waiting
for worker threads to exit.

BUG=none
TEST=unit tests

Change-Id: I6d7e73e1c95a934a4fd80825a9d44187532408b5
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3058842
Auto-Submit: Chirantan Ekbote <chirantan@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Chirantan Ekbote <chirantan@chromium.org>
Reviewed-by: Noah Gold <nkgold@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Reviewed-by: Keiichi Watanabe <keiichiw@chromium.org>
2021-08-12 04:29:18 +00:00
Vikram Auradkar
ed4bd118bc cros_async: name threads + doc fix
BUG=b:191488633
TEST=cargo test && cargo doc && cargo clippy

Change-Id: I61cfb4b547e7ca280ad1726300fd145617a27d69
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3059870
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Vikram Auradkar <auradkar@google.com>
Auto-Submit: Vikram Auradkar <auradkar@google.com>
Reviewed-by: Chirantan Ekbote <chirantan@chromium.org>
2021-07-30 04:41:36 +00:00
Chirantan Ekbote
c950eb92c5 cros_async: Make file offsets optional
p{read,write} cannot be used on sockets even if the file offset is 0.
Make the file offset optional and fall back to regular read/write when
it is not set.

BUG=none
TEST=cargo test

Change-Id: Iff938aabe613b6164782714cfac94743d64f551a
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3033233
Auto-Submit: Chirantan Ekbote <chirantan@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Keiichi Watanabe <keiichiw@chromium.org>
Reviewed-by: Noah Gold <nkgold@google.com>
Reviewed-by: Woody Chow <woodychow@google.com>
Commit-Queue: Chirantan Ekbote <chirantan@chromium.org>
2021-07-20 02:38:48 +00:00
Chirantan Ekbote
2f879fb2be cros_async: Add spawn_blocking()
Add a spawn_blocking() method to the Executor so that users don't need
to manually create a BlockingPool every time they need to do some
blocking work.

BUG=b:179755651
TEST=cargo test

Change-Id: I70d111d98a4c51af4bc8ed8181a4b102bf3c2ffa
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2987586
Commit-Queue: Keiichi Watanabe <keiichiw@chromium.org>
Auto-Submit: Chirantan Ekbote <chirantan@chromium.org>
Reviewed-by: Noah Gold <nkgold@google.com>
Reviewed-by: Keiichi Watanabe <keiichiw@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
2021-07-15 23:18:23 +00:00
Allen Webb
c6fa73d76f Fix clippy warnings and Cargo.lock
This fixes:
* version mismatches in Cargo.lock
* style issues
* implementations of Into that should be From
* deprecated protobuf APIs

It also adds RUST_BACKTRACE=1 to the kokoro tests.

BUG=None
TEST=./bin/preupload-clippy

Change-Id: I8e9157c903f2080a5fdcc4d3e4ed72fbad41c64f
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3024427
Auto-Submit: Allen Webb <allenwebb@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Allen Webb <allenwebb@google.com>
2021-07-15 03:33:17 +00:00
Chirantan Ekbote
6074d45aeb cros_async: Don't use io_uring on kernels < 5.10
Kernels before 5.10 had known bugs in the io_uring implementation.
Don't use io_uring when we detect this.  Also skip all the io_uring
tests in this case.

BUG=none
TEST=cargo test

Change-Id: I5fd6203ad25a6fb85ff28f1a6ddb0181f836ad89
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3006309
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Chirantan Ekbote <chirantan@chromium.org>
Reviewed-by: Keiichi Watanabe <keiichiw@chromium.org>
Reviewed-by: Woody Chow <woodychow@google.com>
2021-07-14 09:42:39 +00:00
Chirantan Ekbote
93e27733d8 cros_async: Add BlockingPool
BlockingPool provides a dedicated thread pool for running blocking
operations.  This is useful when an async task wants to do some
CPU-intensive computation, run some IO operation that cannot be
performed asynchronously, or to call a blocking API in a dependency that
doesn't have an asynchronous variant.

BUG=b:179755651
TEST=cargo test

Change-Id: I389fc504f380d66325739d2d6b7afe58e024194d
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2987585
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Chirantan Ekbote <chirantan@chromium.org>
Reviewed-by: Keiichi Watanabe <keiichiw@chromium.org>
2021-07-13 17:02:37 +00:00
Chirantan Ekbote
eff60328c1 Add a From impl for async errors to io::Error
Sometimes users of cros_async need to return an io::Error.  Provide a
conversion function to convert the various async errors into an
io::Error.  This allows callers to access the underlying error without
forcing us to expose these internal implementation details.

BUG=none
TEST=unit tests

Change-Id: Ie0ab00cb80ea58f628a38c173e28babf30b8d5b4
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3006308
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Keiichi Watanabe <keiichiw@chromium.org>
Commit-Queue: Chirantan Ekbote <chirantan@chromium.org>
2021-07-06 11:35:26 +00:00
Woody Chow
1e3022c922 cros_async: Add Send to async_from
This allows struct using IoSourceExt to be Send

BUG=None
TEST=cargo test

Change-Id: I7541977151537387da59b0ada5592a77a0382998
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3002823
Reviewed-by: Chirantan Ekbote <chirantan@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Woody Chow <woodychow@google.com>
2021-07-02 13:11:46 +00:00
Chirantan Ekbote
79bcb99530 cros_async: Make block_on a top-level function
This doesn't really have anything to do with synchronization so move it
into its own module and export it as a top-level function.

BUG=b:179755651
TEST=cargo test

Change-Id: Icb733c36ee1d4cebcb445e47289c92b9b77a278b
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2987583
Reviewed-by: Keiichi Watanabe <keiichiw@chromium.org>
Reviewed-by: Noah Gold <nkgold@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Chirantan Ekbote <chirantan@chromium.org>
2021-06-29 15:20:58 +00:00
Woody Chow
e6bf005d26 cros_async: Add TimerAsync::sleep
to reduce boilerplate code.

BUG=None
TEST=cargo test

Change-Id: Ia1485b63453145e0b0dfaa6730caee34b52bc0b5
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2960408
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Dylan Reid <dgreid@chromium.org>
Reviewed-by: Chirantan Ekbote <chirantan@chromium.org>
Commit-Queue: Woody Chow <woodychow@google.com>
2021-06-15 08:38:52 +00:00
Chirantan Ekbote
d2399b7d6f cros_async: Add AsyncWrapper type
This makes it easier to implement IntoAsync for foreign types.

BUG=none
TEST=Use it in a later change

Change-Id: I070fe3b63ac7458e21fa38b3a1b1bdb318c44d5b
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2891120
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Chirantan Ekbote <chirantan@chromium.org>
Reviewed-by: Noah Gold <nkgold@google.com>
Reviewed-by: Keiichi Watanabe <keiichiw@chromium.org>
2021-05-17 11:23:19 +00:00
Daniel Verkamp
9901c7509c cros_async: switch to std::hint::spin_loop
Replace the deprecated std::sync::spin_loop_hint with
std::hint::spin_loop.

Fixes this new warning from Rust 1.51:

  warning: use of deprecated function `std::sync::atomic::spin_loop_hint`:
  use hint::spin_loop instead

BUG=None
TEST=bin/clippy

Change-Id: Ic975c77ea19bcf256f73c0b3b89f85866abfe0bb
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2864362
Reviewed-by: Zach Reizner <zachr@chromium.org>
Reviewed-by: Dylan Reid <dgreid@chromium.org>
Tested-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
2021-05-06 22:06:52 +00:00
Daniel Verkamp
2924f8bfb7 kernel_cmdline, cros_async: impl From instead of Into
As recommended by the clippy from_over_into warning, allow conversion to
Vec<u8> by adding a more flexible From impl instead of Into.

https://rust-lang.github.io/rust-clippy/master/index.html#from_over_into

BUG=None
TEST=bin/clippy

Change-Id: Iaa6455b11629846e5ce9bc9be4a6cae0baf87d5b
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2862580
Reviewed-by: Zach Reizner <zachr@chromium.org>
Tested-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
2021-05-06 22:06:50 +00:00
Zach Reizner
6d6efb20c0 remove dependencies on syscall_defines
The syscall_defines crate is redundant with an up to date libc. This
change removes any dependency on syscall_defines. A new libc is required
to bring in some new syscall numbers like the ones for io_uring.

TEST=./test_all
BUG=None

Cq-Depend: chromium:2832000
Change-Id: I6df7fb992bacb5efd54cefca08836d52f4bfcd8c
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2832001
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Allen Webb <allenwebb@google.com>
Reviewed-by: Dylan Reid <dgreid@chromium.org>
Reviewed-by: Chirantan Ekbote <chirantan@chromium.org>
Commit-Queue: Zach Reizner <zachr@chromium.org>
2021-04-21 21:01:38 +00:00
Chirantan Ekbote
3c45e3f7f7 sync: Condvar: Use compare_exchange_weak
This allows the compiler to generate more efficient assembly for arm.
It also maintains consistency for the whole file since we were already
using compare_exchange_weak in some places and not others.

BUG=none
TEST=`FEATURES=test emerge-kukui-arc-r cros_async`.  Also copy the unit
     test binary onto the device and run in a loop to see that there are
     no failures.

Change-Id: Ia8c942c419ac2989a5653875d78c48003fb757d8
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2805754
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Chirantan Ekbote <chirantan@chromium.org>
2021-04-08 07:45:04 +00:00