Commit graph

214 commits

Author SHA1 Message Date
Dennis Kempin
e577c91efc dev_container: Always run as root inside container
Since we default to podman, we can get rid of the _user container and
run as root inside the container, which greatly simplifies the tooling.

This also allows us to provide a single, default devcontainer.json
for local development as well as codespaces.

podman can be used with this devcontainer.json as well, e.g. in
VSCode by setting:

"dev.containers.dockerPath": "podman"

The with_kvm/devcontainer.json can be used to add access to kvm, which
may not be available on all machines. So it's an optional config.

BUG=b:273529620
TEST=Open with devcontainer in VSCode with both docker and podman,
run `presubmit --no-delta`
TEST=tools/dev_container tools/presubmit --no-delta

Change-Id: I10e04898a0e1adb6e4939f6f2e0259dc9b9aa62d
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4419666
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Reviewed-by: Zihan Chen <zihanchen@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
2023-04-25 21:33:42 +00:00
Christian Blichmann
c53b40e455 Fix typos found by common-typos checker
Change-Id: Ie539304c85d0dc54bc1e17892db8bbbfb15b69ef
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4417934
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
Reviewed-by: Dennis Kempin <denniskempin@google.com>
2023-04-12 16:58:51 +00:00
Federico 'Morg' Pareschi
2e93874a25 cros_tracing: Add trace_marker integration tests
This CL adds a new "root" test suite for trace_marker integration tests,
since the tests require root to run in order to access the kernel's
tracefs mounted partition.

BUG=b:273379306
TEST=CARGO_TARGET_X86_64_UNKNOWN_LINUX_GNU_RUNNER='sudo -E' cargo test --features=trace_marker

Change-Id: I4656374023d91d87959f6ce0d59eb9e51bdc77bd
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4349039
Reviewed-by: Keiichi Watanabe <keiichiw@chromium.org>
Auto-Submit: Morg <morg@chromium.org>
Commit-Queue: Morg <morg@chromium.org>
2023-04-12 01:21:39 +00:00
Dennis Kempin
cd98b2fa79 testvm: Pick a random port for SSH
If we want to use the x86vm as a default, we may end up running
multiple instances, e.g one inside the container, another directly
on the host.

Make sure these do not conflict by picking a random port when
starting the VM.

BUG=b:275717656
TEST=x86vm shell - connects to previously started VM on port 9000
TEST=x86vm stop && x86vm up --wait - picks a random port
TEST=x86vm shell - connects to the randomly picked port

Change-Id: If5b1c24ac8c39259254f3fa388224f9c3dbbe1a9
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4397208
Reviewed-by: Zihan Chen <zihanchen@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
2023-04-11 20:30:26 +00:00
Dennis Kempin
1b0c937ce4 testvm: Improved CLI usability
Makes the testvm CLI more intuitive, so we can start using the
x86vm as a default for running crosvm integration tests.

Modernizes the CLI output with colors and spinners for waiting.
Adds status and logs commands to aid debugging.
Improved `up` command to kill VMs if they do not become responsive.

Also merges the wait command into `up --wait`.

BUG=b:275717759
TEST=aarch64vm up && aarch64vm up --wait
TEST=aarch64vm stop && aarch64vm up --wait
TEST=testvm
TEST=aarch64vm logs

Change-Id: I82cf14cff19a0b01cda64718a24d170d564fdcd7
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4390731
Reviewed-by: Zihan Chen <zihanchen@google.com>
2023-04-11 20:30:26 +00:00
Dennis Kempin
2f4504e176 testvm: Refactor testvm to use cmd()
Simplifies the code and brings it in line with other
crosvm tooling.

BUG=b:275717759
TEST=x86vm run
TEST=x86vm up && x86vm wait
TEST=x86vm ssh

Change-Id: I88e21fca0e78eb76e611fdba7664e6b16f246d1e
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4390730
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Zihan Chen <zihanchen@google.com>
2023-04-11 20:30:26 +00:00
Dennis Kempin
6edc4e86d2 testvm: Refactor to use argh
The code has been written before we used argh. The CLI is specified
in tools/testvm. The implementation remains in impl/testvm.py so it
can be shared with run_tests.py for automatically managing the VM
when running tests.

BUG=b:275717759
TEST=x86vm run/up/wait/stop
TEST=run_tests --dut=vm

Change-Id: I2bdb554e7ed17a9f397fd8fb02f754e4ec1e52d7
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4390729
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Zihan Chen <zihanchen@google.com>
2023-04-11 20:30:26 +00:00
Zihan Chen
c01aa9507a tools: Promote cros container out of experimental project
It has been stable and in-use for quite a while, so I decided to no
longer mark it as experimental.
A weekly Cloud Scheduler "cron" job is also setup (and tested) to
update the container every Saturday.

TEST=force run cloud scheduler job, will trigger workflow and create
a new and correctly-defined cloud build run. crosvm-infra/crosvm_cros_cloudbuild
shows up in container registry console after cloud run was finished.

FIXED=b:240692674

Change-Id: I8daf96c05354b24ce43f3b90cd8d59e50eedb129
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4400014
Commit-Queue: Zihan Chen <zihanchen@google.com>
Reviewed-by: Dennis Kempin <denniskempin@google.com>
2023-04-05 18:28:31 +00:00
Junichi Uekawa
d03c99fc0c crosvm: Uprev uuid > 1
Uprev to version available in Debian.

This seem to end up doing uprev to 1.3 now.

BUG=b:265082456
BUG=b:229895468
TEST=build

Change-Id: I550778acb675c9034b9cfcea77f4ae847e2d2ea1
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4364559
Commit-Queue: Junichi Uekawa <uekawa@chromium.org>
Reviewed-by: Keiichi Watanabe <keiichiw@chromium.org>
Auto-Submit: Junichi Uekawa <uekawa@chromium.org>
Reviewed-by: Frederick Mayle <fmayle@google.com>
2023-04-04 23:56:13 +00:00
Dennis Kempin
bfc640f92d testvm: Rebuild VM images
The previous images were corrupted. This change uprevs the image
version and re-builds the images.

BUG=b:274167326
TEST=x86vm run
TEST=run_tests --dut vm
TEST=dev_container run_tests --dut vm
TEST=dev_container run_tests --dut vm -p aarch64

Change-Id: I926acf080ecf29c5d6979bcb714cb06e79159a14
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4377985
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Zihan Chen <zihanchen@google.com>
2023-03-28 21:50:53 +00:00
Zihan Chen
894cf79ba0 dev_container: Add strace to dependency
strace is used to capture seccomp profile during benchmark runs.
This CL also added /usr/lib/wine to PATH as debian has decided
to remove /usr/bin/wine64 alternative entry.

TEST=./tools/bench --log-seccomp --log-seccomp-output-dir /workspace/seccomp_result boot
TEST=./tools/run_tests --dut=host

BUG=b:258316090

Change-Id: Ie362149ef77474ad7d9c9ff918a990d386d7a5ff
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4358600
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Auto-Submit: Zihan Chen <zihanchen@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
2023-03-28 18:25:12 +00:00
Daniel Verkamp
083dcf75f4 tree-wide: apply nightly rustfmt
BUG=None
TEST=tools/fmt --nightly

Change-Id: Ifb08dd55ccf2a74ef739d7517a64970d24a82405
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4375640
Auto-Submit: Daniel Verkamp <dverkamp@chromium.org>
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
2023-03-27 21:30:45 +00:00
Dennis Kempin
20f0604494 dev_container: Add latest tag when pushing images
This will allow devcontainer.json files to always use the latest
container without manually upreving the version.

BUG=b:273529589
TEST=open dev container in vscode

Change-Id: I71d58eb2936097bd81dfcc0872d362b2c6b9e388
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4341238
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Zihan Chen <zihanchen@google.com>
2023-03-15 21:02:13 +00:00
Dennis Kempin
050bc63556 dev_container: Install g++
Required by https://crrev.com/c/4333206

BUG=None
TEST=make -D tools/impl/dev_container

Change-Id: I82c07141bc4b103f66a984edd91abc1631d39c9d
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4337397
Auto-Submit: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
2023-03-14 18:20:36 +00:00
Dennis Kempin
327fd410d1 prebuilds/testvm: Retry downloads
We occasionally see CI jobs fail due to network flakiness when
downloading prebuilts or the test vm image.
Retry both. This covers:
- Test VM images (i.e. the aarch64 vm image for testing)
- e2e_test guest images
- prebuilt libraries for win64 builds

BUG=b:248568518
TEST=dev_container tools/presubmit
TEST=dev_container --clean tools/presubmit

Change-Id: I8e5e185b17738182738bfcc4420890b925871fde
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4322030
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Zihan Chen <zihanchen@google.com>
2023-03-09 23:04:20 +00:00
Dennis Kempin
3e71484dca presubmit: Preserve order of checks
Replaces the set's with list's and uses dict.fromkeys to remove
duplicates while preseving the order of checks.

Also simplifies Group configuration by allowing groups to include
each other.

This allows us to configure the order of checks in CI more easily.

BUG=b:270390103
TEST=tools/presubmit --list-checks all

Change-Id: I06a24c27aa249fda11af9a5a712309d7c7e38341
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4303076
Reviewed-by: Zihan Chen <zihanchen@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
2023-03-07 18:01:47 +00:00
Dennis Kempin
f48639f3fe Automatically register tools/ on PATH
Developers with `direnv` installed will have ./tools added to their
PATH automatically. Also adds it to the PATH in the dev container.

This reduces `tools/dev_container tools/presubmit` to
`dev_container presubmit`.

Note: Does not rebuild the container, the change can be picked
up at a later time.

BUG=None
TEST=cd crosvm

Change-Id: Ia57bd7bb1cf86df2b32d624d97a74518e8b5f423
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4305345
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
2023-03-03 00:33:15 +00:00
Dennis Kempin
60a517f6bf infra: Update linux builders to use new presubmit script
This will keep CI and local test in sync. Failed steps can
also easily be reproduced locally by copy/pasting the
`tools/presubmit {check}` title of each step.

We should be able to merge the health check and linux builders
eventually, but will need to move the binary size calculation
into a separate builder.

BUG=b:271171326
TEST=led jobs below:
health_check: https://ci.chromium.org/swarming/task/60b7c71b340f4b10
x86_64: https://ci.chromium.org/swarming/task/60b7b0b45f572210
x86_64_direct: https://ci.chromium.org/swarming/task/60b7c44415ae7d10
aarch64: https://ci.chromium.org/swarming/task/60b7c50cc7682610
armhf: https://ci.chromium.org/swarming/task/60b7c458dce14310
mingw64: https://ci.chromium.org/swarming/task/60b7c47b3376fc10

Change-Id: Id3f78c5a021db96e42a90cdc77ab89d43b0f49c1
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4299933
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Zihan Chen <zihanchen@google.com>
2023-03-02 19:12:51 +00:00
Dennis Kempin
489119f34f tools: Add warnings about unsupported target platforms
Uses rustup to check if cross-compiliation for the targets are
installed.
Most commonly, this will warn when running tools/presubmit on glinux
and recommend to use tools/dev_container to run all checks.

BUG=b:271171619
TEST=tools/presubmit on glinux and with dev_container

Change-Id: I1f6739253a9c548b853be836ccc73dfd01602607
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4298630
Reviewed-by: Zihan Chen <zihanchen@google.com>
Auto-Submit: Dennis Kempin <denniskempin@google.com>
Commit-Queue: Zihan Chen <zihanchen@google.com>
2023-03-01 19:09:24 +00:00
Dennis Kempin
35144ac49a tools/presubmit: New presubmit checks
This new script merges tools/health-checks with tools/presubmit. It
can now execute checks in parallel and will run crosvm tests and
clippy for all platforms as well.

This reduces presubmit check time for "everything" on a clean state
from 6:30m to 2m.

Time to run checks after a no-op change to the base crate is 1m,
down from 3m.

The default set of checks is optimized to cover all platforms but
prevent blocking on the build cache. This allows it to complete
in ~30s on incremental builds.

Luci build recipes will be updated to use this script, to ensure
that CI and local checks are always in sync.

The old health-check script is forwarded to the new presubmit
script while CI is updated to use the new script.

BUG=b:254317173
TEST=tools/presubmit

Change-Id: Iafde602bab61dbd2ee2f5f74918dc182b2832d47
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4277625
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Zihan Chen <zihanchen@google.com>
Reviewed-by: Vikram Auradkar <auradkar@google.com>
2023-02-28 22:31:48 +00:00
Dennis Kempin
a53130345e tools: Clean up old test runner code
test_runner.py and test_target.py are mostly no longer needed. Some
common utility code has been moved to common.py

BUG=b:268530960
TEST=presubmit

Change-Id: I5f616cd4fdd496eae49adfcae80ad5472efc9562
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4289247
Reviewed-by: Zihan Chen <zihanchen@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
2023-02-28 19:33:44 +00:00
Dennis Kempin
65d0436acc testvm: Use CROSVM_CACHE_DIR variable for storing VM images
They are currently stored in a temporary directory, which is fine
locally, but will not be preserved between runs in CI. So each
CI build has to re-download the files.

BUG=b:254924880
TEST=tools/dev_container tools/x86_vm run

Change-Id: I49916887d5cb2d4c23c315a3d8ee369de1cba88f
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4287767
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Zihan Chen <zihanchen@google.com>
2023-02-28 19:32:24 +00:00
Dennis Kempin
c1456fc1de tools: Do not use pip to install python packages
Debian no longer allows us to install packages into the system
managed location.

So we are replacing the pip packages with the debian packaged
versions.

Also removes the wine64 hack which is no longer needed.

This allows us to build a new dev container again.

BUG=b:270568440
TEST=presubmit

Change-Id: I592ebf13bc566785fcdf4a4fa4badb3e11c8126e
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4285744
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Zihan Chen <zihanchen@google.com>
2023-02-27 19:24:18 +00:00
Zihan Chen
2524a185b6 e2e_tests: Restructure for initial benchmark support
- Restructured to share fixture between tests and benches
- Add bench script to run benchmark target

BUG=b:257303497
BUG=b:258316090
TEST=cargo clean && ./tools/build_release && ./tools/bench boot

Change-Id: I0ea3295839889e570032e03eec033d99b7b8e553
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4064114
Commit-Queue: Zihan Chen <zihanchen@google.com>
Auto-Submit: Zihan Chen <zihanchen@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Dennis Kempin <denniskempin@google.com>
2023-02-23 22:11:48 +00:00
Daniel Verkamp
2712fc59b1 bindgen: update kernel bindings to Linux 6.1
Switch to the ChromeOS v6.1 branch and regenerate all affected bindings
in the crosvm tree.

Some minor fixes were required in users of the bindings:
- KVM_SYSTEM_EVENT_RESET_FLAG_PSCI_RESET2 is available in the kernel now
  and was removed from the manually-added section of kvm_sys.
- The KVM system_event now contains an anonymous union, requiring a few
  renames.
- IORING_OP_* were moved into an enum, which required the enum name to
  be prefixed to the uses of those values.

BUG=None
TEST=tools/presubmit --all

Change-Id: I84568fb76658832130e2e9a631495ba82fd4c3ed
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4285742
Reviewed-by: Elie Kheirallah <khei@google.com>
Reviewed-by: Zihan Chen <zihanchen@google.com>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
2023-02-23 21:31:44 +00:00
Zihan Chen
edc1265c07 infra: Upload binary size to dashboard in postsubmit
Finally all pieces are together and we can start uploading the
metrics.
There's also a typo in catapult converter that's fixed.
Tested via led presubmit with few modifidcations to allow upload
happen in a presubmit runner.

TEST=https://chromeperf.appspot.com/report?sid=a7f57b2395b43aeaa65009589e17bc31548b78cb58daae21f12f70ee24ed9578

BUG=b:257303497

Change-Id: I351586f52ef18be95b14cc3310bb05db4f49a101
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4249502
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Commit-Queue: Zihan Chen <zihanchen@google.com>
2023-02-15 22:41:49 +00:00
Zihan Chen
28ce4e5423 crosvm: Uprev rust toolchain and dev container
- Rust toolchain is updated to 1.65.0
- Catapult dashboard upload tool is added to dev_container
- Bindgen is updated to latest version to support custom derive
- Derive Eq when PartialEq is derived as required by new Clippy

TEST=CQ, bindgen-all-the-things

FIXED=b:260784028
BUG=b:257303497

Change-Id: I2034cd09e0aed84d4e9b30f2e85d84d94a442ea4
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4228427
Auto-Submit: Zihan Chen <zihanchen@google.com>
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Commit-Queue: Zihan Chen <zihanchen@google.com>
2023-02-08 20:26:30 +00:00
Dennis Kempin
356a6ddf39 tools/run_tests2: Various process improvements
- Properly propagate test failures as exit codes
- Add --profile option to select nextest profile
- Add --no-parallel option for more stable test results
- Add --repetitions option to repeat tests to check
  test flakiness

BUG=b:261600801
TEST=tools/run_tests2 [--no-parallel] [--repetitions]

Change-Id: I7eb8617cfef8bc9979ff2115b2936c31a263e6f3
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4228216
Auto-Submit: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Zihan Chen <zihanchen@google.com>
Commit-Queue: Zihan Chen <zihanchen@google.com>
2023-02-07 22:22:37 +00:00
Dennis Kempin
5eea928ef9 New nextest based test runner
We greatly improved the test health of crosvm which allows to
make better use of standard cargo tools instead of custom
tailored implementations for running tests.

The new test runner is available at tools/run_tests2 until it
fully replaces tools/run_tests.

The key difference is that tests are packaged into a tarball
to be executed remotely instead of sending and executing every
binary separately via SSH.

Improvements over the old test runner:

- Faster, nextest runs faster and the new model for remote testing
  is a lot faster when running tests in a VM.
- Simpler, less custom code to maintain.
- More readable output, uses modern TUI functionality.
- Enables future workflows for testing on cloud devices running
  in our luci infrastructure.

How much faster?
- Running all tests on host: 1m -> 9s
- Running all aarch64 tests in vm: 3m20s -> 32s

BUG=b:261600801
TEST=./tools/run_tests2 with args:
-p x86_64
-p x86_64 --dut=host
-p x86_64 --dut=vm
-p aarch64
-p aarch64 --dut=vm
-p armhf
-p mingw64

Change-Id: I842a0ac6f7b288aeccb112b4e2f46a15f3baf54b
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4144641
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Reviewed-by: Zihan Chen <zihanchen@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
2023-02-02 23:39:42 +00:00
Dennis Kempin
3e1ed7926f tools: Add 'rich' as dependency
Updates the code to be able to install multiple packages and show
a useful error message if pip is not available.

Rich will be used by future tools for prettier terminal output
of our tools, most importantly to be able to print status updates
of parallel presubmit checks.

BUG=b:239255137
TEST=uninstall python3-pip, removed all packages. Ran ./tools/cl

Change-Id: I50bfdf2bee279e6d6d02c7dd23131cba1fed2a8c
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/3824003
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
2023-01-24 17:33:29 +00:00
Dennis Kempin
108a335d7d toolchain refactoring: Only support native out of the box
cross-compilation requires additional configuration that depends
heavily on the host platform. Unfortunately cargo cannot be
configured on a per host platform basis so this has to be done
manually.

Most developers are just using native compilation (and using
the containers for everything else). So we can slim our
install-deps scripts a bit.

To verify the updated install scripts, this change includes
containers to simulate a fresh developer workstation that
can be used to verify out of the box workflows.

BUG=b:262829206
BUG=b:265995780
BUG=b:265842137
TEST=tools/contrib/minimal_container/test-all.sh and CQ

Change-Id: I0e803d0224306e5351728e2c80fcbcd8260d029d
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4179290
Reviewed-by: Zihan Chen <zihanchen@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
2023-01-24 17:33:29 +00:00
Dennis Kempin
81697578bd tools/x86vm: Revive unused x86 testvm
The testvm for x86 has not been used recently and was outdated.
To make it usable we need to resize the root partition and
install dependencies needed for some of x86 only tests to pass
(e.g. gcc for plugin tests).

BUG=b:264702133
TEST=./tools/x86vm ssh

Change-Id: Iabc03e7ca91595093da849008e2a88f89dbac862
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4144639
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Zihan Chen <zihanchen@google.com>
2023-01-19 21:36:31 +00:00
Dennis Kempin
acc162000f Add conditional compilation for unix-only crates
Instead of configuring which crates to --exclude in
test_config.py, we can use conditional compilation to
exclude code that is not supported on windows.

This allows more fine-grained control and also allows
us to use plain cargo for building without complicated
configuration and exclusions.

BUG=b:265829867
TEST=cargo test --lib --bins --workspace
	--target=x86_64-pc-windows-gnu
	--features=all-mingw64

Change-Id: I8422c3f08053bc27d9896b220876a56bd25543d6
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4165868
Reviewed-by: Vikram Auradkar <auradkar@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
2023-01-19 21:21:59 +00:00
Dennis Kempin
a6e7d6f139 Enable compilation of fuzzing targets
Adds a fall-back to cros_fuzz when compiled without
fuzzing enabled that will just produce a main fn entrypoint
with the fuzzing code.
This allows the fuzzing code to be compiled, but won't produce
functional fuzzing binaries.

BUG=b:265829867
FIXES=b:244631591
TEST=crosvm CQ
cargo +nightly fuzz run --fuzz-dir crosvm-fuzz --features upstream-fuzz crosvm_block_fuzzer

Change-Id: Ib2602aab5c5373cb2a71dca0d8419640a00c6725
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4167143
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Zihan Chen <zihanchen@google.com>
2023-01-17 23:18:22 +00:00
Daniel Verkamp
c116518b50 aarch64: clean up unix-only dependencies
Drop a few unused kvm dependencies and only require minijail on unix
platforms.

The aarch64 platform won't be functional on Windows yet, but it's at
least possible to drop it from the WIN64_DISABLED_CRATES list.

BUG=None
TEST=tools/dev_container tools/run_tests --platform=mingw64
TEST=tools/dev_container tools/run_tests --platform=aarch64

Change-Id: If48b935afe1295820b6d78cfde3e8634a3e397db
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4167144
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
Reviewed-by: Dennis Kempin <denniskempin@google.com>
2023-01-13 23:25:19 +00:00
Zihan Chen
c62ba4bddf tools/impl/test_target: Fix CROSVM_TARGET_DIR typo
It evaluates to '<function crosvm_target_dir at 0x7f6bbecaf880>'
which is obviously wrong.

TEST=CQ only

Change-Id: Ifaac4690d069685175653e6481adb1397d9c46b3
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4149732
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Auto-Submit: Zihan Chen <zihanchen@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
2023-01-12 19:08:40 +00:00
Vikram Auradkar
15b7d8c8f5 tools: add check for lf line endings
We are planning to add developer tools/scripts for windows platform.
Some of these tools will be windows batch scripts, which don't play
well with lf line endings. The patch forces .bat files to have crlf
line ending.

BUG=b:264555939
TEST=none

Change-Id: I66773f924316c8596361e02aefa2fb368b43acac
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4143459
Commit-Queue: Vikram Auradkar <auradkar@google.com>
Reviewed-by: Dennis Kempin <denniskempin@google.com>
2023-01-06 20:27:39 +00:00
Vikram Auradkar
c629b81de9 tools: make dev_container runnable on downstream repo
We are trying to reuse dev container to run tests on linux downstream.
The patch makes the script to understand downstream repo layout.

BUG=b:263398717
TEST=ran on downstream bots and local machine

Change-Id: I194edc8099d51e707ca034f64e8813c4cab14f23
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4134589
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Commit-Queue: Vikram Auradkar <auradkar@google.com>
2023-01-04 00:01:24 +00:00
Zihan Chen
b860ec82a7 cros_container: Remove incremental setup
We don't need the incremental setup as it's not helpful.

BUG=b:240692674

Change-Id: I7d4720bc56480c4b878415e1172b7df2617e8312
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4034379
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Commit-Queue: Zihan Chen <zihanchen@google.com>
2023-01-03 22:51:36 +00:00
Vikram Auradkar
13f8029329 e2e_tests: Add windows tests
The windows code is build only for now because
- crosvm upstream windows code needs some changes to be runnable
- LUCI windows infra is not ready for nested vm

BUG=b:253498690
TEST=ran `cargo t --features all-msvc64 -p e2e_tests` downstream

Change-Id: I318f6f17fc91bc76cfd28741a526193362ab4e50
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4096606
Commit-Queue: Vikram Auradkar <auradkar@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
2022-12-14 14:21:51 +00:00
Zihan Chen
03b1653276 tools: Add catapult converter
This tools allow us to convert fuchsiaperf formated benchmark
results to catapult dashboard's HistogramSet format.

BUG=b:257303497
TEST=Generate LGTM result from sample fuchsiaperf file in their doc

Change-Id: Iae7ce36a8444bdb436a6126939f6d50997dedf5e
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4084425
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Zihan Chen <zihanchen@google.com>
2022-12-13 21:48:49 +00:00
Dennis Kempin
5f58efc44d tools/run_tests: Preserve environment in sudo run tests
This allows tests to see environment variables specified by the
developer.

BUG=None
TEST=presubmit

Change-Id: I3a12e0652d5bdc68cad937cda66dc5a7bfc61526
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4082147
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
2022-12-08 19:05:29 +00:00
Dennis Kempin
ae17950731 libvda: Ignore tests when using libvda-stub
The tests will not work at runtime when using the stub
implementation.
This replaces the DO_NOT_RUN test option.

BUG=None
TEST=run_tests

Change-Id: Iabab40c70136945b377040712692dae258b4461c
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4072745
Auto-Submit: Dennis Kempin <denniskempin@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Alexandre Courbot <acourbot@chromium.org>
2022-12-02 17:46:18 +00:00
Dennis Kempin
b157bda3ee tools/run_tests: Run some tests as root
Testing tap functionality requires root privileges. The crosvmdev
user of our dev_container is set up for passwordless sudo, so we can
seamlessly execute these tests via sudo.

For running on the developer workstation directly, this will prompt
for a password, which is disruptive to workflows. The --no-root
option can be used to prevent this and skip the tests in question.

BUG=None
TEST=tools/run_tests [--no-root]

Change-Id: I731887837affceb76152466f0006c4ee51a19234
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4063237
Reviewed-by: Zihan Chen <zihanchen@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
2022-11-29 19:40:36 +00:00
Zihan Chen
8458188515 dev_container: Add mdformat-tables to dependency
Now we are using markdown tables in our docs, and mdformat without
extension will do horrible things to them as tables is a GitHub
Flavored Markdown extension.

TEST=./tools/dev_container ./tools/fmt

Change-Id: I735284136a87ea17752bd6732885ff1f4f58a3f4
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4026017
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Auto-Submit: Zihan Chen <zihanchen@google.com>
2022-11-15 22:36:38 +00:00
Dennis Kempin
2ca028d8e2 run_tests: Allow real-time output of integration tests in verbose mode
There are test flakes in our post-submit builders that will
just crash the run_tests process.
To better figure out what's going on, switch verbose mode output
to print output directly instead of caching it to possible hide
the results on success.

This should help us find out where exactly the test is failing.

BUG=b:254923676
TEST=run_tests && run_tests -v

Change-Id: I3a9884db5da9c0973e8b05dea5541a8050c00548
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4018117
Reviewed-by: Zihan Chen <zihanchen@google.com>
Commit-Queue: Zihan Chen <zihanchen@google.com>
2022-11-15 01:56:02 +00:00
Shintaro Kawamura
67390a05fb Reland "swap: add userfaultfd wrapper"
origin:
https://chromium-review.googlesource.com/c/crosvm/crosvm/+/3895235

userfaultfd enables applications to handle page faults on designated
memory area.

vmm-swap feature uses userfaultfd to catch page fault event and swap
in the guest memory from the swap file.

design document: go/tanooki-phase1-dd

BUG=b:215093219
TEST=cargo build --features=swap

Change-Id: I36ecfe9be988a4bc451f8edaf2ab48e25c6600f4
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4016142
Commit-Queue: Shin Kawamura <kawasin@google.com>
Reviewed-by: David Stevens <stevensd@chromium.org>
Reviewed-by: Dennis Kempin <denniskempin@google.com>
2022-11-14 06:14:07 +00:00
Frederick Mayle
e0ab99da0c dev_container: fix arg forwarding in entrypoint.sh
Without this change, the command

    ./tools/dev_container cargo watch -s "cargo check --workspace"

would end up as

    cargo watch -s cargo check --workspace

inside the docker shell.

Change-Id: I79fed9d23804699032adf359a24d7250c1124b21
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4021752
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Commit-Queue: Frederick Mayle <fmayle@google.com>
2022-11-10 23:34:53 +00:00
Clarissa Garvey
54311a8da9 crosvm bindgen: Allow missing_safety_doc
It is not feasible to add safety documentation to generated files, so
this CL adds a suppression to generated files for missing_safety_docs.

BUG=b:157245930, chromium:908640
TEST=Rerun bindgen, cargo clippy reports less errors, cargo build OK

Change-Id: I793f793a0d4996809659f5ba2a6f738e6b3443c9
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4018404
Commit-Queue: Clarissa Garvey <clarissagarvey@chromium.org>
Reviewed-by: Dennis Kempin <denniskempin@google.com>
2022-11-10 19:05:38 +00:00
Dennis Kempin
82315e0a20 base: Move newly added process tests into an integration test
The test needs to be run single threaded and is not suitable
to be a unit tests.
Integration tests are automatically run single threaded.

BUG=None
TEST=presubmit

Change-Id: Idd49af90c7c1858b28ba47a642a8a808319138c3
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4018113
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
2022-11-09 20:45:59 +00:00