This builder will only run clippy using the new
presubmit recipe.
Also updates the health_check builder to use the
same recipe.
BUG=b:349907813
TEST=lucicfg validate main.star
Change-Id: Icf8a721b6eb37675dd289d496888b62de7ea9660
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/5689951
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Zihan Chen <zihanchen@google.com>
This allows cargo to build for android. A new
dev container is built to include this.
To allow us to build the new container, we had to
remove the ffmpeg dependencies for armhf which fail
to install in the current version of debian.
BUG=b:349907813
TEST=dev_container presubmit clippy_android
Change-Id: I7fbcd460e53fbd339a28b451a91b0722d34257b8
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/5689659
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
This change enables clippy builds of android specific code.
Doing so without adding the full Android SDK into our container
is a bit hacky. The CL https://crrev.com/c/5671653 adds env variables
to the minijail build.rs to allow us to skip building the library, and
generate bindings for linux instead of android.
This allow us to build all non-gpu related features of the
android build. It will not link, but it will run clippy.
This CL fixes various clippy issues that come up in this new
configuration
BUG=b:349907813
TEST=tools/clippy -p android
TEST=tools/presubmit clippy_android
Change-Id: I1f51d383051bbeeeff55d716feb7b56c3e24588b
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/5672567
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
This also upgrades the dev_container to package the new Rust version.
A few final clippy fixes are included as well.
BUG=b:344974550
TEST=tools/clippy
Change-Id: Iab4253e9cd88fdae989bb5b5170040b6af85baca
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/5609084
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
This uprevs both testvm and devcontainer and uploads new builds.
This will update them to the latest debian images and packages.
Some minor tweaks were made to satisfy the newer versions of
mdformat and python.
BUG=None
TEST=presubmit all --no-delta
Change-Id: I73e30aa1d9cefb5cf4ac117f513c2a98bf3c9ac5
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/5416093
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Ensure that every Cargo.toml dependency on a third-party crates.io crate
specifies at least a major version, or a minor version for 0.x crates,
to ensure that if a new major version is published, it cannot cause API
breaks.
The versions are selected to match the ones already in Cargo.lock, so
this should have no functional change, but it will help prevent new "*"
versions from being introduced via copy-and-paste.
For rationale, see the Cargo FAQ:
<https://doc.rust-lang.org/cargo/faq.html#can-libraries-use--as-a-version-for-their-dependencies>
`minijail`, `audio_streams`, and `cras` are left as "*" for now, since
they have unusual situations (imported from a submodule and/or replaced
at build time with ebuild magic).
BUG=None
TEST=tools/dev_container tools/presubmit
TEST=verify Cargo.lock is unchanged
Change-Id: Ifa18199f812f01d2d10bfb4146b3353c1a76527c
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/5555656
Reviewed-by: Frederick Mayle <fmayle@google.com>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
Recipe roller has lead to many unwanted merge attempts. Don't start
a merge unless there are some other things to merge.
TEST=copy and paste in REPL for a manual mock test
BUG=b:333576491
Change-Id: Ia11c6706e98d60e6b4a908716a90cd4e04b55527
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/5438909
Auto-Submit: Zihan Chen <zihanchen@google.com>
Commit-Queue: Zihan Chen <zihanchen@google.com>
Reviewed-by: Dennis Kempin <denniskempin@google.com>
cros added another package named crosvm in a different path,
so we need to explicitly specify the full path now.
BUG=None
TEST=None
Change-Id: I2920fa95ad570932757862b1686dc5a63e2eac4f
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/5430298
Reviewed-by: Zihan Chen <zihanchen@google.com>
The mingw64 install script does not install winetricks, so it does not
need these variables. They are probably left over from a previous
refactoring. tools/setup-wine64 has the same variables, but they are
actually used in that case.
TEST=make -C tools/impl/dev_container crosvm_dev
Change-Id: I3993f4e53677752c973fc683d13f9dda68f6aae0
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/5353268
Reviewed-by: Zihan Chen <zihanchen@google.com>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
There don't appear to be any new clippy warnings, and this is the
version of Rust that Android is now using: b/303252546. Work for the
next version is tracked in b/310977762.
This doesn't have any particular features I'm looking for (async traits
only come in 1.75.0). I'm not particularly attached to this change. If
it's easy to upgrade the container though, we could ensure no new clippy
warnings appear.
Android rust versions appear here:
https://cs.android.com/android/platform/superproject/main/+/main:prebuilts/rust/linux-x86/
Also not sure what the ChromeOS release schedule is.
Release docs: https://releases.rs/docs/1.73.0/
Test: tools/dev_container --clean bash -c "rustc --version && tools/presubmit clippy"
Change-Id: I2d3010c3032053709f0c0beb7dcbe2b80d782415
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/5201659
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
libc6:arm64 will call telinit to restart affected services.
This will block indefinitely in the container without init running.
The workaround is to replace telinit with true to make it a
no-op.
Also removes the mount caches from the build process as they seem
don't seem to work with APTs file lock.
BUG=b:322015733
TEST=make -C tools/impl/dev_container
Change-Id: If1079581680ac9edc981ad46afb3173508668335
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/5230818
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Hook rutabaga_gfx/vulkano into a new top-level feature of the same name,
so that it can be detected for conditional compilation of gpu device
creation details.
Also moves the `vulkano` dependency out of windows-specific list and
into generic list for all platforms.
(Note: the existing rutabaga_gfx/Cargo.toml hacks must still be locally
applied and rust-toolchain channel must be manually set to 1.70 or
greater to successfully build with --feature=vulkano)
BUG=b:246334944
BUG=b:244591751
Test=tools/dev_container tools/presubmit
Change-Id: Ic1938f3d8599de1414539abf5a4b57a9fe1764dd
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/5264226
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Ryan Neph <ryanneph@google.com>
This commit addds a tool to find the bottleneck virtio device by
analyzing the output of trace-cmd. This tool analyzes what processes
crosvm's vcpu process waited for, for how long time. For example, if the
vcpu processes are mostly waiting for vitio-blk process, the bottleneck
is likely to be the blk. If the vcpu processes are mostly waiting for
other vcpu processes or often preempted by other processes, the
bottleneck is likely to be the processor.
See README.md for the details.
BUG=b:302240202
TEST=cargo test
Change-Id: I98cff9e1158cf7980d522e549edef002630763b4
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4891880
Reviewed-by: Keiichi Watanabe <keiichiw@chromium.org>
Commit-Queue: Takaya Saeki <takayas@chromium.org>
If running behind proxy, need to pass proxy setting for container as well,
otherwise can't pull crates properly. So this tries to add 'http_proxy'
and 'https_proxy' in environment for container.
Change-Id: I897703f571d9fe5bd03b819fc6b13a9e04384e6f
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/5232200
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
Add -f to the xz command line so the compressor will always overwrite
workspace/profiles/benchmarks.profdata.xz even if it exists instead of
failing:
$ tools/dev_container xz -9e -T 0 /workspace/profiles/benchmarks.profdata
[...]
xz: /workspace/profiles/benchmarks.profdata.xz: File exists
Change-Id: I74aae3ee052adacd2a4f8da69e6264273cfebcc0
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/5226612
Reviewed-by: Zihan Chen <zihanchen@google.com>
Commit-Queue: Zihan Chen <zihanchen@google.com>
When a .dat file contains only "Enter: " trace record but no
corresponding "Exit: " trace record, the calculate_function_time() will
ends up at divede by zero error. This CL fixes the error.
BUG=b:320395473
TEST=run with the edge case .dat file
Change-Id: I79709258d914156f5fcff1a6fd9c2d0be16c8c41
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/5198660
Reviewed-by: Keiichi Watanabe <keiichiw@chromium.org>
Reviewed-by: Morg <morg@google.com>
Commit-Queue: Yuan Yao <yuanyaogoog@chromium.org>
This generates riscv64/bindings.rs from the kernel headers to bring it
in line with the other auto-generated bindings, which is possible now
that we are generating the bindings from new enough kernel headers
(Linux 6.6).
BUG=b:287496832
TEST=tools/dev_container tools/presubmit crosvm_unit_tests_riscv64
Change-Id: I86c17d75a467368dcc99e616a2dfd31ee8cb1071
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4676074
Reviewed-by: Dylan Reid <dgreid@rivosinc.com>
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
The obsolete KVM_SET_MEMORY_ALIAS ioctl was removed from the kernel; it
was already unused in crosvm, so remove the corresponding ioctl macro.
Change-Id: I1ecfce4b8df886680154c567f82ab7f006fbfd1f
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/5178535
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Consolidates the instructions into a README.md file to make it
easier to discover.
BUG=b:273481336
TEST=None
Change-Id: I3b7a4a204ba5802bcfdb24db5a808fadce9c757e
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/5123312
Reviewed-by: Zihan Chen <zihanchen@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
This allows postsubmit to run with the postsubmit nextest profile
to fail flaky tests.
BUG=b:300669562
TEST=export NEXTEST_PROFILE=postsubmit; dev_container run_tests
Change-Id: I4ac9c351e63243cb1c66fabfb755949530768917
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/5121290
Reviewed-by: Zihan Chen <zihanchen@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Generate bindings for the installed version of the ffmpeg libraries at
build time instead of checking in a static version of the bindings.
BUG=b:309563850
TEST=cargo build --features=ffmpeg
Change-Id: I38d922dd7332c4385ae6a0ae93e893538c3b1cce
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/5027711
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
Nigthly is enabled as part of https://crrev.com/c/4950268
This change contains the formatting changes resulting from the switch.
BUG=b:302055317
TEST=dev_container presubmit format --no-delta
Change-Id: Idaf2b8bae2e09c624b19d9cd3dd5fc8e4d099b3c
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/5067088
Reviewed-by: Zihan Chen <zihanchen@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
This enables nightly rustfmt features in CI so we can keep a consistent
imports style.
The resulting formatting changes are in the follow-up CL and will
need to submit together.
This change also uncovered a bug from a previous commit that caused only
the first command in a series to be waited for. Formatting issues would
not be reported by the tool. The bug is fixed as part of this CL.
BUG=b:302055317
TEST=dev_container presubmit format --no-delta
Change-Id: I5d50609cc89d7e78bce66db120737a302f5c57e8
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4950268
Reviewed-by: Zihan Chen <zihanchen@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Dennis Kempin <denniskempin@google.com>
For some reason, when running in production CI the merge bot
script can't find directories created inside the container. This
is the best explanation I can think of for the error I see in CI
log. This CL moves `mkdir` for all directories host need to access
to run from host.
BUG=b:181105093
TEST=untested, as led can't reproduce this issue
Change-Id: I9b10838c06c32ff607b98e21616f09c4b96392f3
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/5064383
Auto-Submit: Zihan Chen <zihanchen@google.com>
Reviewed-by: Alexandre Courbot <acourbot@chromium.org>
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Merge bot will now automatically run crosvm benchmark suite and
produce merged profile data file for PGO use at each merge commit
generated on Monday.
As profile data functions with snapshot of code and generating
them requires a non-trivial amount of compute, we decided on
generate profiles at each "natural" snapshot point (cros merges)
instead of each commit, and thus `merged.profdata` will not be
available in upstream repository.
We decided against generating profile at every merge to avoid
bloating cros repository. The profile is also compressed
with xz to reduce its footprint.
TEST=./tools/chromeos/merge_bot --verbose update-dry-runs origin/main
TEST=https://luci-milo.appspot.com/swarming/task/662c0e8432332c11
Change-Id: I0787f3e01b9a83e9e53f36c36cda1a043ceee0d8
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/5038001
Commit-Queue: Zihan Chen <zihanchen@google.com>
Reviewed-by: Dennis Kempin <denniskempin@google.com>
The current output of the "list" subcommand is like:
```
\#1: create: 1234 usec
\#2: mkdir: 567 usec
```
This output format is not so easy for futher processing.
Also, it doesn't contain how many times a result is sent.
So, this CL update the format to the following one:
```
name,total (us), count
create, 1234, 400
mkdir, 567, 89
```
BUG=none
TEST=cros_trace_analyser list --input trace.dat
Change-Id: Ia0e0d2d9777e8ba7431a8b90e8ec54e65d242166
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/5061774
Auto-Submit: Keiichi Watanabe <keiichiw@chromium.org>
Commit-Queue: Morg <morg@google.com>
Reviewed-by: Morg <morg@google.com>
The tests stopped working due to issues with git.
BUG=b:309148074
Change-Id: I08cd522660927bb4d46d48ea9c6f3cbf8ec51486
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/5005630
Auto-Submit: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Also adds lld, which will be used in a follow-up to improve
linking performance.
Since debian has had a stable release since then, we also
need to upgrade a few library versions in the testvm.
Swap tests that require userfaultfd were able to run without
root before. In the latest version of debian we need to configure
for unprivileged access.
Since this is the case for glinux workstations as well, let's
mark those tests as root-only and let them execute via sudo.
BUG=b:304875018,b:256905223
TEST=presubmit
Change-Id: Ied0b34e248990378a7864b4a394aa21d8f824f71
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/5013631
Reviewed-by: Zihan Chen <zihanchen@google.com>
Reviewed-by: Shin Kawamura <kawasin@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
/dev/kvm is not a normal file, so os.path.isfile("/dev/kvm") returns
False. Use pathlib's is_char_device() check instead.
TEST=tools/dev_container --stop; tools/dev_container
Fixes: 0be7e9757e ("dev_container: Only pass /dev/kvm if present")
Change-Id: I150009beb9b0b438013c2dd86fae7f99c959be75
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/5021441
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Auto-Submit: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Pierre-Clément Tosi <ptosi@google.com>
Running cargo binstall on some packages will install them from source
if no binary release is available.
In that case, the rust-toolchain file of the CWD is used for building,
which may not be the right version required by the binary package
we are trying to install.
The new version of binstall is now also able to pull binaries from
github releases without hitting API quotas.
BUG=b:309563518
TEST=./install-deps
Change-Id: I869002cf766f806f66ba58d5b712b81c0bd30f4f
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/5014783
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Add test for the device tree overlay mechanism.
Bug: b/296796644
Test: cd cros_fdt && cargo test
Change-Id: I8e8d129344e1ad4ddf0d7af7b3f21c0c9d9ca038
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4855966
Commit-Queue: Jakob Vukalović <jakobvukalovic@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Some users of *.googlesource.com access the platform through the
internal Single Sign-On authentication mechanism so consider the sso://
remote as valid.
TEST=tools/cl upload
Change-Id: Ie7ebe4c9e165241645fddcdf1884b7e9e7a7d83d
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/5009550
Auto-Submit: Pierre-Clément Tosi <ptosi@google.com>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Avoid passing the file to Podman/Docker if it isn't present as that
prevents the container from running:
Error: stat /dev/kvm: no such file or directory
Even though the script may be used in cases where KVM isn't necessary.
Remove the check for POSIX as non-POSIX OSes are very unlikely to have a
file at that path.
TEST=tools/dev_container tools/run_tests --platform=aarch64
Change-Id: If46bb42076c79adfeab4baceb01913a9179ac00d
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/5010772
Auto-Submit: Pierre-Clément Tosi <ptosi@google.com>
Reviewed-by: Vikram Auradkar <auradkar@google.com>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
this allows sharing repo on windows running a linux container
BUG=b:308883445
TEST=on windows ran
py .\tools\dev_container tools/presubmit line_endings
Change-Id: Ief73f1ad2da201dac0337711d0c4f706a936f7a6
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/5008193
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Commit-Queue: Vikram Auradkar <auradkar@google.com>
With a fix in libtracemd-rs v0.2.1 [1], we no longer need to rebuild
libtracecmd-rs every time.
[1]: https://github.com/google/libtracecmd-rs/pull/6
BUG=none
TEST=cargo build
Change-Id: Ica1342c9b23ed778fff2c0677717e3953b42dbd9
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/5000825
Auto-Submit: Keiichi Watanabe <keiichiw@chromium.org>
Reviewed-by: Ryuichiro Chiba <chibar@chromium.org>
Reviewed-by: Morg <morg@google.com>
Commit-Queue: Keiichi Watanabe <keiichiw@chromium.org>
We should be using the builder-provided vpython to be able to
reliably testing recipes in the right environment.
Let's remove these to unblock the recipe roller, then add them
as a seprate step in the health check build recipe.
BUG=None
TEST=CQ
Change-Id: I14fa4d744ab83c01c54f9618854e5e5f657d9f60
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/5005627
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
the script may need a few other changes as my dev box already had
podman and potentially other dependencies. I uninstalled podman to test.
BUG=b:308883445
TEST=used script install on a dev box.
Change-Id: Ieb44587cc79b22952bc0c8154c9001f608e51769
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4998699
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Commit-Queue: Vikram Auradkar <auradkar@google.com>
Auto-Submit: Vikram Auradkar <auradkar@google.com>
