This new script merges tools/health-checks with tools/presubmit. It
can now execute checks in parallel and will run crosvm tests and
clippy for all platforms as well.
This reduces presubmit check time for "everything" on a clean state
from 6:30m to 2m.
Time to run checks after a no-op change to the base crate is 1m,
down from 3m.
The default set of checks is optimized to cover all platforms but
prevent blocking on the build cache. This allows it to complete
in ~30s on incremental builds.
Luci build recipes will be updated to use this script, to ensure
that CI and local checks are always in sync.
The old health-check script is forwarded to the new presubmit
script while CI is updated to use the new script.
BUG=b:254317173
TEST=tools/presubmit
Change-Id: Iafde602bab61dbd2ee2f5f74918dc182b2832d47
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4277625
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Zihan Chen <zihanchen@google.com>
Reviewed-by: Vikram Auradkar <auradkar@google.com>
This search/replace updates all copyright notices to drop the
"All rights reserved", Use "ChromiumOS" instead of "Chromium OS"
and drops the trailing dots.
This fulfills the request from legal and unifies our notices.
./tools/health-check has been updated to only accept this style.
BUG=b:246579983
TEST=./tools/health-check
Change-Id: I87a80701dc651f1baf4820e5cc42469d7c5f5bf7
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/3894243
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
The updated health-check will by default only run on modified
files. If you do not make changes to python code, python checks
won't run, etc.
The script also simplifies the writing of those checks so we can
start adding more of them.
Luci will be updated to make use of the --list-checks function to
run each check in a separate luci step. In the meantime, we
keep a compatibility layer to translate the old arguments
to the new style.
BUG=b:239255137
TEST=./tools/health-check in all it's variations
Change-Id: I21b986b46c7cfccf3d13f4c76bbd3d0ec7240c26
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/3827174
Tested-by: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
This allows you to use the nightly version of rustfmt
to reformat imports as well.
BUG=None
TEST=./tools/fmt --nightly
Change-Id: I07c2fd1ad59ddfed3d97b636f991ae50076fd5d3
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/3792430
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
Auto-Submit: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: Dennis Kempin <denniskempin@google.com>
This change adds python type and formatting checks and
consolidates code health checks in ./tools/health-check.
Dealing with relative imports in python is tricky, so
we are making ./tools/impl a proper package with no
directly executable files.
Some of the bash shorthands in ./tools had to be converted
to python for this.
To make the new checks pass, we run the formatter and fix
some mypy type checks.
TEST=./tools/health-check
BUG=b:218559722,b:219965702
Change-Id: Ie18d3d6dd2f5a033141e167a6e1aa762791941d6
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3558592
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
cargo fmt will only format those files that are referenced by
crosvm. To really catch all files in our source tree, we need to
run rustfmt directly.
This will check files multiple times, since rustfmt will follow
included modules. But is still faster that cargo fmt.
To run these in parallel, this change adds some tools to common.py
for parallel execution of commands.
This reduces the fmt check time from ~9s to 1.5s.
I verified that we are actually catching all files now by adding a
newline to all .rs files in our sources tree. The formatter reverted
them all.
BUG=b:218559722
TEST=./tools/fmt
Change-Id: If721d837ebc9eee3ce28fa7f439ab0bcc0b993cf
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3474926
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
We are replacing most of our bash tools with python versions so we
can support cross-platform development.
To facilitate writing shell-like scripts in Python, this CL adds
a common.py file to share common utilities. It curretly contains
tools to simplify shell-like command execution and argument parsing.
BUG=b:218559770
TEST=./tools/clippy && ./tools/fmt && ./tools/impl/common.py
Change-Id: I7f8b3523394973ed5c741b926fdc41e52133189f
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3461240
Reviewed-by: Maciek Swiech <drmasquatch@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
The tool is added to ./tools/fmt which will be called during
./tools/presubmit as well as by Kokoro during testing.
This requires a new dev container version to bundle the mdformat
tool in the container.
Note: mdformat does not have any IDE integrations, but the prettier
plugin with prose-wrap enabled and a line length of 100 provides
almost identical results.
BUG=None
TEST=./tools/fmt --check
Change-Id: I0ae5659eff8555df7c85c70e62095a1e116b98da
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3416098
Reviewed-by: Keiichi Watanabe <keiichiw@chromium.org>
Reviewed-by: Alexandre Courbot <acourbot@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
The updated presubmit script allows parallel execution of checks
with --tmux.
It will also try to detect if the host is set up for aarch64 builds
and use the dev container if needed.
BUG=None
TEST=./tools/presubmit --tmux
Change-Id: I0247c39d826ee38d5f7f689de5e63380fe789cf4
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3292101
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
We have not been running clippy on many crates before. This CL
does not contain any clippy fixes, but documents how many issues
each disabled clippy check and crate currently has.
Many of them should be easy to fix.
BUG=b:192425184
TEST=./tools/presubmit --quick
Change-Id: I9ea25bc9a24d405700027ca4e72fb2a50e376c9c
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3276668
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Includes some fixes for new clippy checks. A rebuilt
dev_container that ships the new toolchain.
This allows us to get rid of the annoying cargo clean
before running clippy.
BUG=b:203142205
TEST=./tools/presubmit
Change-Id: I9d486fbcf7b2d468f6a1375ac7df95091a2c1465
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3232277
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Consolidates the utility scripts from bin/ into tools/.
Adds a 'presubmit' utility script to run a set of checks and
tests.
This won't be a git hook, but can be manually used to verify
changes before uploading.
BUG=b:199951064
TEST=./tools/presubmit --quick
./tools/dev_container ./tools/presubmit
Change-Id: Iac7c11fca0beaa6d4f214319149ef385fa5ced70
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3225139
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
