Commit graph

28 commits

Author SHA1 Message Date
Dennis Kempin
ed7d455a43 Switch main crosvm repo
This updates all references to chromiumos/platform/crosvm to
crosvm/crosvm.
It will also switch ./tools/cl to upload to the new main repository.

BUG=None
TEST=None

Change-Id: I1f71e5d5de2300e57ace1315cb4553f68533d576
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3760903
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: Dennis Kempin <denniskempin@google.com>
2022-07-14 17:58:26 +00:00
Dennis Kempin
5156976b7f merge_bot: Enable on luci
To enable the merge bot on luci we had to solve a couple of problems:

- We cannot use http cookies for auth, so added gcloud auth into
  merge_bot.
- Switch back to original HEAD after running merge bot. Otherwise
  the version of the merge_bot script can change between invocations.
- Do not soft reset when checking out crosvm source.
- For less invasive testing, also added a few more exceptions when
  using MERGE_BOT_TEST so it won't email, etc.
- Rename to update_chromeos_merges, as it is more fitting to what
  the bot is doing

BUG=b:233913643
TEST=https://ci.chromium.org/swarming/task/5b58ed4497fda510

Change-Id: Iaa9b4821b70a1e90d19637d01856196bd7852ed5
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3692686
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
2022-06-08 17:20:27 +00:00
Dennis Kempin
47e25297d5 merge_bot: Handle merge conflicts
The bot cannot resolve those automatically. To ease the process, we
will upload the merge anyway, so the oncall can download and resolve
the conflict.

Dry runs are not created when there are conflicts, they are bound
to fail and there is nothing the oncall can do about conflicts
until the actual merge is generated.

BUG=b:234847583
TEST=MERGE_BOT_TEST=1 ./tools/chromeos/merge_bot update-merges / update-dry-runs

Change-Id: I409e6b99284ff8a98b504b390c35329875d6a8b9
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3689544
Reviewed-by: Anton Romanov <romanton@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
2022-06-03 19:12:04 +00:00
Dennis Kempin
ca66addad0 Update merge_bot to support merging across repos
We will be merging from the upstream repo into the cros repo. The
script can be run from both an upstream and a cros checkout, it
will setup the required remotes.

The CLI changes a little. We now require a revision to merge from
needs to be specified, since the script is not setting up the
upstream remote.

CI jobs will simply use HEAD, as the version to merge is already
checked out by the CI system.

BUG=b:233913643
TEST=
./merge_bot update-merges HEAD
./merge_bot update-merges origin/main
./merge_bot update-dry-runs HEAD

Change-Id: I247e1edc3ad951becf5d59b63efd74914bb2205f
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3687222
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
2022-06-03 19:12:04 +00:00
Junichi Uekawa
3cb328aa10 crosvm: Script for rebasing on main from chromeos branch.
chromeos branch is a stable base to work on, but patches need to land on
the main branch. For uploading to that directory, it's always necessary
to rebase onto that branch.

BUG=b:227535695
TEST=rebase_for_review && repo upload --cbr .

Change-Id: I5f41e52f91adf15ade84ae09f6b34dcdbe733d23
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3565627
Commit-Queue: Junichi Uekawa <uekawa@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Dennis Kempin <denniskempin@google.com>
2022-05-30 07:26:07 +00:00
Dennis Kempin
039936a871 merge_bot: Make script Python 3.8 compatible
Another script that needs to run on python 3.8. I filed b/234173142 to
make this consistent across all scripts and environments.

BUG=b:233913643
TEST=vpython3 tools/chromeos/merge_bot

Change-Id: Ic42423040feaaa5741a123972978af364886dc9e
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3674220
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: Dennis Kempin <denniskempin@google.com>
2022-05-28 01:04:43 +00:00
Dennis Kempin
455d04d0d1 merge_bot: Do not batch dry runs
This just creates unnecessary noise. Just create a single dry run
merge commit each time.

BUG=b:227224259
TEST=None

Change-Id: I54385fd80f01771bb83e02f2b48e604ad2d68d1e
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3673021
Reviewed-by: Anton Romanov <romanton@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
2022-05-27 18:23:48 +00:00
Dennis Kempin
97306a2bfa Add check_cq_depend to PRESUBMIT
This will be run when using repo upload from a chromiumos checkout
and reminds developers that Cq-Depend cannot be used with crosvm.

BUG=b:226975263
TEST=./tools/chromeos/check_cq_depend

Change-Id: Ib9cafaba181cb2103838369123e6c1288e653486
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3651674
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Dmitry Torokhov <dtor@chromium.org>
Tested-by: Dennis Kempin <denniskempin@google.com>
2022-05-18 18:54:34 +00:00
Dennis Kempin
af594fe6a6 Merge bot: A few minor tweaks
- Don't try to download gcompute tools multiple times
- Fix off-by-one error in merge log calculation.
- Fail if gerrit upload fails repeatedly

BUG=None
TEST=./tools/chromeos/merge-bot update-dry-runs

Change-Id: I9e845455f8c5209c38aee1bcd6bb1b2b81a3d99a
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3585019
Reviewed-by: Anton Romanov <romanton@google.com>
Tested-by: Dennis Kempin <denniskempin@google.com>
2022-04-15 20:40:00 +00:00
Dennis Kempin
77569908de Merge bot: Always set up username on gce
Sligthly changes the logic. If we are on GCE, always use the authdaemon
and set up the git user info for the service account.

Outside of GCE just check if a cookie is available.

BUG=None
TEST=Faked out is_gce_instance and ran locally

Change-Id: Icf4c521b650b5d47f395a59695d1f1470621f661
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3585016
Reviewed-by: Anton Romanov <romanton@google.com>
Tested-by: Dennis Kempin <denniskempin@google.com>
2022-04-15 19:59:57 +00:00
Dennis Kempin
a3249a7fef Merge bot: Actually execute clone of gcompute-tools
BUG=None
TEST=Ran locally, but cannot test actually running the tool outside of
GCE.

Change-Id: I4483d8ea75fceb3d289a38d2aa3b8c6f85985dfa
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3588403
Reviewed-by: Anton Romanov <romanton@google.com>
Tested-by: Dennis Kempin <denniskempin@google.com>
2022-04-15 19:31:18 +00:00
Dennis Kempin
0506ac6549 Curl with headers to get GCE metadata
Metadata-Flavor is a header and not part of the body.

BUG=None
TEST=Has to be tested in GCE.

Change-Id: I92711a86502d19601a5f2d63e6a653743b5b2614
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3579815
Reviewed-by: Dmitry Torokhov <dtor@chromium.org>
Tested-by: Dennis Kempin <denniskempin@google.com>
2022-04-14 23:33:54 +00:00
Dennis Kempin
cc6642deff Do not use curl --fail-with-body
The curl version of Kokoro is too old and does not know it.

BUG=None
TEST=testing in prod.

Change-Id: I8e1fe856dcd893944da06f6a9097497a9c320c43
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3584319
Reviewed-by: Anton Romanov <romanton@google.com>
Tested-by: Dennis Kempin <denniskempin@google.com>
2022-04-14 22:51:29 +00:00
Dennis Kempin
f334e19ffc merge bot: Do not check result of git config
If the value is not set, treat it as empty.

BUG=None
TEST=Tested by removing the config setting locally.

Change-Id: Ia160285c03bffcf7a332f3891587e7ada2228623
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3585860
Reviewed-by: Anton Romanov <romanton@google.com>
Tested-by: Dennis Kempin <denniskempin@google.com>
2022-04-14 22:43:59 +00:00
Dennis Kempin
b193738186 New merge bot
The new merge bot is written in python to allow us to improve the
merge process.

The new process is as follows:
- The merge bot will only create a new merge when the previous
  ones have been submitted.
- When the bot creates a new merge, it'll split the merge commits
  into smaller changes to make bisection easier.

This allows us to only ever deal with one set of merge commits
instead of piling on new merges day after day.

For dry runs, the new process is:
- The bot will check existing dry runs and abandon them if the CQ
  is done.
- The bot will vote Verified+-1 to indicate if the dry run passed.
- If no more dry runs are active, it'll create a new one.

This ensures we always have a dry run in the CQ and can easily see
in gerrit which ones have passed.

The python script will also allow us to add convenience features
for oncall for retrying bratches, etc.

BUG=b:227347397
TEST=The script can be run locally:
./tools/chromeos/merge_bot update-merges
./tools/chromeos/merge_bot update-dry-runs

Change-Id: I51fc0741f5032868ad07f5e926e570e29cc461fc
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3583254
Reviewed-by: Junichi Uekawa <uekawa@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
2022-04-14 21:38:07 +00:00
Dennis Kempin
e871ee9601 Kokoro merge bot: Rebase merges before uploading
This allows the bot to account for manually cherry-picked commits.

To make rebases work properly, we had to fix the commit message
trailers to be added in a separate paragraph. This ensures
git inspect-trailers can add and find the Commit-Id trailer.

BUG=b:226975273
TEST=./ci/kokoro/simulate.py ./ci/kokoro/build-merge-into-chromeos.sh

Change-Id: If008be1f1c671ef1d79c6f223f10ea0e8f6742b5
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3561015
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Reviewed-by: Maciek Swiech <drmasquatch@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
2022-03-30 22:57:48 +00:00
Dennis Kempin
ee53dd075d tools/chromeos/create_merge: Automatically resolve conflicts
The merge can trigger conflicts if a change has been cherry picked
ahead of the merge. We always want to resolve conflicts by picking
the incoming version from cros/main.

BUG=None
TEST=./tools/chromeos/create_merge

Change-Id: I06694dd24b051fcdc12dbaccdbab552c72d7f2c4
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3494968
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
2022-02-28 21:30:13 +00:00
Dennis Kempin
eb962a6993 create_merge: Add BUG= lines into merge commit message
This will trigger the usual gerrit automation to notify the bugs
about the merge being submitted, which will help identify when
fixes have landed in chromeos.

BUG=b:219743659
TEST=./tools/chromeos/create_merge

Change-Id: I6365f3c5596a94a6990caf1e1733e8040e5e8fff
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3469495
Reviewed-by: Noah Gold <nkgold@google.com>
Reviewed-by: Junichi Uekawa <uekawa@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
2022-02-17 19:11:03 +00:00
Dennis Kempin
d6c3154b13 merge-into-chromeos: Create a dry run merge as well
The dry run merge is re-created every day and automatically
set to CQ+1. This allows us to check for bad crosvm changes
while we submit parts of the merge chain.

BUG=b:215741244
TEST=./tools/ci/simulate merge-into-chromeos.sh

Change-Id: I0773a99f9858962e9f25bebff1b233d455d33cdd
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3417508
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
2022-01-26 20:47:46 +00:00
Junichi Uekawa
5e91be7064 crosvm: Commit Cargo.toml with a useful description.
This is what I do all the time manually, maybe this can be automated.

BUG=None
TEST=run it and observe.

Change-Id: Ia32790ca7dd0b8313ce05dcff28509b8a21d4f76
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3336823
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Junichi Uekawa <uekawa@chromium.org>
2022-01-06 22:59:57 +00:00
Dennis Kempin
8d8a136eaa create_merge: Collect Cq-Depend of included commits
This passes Cq-Depend lines through to the merge commit.
Right now this is done manually by the oncaller, which then
requires another person to CR+2, slowing down the process.

BUG=b:210863861
TEST=./tools/chromeos/create_merge

Change-Id: I4adacef8b5fb39aa3a300b56fec5ed0ed79980f1
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3342734
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Tested-by: Dennis Kempin <denniskempin@google.com>
2021-12-21 17:21:53 +00:00
Dennis Kempin
fbb2de0188 kokoro: Various fixes to merge-into-chromeos job
- Adds the needed job config
- Makes a copy of the script before git checkout
- Follow naming scheme of other build scripts
- Add reviewer when uploading

BUG=b:209034086
TEST=./ci/kokoro/simulate build-merge-into-chromeos.sh

Change-Id: I279ff6aca7c3c304354a238ebd5c40b4dd272385
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3321726
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: Dennis Kempin <denniskempin@google.com>
2021-12-07 20:50:29 +00:00
Dennis Kempin
e061930706 Add Kokoro build script for merging main into chromeos
The Kokoro build script sets up access to the chromium
gerrit, creates the merge commit and uploads it.

BUG=b:209034086
TEST=./ci/kokoro/simulate merge-to-chromeos.sh

Change-Id: Ia686b937c3d6b0b744d5c2162eb2847c5952e977
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3318793
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
2021-12-06 23:28:26 +00:00
Dennis Kempin
0830e2f11c chromeos: Add script for merging from cros/main
This script can be used to manually create merge
commits to be uploaded to gerrit, and will also
be used by the buildes doing automatic uprevs.

The main purpose of the script is to create a helpful
commit message describing the commits included.

BUG=b:209034086
TEST=./tools/chromeos/create_merge

Change-Id: I65715ebefb182a5287b5052b6af56361e41d8efd
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3315374
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
2021-12-06 21:19:19 +00:00
Keiichi Watanabe
eef484c20a tools: chromeos: Use relative path to Cargo.toml in setup_cargo
This change will allow the script to be executed from any path.

BUG=none
TEST=run tools/chromeos/setup_cargo from outside of crosvm directory

Change-Id: I9752f4a41987259f446431bf89c322095ef26e63
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3280361
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Auto-Submit: Keiichi Watanabe <keiichiw@chromium.org>
2021-11-15 18:18:18 +00:00
Keiichi Watanabe
3a6ec148d6 third_party/vmm_vhost: Add vmm_vhost code into /third_party/
Move rust-vmm/vhost code from chromiumos's third_party directory to
crosvm's one.

Note that we have crrev.com/c/3269863 for license files in Chrome OS
ebuild.

Changes:
* Copied vmm_vhost files at "7c95b4a2 rust-vmm/vhost: switch to upstream
tempfile", whose latest upstream commit is "488b3adc fix warning:
unaligned_references".
* Remove [patch.crates-io] section from its Cargo.toml.
* Add attributes to skip `clippy::bool_assert_comparison` and
`clippy::needless_borrow`, which will be fixed in following CLs.
* Update crosvm's Cargo.toml and tools/chromeos/setup_cargo

BUG=b:205511695
TEST=tools/chromeos/setup_cargo && cargo build

Change-Id: Icee201b39ff1d352df59469a6c8668fc7cb92ab9
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3267912
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Keiichi Watanabe <keiichiw@chromium.org>
Reviewed-by: Dennis Kempin <denniskempin@google.com>
2021-11-10 08:42:11 +00:00
Dennis Kempin
ccb9014afb Deprecate old dev tooling scripts
All of the developer tooling has moved to ./tools to make them
easier to discover.

BUG=None
TEST=None

Change-Id: I4daf6a0ca08c94a0c35ce8bc52f9d86e4cb15de9
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3251781
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
2021-11-02 22:23:03 +00:00
Dennis Kempin
6fdbcacce7 Add uprev_ebuilds script
This script allows us to generate manual uprev CLs the same way
PUpr creates them.

BUG=b:204223210
TEST=./tools/chromeos/uprev_ebuilds

Change-Id: I45f9f0f4d33f8782637455b578688fa9b504fe8e
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3248905
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
2021-10-27 22:51:24 +00:00