crosvm/.config/nextest.toml

[profile.default]
# Detect slow tests and terminate them after 60s.
slow-timeout = { period = "20s", terminate-after = 3 }
# Always finish running all tests instead of terminating after the first failure.
fail-fast = false
# Retry tests in case they are flaky.
retries = 2

[profile.postsubmit]
# Do not retry tests in postsubmit to detect flaky tests.
retries = 0
New nextest based test runner We greatly improved the test health of crosvm which allows to make better use of standard cargo tools instead of custom tailored implementations for running tests. The new test runner is available at tools/run_tests2 until it fully replaces tools/run_tests. The key difference is that tests are packaged into a tarball to be executed remotely instead of sending and executing every binary separately via SSH. Improvements over the old test runner: - Faster, nextest runs faster and the new model for remote testing is a lot faster when running tests in a VM. - Simpler, less custom code to maintain. - More readable output, uses modern TUI functionality. - Enables future workflows for testing on cloud devices running in our luci infrastructure. How much faster? - Running all tests on host: 1m -> 9s - Running all aarch64 tests in vm: 3m20s -> 32s BUG=b:261600801 TEST=./tools/run_tests2 with args: -p x86_64 -p x86_64 --dut=host -p x86_64 --dut=vm -p aarch64 -p aarch64 --dut=vm -p armhf -p mingw64 Change-Id: I842a0ac6f7b288aeccb112b4e2f46a15f3baf54b Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4144641 Reviewed-by: Daniel Verkamp <dverkamp@chromium.org> Reviewed-by: Zihan Chen <zihanchen@google.com> Commit-Queue: Dennis Kempin <denniskempin@google.com> 2023-01-06 22:25:50 +00:00			`[profile.default]`
tools/run_tests2: Prepare for CI usage Ensure that the VM is available and add profiles for pre and post-submit. BUG=b:261600801 TEST=None Change-Id: Ib94bf2e5e770c1a802c44199142f01953865f6f8 Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4226492 Commit-Queue: Dennis Kempin <denniskempin@google.com> Reviewed-by: Zihan Chen <zihanchen@google.com> 2023-02-06 19:14:01 +00:00			`# Detect slow tests and terminate them after 60s.`
			`slow-timeout = { period = "20s", terminate-after = 3 }`
			`# Always finish running all tests instead of terminating after the first failure.`
			`fail-fast = false`
			`# Retry tests in case they are flaky.`
			`retries = 2`

			`[profile.postsubmit]`
			`# Do not retry tests in postsubmit to detect flaky tests.`
			`retries = 0`