Commit graph

86 commits

Author SHA1 Message Date
Suleiman Souhlal
2ac78b9ea0 Add "make_rt" command.
This is used to finally make a "delay-rt" VM's RT VCPUs RT.

BUG=b:142777321
TEST="grep policy /proc/<pid of rt vcpu>/sched" to make sure RT vcpu
    isn't RT until "make_rt".

Change-Id: I73177f8adadf3d1099c2b09871ee9d5feab82587
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2703225
Auto-Submit: Suleiman Souhlal <suleiman@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Chirantan Ekbote <chirantan@chromium.org>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Suleiman Souhlal <suleiman@chromium.org>
2021-08-11 11:39:31 +00:00
Charles William Dick
5404501966 crosvm fix stale balloon stats
Stale balloon stats results can be returned from a stats request for the
following reasons:
* The initial stats buffer from the guest is posted to the
  balloon_host_tube without a request.
* Balloon stats requests can fail because the balloon device isn't
  completely set up yet; writing a stats request to the tube without
  reading the response.
* Balloon stats requests can time out, returning an error. When the
  balloon stats are eventually computed, they will be queued to the tube
  without a read to consume them.

Possibly other reasons too.

This CL fixes this by adding an id to the balloon stats request. The id
is then returned with the computed stats. When consuming stats results
from the balloon_host_tube, we check that the ID is the one we expect,
if not, we keep reading from the tube until we do.

BUG=b:189282316
TEST=tast run dut multivm.Lifecycle.arc_host

Change-Id: I08e50196a45383b30c9e510b3bacbe32888aef80
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3056310
Auto-Submit: Charles William Dick <cwd@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Charles William Dick <cwd@google.com>
Reviewed-by: Chirantan Ekbote <chirantan@chromium.org>
Reviewed-by: Hikaru Nishida <hikalium@chromium.org>
2021-08-06 15:39:15 +00:00
Hikaru Nishida
a0e381b0d8 Change balloon_stats output format to JSON
This CL changes the output format of `crosvm balloon_stats` command to
JSON to ease parsing the result.

BUG=b:181267848
TEST=built and checked the output of crosvm balloon_stats

Change-Id: I4af40237109efdd02b5b70375ef8aa706cf5de55
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2914247
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Hikaru Nishida <hikalium@chromium.org>
Reviewed-by: Chirantan Ekbote <chirantan@chromium.org>
Reviewed-by: Charles William Dick <cwd@google.com>
2021-06-15 17:29:30 +00:00
Daniel Verkamp
f1439d444b devices: add "usb" feature for emulated xHCI
Add a default-enabled "usb" feature to allow compiling out the emulated
USB controller and host device provider when not needed (e.g. for
crosvm-direct).

This reduces the crosvm-direct binary size by about 400 KiB.

BUG=b:173824333
TEST=cargo build --no-default-features
TEST=cargo build # ensure xhci controller is added

Change-Id: I1fc0eeb09c647854e5df57cd2fe7e92140256853
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2913136
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Tomasz Jeznach <tjeznach@chromium.org>
Reviewed-by: Dylan Reid <dgreid@chromium.org>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
2021-05-24 21:37:22 +00:00
Daniel Verkamp
2940980d2e linux: generalize kick_all_vcpus to send any VcpuControl message
Previously, kick_all_vcpus() would only accept VmControl::RunState
messages; extend it to accept any VmControl message type instead.
This required adding the Clone trait to a few types.

BUG=b:174705596
TEST=./test_all

Change-Id: I3c8c42ee8a96ff151fa8f01ab067931bdff2b7b7
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2718281
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
Reviewed-by: Zach Reizner <zachr@chromium.org>
Reviewed-by: Keiichi Watanabe <keiichiw@chromium.org>
2021-05-14 05:24:47 +00:00
Daniel Verkamp
7b17f50df9 devices: usb: claim interfaces on device creation
This initializes the claimed_interfaces list at startup (before the
first Set Configuration request).  It also makes it possible to catch
initialization errors earlier in the process of adding a device, before
it is added to the emulated USB hub.

BUG=b:180238956
BUG=chromium:1030778
TEST=Share USB device with Crostini

Change-Id: I6f27ec9b7ff6cda2ff23cadd77611f8ce1cfc89e
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2811859
Reviewed-by: Zach Reizner <zachr@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
2021-04-22 17:25:22 +00:00
Gurchetan Singh
f860f50b8a rutabaga_gfx: cross-process Vulkan mappings
In multi-process mode, we currently rely on dma-buf mmap() to
map GPU buffers into the guest.  We usually have to fix the
Mesa driver, and maybe even the kernel to get to work.  That's
"kind of" fine for ChromeOS, which owns the entire stack.

For their Virtual Graphics Interface (VGI) initiative, Android
upstream has requested multi-process mode to work in a
cross-platform, generic way.  Using Vulkan is the only option
that meets the rigorous, uncompromising, strict, meticulous and
bone-crushing requirements of Android upstream.

This has possible two benefits:
  1) We can enable multi-process mode on Nvidia or other
     closed-source drivers, which is nice for Cuttlefish.

  2) On open-source drivers, dma-buf memory is pinned to the
     GTT (amdgpu), even when ideally it can be moved into faster
     vram regions.  This atleast gives the implementation the
     chance to do the smarter and faster option.

We shouldn't run into any SELinux issues since the main crosvm
process is not sandboxed.

Incidentals:
   * Changes vulkano_gralloc to consider integrated GPUs and dGPUs.
     Metadata query is preferred done on the integrated GPU.
   * Update vulkano_gralloc to match top of tree vulkano.

BUG=b:173630595
TEST=used Vulkano allocator and mapped memory into the guest

Change-Id: I78b069c7478d11b3201397894dcccd13bdc61f2c
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2792042
Tested-by: Gurchetan Singh <gurchetansingh@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Gurchetan Singh <gurchetansingh@chromium.org>
Reviewed-by: Zach Reizner <zachr@chromium.org>
2021-04-15 17:58:25 +00:00
Noah Gold
9af97d72fa [base] update/clean mmap interfaces.
This CL addresses some minor issues with the existing interface:
1. from_descriptor is too generic for some platforms that require
   special handling for file/File backed mappings.
2. Nearly all call sites pass either File or SharedMemory. Now
   we just have from_ methods for those types to preserve type
   information.
3. Other platforms require additional fields in MemoryMapping, so a
   tuple struct no longer makes sense.
4. The mmap syscall error message was misleading as we use it for more
   than just the mmap syscall.

BUG=None
TEST=builds

Change-Id: I74c41bad52bb81880a11231cd18f47e233548a24
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2815614
Reviewed-by: Udam Saini <udam@google.com>
Reviewed-by: Zach Reizner <zachr@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Noah Gold <nkgold@google.com>
2021-04-15 02:10:35 +00:00
Kevin Hamacher
6fc5f20fd9 Add FFI library providing control socket access
This allows other languages to communicate directly with the control
socket without having to invoke `crosvm`

BUG=None
TEST=Ran ./run_tests

Change-Id: Icbf5905c41643b080bae3613b73a032467db1c4c
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2772798
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Kevin Hamacher <hamacher@google.com>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
2021-04-08 00:20:01 +00:00
Zach Reizner
d49bcdb081 replace all usage of MsgOnSocket derives
Replaced all usage of MsgSocket with Tube.

BUG=b:176847362
TEST=run_tests

Change-Id: Ie647f79926bc0414c125a776eafe7f60c071bab2
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2710324
Auto-Submit: Zach Reizner <zachr@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Zach Reizner <zachr@chromium.org>
Reviewed-by: Dylan Reid <dgreid@chromium.org>
2021-04-02 15:40:41 +00:00
Gurchetan Singh
293913c01a rutabaga_gfx: rutabaga_gralloc: a shimmering beacon of hope
rutabaga_gralloc is a cross-platform, Rust-based buffer
manager.

The rationale for this change is:

1) For the {cross-domain, wayland} context type, we need to
have a good story for the crucial "wl-dmabuf" feature.  As
minigbm has been thoroughly tested on ChromeOS and currently
powers the "wl-dmabuf" feature, it only makes sense for us to
have a path to minigbm for the cross-domain prototype.  This
will be used by Sommelier.

2) While minigbm allocation works well on Chromebooks, it is
not sufficient for cross-platform purposes.  For their Virtual
Graphics Interface (VGI) initiative, Android graphics
virtualization experts have expressed their desire for a Vulkan
based allocator.  This will to go alongside cros_gralloc in
minigbm, which is considered by many to be the ""world's
premiere gralloc implementation".

3) Android graphics virtualization experts have expressed their
desire for vkMapMemory(..) to be used when crosvm is in
multi-process mode.  Currently, only dma-buf mmap() is supported
for zero-copy blobs in multi-process mode.  dma-buf mmap() is not
guaranteed to work on Nvidia (a "must have" for Cuttlefish) or
any other driver for that matter (we *make* it work for ChromeOS).
Possibly only solution: vkMapMemory ;-)

With these goals in mind, here's a summary of the revelant changes:

* Renamed the {gpu_allocator.rs, GpuMemoryAllocator trait} to be
  {gralloc.rs, Gralloc trait}.

* Moved all GPU allocation out of the resources crate and into
  the rutabaga_gfx crate.  This will allow the resources crate to
  be focused on managing resources for virtual machines.

* Moved the gpu_buffer crate into the gralloc module in the
  rutabaga_gfx crate.  The same functionality is now under
  "minigbm.rs", "minigbm_bindings.rs" and "rendernode.rs"

* Added an optional dependency on vulkano.rs.  vulkano.rs is a safe
  Rust wrapper around the Vulkan api [a].  It's emphasis on type
  safety makes a good fit for crosvm, though there are other high
  quality crates out there (gfx-rs, ash.rs).  Though development
  has slowed down, it should satisfy goals (2) and (3) quite easily.

* Added a system_gralloc implementation based on memfd.  This can be
  used when minigbm or Vulkano features are not used, to replicate the
  highly useful "wl-shm" feature in Sommelier.  Astute observers will
  note this can also enable seamless Wayland windowing without GPU
  features for Android too.  Some minor changes to the base crate were
  needed.

* Cut down on the amount of DrmFormats to the subset needed by
  Sommelier and cros_gralloc.

* Moved checked arithmetic into it's own file.

* Internally renamed to "wl-dmabuf" feature to be the "minigbm"
  feature.  This is because "wl-dmabuf" has a dependency on minigbm.

* Small rutabaga_gfx cleanups

[a] https://github.com/vulkano-rs/vulkano/blob/master/DESIGN.md

BUG=b:146066070, b:173630595, b:150239451
TEST=launch virtual machine with 2D mode
TEST=launch virtual machine with 3D mode
TEST=run sommelier with "wl-dmabuf" and "wl-shm"

Change-Id: I693a39cef64cd98e56d843d3c60caa7983d4d6e1
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2626487
Tested-by: Gurchetan Singh <gurchetansingh@chromium.org>
Reviewed-by: Zach Reizner <zachr@chromium.org>
Commit-Queue: Gurchetan Singh <gurchetansingh@chromium.org>
2021-02-04 02:35:52 +00:00
Keiichi Watanabe
eefe7fb19e devices: virtio: fs: DAX based shared memory support
Support virtio-fs's DAX (direct memory access) operation which allows the guest
to directly access file pages.
Specifically, FUSE_SETUP_MAPPING and FUSE_REMOVE_MAPPING operations are
supported.

This option can be used by specifing `dax` option when mount a file system in
the guest.

The DAX optoin improved file I/O performance in most cases.
In Fio tests, both of read and write score were improved by 1.3-14x depending on
test cases.
In Blogbench tests, which create many small files, DAX improved the write score
by 1.5x while the read score was reduced to ~25% (20391 -> 4593).

Here is an excerpt of results:
Fio
* seq_read:     10.2x (143528 -> 1464911)
* seq_write:     3.3x (61253 -> 896791)
* rand_read:    11.6x (138753 -> 1612739)
* rand_write:   14.6x (61253 -> 896791)
* surfing_read:  1.3x (98473 -> 127907)
* surfing_write: 1.3x (83309 -> 108089)

Blogbench
* read:  0.23x (20391 -> 4593)
* write: 1.50x (248 -> 373)

BUG=b:147341783
TEST=Run vm.{Blogbench, Fio} with CL:2291856

Change-Id: I4a47c601412ed32d926de6304337e1594252d258
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2108315
Tested-by: kokoro <noreply+kokoro@google.com>
Tested-by: Keiichi Watanabe <keiichiw@chromium.org>
Commit-Queue: Keiichi Watanabe <keiichiw@chromium.org>
Reviewed-by: Chirantan Ekbote <chirantan@chromium.org>
2020-12-16 06:03:54 +00:00
Camilo Jacomet
a9bfcb485c Fixed Clippy or_fun_call warnings in devices/
BUG=chromium:1111728
TEST=cargo clippy -- -A clippy::all -W clippy::or_fun_call passes under
devices directory

Change-Id: I4c9e079df98b50e55a5489cd925fb7b621caf4dd
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2587191
Reviewed-by: Dylan Reid <dgreid@chromium.org>
Reviewed-by: Zach Reizner <zachr@chromium.org>
Tested-by: Camilo Jacomet <cjacomet@google.com>
Commit-Queue: Camilo Jacomet <cjacomet@google.com>
2020-12-15 05:15:00 +00:00
Chuanxiao Dong
256be3a144 Goldfish battery: add external command interface to set the state
Add a new command "battery" which is used to modify the virtual
battery/ac status.

When there is goldfish battery device model created, the command
is able to sent to the goldfish battery monitoring thread. If no,
the command won't be sent.

The supported commands are:
1. crosvm battery goldfish status <status string> crosvm.sock
2. crosvm battery goldfish health <health string> crosvm.sock
3. crosvm battery goldfish present <number> crosvm.sock
4. crosvm battery goldfish capacity <number> crosvm.sock
5. crosvm battery goldfish aconline <number> crosvm.sock

"goldfish" is the battery identifier to specify which virtual battery
is going to be modified by this command in case there are multiple
virtual batteries in the guest in future. Right now only one goldfish
battery is supported.

BUG=chromium:1050432
BUG=b:137890633
TEST=create VM with parameter "--battery" or "--battery=type=goldfish"
and boot linux guest with goldfish_battery driver enabled. From host
side, execute command "#crosvm battery goldfish status discharging
crosvm.sock" can make the guest receive an interrupt, and the status
sysfs is changed to "Discharging". Also tested the other commands
as well.
TEST=modified values are visible in ARCVM and interrupts are triggered

Change-Id: I82177811a6f2b1960f7895522760ff3b8143163f
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2119574
Tested-by: Alex Lau <alexlau@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Alex Lau <alexlau@chromium.org>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
2020-11-12 19:34:04 +00:00
Keiichi Watanabe
23f9471bc3 gdb: Support single-step execution and HW breakpoints for x86_64
Support single-step execution command and hardware-associated breakpoints
command in GDB.

BUG=chromium:1141812
TEST=Set breakpoint with `hbreak start_kernel` and show backtrace

Change-Id: I762565ffc9fb055dd3891e172799c3fc87f7604c
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2494327
Commit-Queue: Keiichi Watanabe <keiichiw@chromium.org>
Tested-by: Keiichi Watanabe <keiichiw@chromium.org>
Auto-Submit: Keiichi Watanabe <keiichiw@chromium.org>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
2020-11-12 08:48:01 +00:00
Keiichi Watanabe
c5262e9fad linux: Support GDB remote serial protocol for x86_64
Add a flag '--gdb <port>' to provide GDB remote protocol interface so
a developer can attach GDB to the guest kernel.
In this CL, we support read/write operations for registers and memories.

BUG=chromium:1141812
TEST=Attach gdb and see register values on workstation and intel DUT

Change-Id: Ia07763870d94e87867f6df43f039196aa703ee59
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2440221
Commit-Queue: Keiichi Watanabe <keiichiw@chromium.org>
Tested-by: Keiichi Watanabe <keiichiw@chromium.org>
Auto-Submit: Keiichi Watanabe <keiichiw@chromium.org>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
2020-11-12 08:47:59 +00:00
Chuanxiao Dong
fd5626c523 Goldfish battery: add the basic battery/ac property emulation
Create goldfish battery device model for guest. Provide the
minimal properties and basic accessing for the golfish_battery
driver in the guest.

BUG=chromium:1050432
BUG=b:137890633
TEST=create VM with parameter "--battery" or "--battery=type=
goldfish" and boot linux guest with goldfish_battery driver
enabled. The goldfish_battery driver is loaded successfully.
The /sys/class/power_supply/battery and
/sys/class/power_supply/ac are created successfully.
TEST=cargo test -p crosvm
TEST=goldfish_battery driver is loaded on ARCVM after crrev.com/c/2519336

Change-Id: Iaafa09e3d7e5664d3db8b7df5295f077ec3f6da9
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2119572
Commit-Queue: Alex Lau <alexlau@chromium.org>
Tested-by: Alex Lau <alexlau@chromium.org>
Reviewed-by: Tomasz Jeznach <tjeznach@chromium.org>
2020-11-11 13:15:11 +00:00
Dylan Reid
b04926680c linux: Switch from a run mutex to a channel
Using a channel will allow for more complicated messages to be sent to
the vcpu thread. Adding debug support will require being able to process
each message from the main thread. The atomic state worked well when
every CPU needed a message and when transient state changes can be
dropped. Neither of those assumptions will hold true when using this
communication channel for debugging message such as setting and getting
vcpu registers or single stepping a particular cpu.

The run state is now per-cpu so that individual CPUs can be stopped and
stepped by GDB in later commits.

BUG=chromium:1141812
TEST=run a VM

Change-Id: Ibe3b71af0c99057666d30e302090a8a4364c064a
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2440219
Tested-by: Keiichi Watanabe <keiichiw@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Keiichi Watanabe <keiichiw@chromium.org>
2020-11-11 08:08:40 +00:00
Michael Hoyle
6a0960a4a7 base: First steps towards universal RawDescriptor
This involves removing RawFd trait implementations from
Event and Timer, and resolving the echoing dependencies from
there.

Ultimately, this results mostly in changes across ioctl
(new thin layer in base), kvm, msg_on_socket, and a few other
areas. As usual, most changes are negligible.

BUG=b:162363783
TEST=./build_test

Change-Id: I47dec94666bc3430fed627b4767919c9cb4fce6f
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2462330
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Michael Hoyle <mikehoyle@google.com>
2020-10-31 07:12:34 +00:00
Michael Hoyle
3038e40a6b base: Refactor mmap to use builder pattern.
BUG=b:165423256
TEST=./build_test

Change-Id: Ia67c1a7fe29b66c9cab38476eecde8c25a55617b
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2442569
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Michael Hoyle <mikehoyle@google.com>
2020-10-16 07:15:24 +00:00
Michael Hoyle
685316f0bd base: EventFd -> Event renaming
Note the CL size is large entirely due to the rename,
the changes are mostly negligible.

Also making a few small additional changes in sys_util
areas that don't need much attention in base. This includes
typedefing and adding specific imports for areas that don't
require significant interface changes.

BUG=b:162363783
TEST=./build_test

Change-Id: I4a2c9c4cdce7565806ed338e241c6b8c82c855c7
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2415180
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Michael Hoyle <mikehoyle@google.com>
2020-10-06 13:50:09 +00:00
Michael Hoyle
a7e38ab421 base: Add shared memory layer to base.
This is very largely boilerplate, but changes some interfaces
to accept different parameters, esp using Descriptor.

BUG=b:162363783
TEST=./build_test

Change-Id: I81b513c0de9448c2de505fe5d3a61a29f4eccb78
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2342591
Reviewed-by: Zach Reizner <zachr@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Michael Hoyle <mikehoyle@google.com>
2020-09-30 19:44:40 +00:00
Steven Richman
f32d0b48fd Use hypervisor abstraction to run VMs
Calls to the kvm crate have been replaced by calls to hypervisor and
devices::irqchip.  Plugin is unchanged and still KVM-specific.

x86 creates and configures vcpus on the vcpu thread instead of the main
thread.

Code that still depends on kvm or kvm_sys:

  - crosvm_plugin, plugin, and protos use kvm and kvm_sys if built with
    the plugin feature

  - devices::VfioGroup does a kvm_sys ioctl

  - devices::irqchip and hypervisor use kvm_sys.  hypervisor uses
    kvm::Cap and kvm::CpuId internally for now.

BUG=chromium:1077058
TEST=runs es2gears_wayland on kukui
TEST=runs evince on amd64-generic with kernel and split irqchip
TEST=build_test excluding tpm2
TEST=cargo check --features plugin

Change-Id: I0068e34da6a5ff0079b1237be1958933bf12ffe4
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2307711
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Steven Richman <srichman@google.com>
2020-08-11 04:51:34 +00:00
Michael Hoyle
6b19695c81 Add "base" crate and transition crosvm usages to it from sys_util
For now, this crate simply re-exports all of sys_util, but it will
be updated to provide new interfaces when needed. This is the
first step to making crosvm not directly depend on sys_util, so
that we can make the interface changes we need without fear of
negatively affecting (i.e. completely breaking) other usages
within chromeos.

BUG=b:162363783
TEST=./build_test

Change-Id: I7d0aa3d8a1f66af1c7fee8fd649723ef17027150
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2325168
Tested-by: Michael Hoyle <mikehoyle@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Michael Hoyle <mikehoyle@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Reviewed-by: Zach Reizner <zachr@chromium.org>
2020-08-06 18:19:44 +00:00
Dylan Reid
ec058d6c46 vm_memory: A crate to hold vm-specific memory objects
Move GuestAddress and GuestMemory to a new crate for VM memory. This
will make separating sys_util and crosvm independent making it easier
to use sys_util functions outside of crosvm.

Change-Id: I12e14948ea85754dfa6267b3a3fb32b77ef6796e
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2311251
Auto-Submit: Dylan Reid <dgreid@chromium.org>
Commit-Queue: Dylan Reid <dgreid@chromium.org>
Tested-by: Dylan Reid <dgreid@chromium.org>
Reviewed-by: Zach Reizner <zachr@chromium.org>
2020-07-24 21:42:34 +00:00
Lingfeng Yang
d6ac1ab3a7 kvm: allow registering memory from current address space
This is good for enabling non-exportable Vulkan host coherent memory,
along with anything else where an exportable OS object isn't supported.

This CL introduces:

1. ExternalMapping, which wraps an external library mapping using
function callbacks, for purposes of sharing device memory to the guest
in the case where the device memory is not compatible with the mmap interface.
This is common in Vulkan when VkDeviceMemory is host visible but not
external, or external but based on an opaque fd.

The lifetime of the library mapping is tied to the lifetime of the
ExternalMapping.

2. Usually, we would send such memory requests over a socket to the main
thread. However, since these new objects require more metadata than
other requests that are sent over the wire (because there's information
about inheritance and refcounts), we also plumb the "map_request" field,
which wraps a single ExternalMapping.

Note that this ExternalMapping will not work in the sandbox case. In the
sandbox case, we will then have to figure out how to serialize/deserialize
ExternalMapping requests over a socket.

BUG=b/146066070, b/153580313
TEST=compile and test

Change-Id: I3b099b308aec45a313a8278ed6274f9dec66c30b
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2034029
Tested-by: Gurchetan Singh <gurchetansingh@chromium.org>
Commit-Queue: Gurchetan Singh <gurchetansingh@chromium.org>
Reviewed-by: Zach Reizner <zachr@chromium.org>
2020-07-24 00:19:11 +00:00
Gurchetan Singh
014b351f58 resources: add address_from_pci_offset function
Refactor current code and add tests.

BUG=chromium:924405
TEST=compile and test

Change-Id: I9476f3a4ffd8ae85fc95d6889ada6b056613bbfa
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2216447
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Gurchetan Singh <gurchetansingh@chromium.org>
Tested-by: Gurchetan Singh <gurchetansingh@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Auto-Submit: Gurchetan Singh <gurchetansingh@chromium.org>
2020-06-12 05:00:42 +00:00
Gurchetan Singh
173fe62df2 kvm: use MappedRegion trait
- Reduces code duplication between MMIO and mmap arenas
- Makes adding future types easier
- Makes upcoming deprecation of kvm crate easier
- Use BTreeMap instead of HashMap since it's more efficient

BUG=chromium:924405
TEST=compile and test

Change-Id: I520abed0926489e64aac046e0dc0cfeb72fae7b2
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2216446
Tested-by: Gurchetan Singh <gurchetansingh@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Gurchetan Singh <gurchetansingh@chromium.org>
Reviewed-by: Steven Richman <srichman@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Auto-Submit: Gurchetan Singh <gurchetansingh@chromium.org>
2020-06-12 05:00:37 +00:00
Steven Richman
7c359d617f hypervisor: add Vm user memory region functions
The separate Vm functions for MemoryMappings and MemoryMappingArenas
have been combined and now use a MappedRegion trait that the mappings
implement.  msync_memory_region replaces the get_mmap_arena function,
which is used by VmMsyncRequest.  Since Vm uses mutexes for cloning, it
can't return mem region references.

BUG=chromium:1077058
TEST=cargo test, cargo test -p sys_util, cargo test -p hypervisor

Change-Id: If257b16ee34d07820ae7ebdb9a3a598a41df013c
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2202845
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Reviewed-by: Gurchetan Singh <gurchetansingh@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Gurchetan Singh <gurchetansingh@chromium.org>
2020-06-10 17:08:55 +00:00
Zach Reizner
73665866a0 msg_socket: remove MsgOnSocket impl for RawFd, an alias for i32
This implementation makes it impossible to use i32 as ordinary data in a
MsgOnSocket implementation. It is also error prone because i32 is the
default type for integer literals in Rust, and #[derive(MsgOnSocket)] on
structs with an i32 would suddenly be transmitting a RawFd.

This change also replaces the uses of RawFd's MsgOnSocket impl for one
based on File.

TEST=cargo build
BUG=None

Change-Id: I9da392e34e62fe61f773f24d656dbcbf4716f1a9
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2197777
Tested-by: Zach Reizner <zachr@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Zach Reizner <zachr@chromium.org>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
2020-05-16 02:19:37 +00:00
Tomasz Jeznach
da0e0f939b devices: pci: refactor PCI devices to use PciAddress.
Simple refactor of PCI device addressing to use
PciAddress type providing bus:device.function number.

BUG=None
TEST=build_test & tast run crostini.Sanity

Change-Id: I7755ad6b31aa8c882475cd8212630e1cc86ef49e
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2172766
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Tomasz Jeznach <tjeznach@chromium.org>
2020-05-01 05:49:39 +00:00
Chirantan Ekbote
2d9dde9ee7 Stop tracking sub-mappings in MemoryMappingArena
The kernel already takes care of tracking all our memory mappings.
Doing it again ourselves doesn't provide any benefit and also adds
additional restrictions (like not being able to overlap with existing
mappings or partially remove mappings).  Additionally, the
`MemoryMappingArena` will already unmap the entire memory mapped region
so there is no need to individually unmap the sub-mappings.

The kernel's mmap api doesn't have these restrictions and as far as I
can tell there are no safety concerns with allowing this behavior so
just stop tracking the sub-mappings.

Safe use of MAP_FIXED only requires that the address is part of a
previously mmaped region so allow any MemoryMapping to be converted into
a MemoryMappedArena.

BUG=b:147341783
TEST=unit tests

Change-Id: Iaf944a971b8ba9333802aab73c1d184fe388af89
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2162542
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Chirantan Ekbote <chirantan@chromium.org>
Reviewed-by: Stephen Barber <smbarber@chromium.org>
2020-04-29 07:36:36 +00:00
Charles William Dick
ed22f6b611 crosvm balloon_stats command
In preparation for moving balloon sizing logic from crosvm to concierge,
expose a balloon_stats command in crosvm. This will allow concierge to
query the actual balloon size and available memory of VMs.

BUG=b:153134684
TEST=(chroot)$ tast run <DUT> arc.Boot.vm; (vm)$ crosvm balloon_stats
<pipe>; See stats are reported.

Change-Id: I1f544526ee728a085d842035754a0c17cf41ed3f
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2141752
Tested-by: Charles Dueck <cwd@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Chirantan Ekbote <chirantan@chromium.org>
Commit-Queue: Charles Dueck <cwd@chromium.org>
2020-04-13 02:59:02 +00:00
Daniel Verkamp
e1980a9c36 devices: pmem: implement flush using msync()
Previously, writable pmem devices implemented the flush command using
fsync(); however, this does not guarantee synchronization of memory
mappings via mmap() to the file on disk.  What we actually need is
msync() on the pmem file mapping, but we don't have access to that
mapping in the pmem child process, and it isn't trivial to pass it along
since it is owned by the Vm object once it has been added as a
mmap_arena.

In order to call msync() on the mapping, add a new VmControl socket so
that the pmem device can request that the main process issues an msync()
on the MemoryMappingArena identified by its slot number.

BUG=chromium:1007535
TEST=mount filesystem on /dev/pmem0 and sync; verify msync in strace

Change-Id: Id0484757c422cf81d454fd54012a12dbcc1baaf6
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2044365
Reviewed-by: Stephen Barber <smbarber@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
2020-04-08 06:09:25 +00:00
Dylan Reid
252d5b3cf3 handle mmap of large offsets on 32 bit systems
While only 32 bits of address can be mapped, that 32 bits can be offset
by further than 32 bits in to a large file. As chirantan points out, the
try_mmap call was already casting the usize to u64 on all architectures.

Convert the usize offset in mmap to u64 and address users of the API as
well.

Change-Id: I67aed928ea521049fb51eb7aa61ea4de8b4d096c
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2124879
Reviewed-by: Dylan Reid <dgreid@chromium.org>
Tested-by: Dylan Reid <dgreid@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Dylan Reid <dgreid@chromium.org>
2020-04-05 21:32:17 +00:00
Zach Reizner
146450b456 msg_socket: support dynamically sized types
This change is a major shift in how the MsgOnSocket trait works to allow
`self` to be used to determine the result `msg_size()`. This is to
support data structures with `Vec` or other dynamically sized type.

TEST=./build_test
     cargo test -p msg_socket
     tast run <DUT> crostini.CopyPaste.*
BUG=None

Cq-Depend: chromium:2025907
Change-Id: Ibdb51b377b2a2a77892f6c75e1a9f30b2f8b0240
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2029930
Tested-by: Zach Reizner <zachr@chromium.org>
Auto-Submit: Zach Reizner <zachr@chromium.org>
Reviewed-by: Zach Reizner <zachr@chromium.org>
Commit-Queue: Zach Reizner <zachr@chromium.org>
2020-04-05 15:10:47 +00:00
Charles William Dick
664cc3ca49 crosvm virtio balloon stats
Introduces the ability to request BalloonStats from a
BalloonControlCommand.

BUG=b:147334004
TEST=tast run <DUT> arc.Boot.vm, and the balance available changes based
on this.

Change-Id: I808c4024f8c644c9cc4e30cc455ceda5f477bff3
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2061517
Reviewed-by: Chirantan Ekbote <chirantan@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Charles Dueck <cwd@chromium.org>
2020-03-31 05:59:32 +00:00
Gurchetan Singh
83fc5c49ba devices: gpu: complete resource V2 rebase
* Remove RESOURCE_V2_UNREF
* Add RESOURCE_MAP/RESOURCE_UNMAP to enable resources without guest
  storage that don't need to be mapped directly either

BUG=chromium:924405
TEST=compile and test

Change-Id: I10d6cd120d86131fa7ed8917ddad25cdb99ae50c
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2015587
Tested-by: Gurchetan Singh <gurchetansingh@chromium.org>
Auto-Submit: Gurchetan Singh <gurchetansingh@chromium.org>
Reviewed-by: Zach Reizner <zachr@chromium.org>
Commit-Queue: Gurchetan Singh <gurchetansingh@chromium.org>
2020-03-23 20:37:12 +00:00
Zach Reizner
bc7728f69b vm_control: fix double-close on VmIrqRequest::AllocateOneMsi
The EventFd that wraps the MaybeOwnedFd will close the fd, but so will
MaybeOwnedFd, causing a double-close.

BUG=None
TEST=strace crosvm run

Change-Id: I277386cd20eaa1a8187274cc16084b1936355012
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2034026
Commit-Queue: Zach Reizner <zachr@chromium.org>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Reviewed-by: Xiong  Zhang <xiong.y.zhang@intel.corp-partner.google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
2020-02-06 21:56:39 +00:00
Xiong Zhang
85abeff27f vfio: Implement bar mappable
if device bar is mappable, map bar's gpa to hpa in EPT, guest vcpu
could access this bar directly through EPT without trapping. This
could improve performance.

vm.add_mmio_memory could help do this, here vfio_pci send
RegisterMmapMemory request through vm_control socket to do this.

BUG=chromium:992270
TEST=none

Change-Id: I3b4274372f7dcd32e18084d55f037b6fe45ed422
Signed-off-by: Xiong Zhang <xiong.y.zhang@intel.corp-partner.google.com>
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/1581147
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
2019-12-06 08:47:29 +00:00
Xiong Zhang
383b3b520a resource: Rename device memory to mmio
Since unified allocator is used to allocate mmio, this patch remove the
device memory name, and rename device to mmio.

BUG=chromium:992270
TEST=this patch doesn't change function, run build_test

Change-Id: I234b0db4b3c5de8cfee372ace5212a980564d0c7
Signed-off-by: Xiong Zhang <xiong.y.zhang@intel.corp-partner.google.com>
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/1895234
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
2019-11-10 06:39:34 +00:00
Xiong Zhang
87a3b44d9e Resource: Unify mmio allocator
Current mmio and device two allocators exist, the purpose to define
two allocator is:
Accessing to gpa from mmio allocator cause vm exit, while gpa from
device allocator doesn't cause vm exit.

Whether vm exits exist or not, dependency on whether
vm->add_device_memory() is called with gpa from allocator or not.Even
if gpa is from mmio alloator, and vm->add_device_memory() is called
with this gpa, accessing this gpa won't cause vm exit. So mmio allocator
and device allocator couldn't guarantee the original purpose.

This patch unify mmio allocator and device allocator into one mmio
allocator.

BUG=chromium:992270
TEST=this patch doesn't change function, so just run build_test

Change-Id: If87d5c2838eb122ef627fa45c394b1b3ccfafeb0
Signed-off-by: Xiong Zhang <xiong.y.zhang@intel.corp-partner.google.com>
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/1895233
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Zach Reizner <zachr@chromium.org>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
2019-11-10 06:39:31 +00:00
Daniel Verkamp
6494117e17 usb: replace libusb with Rust usb_util library
Drop the dependency on libusb and reimplement the host USB backend using
usb_sys to wrap the Linux usbdevfs ioctls.

This allows sandboxing to work without any dependency on libusb patches,
and it gives us the flexibility to modify and update the USB backend
without depending on an external third-party library.

BUG=chromium:987833
TEST=`adb logcat` on nami with Nexus 5 attached
TEST=deploy app to phone with Android Studio
TEST=Run EdgeTPU USB accelerator demo (including DFU mode transition)

Cq-Depend: chromium:1773695
Change-Id: I4321c2b6142caac15f48f197795a37d59d268831
Signed-off-by: Daniel Verkamp <dverkamp@chromium.org>
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/1783601
Reviewed-by: Zach Reizner <zachr@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
2019-10-17 00:20:24 +00:00
Xiong Zhang
2515b75630 vm_control: Add VmIrqRequest Socket
When vfio device's msi/msi-x or virtio device's msi-x is enabled,
its irq routing info should be notified to kvm. But this is a runtime
vm service call, so vm_control is used to call vm service.

VmIrqRequest->AllocateOneMsi() is used to allocate one gsi for a msi
and a msi-x vector, and link gsi with irqfd through vm->register_irqfd.
The orignal interrupt_evt and interrupt_resample_interrupt is used for
INTX only.

VmIrqRequest->AddMsiRoute is used to add msi routing info into kvm
route table.

BUG=chromium:992270
TEST=none

Change-Id: I4f1beeb791943e09d957573dd2a58d55bf895d16
Signed-off-by: Xiong Zhang <xiong.y.zhang@intel.com>
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/1846603
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Reviewed-by: Zach Reizner <zachr@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Xiong  Zhang <xiong.y.zhang@intel.corp-partner.google.com>
2019-10-15 11:41:38 +00:00
Gurchetan Singh
7ca5e3dba3 vm_control: support registering memory at a specific address
The guest kernel will allocate from PCI range of the virtgpu
device, and send physical addresses via a hypercall.

Right now, only support buffers that can be mmap'ed.  We could
add optimizations for GBM buffers later if needed.

BUG=chromium:924405
TEST=compile

Change-Id: I094de96a2c35bcd2e18c8a6a2d8cdc39bb392e36
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/1626794
Tested-by: kokoro <noreply+kokoro@google.com>
Tested-by: Gurchetan Singh <gurchetansingh@chromium.org>
Commit-Queue: Gurchetan Singh <gurchetansingh@chromium.org>
Reviewed-by: Zach Reizner <zachr@chromium.org>
2019-10-01 07:53:32 +00:00
Daniel Verkamp
a445ae1fbe Remove remaining byteorder crate references
Remove the last (unused) reference to byteorder in vm_control/Cargo.toml
and remove it from the list of external crates in README.md.

BUG=None
TEST=./build_test

Change-Id: Ie70c0bc2f96c19b34ef1596885245eca7bc202f4
Signed-off-by: Daniel Verkamp <dverkamp@chromium.org>
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/1761156
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Zach Reizner <zachr@chromium.org>
2019-08-26 23:41:54 +00:00
Gurchetan Singh
53edb817da crosvm: {WlDriverRequest, WlDriverResponse} --> {VmMemoryRequest, VmMemoryResponse}
These type of requests are not necessarily specific to the virtio-wl,
and other devices (virtio-gpu) may want to use them.

BUG=chromium:924405
TEST=compile

Change-Id: Iad0889da8ab3d23bb2378448fc05e3c840a93d93
Reviewed-on: https://chromium-review.googlesource.com/1626791
Commit-Ready: ChromeOS CL Exonerator Bot <chromiumos-cl-exonerator@appspot.gserviceaccount.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Legacy-Commit-Queue: Commit Bot <commit-bot@chromium.org>
Reviewed-by: Zach Reizner <zachr@chromium.org>
2019-05-24 15:09:26 -07:00
Zach Reizner
aff94ca6da usb: support for listing attached usb devices
Originally, crosvm would list details about an attached usb device for a
given port. This change allows getting details about multiple ports at
once. This is intended to simplify command line usage and downstream
consumers like concierge.

TEST=various vmc commands
     Chrome UI for handling USB devices
BUG=chromium:831850

Change-Id: I55681a7fea7425c897a22a579dcc15567683ef54
Reviewed-on: https://chromium-review.googlesource.com/1529765
Commit-Ready: Zach Reizner <zachr@chromium.org>
Tested-by: Zach Reizner <zachr@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Zach Reizner <zachr@chromium.org>
2019-04-27 01:36:47 -07:00
Jakub Staron
d99cd0ae0b crosvm: Extracts Wayland commands from from VmRequest.
BUG=None
TEST=cargo test
TEST=cargo test --package msg_socket
TEST=cargo test --package devices
TEST=cargo test --package vm_control
TEST=tast -verbose run ${IP} vm.CrostiniStartEverything

Change-Id: I07f034b1cc41e30b9deae68ea9c510b0923e17a8
Reviewed-on: https://chromium-review.googlesource.com/1565299
Commit-Ready: Jakub Staroń <jstaron@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Zach Reizner <zachr@chromium.org>
2019-04-27 01:36:38 -07:00
Jakub Staron
1f828d7cd7 Extracts BalloonAdjust from VmRequest.
Extracts BalloonAdjust from VmRequest into BalloonControlCommand.

BUG=None
TEST=cargo test
TEST=cargo test --package msg_socket
TEST=cargo test --package devices
TEST=cargo test --package vm_control
TEST=tast -verbose run ${IP} vm.CrostiniStartEverything

Change-Id: Ia9f5778c37c8fd4fa560df413134d1b441142f64
Reviewed-on: https://chromium-review.googlesource.com/1565298
Commit-Ready: Jakub Staroń <jstaron@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Stephen Barber <smbarber@chromium.org>
2019-04-25 00:59:51 -07:00