Commit graph

21 commits

Author SHA1 Message Date
Michael Hoyle
6b19695c81 Add "base" crate and transition crosvm usages to it from sys_util
For now, this crate simply re-exports all of sys_util, but it will
be updated to provide new interfaces when needed. This is the
first step to making crosvm not directly depend on sys_util, so
that we can make the interface changes we need without fear of
negatively affecting (i.e. completely breaking) other usages
within chromeos.

BUG=b:162363783
TEST=./build_test

Change-Id: I7d0aa3d8a1f66af1c7fee8fd649723ef17027150
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2325168
Tested-by: Michael Hoyle <mikehoyle@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Michael Hoyle <mikehoyle@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Reviewed-by: Zach Reizner <zachr@chromium.org>
2020-08-06 18:19:44 +00:00
Daniel Verkamp
1ba620d821 qcow_utils: use DiskFile trait from disk crate
Drop the local DiskFile trait definition from qcow_utils and use the one
defined by the disk crate, since qcow_utils already depends on disk.

In order to make the switch, use the DiskGetLen::get_len function
instead of seeking to the end of the file to get the current file size.

BUG=None
TEST=emerge-nami crosvm
TEST=cargo build -p qcow_utils

Change-Id: Ie4b3b8ee0fb11ef02fbc322c5b0f9e22b0345bb0
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2056991
Reviewed-by: Dylan Reid <dgreid@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
2020-02-19 00:46:39 +00:00
Daniel Verkamp
f2eecc4152 disk: move qcow into disk crate
Move qcow from being its own crate into a module of the disk crate,
similar to the composite disk module.

This will allow use of qcow from disk and vice versa without introducing
a circular crate dependency.

BUG=None
TEST=./build_test.py
TEST=USE='asan fuzzer' emerge-nami crosvm

Change-Id: I77512bbe4b94faff1b5460f9796ee56505135580
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/1972477
Reviewed-by: Cody Schuffelen <schuffelen@google.com>
Reviewed-by: Zach Reizner <zachr@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
2020-01-06 21:26:38 +00:00
Daniel Verkamp
521cf5d6d3 qcow_utils: remove unused convert functions
The convert_to_qcow2 and convert_to_raw functions are no longer used
now that concierge's export operation exports the unmodified disk image
in a tarball instead of converting it to qcow2.  Remove the unused
functions to clean up unreachable code.

BUG=None
TEST=cargo build -p qcow_utils
TEST=emerge-nami crosvm vm_host_tools

Change-Id: I525a9123481bd8cb6ebf022a289ecdf6e7ceaff2
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/1972476
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Dylan Reid <dgreid@chromium.org>
2020-01-06 21:26:37 +00:00
Daniel Verkamp
130fbbe71d Add explicit dyn for trait objects
Fix "trait objects without an explicit `dyn` are deprecated" warnings
introduced in Rust 1.38.

BUG=None
TEST=emerge-nami crosvm

Change-Id: I8ca6aa747475268ae898adddd5d091d401326ceb
Signed-off-by: Daniel Verkamp <dverkamp@chromium.org>
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/1862999
Reviewed-by: Dylan Reid <dgreid@chromium.org>
Reviewed-by: Stephen Barber <smbarber@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
2019-10-17 03:35:35 +00:00
Cody Schuffelen
f9b035d50c Support composite disks.
This adds a new disk file type next to raw files and qcow images that
represent an indirection to further raw disk files. The disk file
itself is a proto file with references to file paths for other disks to
open and their virtual offsets and lengths.

The intention is to make it easy to assemble a single virtual hard disk
out of several distinct partition files. In the particular case of
Cuttlefish running Android in a VM, this is relevant as the Android
build system distributes partitions as separate raw files. While the
simple solution is to pass each partition as a separate raw disk, some
functionality (like the bootloader) assumes there is a partition table
with multiple distinct partitions on a single disk.

Implementing composite disk support in the VMM bridges this gap through
supporting the general-purpose case of a disk built out of multiple
component files.

If desired, this can be extended to support qcow files to support
unusual configurations like a mixed qcow/raw disk.

Enabled with the "composite-disk" feature.

Bug: b/133432409
Change-Id: I2b0c47d92fab13b5dc0ca5a960c7cfd2b7145b87
Signed-off-by: Cody Schuffelen <schuffelen@google.com>
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/1667767
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
2019-10-03 00:59:14 +00:00
Daniel Verkamp
71a6f0a790 sys_util: add write_zeroes_all() function
In the same spirit as write_all() for the standard io::Write::write()
function, add a write_zeroes_all() function with a default
implementation that calls write_zeroes() in a loop until the requested
length is met.  This will allow write_zeroes implementations that don't
necessarily fulfill the entire requested length.

BUG=None
TEST=cargo test -p sys_util write_zeroes

Change-Id: I0fc3a4b3fe8904946e253ab8a2687555b12657be
Signed-off-by: Daniel Verkamp <dverkamp@chromium.org>
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/1811466
Reviewed-by: Zach Reizner <zachr@chromium.org>
Reviewed-by: Cody Schuffelen <schuffelen@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
2019-09-25 00:06:27 +00:00
Cody Schuffelen
7d533e5952 Extract disk creation logic out of qcow and src.
Bug: b/133432409
Change-Id: Iba25d5f6bb5f60619bb2f5a3d72ddfd3a81650b4
Signed-off-by: Cody Schuffelen <schuffelen@google.com>
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/1691460
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
2019-08-28 23:52:36 +00:00
David Tolnay
aecf9a4dee edition: Remove extern crate lines
In Rust 2018 edition, `extern crate` is no longer required for importing
from other crates. Instead of writing:

    extern crate dep;
    use dep::Thing;

we write:

    use dep::Thing;

In this approach, macros are imported individually from the declaring
crate rather than through #[macro_use]. Before:

    #[macro_use]
    extern crate sys_util;

After:

    use sys_util::{debug, error};

The only place that `extern crate` continues to be required is in
importing the compiler's proc_macro API into a procedural macro crate.
This will hopefully be fixed in a future Rust release.

    extern crate proc_macro;

TEST=cargo check
TEST=cargo check --all-features
TEST=cargo check --target aarch64-unknown-linux-gnu
TEST=local kokoro

Change-Id: I0b43768c0d81f2a250b1959fb97ba35cbac56293
Reviewed-on: https://chromium-review.googlesource.com/1565302
Commit-Ready: David Tolnay <dtolnay@chromium.org>
Commit-Ready: ChromeOS CL Exonerator Bot <chromiumos-cl-exonerator@appspot.gserviceaccount.com>
Tested-by: David Tolnay <dtolnay@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: David Tolnay <dtolnay@chromium.org>
2019-04-15 02:06:08 -07:00
David Tolnay
59bb712992 edition: Update qcow and qcow_util to 2018 edition
Separated out of CL:1513058 to make it possible to land parts
individually while the affected crate has no other significant CLs
pending. This avoids repeatedly introducing non-textual conflicts with
new code that adds `use` statements.

TEST=cargo check
TEST=cargo check --all-features
TEST=cargo check --target aarch64-unknown-linux-gnu

Change-Id: Ief3fb967df340a99b8263ac185207e30e096105a
Reviewed-on: https://chromium-review.googlesource.com/1519704
Commit-Ready: David Tolnay <dtolnay@chromium.org>
Tested-by: David Tolnay <dtolnay@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: David Tolnay <dtolnay@chromium.org>
2019-04-08 22:30:36 -07:00
Dylan Reid
3d00b70eb5 Revert "qcow_utils: Add tests"
This test breaks the asan builders

BUG=945423

This reverts commit fbbcf7ad14.

Signed-off-by: Dylan Reid <dgreid@chromium.org>
Change-Id: I4db9720118ea6e3002045fad997774ab84d3b59a
Reviewed-on: https://chromium-review.googlesource.com/1540339
Commit-Ready: ChromeOS CL Exonerator Bot <chromiumos-cl-exonerator@appspot.gserviceaccount.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Chirantan Ekbote <chirantan@chromium.org>
2019-03-28 02:11:02 -07:00
Dylan Reid
fbbcf7ad14 qcow_utils: Add tests
Add basic tests that qcow_utils compiles.

Change-Id: I433dc7cb55d42997ba060f9bd989ca3b5c8b0045
Reviewed-on: https://chromium-review.googlesource.com/895189
Commit-Ready: Dylan Reid <dgreid@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Tested-by: Dylan Reid <dgreid@chromium.org>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
2019-03-16 18:05:03 -07:00
Daniel Verkamp
348ccf1102 qcow_utils: add disk image expand function
This exports a new C API to resize a disk image.  The new function is
intended to only expand (increase in size) to avoid accidentally
truncating user data due to bugs elsewhere.

BUG=chromium:858815
TEST=build_test.py

Change-Id: I6f834209aba693618e0f51d920e7b73d4f2a9dfc
Signed-off-by: Daniel Verkamp <dverkamp@chromium.org>
Reviewed-on: https://chromium-review.googlesource.com/1464384
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Zach Reizner <zachr@chromium.org>
2019-02-14 05:27:21 -08:00
David Tolnay
2bac1e7a9c toolchain: Update to Rust 1.31.0
We updated the production toolchain from 1.30 to 1.31 in CL:1366446.
This CL does the same upgrade for the local developer toolchain and
Kokoro.

The relevant changes are in rust-toolchain and kokoro/Dockerfile.
The rest are from rustfmt.

TEST=cargo fmt --all -- --check
TEST=as described in kokoro/README.md

Change-Id: I3b4913f3e237baa36c664b4953be360c09efffd4
Reviewed-on: https://chromium-review.googlesource.com/1374376
Commit-Ready: ChromeOS CL Exonerator Bot <chromiumos-cl-exonerator@appspot.gserviceaccount.com>
Tested-by: David Tolnay <dtolnay@chromium.org>
Reviewed-by: Zach Reizner <zachr@chromium.org>
2018-12-13 19:28:04 -08:00
Zach Reizner
94923406ae qcow_utils: do not close given fds in convert_to_* functions
The `convert_to_*` functions take ownership of the passed FDs even
though they should not according to the function's contract. This change
clones the passed FDs so that the caller can retain ownership of its
FDs.

This change also wraps most of the implementations in catch_unwind so
that panics do not unwind past FFI boundaries, which is undefined
behavior.

BUG=chromium:905799
TEST=in crosh: `vmc export <vm name> <file name>`

Change-Id: I2f65ebff51243675d0854574d8fd02cec1b237a4
Reviewed-on: https://chromium-review.googlesource.com/1338501
Commit-Ready: Zach Reizner <zachr@chromium.org>
Tested-by: Zach Reizner <zachr@chromium.org>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
2018-11-16 05:01:42 -08:00
Daniel Verkamp
7f9b9ea344 qcow: add convert API and export it in qcow_utils
This will be used in vm_concierge's ExportDiskImage function in order to
allow a minimal qcow2 image to be written on the fly (containing only
the required clusters in a tightly-packed image file). It also allows
future flexibility to change the underlying disk image file format while
still exporting qcow2 images (e.g. via `vmc export`).

For testing, add a qcow_img `convert` command, which can convert
between raw and qcow2 as both source and destination.

BUG=None
TEST=Use qcow_img to convert a raw image to qcow2 and back and verify
     its contents are the same as the original.

Change-Id: I74167aca9a9c857d892e24adf5ee17afc0f6e6b5
Signed-off-by: Daniel Verkamp <dverkamp@chromium.org>
Reviewed-on: https://chromium-review.googlesource.com/1272060
Commit-Ready: ChromeOS CL Exonerator Bot <chromiumos-cl-exonerator@appspot.gserviceaccount.com>
Reviewed-by: Dylan Reid <dgreid@chromium.org>
2018-10-18 19:01:06 -07:00
Dylan Reid
cd9f86b299 qcow: Add a utility program for qcow analysis
This program makes figuring out the state of a qcow file easier.

Change-Id: If297eb0cd835a86d8f284d3aef3d7e962e095726
Signed-off-by: Dylan Reid <dgreid@chromium.org>
Reviewed-on: https://chromium-review.googlesource.com/1207455
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
2018-09-19 15:40:06 -07:00
Dylan Reid
2dcb632405 qcow: Set refcounts for initial clusters.
All qcow clusters need to have their refcounts set. Add a `new` method
to `Qcowfile` and use it instead of just headers from the library.

The new method will loop over the initial clusters and initialize their
refcounts.

Add a `create_qcow2` option to the main executable so there is a way to
test image creation that doesn't require DBUS and Concierge.

BUG=none
TEST='crosvm create_qcow2 /tmp/file.qcow2 1000000'
'qemu-img check /tmp/file.qcow2'
no errors reported.

Change-Id: I8798df5942fb23f79cc7ca86820d0783d1f2b608
Signed-off-by: Dylan Reid <dgreid@chromium.org>
Reviewed-on: https://chromium-review.googlesource.com/1136900
Reviewed-by: Stephen Barber <smbarber@chromium.org>
2018-07-16 03:42:07 -07:00
Zach Reizner
d1ab493672 fix some clippy errors that are default deny
Clippy only has 6 lints that are defaulted to deny, which this CL fixes.
The first step to running clippy and establishing our own set of default
deny lints, is to ensure we aren't running afoul of the most common
deny lints.

BUG=None
TEST=cargo clippy --all

Change-Id: I225801357d76a8a9e246e3842bc9bf550fcd7207
Reviewed-on: https://chromium-review.googlesource.com/952045
Commit-Ready: Zach Reizner <zachr@chromium.org>
Tested-by: Zach Reizner <zachr@chromium.org>
Reviewed-by: Stephen Barber <smbarber@chromium.org>
Reviewed-by: Dylan Reid <dgreid@chromium.org>
2018-03-07 19:47:11 -08:00
Stephen Barber
19e57b9532 qcow_utils: add pkgconfig script
Add a pkgconfig file and script to fix it up at build time.

Also fix extern "C" for the qcow_utils header.

BUG=chromium:806119
TEST=emerge-eve-kvm crosvm

Change-Id: Ib69d9e88b42d2f2c8661798c37537a4236e0506e
Reviewed-on: https://chromium-review.googlesource.com/891572
Commit-Ready: Stephen Barber <smbarber@chromium.org>
Tested-by: Stephen Barber <smbarber@chromium.org>
Reviewed-by: Dylan Reid <dgreid@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>
2018-02-01 21:29:32 -08:00
Dylan Reid
6ecbbfd723 Add qcow_utils for exposing qcow functions as a C library
This C library will be use by the VM launcher to create the qcow2 files
used for persistent VM data.

CQ-DEPEND=CL:884263
BUG=none
TEST=cargo test --all -- --test-threads=1

Change-Id: Ibd7f71d2e3f1f72f781978f014865d2161f033f5
Signed-off-by: Dylan Reid <dgreid@chromium.org>
Reviewed-on: https://chromium-review.googlesource.com/875116
2018-02-01 21:29:31 -08:00