I've wanted to make the Git support optional for a long time. However,
since everyone uses the Git backend (and we want to support it even in
the custom binary at Google), there hasn't been much practical reason
to make Git support optional.
Since we now use jj-lib on the server at Google, it does make sense to
have the server not include Git support. In addition to making the
server binary smaller, it would make it easier for us (jj team at
Googlle) to prove that our server is not affected by some libgit2 or
Gitoxide vulnerability. But to be honest, neither of those problems
have come up, so it's more of an excuse to make the Git support
optional at this point.
It turned out to be much simpler than I expected to make Git support
in the lib crate optional. We have done a pretty good job of keeping
Git-related logic separated there.
If we make Git support optional in the lib crate, it's going to make
it a bit harder to move logic from the CLI crate into the lib crate
(as we have planned to do). Maybe that's good, though, since it helps
remind us to keep Git-related logic separated.
This is an alternative to https://github.com/martinvonz/jj/pull/3732. I
think it's pretty important to show the "latest" alias. I don't know a
way to not show any alias for "prerelease" in the version selector
without deleting all of them.
I'll have to then separately delete the directories for the aliases on
the gh-pages branch, so that the URL does not resolve to outdated pages.
The downsides are: this will break https://martinvonz.github.io/jj/main/... and
(only recently created) https://martinvonz.github.io/jj/nightly/... links. We
won't have an always-on demo of how well aliases work for when and if we adjust
how aliases (including the all-important latest alias) work in the future.
However, expanding on #3732 (comment), mike is now randomly alternating between
"nightly" and "main" being the first alias of prerelease (it never promised that
the set of aliases would be ordered). So, the version selector keeps switching
in which version it shows. Previously, this only affected the prerelease docs,
but now it also affects https://martinvonz.github.io/jj/latest/.
I feel like recommending people install Poetry via a package manager was
a mistake. Poetry only supports its latest version, and while newer
versions are sort-of backward-compatible, they print warnings in
different situations and have different bugs. Installing `pipx` via a
package manager, OTOH, works fine, and its older versions work fine.
Using Poetry 1.8+ allows us to use Poetry's new "non-package" mode,
which would no longer print warnings if the user does `poetry install`
instead of `poetry install --no-root`. It's likely that in newer
versions of Poetry, it will become an error.
This means that <https://martinvonz.github.io/jj/latest/index.html> will
not longer redirect to
<https://martinvonz.github.io/jj/v0.18.0/index.html> (or similar) after
the next release, as requested by @jyn514 on Discord. Instead, it will
be a copy.
Originally, the plan was to use mike's symlink functionality and make
`latest` a symlink. See a comment inline for how that didn't work; we
could do that after changing how GitHub Pages are built.
Before the next `jj` release, and after this PR is merged, you can play
with this by going to <https://martinvonz.github.io/jj/main/> which is
an alias for <https://martinvonz.github.io/jj/prerelease/>.
Because of previous commits in this PR, the latest version will be
marked in the version selector as "latest", so hopefully this won't
confuse users.
Due to the parent commit, the "latest" URL will be considered canonical.
See https://github.com/martinvonz/jj/pull/3723 for a screenshot.
This will make it clearer to the users that the "latest" and "v0.??.?"
are the same even if the former does not redirect to the latter (but has
identical contents).
MkDocs material lists the *first* alias for each version, so I also
added a "nightly" alias for "prerelease". Otherwise, users would see the
"main" alias, and might get confused whether they should use "latest" or
"main" docs if they don't realize it's named after the "main" branch.
(Upon testing, it still shows "main" for now, but it should be possible
to fix this after this PR is merged by reordering the aliases directly
in `versions.json` on the gh-pages branch)
The aarch64-darwin macOS runners are actually quite speedy, often the fastest
builders anyway. With the recent improvements to the Nix CI speed in 3f7b5a75e,
this shouldn't be a bottleneck, and will catch build regressions.
When running the `nix build`, the `buildRustPackage` function -- which builds
the `jj` crates -- calls `cargo build --release` with flags like `HOST_CXX`
set. This is called the `buildPhase`. Then, it runs the `checkPhase`, which
calls `cargo nextest`, in our case. However, it does not set `HOST_CXX`, for
some reason.
The intent of `buildRustPackage` is that the `buildPhase` and `checkPhase`
have their compilation options fully aligned so that they reuse the local cargo
`target/` cache directory, avoiding a full recompilation. However, the lack
of `HOST_CXX` above among others causes a cache miss, and a bunch of cascading
recompilations. The net impact is that we compile all of the codebase once in
`buildPhase`, then again in `checkPhase`, making the Nix CI build 2x slower on
average than the other Linux runners; 2-3 minutes versus 7 minutes, on average.
Instead, re-introduce a 'check' into the Flake directly, which overrides the
`jujustsu` package, but stubs out the `buildPhase`. This means it will only run
`checkPhase`, which is what we want. Then, modify the CI to run `nix flake check`
again, like it used to in the past.
Unfortunately, this doesn't fix the cache miss when running `nix build`
yourself, it recompiles from scratch in both phases still. That needs to be
fixed in the future, but it's tolerable for now.
This reverts most of 71a3045032.
Signed-off-by: Austin Seipp <aseipp@pobox.com>
This should nerf some of the impact of the flaky Windows GPG tests right now,
which are currently non-deterministically running into hangs, causing a lot of
PRs to stall out. 15 minutes was chosen as a good trade-off between "pretty far
from our current build times" while also catching obvious bugs and flakes.
Signed-off-by: Austin Seipp <aseipp@pobox.com>
GitHub announced these new Apple Silicon based runners today. Let's take them
for a spin.
Let's also add an entry in the release matrix to build and publish `aarch64-
apple-darwin` binaries, too. This doesn't migrate the old release matrix entry;
it still uses a `macos-11` runner. This means the x86 binaries should work on a
few older macOS versions if users need it.
Signed-off-by: Austin Seipp <aseipp@pobox.com>
Summary: It needs new credentials to act as a robot account on my behalf, rather
than the 'github-actions' user, as that one can't be approved for security/legal
reasons from Google.
Signed-off-by: Austin Seipp <aseipp@pobox.com>
Change-Id: I72e303b5625a6417d3444fc65cb79a779a5d9d53
This is quite minor, but it took me a few minutes to figure out
the correct command.
It might be slightly better to print this text inside the build
logs, where people will be looking for certain if the CI fails,
but I didn't immediately find a good way to do so without
complicating the config too much.
The dependabot refused to update some dependencies anyway. Maybe it
conservatively checks if all intra dependencies meet a certain version?
```
updater | 2023/12/15 15:56:45 INFO <job_762807265> No update possible for cargo_metadata 0.17.0
updater | 2023/12/15 15:56:54 INFO <job_762807265> No update possible for crossterm 0.26.1
updater | 2023/12/15 15:57:04 INFO <job_762807265> No update possible for itertools 0.11.0
updater | 2023/12/15 15:57:16 INFO <job_762807265> No update possible for zstd 0.12.4
updater | 2023/12/15 15:57:16 INFO <job_762807265> No update possible for jj-cli 0.12.0
updater | 2023/12/15 15:57:27 INFO <job_762807265> No update possible for toml_edit 0.19.15
updater | 2023/12/15 15:57:38 INFO <job_762807265> No update possible for prost-build 0.11.9
updater | 2023/12/15 15:57:49 INFO <job_762807265> No update possible for prost 0.11.9
```
backout of commit 58744d9573
I noticed some cargo dependencies aren't caught by the dependabot. For example,
there are gix updates, but the dependabot somehow thinks it's not possible to
update.
```
updater | 2023/12/14 15:57:52 INFO <job_762380319> Checking if gix 0.55.2 needs updating
proxy | 2023/12/14 15:57:52 [063] GET https://crates.io:443/api/v1/crates/gix
proxy | 2023/12/14 15:57:52 [063] 200 https://crates.io:443/api/v1/crates/gix
updater | 2023/12/14 15:57:53 INFO <job_762380319> Latest version is 0.56.0
...
updater | 2023/12/14 15:58:00 INFO <job_762380319> Requirements to unlock update_not_possible
updater | 2023/12/14 15:58:00 INFO <job_762380319> Requirements update strategy bump_versions
updater | 2023/12/14 15:58:00 INFO <job_762380319> No update possible for gix 0.55.2
```
I don't know what's wrong, but let's try without the grouped updates as it was
working before.
FWIW, this issue looks similar:
https://github.com/dependabot/dependabot-core/issues/7896
Originally, my motivation was to try again to get `mike` to not push empty
commits (which this should do). I'm now reconsidering this, since *not* pushing
empty commits will make the output of the CI job a little harder to read. If
this becomes an issue, I might even add `--allow-empty` to the `mike`
invocations later.
A more important motivation is that even for a 400-byte file, changing it for
every PR blows up the size of the repo eventually.
The cause for the changes to this file was that `gzip` stores a timestamp
inside the `.gz` file.
https://github.com/jimporter/mike/releases/tag/v2.0.0
The main immediate advantage of this is that `mike` will stop pushing empty
commits.
Also, we can consider switching to using symlinks instead of redirects for
mapping the "latest" version to "v0.11.0". This would make
`https://martinvonz.github.io/jj/latest/` have the same content as
`https://martinvonz.github.io/jj/v0.11.0/` (until the next version is out), but
the user would see `latest` in the URL.
For now, I set an option to keep using redirects.
I did a bit of non-exhaustive testing; it seems to work.
Summary: Keeping the flake.lock up to date and 'fresh' is nice for
all the same reasons that apply to things like Cargo, Poetry, etc.
Unfortunately, dependabot doesn't have support for Nix flakes. There is also
no mechanism to add 'out of band' updates through dependabot, at least not yet.
Instead, we use the `update-flake-lock` action from Determinate Systems, which
can paper over it for us.
This updates once a week on Sunday, which is pretty fine, I think.
A theoretical downside of this approach is that we can't group updates together
like dependabot does; but dependabot only groups 'related' updates together,
i.e. updates to Cargo dependencies. If it also detected updates for e.g. Poetry
or Nix, it would make separate PRs for those.
Signed-off-by: Austin Seipp <aseipp@pobox.com>
Change-Id: I6f447deffc545da77fb320519abcf437
Summary: Codespell actually matches local files at the root of the repo with the
`./` prefix, so without it the `skip` field won't match. Fix this for `./target`
and `./.jj` directories.
Signed-off-by: Austin Seipp <aseipp@pobox.com>
Change-Id: Ibeafd7e400ff3bca9187d62241296060
Summary: Since 066032b6e6 was merged, the `nix flake check` build no longer
overrides the 'cargo test' profile explicitly, to save disk space. The CI seems
to be in a better spot. This will stem the tide for a while hopefully.
However, with that change in place, the `nix flake check` build was
essentially a redundant, nearly-identical copy of a normal `nix build` with no
differentiating features, except: `RUST_BACKTRACE` is set to 1.
Delete all this code, and remove it from the CI matrix, and instead just export
`RUST_BACKTRACE` on the `checkPhase` of the normal `nix build` instead, which is
functionally equivalent.
Also does some minor, no-functional-change touchups to `flake.nix` while I was
there (whitespace, etc.)
Signed-off-by: Austin Seipp <aseipp@pobox.com>
Change-Id: I87336b16e2a0b973343ecbde8ffd7b8f
Perhaps this will stop Github from showing jj-docs-bot as a very active
contributor in https://github.com/martinvonz/jj/pulse. This would
probably fair, even though jj-docs-bot tries its best to be a good and
helpful bot.
Regardless, this seems to be the standard on Github which has
`dependabot[bot]` and `github-actions[bot]`.
One action publishes the 'prerelease' version on every push
to `main`.
The other publishes a 'latest' version on every releasse.
I tested both of them, but not with branch protection rules.
I initially was thinking of using `mdbook`, which looks a little better, but I
think versioning the docs is important, and the features I want are pretty much
only supported by the Mkdocs' "Material" theme.
Mkdocs is written in Python. The prerequesites for building docs on your
machine should be to install Python and Poetry, everything else should be
installed automatically by Poetry. See the edits to `contributing.md` for more details.
We run a separate build on CI with the MSRV so we notice if we
accidentally break the MSRV. However, as we talked about on Discord,
the opposite is very unlikely - that we accidentally break the build
with the stable release without breaking the MSRV build. Also, we
explicitly run Clippy with the stable release, and formatting with the
nightly release. So, let's just do the regular build and tests with
the MSRV.
Summary: Here's a fun fact — macos-latest doesn't actually refer to the latest
macOS version used in your GHA runner! It apparently means, tautologically,
"latest macos version that we have chosen as the default." OK?
But why bother? Because word on the street is that the new macos-13 builders
from GitHub are on new hardware, and much faster than previous ones. Let's test
that rumor.
Signed-off-by: Austin Seipp <aseipp@pobox.com>
Change-Id: Ifdbab62e085adbda41a7edb9fc7038f0
Summary: Let's be more aggressive about tracking the latest stable Rust release.
There's little benefit to being conservative so early on, especially when no
users seem to have faced any issue with upgrading, or strictly required an old
Rust version.
Right now, just lagging Rust by 1 major release probably seems fine. We're
targeting 1.71.0 to get ahead of the curve, since 1.72.0 will likely release
sometime before the next `jj` release.
Signed-off-by: Austin Seipp <aseipp@pobox.com>
Change-Id: I4e691b6ba63b5b9023a75ae0a6917672
Summary: The Determinate Systems Nix installer is fast and importantly includes
Flake support by default on new installs.
Magic Nix Cache will help ease the build time a little by making sure things
like downloads from crates.io remain cached between builds; in the future if we
ever need to add any custom builds to the Nix expression it will come really in
handy. But in my experience I don't think it should slow down much, and might
speed some things up a whole lot.
Signed-off-by: Austin Seipp <aseipp@pobox.com>
Change-Id: I56e1592cbaa6c7a43b5bd57972b98ada
Summary: While I was developing #1846, I found a case where `nix flake check`
succeeded but `nix build` on the jujutsu expression failed, which was confusing
to me. If it had been merged, it would have broke the ability to use Nix to
install things.
Let's try to attack this, and punch a parallel job into the build matrix;
hopefully running the two jobs concurrently can help catch this without
making build time much worse.
Signed-off-by: Austin Seipp <aseipp@pobox.com>
Change-Id: If2520d69492fa6abe499c1cb1d51d6e1
Summary: When building in CI, we just build everything from scratch in the `dev`
profile, so just turn off debuginfo and incremental compilation data. These
aren't reused, but more importantly take up more space and CPU time to generate;
that's more expensive on slower machines and when using The Cloud(TM) to build.
Together, these take a fresh build (`rm -rf target/` ahead of time) from about
40s to 30s on my machine.
Signed-off-by: Austin Seipp <aseipp@pobox.com>
Change-Id: Id6e706634dbaf88f6ccd7f457409ad7c
