Radish alpha
r
rad:zwTxygwuz5LDGBq255RA2CbNGrz8
Radicle CI broker
Radicle
Git
tests(ci-broker.md): document what to upgrade test, use commits
Merged liw opened 1 year ago

Signed-off-by: Lars Wirzenius liw@liw.fi

1 file changed +35 -5 ba9b4365 f736bdf8
modified ci-broker.md
@@ -1663,8 +1663,38 @@ The overall approach is as follows:
Note that because this scenario may be run outside the developer's
development environment, it is currently difficult to access the Git
tags that represent the CI broker releases. Thus we verify upgrades to
-
the Git commit identifiers instead. In any case, this scenario needs
-
to be updated when a new release is made.
+
the Git commit identifiers instead. Note that this should be commits,
+
not tag objects, as the tests may need to run in clone of the a Git
+
repository without tags.
+

+
This scenario needs to be updated when a new release has been made, to
+
avoid the test suite taking too long to run. The goal is to verify,
+
across releases, that upgrades from each release to the next is
+
verified to work. Thus, given releases 1, 2, 3, etc, we amend the
+
scenario to drop all but latest release, and add any missing release.
+
However, if we've neglected to update the scenario for a release, we
+
make sure we don't break the chain.
+

+

+
|release|scenario has|
+
|-|:-|
+
|1|none|
+
|2|1 HEAD|
+
|3|1 2 HEAD|
+
|4|2 3 HEAD|
+
|5|2 3 HEAD|
+
|6|3 4 5 HEAD|
+
|7|5 6 HEAD|
+

+
Release can't do upgrade tests, but it's long in the past so that's
+
OK. Release 2 upgrades from release 1 to HEAD, the current tip of the
+
branch. Release 3 upgrades from 1 to 2 to HEAD. Release 4 can drop
+
release 1, but adds 3. After release 5 we forgot to update the
+
scenario, so for release 6 we include testing upgrade to release 4.
+
For release 7 we can again trip the list.
+

+
This doesn't verify that upgrades work if we skip releases. We're OK
+
with that, until users say they want to skip and are having trouble.

~~~scenario
given a Radicle node, with CI configured with broker.yaml and adapter dummy.sh
@@ -1675,9 +1705,9 @@ given a directory reports

when I touch file run-list.txt
when I run ./env.sh bash -x verify-upgrade run-list.txt 869b451728d16719b560df142b2d901cbaf3764c
-
when I run ./env.sh bash -x verify-upgrade run-list.txt 1dd6fce04ae972bf82c787177c1baa942c77c744
-
when I run ./env.sh bash -x verify-upgrade run-list.txt f3c969511eb4df2aa5dfbdb00bf6a8b46263138f
-
when I run ./env.sh bash -x verify-upgrade run-list.txt 2143072e10f106c06fba6fd908e6bf1c6d18e6eb
+
when I run ./env.sh bash -x verify-upgrade run-list.txt 70f1ba7de5950c8e544fc35e49a7f1123c34d96f
+
when I run ./env.sh bash -x verify-upgrade run-list.txt 6dc8f8e2024d450a7bb6ca5e13cfc31de49d5b6e
+
when I run ./env.sh bash -x verify-upgrade run-list.txt 14d7b5322b87eac37af183c08437534bb20f19f5
when I run ./env.sh bash -x verify-upgrade run-list.txt HEAD
~~~