In the release candidate period leading to the 1.0 release we had a few cases where one component (radicle-node, radicle-httpd, or radicle-explorer especially) changed and was released before the other components were changed and released to cope. This led to things like being unable to see what’s on a node via the explorer, until the explorer on server where it runs had been upgraded. That’s unfortunate from a user point of view, as it seems like Radicle is “down” or “broken”.

To guard against this in the future, we could to some rudimentary smoke testing by making sure different versions of Radicle components on different nodes can communicate with each other, at least at some minimal level.

Specifically, I’m thinking of a process like this:

• pick a suitable host, such as a virtual machine
• install a chosen set of versions to test
  • first cut, radicle-node and rad only
  • later add radicle-httpd and radicle-explorer, but start as small as possible
  • the set should probably be the two latest releases and whatever new version is being prepared for the release
  • ideally install release binaries from the official download location, using the official installer
• set up a node for each radicle-node version
  • these can run on the same host, as long as they use different ports
  • the test nodes should be isolated from the main Radicle network: there’s no point in replicating test repositories to other nodes
  • the nodes should all connect to each other
• on each node:
  • create a new repository
  • seed it on every other node
  • verify the other nodes actually have the repository by running rad checkout on that node
  • on the origin node, push a change to the repository
  • verify it reaches every other node
  • verify the HTTP API on every other node works and is reachable
• if all of this succeeds, good; otherwise report what’s wrong

This would be launched with a command something like:

radicle-smoke --use radicle-node=1.0 --use radicle-node-1.1 --use radicle-node=.

This would set up a test network with three nodes: versions 1.0 and 1.1, as well as the version whose source is in the current working directory. The first two would be installed the Radicle installer (--version=1.1), the last one with cargo install --page=..

I propose to implement this, and do it in such a way the administration of the host where this is run is out of scope for the testing. This allows us to run the tests on our development work stations, in a dedicated VM, or in a container.

I’m not experienced with testing web UIs, which may be needed for testing radicle-explorer, but when we get to that point, I’m sure others can help.