This patch is not yet ready for merging, but as I’m going away for a few days, I’ll post it for review. The changes touch so many parts of the CI broker that it’s been much too much work to make them. I will try to split changes into smaller patches in the future.

The purpose of these changes it to change the way the CI broker deals with changes in the repositories in its local nodes. The node sends “node events”, via its control socket, which basically say “we got a change from this node, in this repo, which changes these refs to point at these Git objects”. This is simple, but its at a level of abstraction that makes it harder than necessary to think about when combined with continuous integration tasks. Those tasks would need things like “a branch was created”, “a patch was updated”, and so on.

Previously, the CI broker would split the node events (each of which can have any number of ref updates) into simpler “broker events” (which have exactly one ref update).

This patch changes the way the CI broker deals with node events. After these changes, the CI broker turns node events into “CI events”, which are meant to more easily used for CI tasks. https://files.liw.fi/radicle/cub-userguide-draft.html has an initial version of a user guide for the CI broker, also included in this patch, which explains the new CI events.

The fundamental change in the implementation is that CI events are constructed earlier in the process. The updated refs are parsed to determine if a change is for a branch of a patch. This is sub-optimal, but not a change from before. The better way would be to look at COBs, but that’s a bigger change than I’m willing to include in this patch.

(A future patch will add parsing of COBs to allow CI events like “a comment was added to a patch”. Those changes will require the new mechanisms in this patch.)

The broker events are dropped from the code by this patch. To use this new version will require that the queue of broker events in the CI broker database is empty: the new CI broke does not process them.

I’ve not yet tested upgrades, but I expect the process go something like this:

• stop cib
• run cib queued to empty the broker event queue
• run cibtool event list to check the queue is empty
• upgrade
• update the cib config to use new CI event filter syntax
• start cib

Existing CI adapters will work as-is with this patch. The CI broker/adapter interface has not changed in this patch.

Don’t upgrade production instances of the CI broker to this patch.

I hope to merge this next week, and then resume weekly releases of the CI broker. This big change has prevented me from making smaller, unrelated changes, to avoid merge conflicts all over the place.

Fixes to make CI broker run on my production instance.