## Architecture

See the `doc` directory for architecture documentation.

## Binaries

The crate contains several binaries:

* `ci-broker` --- the actual CI broker

  - this is probably the only one you need to care about

* `broker-messages`  --- helper program to print sample messages

  between the broker and adapters

* `filter-events` --- a helper program to see what events a CI broker

  config allows

* `list_runs` --- list all CI runs in a broker database

* `pagegen` -- helper program to produce sample report pages

  - mostly only useful for testing changes to the page generating code

## Packaging

There is simple, simplistic, rudimentary, personal-use-only packaging

for Debian in the `debian` directory, used by Lars to build packages

for his own use.

## Running tests

To run the test suite for the CI broker:

~~~sh

cargo test

~~~

(As usual, for a Rust program.)

.SUFFIXES: .uml .svg .dot .pik .md .html

	pandoc -V date="Version: $$(git describe --long --dirty --all)" --toc --number-sections --standalone --self-contained $< -o $@

.dot.svg:

	dot -Tsvg $< > $@.tmp

	mv $@.tmp $@

architecture.html: architecture.svg architecture-ext.svg comp.svg comp-ext.svg Makefile

engine -> adapter : web hook?

author: The Radicle Team

[Radicle](https://radicle.xyz/) is a peer-to-peer collaboration system

built on top of the git version control system. Radicle has support

for integrating with continuous integration (CI) systems, using an

architecture where a "broker" listens to events about changes to

repositories stored in a node, and launching the appropriate "adapter"

for each change, according to its configuration.

This means each node can opt into running CI for what projects and

changes according to the interests of the person whose node it is.

* The delegates for a repository might run CI on all patches to make

  merge decisions with more confidences.

* Someone whose contributing to a project might only care about

  patches they themselves created, and only run CI for those.

* A third party might run CI for projects they use, to know if it's OK

  to deploy to production.

Radicle provides its own, very simple "native CI" solution. It's just

good enough for the Radicle project to use itself. In addition, there

are adapters that integrate with external CI systems.

## Components in native CI

![Components for native CI](comp.svg)

![Sequence diagram for native CI](architecture.svg)

## Components when integrating an external CI system

instance can run anywhere. The adapter talks to the CI instance using

whatever protocol the CI instance supports, such as HTTP.

External CI integration works like this:

* the CI broker listens to node events

  - the broker subscribes to node events via the node control socket,

  appropriate adapter process

  - there are different adapter for different CI implementations

* the adapter communicates with the external CI instance in whatever

  way is suitable for that instance

  - this is usually over HTTP

  - it may involve the CI instance making a web hook request back to

    the adapter

![Components for external CI](comp-ext.svg)

![Sequence diagram for external CI](architecture-ext.svg)

updated`, etc.), to satisfy different workflow needs. (Only a few of

these are actually implemented yet, but the scaffolding to support

more is there.)

Some examples of real-world use-cases:

Run the `broker-messages` binary to get actual examples produced by

code.

~~~{.sh .numberLines}

$ cargo run -q --bin broker-messages

Trigger request:

{"request":"trigger","event_type":"push","repository":{"id":"rad:zwTxygwuz5LDGBq255RA2CbNGrz8","name":"radicle-ci-broker","description":"Radicle CI broker","private":false,"default_branch":"main","delegates":["did:key:z6MkgEMYod7Hxfy9qCvDv5hYHkZ4ciWmLFgfvm3Wn1b2w2FV"]},"pusher":{"id":"did:key:z6MkgEMYod7Hxfy9qCvDv5hYHkZ4ciWmLFgfvm3Wn1b2w2FV","alias":"liw"},"before":"b4fb1e347be7db19f0859717062f94116b5bec9f","after":"b4fb1e347be7db19f0859717062f94116b5bec9f","branch":"patches/8d8232ddcb217fa1402eec4d955e227ef3bb5881","commits":[]}

Triggered response:

{"response":"triggered","run_id":{"id":"any-string-works-as-run-id"}}

Successful response:

{"response":"finished","result":"success"}

Failure response:

{"response":"finished","result":"failure"}

Error response:

{"response":"finished","result":{"error":"error message\nsecond line"}}

~~~

    "branch": "<BRANCH_NAME>",

  - `<BEFORE_COMMIT>` is the commit id of the **parent** of the first

     commit being pushed (i.e. ` <SOME_OTHER_COMMIT_BEING_PUSHED>`),

     (the SHA checksum).

  - `<BEFORE_COMMIT>` is the commit id of the **parent** of the first

    commit being pushed (i.e. ` <SOME_OTHER_COMMIT_BEING_PUSHED>`),

    (the SHA checksum).

that the run id is not repeated as the context makes this clear: the

response comes from the same process, via the same stdout pipe, as the

previous message.

# Report generation

The CI broker has an SQLite database file for persistent storage of

information of the CI runs it triggers. This is used to generate

report pages.

The report pages are HTML, generated from the information in the

database. The broker loads information for all runs when it starts,

from the database, and then pushed information about new runs when

they happens. This avoids the broker having to read everything every

time it updates the report pages.

The report page generation is done in its own thread, separate from

the main thread of the CI broker. This allows the reporting to happens

independently from what the main thread is doing. In particular, it

means the main thread does not need to do anything to trigger reports

from being updated.

When it comes to per-run logs, for external CI these are kept by the

external CI instance and the broker never sees them. For native CI,

the native CI adapter writes them to the report directory, as

`$RUNID/log.html`, and the broker generates report pages that link to

those files.

digraph "" {

   radicle_node [label="Radicle node"];

   broker [label="CI broker"];

   adapter [label="Adapter"];

   engine [label="External CI system"];

   radicle_node -> broker [label="change event"];

   broker -> adapter [label="invoke"];

   adapter -> engine [label="run"];

   engine -> adapter [label="web hook?"];

   adapter -> broker [label="result"];

}

digraph "" {

   radicle_node [label="Radicle node"];

   broker [label="CI broker"];

   native [label="Native CI"];

   commands [label="Shell commands"];

   radicle_node -> broker [label="change event"];

   broker -> native [label="invoke"];

   native -> commands [label="run"];

   native -> broker [label="result"];

}