doc/messages.md

tempfile = { version = "3.10.1" }

.SUFFIXES: .uml .svg .dot .pik .md .html .subplot

.subplot.html:

	subplot docgen --date "Version: $$(git describe --long --dirty)" $< -o $@

architecture.html: architecture.subplot architecture.md Makefile messages.md

messages.md: messages.sh messages.txt

	./messages.sh > messages.md

[Radicle]: https://radicle.xyz/

[Git]: https://git-scm.com/

[CI]: https://en.wikipedia.org/wiki/Continuous_integration

[Radicle] is a peer-to-peer collaboration system

built on top of the [Git] version control system. Radicle has support

for integrating with continuous integration ([CI]) systems, using an

* Someone else, who is contributing to a project, might only care about

  to deploy to their production systems.

## Goal of Radicle CI

Context: The user is a software developer working on a project that

uses Radicle for version control. The project has an automated test

suite, and in-repository configuration for how to build the project

and run the test suite, in a format suitable for the CI engine being

used.

In the long run, the goal for CI in Radicle is "anything that makes it

easier, more fun, and faster to produce working software", but that's

not a concrete goal.

At this stage in the development of Radicle CI has two concrete goals:

* When I create a patch to propose a change, I am automatically told

  if the project branch with my committed changes fails to build or

  pass its test suite. I can also manually check what the status of

  that process ("CI run") is, and find out what the build log is, to

  investigate any problems.

  - This is "build and test the patch branch".

* When a project delegate merges my patch, both they and I are

  automatically told if the merge fails due to a merge conflict, or

  if, after the merge the project no longer builds or its test suite

  fails.

  - This is "build and test the master branch after the merge". This

    is useful, because sometimes a merged change breaks the build or

    the test suite, even when there are no merge conflicts.

It is not yet clear how notifications will work.

~~~dot

digraph "" {

   radicle_node [label="Radicle node"];

   broker [label="CI broker"];

   adapter [label="Adapter"];

   engine [label="External CI system"];

   radicle_node -> broker [label="change event"];

   broker -> adapter [label="invoke"];

   adapter -> engine [label="run"];

   engine -> adapter [label="web hook?"];

   adapter -> broker [label="result"];

}

~~~

  - a Git ref is updated

  - there is a different adapter for each different CI implementation

~~~plantuml

@startuml

node -> broker : RefsFetched event

broker -> adapter : spawn

broker -> adapter : send request

adapter-> engine : trigger run

engine -> worker : start run

note over worker : perform the run

engine -> adapter : response with run id

adapter -> broker : response: run id

worker -> engine : run finished

engine -> adapter : web hook?

adapter -> broker : response: result

@enduml

~~~

## Components in native CI

CI support in Radicle consists of several components. For native CI

they are:

* the Radicle node

* the CI broker

* the native CI executable

These all run on the same host.

~~~dot

digraph "" {

   radicle_node [label="Radicle node"];

   broker [label="CI broker"];

   native [label="Native CI"];

   commands [label="Shell commands \n from .radicle/native.yaml"];

   radicle_node -> broker [label="change event"];

   broker -> native [label="invoke"];

   native -> commands [label="run"];

   native -> broker [label="result"];

}

~~~

~~~plantuml

@startuml

node -> broker : RefsFetched event

broker -> adapter : spawn

broker -> adapter : send request

adapter -> broker : response: run id

note over adapter : perform the run

adapter -> broker : response: result

@enduml

~~~

report pages, among other things.

means the report generation happens even while the main thread is busy

running an adapter.

When it comes to per-run logs, the adapter can include a URL to one in

the first response message. The URL will be included as a link in the

report HTML.

title: "Radicle CI broker"

subtitle: Architecture

authors:

  - The Radicle Project

markdowns:

  - architecture.md

  - messages.md

classes:

  - json

#!/bin/bash

set -euo pipefail

message() {

	echo "~~~json"

	cargo run -q --bin cibtool -- message "$@" | jq .

	echo "~~~"

}

while read -r line; do

	case "$line" in

	@PUSH)

		message --kind=push

		;;

	@PATCH)

		message --kind=patch

		;;

	@TRIGGERED)

		message --kind=triggered

		;;

	@TRIGGEREDURL)

		message --kind=triggered --info-url=https://ci.example.com/runid/log.html

		;;

	@SUCCESS)

		message --kind=success

		;;

	@FAILURE)

		message --kind=failure

		;;

	@*)

		echo "Unknown directive: $line" 1>&2

		a exit 1

		;;

	*)

		echo "$line"

		;;

	esac

done <messages.txt

# Sample messages to and from CI adapter

Note: the JSON objects below are formatted on multiple lines to make

them easier to read. The actual wire format is one line per message.

## Trigger on branch create or update

@PUSH

## Trigger on patch create or update

@PATCH

## Response when run has started

@TRIGGERED

## Response when run has started, with info URL

@TRIGGEREDURL

## Response on successful finish

@SUCCESS

## Response on failure finish

@FAILURE

    #[clap(hide = true)]

    Message(cibtoolcmd::MessageCmd),

            Self::Message(x) => x.run(args),

    #[error(transparent)]

    Message(#[from] cibtoolcmd::MessageError),

use clap::ValueEnum;

use radicle_ci_broker::msg::{EventType, Request, RequestBuilder, Response, RunId, RunResult};

use radicle_ci_broker::test::{MockNode, TestResult};

use radicle::crypto::ssh::Keystore;

use radicle::crypto::test::signer::MockSigner;

use radicle::crypto::Signer;

use radicle::git::RefString;

use radicle::patch::{MergeTarget, Patches};

use radicle::prelude::Did;

use radicle::profile::{Config, Home};

use radicle::storage::ReadRepository;

use radicle::test::setup::Node;

use radicle::Profile;

use super::*;

/// Output sample adapter messages.

#[derive(Parser)]

pub struct MessageCmd {

    /// What kind of message should be output?

    #[clap(long)]

    #[arg(value_enum)]

    kind: MessageKind,

    /// Info URL to include in a `triggered` message.

    #[clap(long)]

    info_url: Option<String>,

}

#[derive(Clone, Debug, Eq, PartialEq, ValueEnum)]

enum MessageKind {

    Push,

    Patch,

    Triggered,

    Success,

    Failure,

}

impl Leaf for MessageCmd {

    fn run(&self, _args: &Args) -> Result<(), CibToolError> {

        let msg = match self.kind {

            MessageKind::Push => request_json(push()?)?,

            MessageKind::Patch => request_json(patch()?)?,

            MessageKind::Triggered => response_json(triggered(&self.info_url)?)?,

            MessageKind::Success => response_json(success()?)?,

            MessageKind::Failure => response_json(failure()?)?,

        };

        println!("{}", msg);

        Ok(())

    }

}

fn request_json(req: Request) -> Result<String, MessageError> {

    req.to_json_pretty().map_err(MessageError::Message)

}

fn response_json(resp: Response) -> Result<String, MessageError> {

    resp.to_json_pretty().map_err(MessageError::Message)

}

fn push() -> Result<Request, MessageError> {

    let mock_node = MockNode::new().map_err(MessageError::Test)?;

    let profile = mock_node.profile().map_err(MessageError::Test)?;

    let project = mock_node.node().project();

    let (_, repo_head) = project.repo.head().map_err(MessageError::Repo)?;

    let cmt =

        radicle::test::fixtures::commit("my test commit", &[repo_head.into()], &project.backend);

    let ci_event = CiEvent::V1(CiEventV1::BranchCreated {

        from_node: *profile.id(),

        repo: project.id,

        branch: RefString::try_from(

            "refs/namespaces/$nid/refs/heads/master".replace("$nid", &profile.id().to_string()),

        )

        .map_err(MessageError::Git)?,

        tip: cmt,

    });

    let req = RequestBuilder::default()

        .profile(&profile)

        .ci_event(&ci_event)

        .build_trigger_from_ci_event()

        .map_err(MessageError::Message)?;

    Ok(req)

}

fn patch() -> Result<Request, MessageError> {

    let mock_node = MockNode::new()?;

    let profile = mock_node.profile()?;

    let project = mock_node.node().project();

    let (_, repo_head) = project.repo.head()?;

    let cmt =

        radicle::test::fixtures::commit("my test commit", &[repo_head.into()], &project.backend);

    let node = mock_node.node();

    let mut patches = Patches::open(&project.repo)?;

    let mut cache = radicle::cob::cache::NoCache;

    let patch_cob = patches

        .create(

            "my patch title",

            "my patch description",

            MergeTarget::Delegates,

            repo_head,

            cmt,

            &[],

            &mut cache,

            &node.signer,

        )

        .map_err(MessageError::Patch)?;

    let ci_event = CiEvent::V1(CiEventV1::PatchCreated {

        from_node: *profile.id(),

        repo: project.id,

        patch: *patch_cob.id(),

        new_tip: cmt,

    });

    let req = RequestBuilder::default()

        .profile(&profile)

        .ci_event(&ci_event)

        .build_trigger_from_ci_event()?;

    Ok(req)

}

fn triggered(info_url: &Option<String>) -> Result<Response, MessageError> {

    let run_id = RunId::from("xyzzy");

    if let Some(url) = info_url {

        Ok(Response::triggered_with_url(run_id, url))

    } else {

        Ok(Response::triggered(run_id))

    }

}

fn success() -> Result<Response, MessageError> {

    Ok(Response::finished(RunResult::Success))

}

fn failure() -> Result<Response, MessageError> {

    Ok(Response::finished(RunResult::Failure))

}

#[derive(Debug, thiserror::Error)]

pub enum MessageError {

    #[error(transparent)]

    Test(#[from] Box<dyn std::error::Error>),

    #[error(transparent)]

    Repo(#[from] radicle::storage::RepositoryError),

    #[error(transparent)]

    Git(#[from] radicle_git_ext::ref_format::Error),

    #[error(transparent)]

    Message(#[from] radicle_ci_broker::msg::MessageError),

    #[error(transparent)]

    Patch(#[from] radicle::patch::Error),

}

mod message;

pub use message::*;

    /// Serialize the request as a pretty JSON, including the newline.

    /// This is meant for the broker to use.

    pub fn to_json_pretty(&self) -> Result<String, MessageError> {

        serde_json::to_string_pretty(&self).map_err(MessageError::SerializeRequest)

    }

    /// Serialize the response as a pretty JSON, including the newline.

    /// This is meant for the broker to use.

    pub fn to_json_pretty(&self) -> Result<String, MessageError> {

        serde_json::to_string_pretty(&self).map_err(MessageError::SerializeResponse)

    }