Building on the work that Lorenz Leutgeb did in patch rad:z3trNYnLWS11cJWC6BbxDs5niGo82/patch/19ba16634fbde97156255ed97069d0aa3ab2bf12, the URI scheme is brought to a final RIP document, with full ABNF and Nearley grammars as well as examples, and a couple of scripts to extract and test those grammars against those examples.

Compared to the URL scheme in RIP 3, this RIP brings a subtly different URL form, while also supporting the RIP 3 scheme (now considered legacy). In short, these two forms are supported by this RIP:

• rad://{rid}/{nid} (considered legacy)
• rad://{rad-auth}/{rid}{object-and-query}

(rad-auth is the authority part, which can take the form of {nid}@{host-and-port} or parts thereof)

This works, because the {rid} and the {nid} are distinguishable, as a {nid} always starts with z6Mk, while a {rid} does start with a z, but never with z6Mk, and they have different lengths (a {rid} is always 28 or 29 characters long, which a {nid} is always 48 characters long).

The test scripts are found in tests/0004-uri-scheme

Get rid of the ellipsis and incorrect RID length in the grammar

The test script resorts to a hack to convert the ellipsis to a suitable number of characters for each part that has it

Add docs/rad-uri.abnf

When rewriting the nearley grammar, I started by writing this file, and then translating. I think this is what should end up in the RIP, unless someone has a strong opinion otherwise

Simplify for stricter URI compatible ‘authority’

• Rethink the path= and file= queries

  This changes path= to allow multiple queries, and reserves the file= query to be singular and for “blob” only.

• Put it all together into 0004-uri-scheme.md

  [note: I still need to verify that everything holds, i.e. perform all tests from this document, check that the ABNF and Nearly grammars correspond to each other, and give fragments a thought]

I’m nearly there

Rebase, squash, etc. Basically, clean up this branch

• Proof-reading fixes and typographical adjustments
• Added a word on fragments with the “blob” object
• Added credits and closing thoughts

A few “last” fixes

Updates following recent commentary on https://radicle.zulipchat.com/#narrow/channel/369876-rips/topic/.60rad.3A.60-URI/with/535054321

Fixing a typo

This revision is a rewrite of the desciption of all object types. The git objects are now more strictly git objects and nothing else, and their descriptions are hopefully easier to understand.

Bah, typos…

Correcting the job COB type name, xyz.radicle.job → xyz.radworks.job

See REVIEW Commits

Integrated Fintan’s review commits

Thank you, they all got integrated

A number of updates

This now includes namespaces as a resource, more examples, and the examples are now sprinkled among the diverse topics.

Change the authority to require a NID with a host+port

This is a requirement of Radicle’s node connection protocol

Drop all ideas of references to lists

There is no real agreement whether they should be part of this URI scheme, at least for now, so the easiest path forward it to simply drop that idea.

Major rework

This revision is a major rework of what was before, partially based on conversations in Zulip, and a good deal of conceptual rethinking.

Something to remember is that these URIs are conceptually references to all sorts of resources. Another thing to remember is that humans read URIs, and will interpret them as best they can, not necessarily just pass them to a software.

This rework isn’t quite done yet… the ABNF details are in disarray and need editing. The next revision will have all that updated.

ABNF and nearley mess fixed up!

Following a discussion on zulip, the Generic rad: URI scheme RIP is rewritten in Asciidoc.

I added an example to refer to the git notes for a specific commit

This is an update where I change ‘object’ types to become ‘resource’ types. This is more consistent with URI descriptions, and allows us to mentally separate generic resource references from objects, where the latter has specific semantics in git and Radicle.

Abbreviations for known COB types removed, courtesy @fintohaps

Reminder to myself: in zulip, @fintohaps remarked that asciidoc’s include functionality could be useful. That’s an excellent idea.

The authority component in URIs typically represents a hierarchical naming authority, i.e. DNS-based authority.

Many URI schemes include a hierarchical element for a naming authority so that governance of the name space defined by the remainder of the URI is delegated to that authority (which may, in turn, delegate it further).

• source: https://datatracker.ietf.org/doc/html/rfc3986#section-3.2

The “authority” in Radicle is cryptographic (the identity document and its delegates), not locational. It is downstream of the identity document which maps to a set of delegates. Git and by extension Radicle have many benefits stemming from their use of hashes as immutable pointers that decouple resources from a specific network location.

This may seem like a subtle distinction, but I think it’s worth delibrating. The utility of encoding a host and port into the URI is clear for bootstrapping, but it also risks linkrot, and some coupling between never-changing resources and transient network locations (on a long enough timeline). In other words, it conflates “where I first heard about this” with “who controls this”.

Some ideas for how this could be addressed while keeping the authority as is:

• Specify that the canonical form omits authority: i.e., rad:z3gqcJUoA1n9HaHKufZs5FCSGazv5 is the canonical identifier
• Allow authority as a non-normative hint for bootstrapping, but specify that implementations should not treat two URIs as different resources just because they have different authorities
• Document that RID is the true authority, i.e. the cryptographic identity, not the network

Alternatively, if we decide to use the RID as the authority —which intuitively feels like a better approach— the seed node for bootstrapping could be encoded in a query parameter: rad://z3gqcJUoA1n9HaHKufZs5FCSGazv5/path?seed=z6Mkux1...@seed.radicle.xyz:8776

Small fixes

• Clarified that the path= could appear multiple times, and that the values should be accumulated, thereby refering to multiple paths.
• Subtle change of the git-obj ABNF rule and git_obj nearley rule to make it clear that it currently supports git’s sha1 IDs, using the git-sha1 / git_sha1 sub-rule. This implies that git-obj / git_obj can be extended to support newer hashes by adding new sub-rules for them.

Move the ABNF and Nearley documents to their own file, and include them in the adoc

The files (rad-uri.abnf and rad-uri.ne) were generated by tests/general-uri-scheme/uri-scheme-extractor.sh, and moved to data/general-uri-scheme.

Then, 000X-general-uri-scheme.adoc was modified by removing almost all ABNF and Nearley code and replace them with file inclusions. For the ABNF, [tagged inclusions](see https://docs.asciidoctor.org/asciidoc/latest/directives/include-tagged-regions/) are used, and the corresponding tags were added to rad-uri.abnf.

The test script in tests/general-uri-scheme were adapted so they use data/general-uri-scheme/rad-uri.{abnf,ne} instead of trying to extract them from 000X-general-uri-scheme.adoc.

I finished up by fixing some inconsistencies between duplicates of the ABNF code, as well as in the Nearley spec compared to the ABNF spec.

Move to a new directory structure, where the top directory is the RIP name

Ref: https://radicle.zulipchat.com/#narrow/channel/369876-RIPs/topic/.60rad.3A.60-URI/near/573958689

REVIEW

There are a few easy REVIEW commits:

• Typo fixes
• Links fixes

Some REVIEW commits might be a bit more bikeshedding:

• Preferring “RFC 3986” rather than “URI”
• Code fencing usage

There is a fix to the grammar of NID and RID, noting that they use base58btc, which omits 0,O,I,l from its alphabet. I also came across an issue with rid in the nearley file.

IMPORTANT: I removed the option revision in cob. Not every COB is expected to have a revision, xyz.radicle.issue is already a good example of this.

Then there is the introduction of flake.nix so that I could have an easy way of dropping myself into a development shell with nearley* tools. I also brought in linkspector to make sure links look correct. I then took this a step further and introduced nix flake check’s so that the grammars and links can easily be checked. Since these test examples now live in files, they are now referenced in the tables, and their expected output is is referenced as well.

Incorporated all of @fintohaps’ latest revision, plus some text adjustments

Rewrote the talking point about COB metadata, and made it a proper aside/sidebar

Review

Update

All of @lorenz’ review commits were incorporated.

Following that, I made lots of editorial fixups, but hopefully kept the spirit of the changes that were recently discussed.

• The grammars are updated to reflect current consensus

  The nearly grammar (data/rad-uri.ne) was far behind, still having old rules that were dropped from the ABNF grammar a long time ago. I also copied all the latest commentary additions from the ABNF grammar.

  In both grammars, git related rules are also updated to correspond to the conclusion of the latest discussions (the .adoc changes will follow)

• Converted the section about addressing git resources into a sidebar

  A “sidebar” is the best that asciidoctor offers in terms of remark box.

• Turned some of the TODOs into more sidebars

  Those TODOs were quite nice, actually. They gave food for thought.

• Defined some query parameters that we want support for

• Rewrote the whole section on COBs

• Fixed a bunch of the tests to conform to all the updates

• Fixed a few typos

Changes:

• Update test script to use git rev-parse --show-toplevel to get base directory, and build directory paths from this.
• Remove the non-accepted linkspector changes from flake.nix.
• Introduce wrapper for nearley for dev shell and flake check.

REVIEW