.\"
.\" FreeBSD pkg - a next generation package for the installation and maintenance
.\" of non-core utilities.
.\"
.\" Redistribution and use in source and binary forms, with or without
.\" modification, are permitted provided that the following conditions
.\" are met:
.\" 1. Redistributions of source code must retain the above copyright
.\"    notice, this list of conditions and the following disclaimer.
.\" 2. Redistributions in binary form must reproduce the above copyright
.\"    notice, this list of conditions and the following disclaimer in the
.\"    documentation and/or other materials provided with the distribution.
.\"
.\"
.\"     @(#)pkg.8
.\"
.Dd January 17, 2021
.Dt PKG-REPO 8
.Os
.Sh NAME
.Nm "pkg repo"
.Nd create a package repository catalogue
.Sh SYNOPSIS
.Nm
.Op Fl hlqs
.Op Fl g Ar groups
.Op Fl m Ar meta-file
.Op Fl o Ar output-dir
.Ao Ar repo-path Ac Op Ar signer-type: Ns Ao Ar keyfile Ac | signing_command: Ao Ar the command Ac
.Pp
.Nm
.Op Cm --{hash,list-files,quiet,symlink}
.Op Cm --groups Ar groups
.Op Cm --meta-file Ar meta-file
.Op Cm --output-dir Ar output-dir
.Ao Ar repo-path Ac Op Ar signer-type: Ns Ao Ar keyfile Ac | signing_command: Ao Ar the command Ac
.Sh DESCRIPTION
.Nm
is used to create a catalogue of the available
packages in a repository.
.Nm
catalogues are necessary for sharing your package repository,
and are intrinsic to the operation of
.Nm "pkg install"
or
.Nm "pkg upgrade" .
.Pp
The current repository format is version 2.
The repository files created by
.Nm
consist of a UCL metadata file and a set of compressed
.Pa .pkg
archives stored at the top level of the repository filesystem.
.Pp
.Pa meta.conf
is a plain-text UCL file that must exist at the apex of the
repository filesystem.
It describes the repository version, compression format, and the
names of the catalogue archives.
See
.Xr pkg-repository 5
for a detailed description of its fields.
.Pp
.Pa packagesite.pkg
is a deprecated compressed archive containing
.Pa packagesite.yaml ,
which lists the metadata for each of the packages in the repository.
Each package manifest is represented as a single-line compact JSON
text, with manifests separated by newlines.
It is still generated for backward compatibility but is superseded by
.Pa data.pkg .
.Pp
.Pa data.pkg
is the primary catalogue archive.
It contains a JSON file with package manifests, package group definitions, and
expired package entries.
See
.Xr pkg-repository 5
for details.
.Pp
.Pa files.pkg
is an optional compressed archive containing
.Pa files ,
a directory-grouped listing of all files present in all packages in the
repository.
It is only generated when the
.Fl l
flag is used.
The file uses a compact line-based text format where paths are grouped by
directory.
When fetched by
.Xr pkg-update 8 ,
this data enables
.Xr pkg-rwhich 8
to look up which remote package provides a given file.
See
.Xr pkg-repository 5
for a full description of the format.
.Pp
The compressed archives may also contain cryptographic signatures
when the signing mechanism of
.Nm
is enabled.
.Pp
Repository users download these files to their local machines, where
they are processed into per-repository sqlite databases for fast
lookup of available packages by programs such as
.Xr pkg-install 8 .
.Pp
To create a package repository catalogue, specify the top-level
directory beneath which all the packages are stored as
.Ar repo-path .
.Nm
will search the filesystem beneath
.Ar repo-path
to find all
.Pa .pkg
files it contains.
Directories starting with
.Sq \&.
or named
.Pa Latest
are not traversed.
.Pp
The repository files will be created in the top-level repository
directory unless relocated by specifying
.Fl o Ar output-dir
or
.Cm --output-dir Ar output-dir .
.Pp
Optionally, the repository catalogue may be cryptographically signed.
This is enabled either by specifying the path to a private key as the
.Ar keyfile
argument or by using an external command.
When a
.Ar keyfile
is being used, it may be prefixed by the signer type.
Currently, this may be one of
.Sy rsa ,
.Sy ecdsa ,
or
.Sy eddsa .
.Sy ecc
is also accepted as an alias of
.Sy eddsa .
Keys for the
.Sy rsa
and
.Sy ecdsa
signers may be generated by OpenSSL or by
.Xr pkg-key 8 .
Keys for the
.Dq eddsa
signer may only be generated by
.Xr pkg-key 8 .
.Pp
If the
.Ar key
is used, a hash of the repository is signed using the provided key.
The
.Sy rsa
signer will sign the SHA256 hash of the repository, while the
.Sy ecdsa
and
.Sy eddsa
signers will sign the BLAKE2 hash of the repository.
The signature is added into the repository catalogue.
The client side should use
.Sy SIGNATURE_TYPE
set to
.Dv PUBKEY
and
.Sy PUBKEY
set to a local path of the public key in its repository configuration file.
.Pp
An external command can be useful to create a signing server to keep the
private key separate from the repository.
The external command is passed the SHA256 of the repository
catalogue on its stdin.
It should output the following format:
.Bd -literal -offset indent
TYPE
signer type here (rsa, ecdsa, eddsa)
SIGNATURE
signature data here
CERT
public key data here
END
.Ed
.Pp
The
.Sy TYPE
field is optional if using
.Sy rsa ,
to remain compatible with external signing commands historically in use.
Note that the
.Sy SIGNATURE
field's data will may require an extra newline after it if the signature is
output in a binary format.
The
.Sy CERT
field may contain binary data, but
.Xr pkg 8
will search the tail of it for the missing
.Sy END
if it runs together.
.Pp
When using an external command, the client's
.Pa pkg.conf
must have
.Sy SIGNATURE_TYPE
set to
.Dv FINGERPRINTS
and
.Sy FINGERPRINTS
set to a directory having a
.Pa trusted/myrepo
containing a fingerprint style representation of the public key:
.Bd -literal -offset indent
function: sha256
fingerprint: \\"sha256_representation_of_the_public_key\\"
.Ed
.Pp
See the
.Sx EXAMPLES
section and
.Xr pkg.conf 5
for more information.
.Pp
Signing the catalogue is strongly recommended.
.Sh OPTIONS
The following options are supported by
.Nm :
.Bl -tag -width quiet
.It Fl g Ar groups , Cm --groups Ar groups
Set the group ownership of the repository files to
.Ar groups .
.It Fl h , Cm --hash
Append a short hash of the package contents to the package filename.
This is the same as setting the
.Ev PKG_REPO_HASH
environment variable.
.It Fl l , Cm --list-files
Generate list of all files in repo as files.pkg archive.
.It Fl m Ar meta-file , Cm --meta-file Ar meta-file
Use the specified file as repository meta file instead of the default settings.
.It Fl o Ar output-dir , Cm --output-dir Ar output-dir
Create the repository in the specified directory instead of the package directory.
.It Fl q , Cm --quiet
Force quiet output.
.It Fl s , Cm --symlink
Create a symlink between the hashed filename and the regular filename.
Only useful in combination with
.Fl h .
This is the same as setting the
.Ev PKG_REPO_SYMLINK
environment variable.
.El
.Sh FILES
See
.Xr pkg.conf 5 .
.Sh ENVIRONMENT
.Bl -tag -width "PKG_REPO_SYMLINK"
.It Ev PKG_REPO_HASH
When set, rename packages with the short hash of contents appended to the
filename.
.It Ev PKG_REPO_SYMLINK
When set, create a symlink between the short hash filename and the regular
filename.
.El
.Sh SEE ALSO
.Xr pkg_checksum 3 ,
.Xr pkg_create 3 ,
.Xr pkg_printf 3 ,
.Xr pkg_repo_create 3 ,
.Xr pkg_repos 3 ,
.Xr pkg-keywords 5 ,
.Xr pkg-lua-script 5 ,
.Xr pkg-repository 5 ,
.Xr pkg-script 5 ,
.Xr pkg-triggers 5 ,
.Xr pkg.conf 5 ,
.Xr pkg 8 ,
.Xr pkg-add 8 ,
.Xr pkg-alias 8 ,
.Xr pkg-annotate 8 ,
.Xr pkg-audit 8 ,
.Xr pkg-autoremove 8 ,
.Xr pkg-check 8 ,
.Xr pkg-checksum 8 ,
.Xr pkg-clean 8 ,
.Xr pkg-config 8 ,
.Xr pkg-create 8 ,
.Xr pkg-delete 8 ,
.Xr pkg-fetch 8 ,
.Xr pkg-help 8 ,
.Xr pkg-info 8 ,
.Xr pkg-install 8 ,
.Xr pkg-key 8 ,
.Xr pkg-lock 8 ,
.Xr pkg-plugins 8 ,
.Xr pkg-query 8 ,
.Xr pkg-register 8 ,
.Xr pkg-repositories 8 ,
.Xr pkg-rquery 8 ,
.Xr pkg-rwhich 8 ,
.Xr pkg-search 8 ,
.Xr pkg-set 8 ,
.Xr pkg-shell 8 ,
.Xr pkg-shlib 8 ,
.Xr pkg-ssh 8 ,
.Xr pkg-stats 8 ,
.Xr pkg-triggers 8 ,
.Xr pkg-unregister 8 ,
.Xr pkg-update 8 ,
.Xr pkg-updating 8 ,
.Xr pkg-upgrade 8 ,
.Xr pkg-version 8 ,
.Xr pkg-which 8
.Sh EXAMPLES
Create an RSA key pair:
.Bd -literal -offset indent
% openssl genrsa -out repo.key 2048
% chmod 0400 repo.key
% openssl rsa -in repo.key -out repo.pub -pubout
.Ed
.Pp
Create a repository and sign it with a local RSA key.
The public key would be shared on all client servers with
.Sy SIGNATURE_TYPE
set to
.Dv PUBKEY
and its path set via
.Sy PUBKEY
setting in the repository configuration file:
.Pp
.Dl pkg repo /usr/ports/packages repo.key
.Pp
Create a repository and sign it with an external command.
The client should set, via the repository configuration file,
.Sy SIGNATURE_TYPE
to
.Dv FINGERPRINTS
and
.Sy FINGERPRINTS
to a path containing a file with the SHA256 of the public key:
.Bd -literal -offset indent
# On signing server:
% cat > sign.sh << EOF
#!/bin/sh
read -t 2 sum
[ -z "$sum" ] && exit 1
echo SIGNATURE
echo -n $sum | /usr/bin/openssl dgst -sign repo.key -sha256 -binary
echo
echo CERT
cat repo.pub
echo END
EOF

# On package server:
% pkg repo /usr/ports/packages signing_command: ssh signing-server sign.sh
# Generate fingerprint for sharing with clients
% sh -c '( echo "function: sha256"; echo "fingerprint: \\"$(sha256 -q repo.pub)\\""; ) > fingerprint'
# The 'fingerprint' file should be distributed to all clients.

# On clients with FINGERPRINTS: /usr/local/etc/pkg/fingerprints/myrepo:
$ mkdir -p /usr/local/etc/pkg/fingerprints/myrepo/trusted
# Add 'fingerprint' into /usr/local/etc/pkg/fingerprints/myrepo/trusted

.Ed
.Pp
The above examples can be repeated with OpenSSL creating a key pair for ECDSA:
.Bd -literal -offset indent
% openssl ecparam -genkey -name secp256k1 -out repo.key -outform DER
% chmod 0400 repo.key
% openssl ec -in repo.key -out repo.pub -pubout -outform DER
.Ed
.Pp
Prefixing the later
.Pa repo.key
reference with
.Dq ecdsa :
.Pp
.Dl pkg repo /usr/ports/packages ecdsa:repo.key
.Pp
The signing server example can be used mostly as-is, but with the following text
placed before the
.Sy SIGNATURE
section in the signing server output:
.Bd -literal -offset indent
TYPE
ecdsa
.Ed
.Pp
For EdDSA instead, create an EdDSA key pair:
.Bd -literal -offset indent
% pkg key --create -t eddsa repo.key > repo.pub
.Ed
.Pp
Create a repository and sign it with a local key.
As with the RSA example above, the public key would be shared on all client
servers with
.Sy SIGNATURE_TYPE
set to
.Dv PUBKEY
and its path set via the
.Sy PUBKEY
option in the repository configuration file:
.Pp
.Dl pkg repo /usr/ports/packages eddsa:repo.key
.Pp
A signing server for EdDSA could be constructed with the
.Fl -sign
option.