Setup of the build/deployment workflow of this documentation

Setup of the build/deployment workflow of this documentation#

Deployment is done on two platforms: github-pages and readthedocs.

CI builds run on via github Actions, for build time reasons: because of the doxygen documentation included in sphinx, build takes a long time, around 20 minutes, and cannot be done directly on readthedocs. The doc is then deployed to gh pages, because that is easy.

But we also want to take advantage of readthedocs feature of hosting multiple versions of the doc, for anterior versions of the code. To enable that, the github workflow sends a HTTP request to ReadTheDocs, which triggers a custom build that will only download a github artifact containing the generated html for the current commit.

This whole procedure and the required configuration steps are detailed below for reproducibility and maintenance.

Github workflow#

The workflow file is located at .github/workflows/sphinx.yml.

It is configured to trigger on pushes to master branch, and to a special branch test_pages that should be used when working on fixes or modification to the CI procedure. It can also be launched manually from the GitHub Actions page, but that should not be needed (unless maybe when making changes only on the readthedocs side of configuration).

In the workflow, the first steps are relatively standard: checkout, setup python, configure pages, and build the documentation.

After the build, two artifacts are uploaded, both containing the html directory generated by sphinx. The first one is a special version for github-pages, and the second one contains the html directory as a zip archive. This last one must is named html followed by the commit sha ID. This is needed to recognize the artifact during the readthedocs procedure, to dowload the correct artifact.

Then, a last step named Trigger RTDs build will send an HTTP request to readthedocs using curl, that will trigger the build on the side of ReadTheDocs.

On readthedocs side, a generic API webhook must be configured in readthedocs integrations (generic url is https://app.readthedocs.org/dashboard/<project_name>/integrations/)

The form of the HTTP request is documented at RTD generic API integration. It uses three parameters:

  • branches must contain the name of the branches to trigger build. We use ${{ github.ref_name }} in the workflow to use the current branch (will be either master or test_pages)

  • token, which is a secret token generated by readthedocs, obtainable in the integrations section of project settings.

  • the url of the webhook, also obtainable in the integrations section of project settings.

The last two parameters (token and webhook url) are stored as secrets of the github repository, making them accessible in the workflow as environment variables. These must be configured here. They are named RTDS_WEBHOOK_TOKEN and RTDS_WEBHOOK_URL.

RTD custom build procedure#

On the readthedocs side, the procedure is kinda special, as there is nothing to build. We only want to dowload the artifact from github pages.

Because of that, we want to override the entire build procedure. See RTD custom build for the documentation.

Thus, in the .readthedocs.yaml, the build procedure is completely overriden to only launch the script rtd/download_github_artifact.py, which will find the correct artifact, download it and extract it in the correct location expected by readthedocs.

This python script uses the GitHub API to get data about available artifacts and find the correct one. This API is documented here.

For the script to work, it must have access to the public github api for the repository. This is the case even if the repo is public (I think). This is done using personal acess tokens. You must go to this link configure an access token with permission Public repositories (gives read-only access to public repos) and store the generated token as an environment variable named GITHUB_TOKEN on readthedocs (here).

The script searches for an artifact whose name is html followed by the current commit SHA ID (as we uploaded earlier during the github workflow). It should be possible (and easy) to add other Sphinx build formats (such as pdf) by duplicating this part of the procedure with an artifact whose name starts with pdf for example.

Contents