RAP Maturity Framework

NHS
RAP
Lets make nhspy-plotthedots fully RAP complient
Author

Craig R Shenton

Published

January 23, 2023

The ‘Levels of RAP’ Maturity Framework

We are going to be developing nhspy-plotthedots in a Reproducible Analytical Pipeline (RAP) way by following the maturity framework developed by NHS Digital RAP community.

There are three levels to RAP:

  1. Baseline - RAP fundamentals offering resilience against future change.
  2. Silver - Implementing best practice by following good analytical and software engineering standards.
  3. Gold - Analysis as a product to further elevate your analytical work and enhance its reusability to the public.

Baseline RAP - getting the fundamentals right

In order for a publication to be considered a reproducible analytical pipeline, it must at least meet all of the requirements of Baseline RAP:

  • Data produced by code in an open-source language (e.g., Python, R, SQL).
  • Code is version controlled (i.e., Git & GitHub).
  • Repository includes a README.md file (or equivalent) that clearly details steps a user must follow to reproduce the code.
  • Code has been peer reviewed (i.e., use PRs and code reviews)
  • Code is published in the open and linked to & from accompanying publication (if relevant).

Silver RAP - implementing best practice

Meeting all of the above requirements, plus:

  • Outputs are produced by code with minimal manual intervention.
  • Code is well-documented including user guidance, explanation of code structure & methodology and docstrings for functions.
  • Code is well-organised following standard directory format.
  • Reusable functions and/or classes are used where appropriate.
  • Code adheres to agreed coding standards (e.g., PEP8).
  • Pipeline includes a testing framework (unit tests, back tests).
  • Repository includes package dependency information.
  • Logs are automatically recorded by the pipeline to ensure outputs are as expected.
  • Data is handled and output in a Tidy data format.

Gold RAP - analysis as a product

Meeting all of the above requirements, plus:

  • Code is fully packaged.
  • Repository automatically runs tests etc. via CI/CD or a different integration/deployment tool e.g. GitHub Actions.
  • Process runs based on event-based triggers (e.g., new data in database) or on a schedule.
  • Changes to the RAP are clearly signposted. E.g. a changelog in the package, releases etc. (See gov.uk info on Semantic Versioning)