report_generator
Builds the PDF report of one stored scan, from the artifacts that scan put in the bucket.
Installing
The package lives in report_generator/, beside the pyproject.toml that builds it. Install it
from the repository root — the directory holding this file, not the package directory:
pip install -e . # add [dev] for the test and lint tooling
Installing from inside report_generator/ does not work: packages.find looks for the package as a
subdirectory, so it finds nothing, and the report_generator console script is then installed
pointing at a module that was never put on the path.
Running
report_generator --scan_id <uuid> [--path <file-or-directory>] --config config.json
Without --path the report is written into reports/. The S3 secret comes from the environment:
PERSISTENT_STORAGE_STORAGE_CONFIG_S3_SECRET_ACCESS_KEY (and the database password from
PERSISTENT_STORAGE_STORAGE_CONFIG_DATABASE_PASSWORD). The name is not this tool's to choose: it is
derived from the config path inject_secrets fills in under — persistent_storage.storage_config —
by the one naming convention in cc_common.config.env_var_name, so it stays whatever the monorepo
calls it. A .env beside the pyproject.toml is read too. --config is resolved relative to the
working directory.
How it is put together
Four steps, each its own module, so that the one that will change is the one that is separate:
| Module | Does |
|---|---|
report_generator/sources.py |
Reads the scan and its files out of the store. The only module that knows which store. |
report_generator/rendering.py |
Decides which stored file feeds which part of the report, and parses it. |
report_generator/context.py |
Maps the parsed artifacts onto the page, and writes <xxx> where they say nothing. |
report_generator/layout.py |
Draws template/report.html.j2 through WeasyPrint. |
No parsing of its own. SARIF and the vulnerability report are read by the summarisers in
cipherverifier.core.services, which are written to be free of Django and of any store; the CVSS
vectors by persistent_storage.utils.cvss_utils.
What it cannot fill in
A field the artifacts do not answer is written as <xxx>, never guessed or quietly dropped. Today
that is the report and artifact ids, the manufacturer, the analysed lines of code, the language
breakdown, and the tools and profile of each pipeline stage — the metadata records that a stage ran
and how long it took, not what it ran.
A measured zero survives: 0 filtered is a reading, and a blank is a gap.
Moving it to archon
sources.collect() is the whole of it. It returns files by name and the rows a report lists;
everything after that point works on bytes and cannot tell which store handed them over. Pointing it
at archon's tables is a change to that function and to nothing else.
Its relationship to the monorepo
This repository is the split of src/persistent_storage/report_generator/ out of the monorepo. The
package names nothing about itself — every import inside it is relative — so the split was a move
and not a rewrite: the sources sit unchanged under report_generator/, with pyproject.toml beside
them rather than inside them.
report_generator/vendor/ supplies everything the package borrows from the monorepo — cc_common,
persistent_storage.shared, and the two summarisers. vendor/__init__.py puts the copies on the
path when the real packages are not installed, and stays out of the way when they are, so the same
sources run in both places. See report_generator/vendor/README.md.
Still to come across: tests/persistent_storage/test_report_*.py from the monorepo, with
persistent_storage.report_generator changed to report_generator in their imports. Until they
land, testpaths = ["tests"] in pyproject.toml points at a directory that does not exist yet.
On the monorepo side, tests/persistent_storage/test_report_vendor.py performs this same split into
a temporary directory on every run and renders a report from it, so the copies cannot quietly stop
being sufficient.
Native dependencies
WeasyPrint needs Pango and cairo, which pip does not install:
brew install pango # macOS
apt-get install libpango-1.0-0 libpangoft2-1.0-0 # Debian/Ubuntu