report_generator

Builds the PDF report of one stored scan, from the artifacts that scan put in the bucket.

Installing

The package lives in report_generator/, beside the pyproject.toml that builds it. Install it from the repository root — the directory holding this file, not the package directory:

pip install -e .            # add [dev] for the test and lint tooling

Installing from inside report_generator/ does not work: packages.find looks for the package as a subdirectory, so it finds nothing, and the report_generator console script is then installed pointing at a module that was never put on the path.

Running

report_generator --scan_id <uuid> [--path <file-or-directory>] --config config.json

Without --path the report is written into reports/. The S3 secret comes from the environment: PERSISTENT_STORAGE_STORAGE_CONFIG_S3_SECRET_ACCESS_KEY (and the database password from PERSISTENT_STORAGE_STORAGE_CONFIG_DATABASE_PASSWORD). The name is not this tool's to choose: it is derived from the config path inject_secrets fills in under — persistent_storage.storage_config — by the one naming convention in cc_common.config.env_var_name, so it stays whatever the monorepo calls it. A .env beside the pyproject.toml is read too. --config is resolved relative to the working directory.

How it is put together

Four steps, each its own module, so that the one that will change is the one that is separate:

Module Does
report_generator/sources.py Reads the scan and its files out of the store. The only module that knows which store.
report_generator/rendering.py Decides which stored file feeds which part of the report, and parses it.
report_generator/context.py Maps the parsed artifacts onto the page, and writes <xxx> where they say nothing.
report_generator/layout.py Draws template/report.html.j2 through WeasyPrint.

No parsing of its own. SARIF and the vulnerability report are read by the summarisers in cipherverifier.core.services, which are written to be free of Django and of any store; the CVSS vectors by persistent_storage.utils.cvss_utils.

What it cannot fill in

A field the artifacts do not answer is written as <xxx>, never guessed or quietly dropped. Today that is the report and artifact ids, the manufacturer, the analysed lines of code, the language breakdown, and the tools and profile of each pipeline stage — the metadata records that a stage ran and how long it took, not what it ran.

A measured zero survives: 0 filtered is a reading, and a blank is a gap.

Moving it to archon

sources.collect() is the whole of it. It returns files by name and the rows a report lists; everything after that point works on bytes and cannot tell which store handed them over. Pointing it at archon's tables is a change to that function and to nothing else.

Its relationship to the monorepo

This repository is the split of src/persistent_storage/report_generator/ out of the monorepo. The package names nothing about itself — every import inside it is relative — so the split was a move and not a rewrite: the sources sit unchanged under report_generator/, with pyproject.toml beside them rather than inside them.

report_generator/vendor/ supplies everything the package borrows from the monorepo — cc_common, persistent_storage.shared, and the two summarisers. vendor/__init__.py puts the copies on the path when the real packages are not installed, and stays out of the way when they are, so the same sources run in both places. See report_generator/vendor/README.md.

Still to come across: tests/persistent_storage/test_report_*.py from the monorepo, with persistent_storage.report_generator changed to report_generator in their imports. Until they land, testpaths = ["tests"] in pyproject.toml points at a directory that does not exist yet.

On the monorepo side, tests/persistent_storage/test_report_vendor.py performs this same split into a temporary directory on every run and renders a report from it, so the copies cannot quietly stop being sufficient.

Native dependencies

WeasyPrint needs Pango and cairo, which pip does not install:

brew install pango          # macOS
apt-get install libpango-1.0-0 libpangoft2-1.0-0    # Debian/Ubuntu
S
Description
Standalone PDF reporting tool in order to generate reports from SAST and SCA artifacts which are stored on the S3 storage, as well as data found in the postgres database. Should later be integrated into archon.
Readme
1 MiB
0 Stars 1 Watchers 1 Forks
Languages
Python 88.6%
CSS 6.8%
Jinja 4.6%