pdf-reporting/README.md

report_generator

Builds the PDF report of one stored scan, from the artifacts that scan put in the bucket.

Installing

The package lives in report_generator/, beside the pyproject.toml that builds it. Install it from the repository root — the directory holding this file, not the package directory:

pip install -e .            # add [dev] for the test and lint tooling

Installing from inside report_generator/ does not work: packages.find looks for the package as a subdirectory, so it finds nothing, and the report_generator console script is then installed pointing at a module that was never put on the path.

Running

Two ways to say where the artifacts are.

Out of the store — Postgres for the rows, the bucket for the files:

report_generator generate --scan_id <uuid> [--path <file-or-directory>] --config config.json
    [--project <name>] [--manufacturer <name>]

Out of a folder — neither Postgres nor the bucket, just the artifacts as the pipeline exported them:

report_generator from-folder --artifacts ./sample [--path <file-or-directory>]
    [--project <name>] [--manufacturer <name>]

--scan_id is optional there: the pipeline writes its own id into the paths it records in metadata.json, so an export names its own run. Pass it for a folder assembled by hand, or one whose metadata is gone.

The inputs are one SARIF file (merged.sarif, with one run per tool) and the vulnerability report (vdr.json); either is enough on its own. metadata.json is optional, and without it the scan and project pages are left out. The report has these parts:

1. the cover and a table of contents,
2. a summary and the most critical issues across all sources,
3. one section per SARIF run, named after its tool (e.g. CipherCell SAST, CipherCell Lynkeus),
4. the dependency vulnerabilities from the VDR,
5. at the end, False Positives and Suppressed Findings: every result Polygraph classified as a false positive or that carries a SARIF suppression (such as the areas Lynkeus ruled out), with the reason.

Every finding is listed; only the summary charts and the most-critical table are capped.

The two produce the same document. Everything after the artifacts are in hand is shared code, and the sample export in sample/ was checked against the report built from the store for the same scan: 17 of 17 pages identical. The tests cover the folder path; comparing the two needs a store and so is not among them.

--scan_id takes either id a run is known by. A run is recorded under two: the pipeline id, which the GUI shows, cipherverifier names its working directories after, and archon's core_scan row is keyed by; and the stored scan id, minted when the artifacts are uploaded, which scans, artifacts and the S3 keys are keyed by. core_scanresult is where they meet — its own id is the stored scan, its scan_id the pipeline — and it is translated in both directions:

• Reading always uses the stored scan id, because that is what the store is keyed by.
• The report always identifies itself by the pipeline id — the file name, the Scan Identifier field and the page footers — because that is the id the GUI shows and so the only one a reader can match against anything else.

So both ids produce the same report, filed under the pipeline id. A run with no pipeline row falls back to being reported under its stored id, which is then the only name it has. Passing a pipeline id used to fail with a bare No row was found when one was required.

Without --path the report is written into reports/. The S3 secret comes from the environment: PERSISTENT_STORAGE_STORAGE_CONFIG_S3_SECRET_ACCESS_KEY (and the database password from PERSISTENT_STORAGE_STORAGE_CONFIG_DATABASE_PASSWORD). The name is not this tool's to choose: it is derived from the config path inject_secrets fills in under — persistent_storage.storage_config — by the one naming convention in cc_common.config.env_var_name, so it stays whatever the monorepo calls it. A .env beside the pyproject.toml is read too. --config is resolved relative to the working directory.

How it is put together

Four steps, each its own module, so that the one that will change is the one that is separate:

Module	Does
report_generator/sources.py	Reads the scan and its files out of the store. The only module that knows which store.
report_generator/rendering.py	Decides which stored file feeds which part of the report, and parses it.
report_generator/context.py	Maps the parsed artifacts onto the page, and writes <xxx> where they say nothing.
report_generator/layout.py	Draws template/report.html.j2 through WeasyPrint.

No parsing of its own. SARIF and the vulnerability report are read by the summarisers in cipherverifier.core.services, which are written to be free of Django and of any store; the CVSS vectors by persistent_storage.utils.cvss_utils.

What it cannot fill in

A field the artifacts do not answer is written as <xxx>, never guessed or quietly dropped. Today that is the manufacturer (unless given with --manufacturer), the analysed lines of code, the language breakdown, and the tools and profile of each pipeline stage — the metadata records that a stage ran and how long it took, not what it ran.

A measured zero survives: 0 filtered is a reading, and a blank is a gap.

Moving it to archon

sources.collect() is the whole of it. It returns files by name and the rows a report lists; everything after that point works on bytes and cannot tell which store handed them over. Pointing it at archon's tables is a change to that function and to nothing else.

Its relationship to the monorepo

This repository is the split of src/persistent_storage/report_generator/ out of the monorepo. The package names nothing about itself — every import inside it is relative — so the split was a move and not a rewrite: the sources sit unchanged under report_generator/, with pyproject.toml beside them rather than inside them.

report_generator/vendor/ supplies everything the package borrows from the monorepo — cc_common, persistent_storage.shared, and the two summarisers. vendor/__init__.py puts the copies on the path when the real packages are not installed, and stays out of the way when they are, so the same sources run in both places. See report_generator/vendor/README.md.

Still to come across: tests/persistent_storage/test_report_*.py from the monorepo, with persistent_storage.report_generator changed to report_generator in their imports. Until they land, testpaths = ["tests"] in pyproject.toml points at a directory that does not exist yet.

On the monorepo side, tests/persistent_storage/test_report_vendor.py performs this same split into a temporary directory on every run and renders a report from it, so the copies cannot quietly stop being sufficient.

Native dependencies

WeasyPrint needs Pango and cairo, which pip does not install:

brew install pango          # macOS
apt-get install libpango-1.0-0 libpangoft2-1.0-0    # Debian/Ubuntu