Updates to new format #1

Open
fklement wants to merge 17 commits from CipherCell/pdf-reporting:updates-to-new-format into main
pull from: CipherCell/pdf-reporting:updates-to-new-format
17 changed files with 1281 additions and 489 deletions

No files matched your search

+50 -3
View File
@@ -23,6 +23,7 @@ Two ways to say where the artifacts are.
``` ```
report_generator generate --scan_id <uuid> [--path <file-or-directory>] --config config.json report_generator generate --scan_id <uuid> [--path <file-or-directory>] --config config.json
[--project <name>] [--manufacturer <name>] [--version <version>]
``` ```
**Out of a folder** — neither Postgres nor the bucket, just the artifacts as the pipeline exported **Out of a folder** — neither Postgres nor the bucket, just the artifacts as the pipeline exported
@@ -30,12 +31,32 @@ them:
``` ```
report_generator from-folder --artifacts ./sample [--path <file-or-directory>] report_generator from-folder --artifacts ./sample [--path <file-or-directory>]
[--project <name>] [--manufacturer <name>] [--version <version>]
``` ```
`--version` sets the version shown on the cover and the project page. Without it the report uses
`repository_version` from `metadata.json`, else the commit the scan was run on.
`--scan_id` is optional there: the pipeline writes its own id into the paths it records in `--scan_id` is optional there: the pipeline writes its own id into the paths it records in
`metadata.json`, so an export names its own run. Pass it for a folder assembled by hand, or one `metadata.json`, so an export names its own run. Pass it for a folder assembled by hand, or one
whose metadata is gone. whose metadata is gone.
The inputs are one SARIF file (`merged.sarif`, with one run per tool) and the vulnerability report
(`vdr.json`); either is enough on its own. `metadata.json` and `cloc.txt` (cloc's plain
table, for the lines of code and the three to five main languages) are optional; without either the
project page is left out. The report has these parts:
1. the cover and a table of contents,
2. a summary and the most critical issues across all sources,
3. the dependency vulnerabilities from the VDR, as CipherCell SCA,
4. one section per SARIF run, named after its tool: CipherCell Lynkeus, then CipherCell SAST, then
any other tool in the order its run appears,
5. at the end, *False Positives and Suppressed Findings*: every result Polygraph classified as a
false positive or that carries a SARIF suppression (such as the areas Lynkeus ruled out), with
the reason.
Every finding is listed; only the summary charts and the most-critical table are capped.
The two produce the same document. Everything after the artifacts are in hand is shared code, and The two produce the same document. Everything after the artifacts are in hand is shared code, and
the sample export in `sample/` was checked against the report built from the store for the same the sample export in `sample/` was checked against the report built from the store for the same
scan: 17 of 17 pages identical. The tests cover the folder path; comparing the two needs a store scan: 17 of 17 pages identical. The tests cover the folder path; comparing the two needs a store
@@ -64,6 +85,32 @@ by the one naming convention in `cc_common.config.env_var_name`, so it stays wha
calls it. A `.env` beside the `pyproject.toml` is read too. `--config` is resolved relative to the calls it. A `.env` beside the `pyproject.toml` is read too. `--config` is resolved relative to the
working directory. working directory.
## The summary report
A shorter document, written by hand in Markdown (like `SUMMARY.md` beside the scan's artifacts) and
set in the dress of the full report: the same cover, header, footers, table of contents, headings
and tables.
```
report_generator from-markdown --markdown SUMMARY.md [--path <file-or-directory>]
[--project <name>] [--manufacturer <name>] [--scan_id <uuid>]
```
How the Markdown is read (`report_generator/summary.py`):
- The first `#` heading is the title. Written as `<title>: <subject>`, the subject is the project
unless `--project` is given.
- What comes before the first section is the introduction, and goes on the cover.
- Every further `#` or `##` heading is a section: it starts a new page, is listed in the table of
contents and keeps the anchor GitHub gives it, so links such as `[…](#1-static-analysis-sast)`
work in the PDF. `###` and below become the report's subheadings.
- A paragraph that is only bold text (`**Remaining findings**`) is set as a subheading.
- Tables are drawn as the report's grids. A number in a column headed *Severity*, *CVSS* or *Score*
carries its severity colour, and a row starting with `**Total**` is set as a total.
The scan id is optional here; without it the cover leaves it out and the footer names the project.
The file is written as `summary_<scan-id>.pdf`, or `summary_<project>.pdf` without one.
## How it is put together ## How it is put together
Four steps, each its own module, so that the one that will change is the one that is separate: Four steps, each its own module, so that the one that will change is the one that is separate:
@@ -74,6 +121,7 @@ Four steps, each its own module, so that the one that will change is the one tha
| `report_generator/rendering.py` | Decides which stored file feeds which part of the report, and parses it. | | `report_generator/rendering.py` | Decides which stored file feeds which part of the report, and parses it. |
| `report_generator/context.py` | Maps the parsed artifacts onto the page, and writes `<xxx>` where they say nothing. | | `report_generator/context.py` | Maps the parsed artifacts onto the page, and writes `<xxx>` where they say nothing. |
| `report_generator/layout.py` | Draws `template/report.html.j2` through WeasyPrint. | | `report_generator/layout.py` | Draws `template/report.html.j2` through WeasyPrint. |
| `report_generator/summary.py` | Sets a Markdown summary in `template/summary.html.j2`, drawn the same way. |
No parsing of its own. SARIF and the vulnerability report are read by the summarisers in No parsing of its own. SARIF and the vulnerability report are read by the summarisers in
`cipherverifier.core.services`, which are written to be free of Django and of any store; the CVSS `cipherverifier.core.services`, which are written to be free of Django and of any store; the CVSS
@@ -82,9 +130,8 @@ vectors by `persistent_storage.utils.cvss_utils`.
### What it cannot fill in ### What it cannot fill in
A field the artifacts do not answer is written as `<xxx>`, never guessed or quietly dropped. Today A field the artifacts do not answer is written as `<xxx>`, never guessed or quietly dropped. Today
that is the report and artifact ids, the manufacturer, the analysed lines of code, the language that is the manufacturer (unless given with `--manufacturer`), and, when there is no `cloc.txt`, the
breakdown, and the tools and profile of each pipeline stage — the metadata records that a stage ran analysed lines of code and the language breakdown.
and how long it took, not what it ran.
A measured zero survives: `0 filtered` is a reading, and a blank is a gap. A measured zero survives: `0 filtered` is a reading, and a blank is a gap.
+2
View File
@@ -0,0 +1,2 @@
[tools]
python = "3.14"
+2
View File
@@ -40,6 +40,8 @@ dependencies = [
# needs the native Pango and cairo libraries, which pip does not install -- see the README. # needs the native Pango and cairo libraries, which pip does not install -- see the README.
"jinja2>=3.1.6", "jinja2>=3.1.6",
"weasyprint>=69.0", "weasyprint>=69.0",
# The summary report, which is written in Markdown and set in the same template.
"markdown-it-py>=4.0.0",
] ]
[project.optional-dependencies] [project.optional-dependencies]
+7
View File
@@ -5,6 +5,7 @@ from .vendor.cc_common.config import load_environment
from .vendor.persistent_storage.shared import inject_secrets from .vendor.persistent_storage.shared import inject_secrets
from .commands.generate import FolderProperties, GenerateProperties, from_folder, generate from .commands.generate import FolderProperties, GenerateProperties, from_folder, generate
from .commands.summary import SummaryProperties, from_markdown
def run() -> None: def run() -> None:
@@ -25,6 +26,12 @@ def run() -> None:
FolderProperties, FolderProperties,
help="Builds the report from a folder of exported artifacts, without Postgres or the bucket.", help="Builds the report from a folder of exported artifacts, without Postgres or the bucket.",
) )
cli.register_command(
"from-markdown",
from_markdown,
SummaryProperties,
help="Builds a summary report from a Markdown file, in the style of the full report.",
)
cli.run() cli.run()
+90
View File
@@ -0,0 +1,90 @@
"""Reads the project's size and languages out of the table cloc prints.
cloc writes one row per language -- files, blank, comment and code lines -- and a ``SUM:`` row under
them. Only the code lines are reported: that is what "lines of code" means to a reader, and blanks
and comments say nothing about how much there is to analyse.
"""
from __future__ import annotations
import re
from typing import Any
# How many languages the report names at most, and how many it names at least where there are that
# many. Between the two, a language has to carry :data:`MINIMUM_SHARE` of the code to be named.
MOST_LANGUAGES = 5
FEWEST_LANGUAGES = 3
MINIMUM_SHARE = 0.01
# What cloc counts that is not code anyone analyses: prose, and data files. Configuration and build
# files -- YAML, Dockerfiles, TOML -- are scanned like code and so stay in.
NOT_CODE = frozenset(
{
"AsciiDoc",
"CSV",
"JSON",
"JSON5",
"Markdown",
"reStructuredText",
"SVG",
"Text",
"TSV",
}
)
# A row of the table: a name, which may contain spaces, then the four counts.
_ROW = re.compile(r"^(?P<name>\S.*?)\s+(?P<files>\d+)\s+\d+\s+\d+\s+(?P<code>\d+)\s*$")
def parse_cloc(content: bytes | None) -> dict[str, Any] | None:
"""Read cloc's table.
Args:
content: The table as cloc printed it, or nothing where the scan stored none.
Returns:
The ``files`` and ``code`` lines of the whole project, and its ``languages``
with the same two counts each, largest first; ``None`` where there is no
table to read.
"""
if content is None:
return None
languages = []
total = None
for line in content.decode("utf-8", errors="replace").splitlines():
row = _ROW.match(line.strip())
if row is None:
continue
counts = {"files": int(row["files"]), "code": int(row["code"])}
if row["name"] == "SUM:":
total = counts
else:
languages.append({"name": row["name"], **counts})
if not languages:
return None
# cloc leaves the sum out when it counted a single language.
total = total or {
"files": sum(one["files"] for one in languages),
"code": sum(one["code"] for one in languages),
}
return {**total, "languages": sorted(languages, key=lambda one: -one["code"])}
def main_languages(cloc: dict[str, Any]) -> list[dict[str, Any]]:
"""The languages worth naming: the largest few that are code, leaving out the negligible ones."""
code = [one for one in cloc["languages"] if one["name"] not in NOT_CODE]
counted = sum(one["code"] for one in code) or 1
return [
one
for rank, one in enumerate(code[:MOST_LANGUAGES])
if rank < FEWEST_LANGUAGES or one["code"] / counted >= MINIMUM_SHARE
]
+46 -4
View File
@@ -32,11 +32,20 @@ class GenerateProperties(BaseModel):
scan_id: The scan to report on. The store is keyed by it. scan_id: The scan to report on. The store is keyed by it.
path: Where to write the PDF. A directory is written into under the name path: Where to write the PDF. A directory is written into under the name
the renderer chose; leaving it out writes into ``reports/``. the renderer chose; leaving it out writes into ``reports/``.
project: The name of the scanned project, as the report states it.
manufacturer: Who makes the scanned product, as the report states it.
version: The version of the scanned project, as the report states it.
Replaces the commit hash the metadata names when given.
storage_config: The store the scan's artifacts are read from. storage_config: The store the scan's artifacts are read from.
""" """
scan_id: UUID = Field(description="The scan the report should be generated for.") scan_id: UUID = Field(description="The scan the report should be generated for.")
path: Path | None = Field(default=None, description="Where to write the PDF report.") path: Path | None = Field(default=None, description="Where to write the PDF report.")
project: str | None = Field(default=None, description="The name of the scanned project.")
manufacturer: str | None = Field(default=None, description="The manufacturer of the scanned product.")
version: str | None = Field(
default=None, description="The version of the scanned project. Replaces the commit hash when given."
)
storage_config: StorageConfiguration storage_config: StorageConfiguration
@@ -48,6 +57,9 @@ class FolderProperties(BaseModel):
scan_id: The scan to report under. Read out of the artifacts when not scan_id: The scan to report under. Read out of the artifacts when not
given -- an export names its own run. given -- an export names its own run.
path: Where to write the PDF, as above. path: Where to write the PDF, as above.
project: The name of the scanned project, as above.
manufacturer: Who makes the scanned product, as above.
version: The version of the scanned project, as above.
""" """
artifacts: Path = Field(description="Folder of exported artifacts to build the report from.") artifacts: Path = Field(description="Folder of exported artifacts to build the report from.")
@@ -55,6 +67,11 @@ class FolderProperties(BaseModel):
default=None, description="The scan to report under. Read from the artifacts when omitted." default=None, description="The scan to report under. Read from the artifacts when omitted."
) )
path: Path | None = Field(default=None, description="Where to write the PDF report.") path: Path | None = Field(default=None, description="Where to write the PDF report.")
project: str | None = Field(default=None, description="The name of the scanned project.")
manufacturer: str | None = Field(default=None, description="The manufacturer of the scanned product.")
version: str | None = Field(
default=None, description="The version of the scanned project. Replaces the commit hash when given."
)
class GeneratedReport(BaseModel): class GeneratedReport(BaseModel):
@@ -89,7 +106,13 @@ def generate(properties: GenerateProperties) -> GeneratedReport:
PipelineError: The scan cannot be read, is missing an input the report PipelineError: The scan cannot be read, is missing an input the report
needs, or the destination cannot be written. needs, or the destination cannot be written.
""" """
return _report(collect(properties.storage_config, properties.scan_id), properties.path) return _report(
collect(properties.storage_config, properties.scan_id),
properties.path,
project=properties.project,
manufacturer=properties.manufacturer,
version=properties.version,
)
def from_folder(properties: FolderProperties) -> GeneratedReport: def from_folder(properties: FolderProperties) -> GeneratedReport:
@@ -107,10 +130,25 @@ def from_folder(properties: FolderProperties) -> GeneratedReport:
PipelineError: The folder cannot be read, names no scan, is missing an PipelineError: The folder cannot be read, names no scan, is missing an
input the report needs, or the destination cannot be written. input the report needs, or the destination cannot be written.
""" """
return _report(collect_from_directory(properties.artifacts, properties.scan_id), properties.path, write_metrics=True) return _report(
collect_from_directory(properties.artifacts, properties.scan_id),
properties.path,
project=properties.project,
manufacturer=properties.manufacturer,
version=properties.version,
write_metrics=True,
)
def _report(sources: ReportSources, path: Path | None, *, write_metrics: bool = False) -> GeneratedReport: def _report(
sources: ReportSources,
path: Path | None,
*,
project: str | None = None,
manufacturer: str | None = None,
version: str | None = None,
write_metrics: bool = False,
) -> GeneratedReport:
"""Render the collected artifacts and write the result. """Render the collected artifacts and write the result.
Everything after the artifacts are in hand, which is the same whichever of the two collected Everything after the artifacts are in hand, which is the same whichever of the two collected
@@ -119,6 +157,10 @@ def _report(sources: ReportSources, path: Path | None, *, write_metrics: bool =
Args: Args:
sources: The scan and its files. sources: The scan and its files.
path: Where the caller asked for the PDF, if anywhere in particular. path: Where the caller asked for the PDF, if anywhere in particular.
project: The project name the caller supplied, if any.
manufacturer: The manufacturer the caller supplied, if any.
version: The version the caller supplied, if any.
write_metrics: Whether to write ``metrics.json`` beside the PDF.
Returns: Returns:
The report that was written. The report that was written.
@@ -127,7 +169,7 @@ def _report(sources: ReportSources, path: Path | None, *, write_metrics: bool =
PipelineError: The report cannot be built or the destination not written. PipelineError: The report cannot be built or the destination not written.
""" """
metrics = build_metrics(sources) metrics = build_metrics(sources)
name, content = render(sources, metrics) name, content = render(sources, metrics, project=project, manufacturer=manufacturer, version=version)
destination = _destination(path, name) destination = _destination(path, name)
try: try:
+94
View File
@@ -0,0 +1,94 @@
"""The summary report: a Markdown summary, set as a PDF in the style of the full report."""
from pathlib import Path
from uuid import UUID
from pydantic import BaseModel, Field
from ..vendor.cc_common.api import PipelineError, context
from ..vendor.cc_common.utils import print
from ..summary import render_summary
from .generate import _destination
class SummaryProperties(BaseModel):
"""Configuration properties for the from-markdown CLI command.
Attributes:
markdown: The Markdown summary to set.
scan_id: The scan it is about. Named on the cover and in the footers
when given, and left out when not.
path: Where to write the PDF. A directory is written into under the name
the renderer chose; leaving it out writes into ``reports/``.
project: The name of the scanned project. Taken from the summary's title
when not given.
manufacturer: Who makes the scanned product, as the summary states it.
"""
markdown: Path = Field(description="The Markdown summary to build the PDF from.")
scan_id: UUID | None = Field(default=None, description="The scan the summary is about.")
path: Path | None = Field(default=None, description="Where to write the PDF summary.")
project: str | None = Field(
default=None, description="The name of the scanned project. Read from the title when omitted."
)
manufacturer: str | None = Field(default=None, description="The manufacturer of the scanned product.")
class GeneratedSummary(BaseModel):
"""The summary that was written.
Attributes:
source: The Markdown it was set from.
path: The file written.
size: How large it came out, in bytes.
"""
source: Path
path: Path
size: int
def print_as_cc_common_result(self) -> None:
"""Say what was written and where."""
print(f"[bold]Summary from {self.source}[/bold]")
print(f" {self.size / 1024:.0f} kB written to {self.path}")
def from_markdown(properties: SummaryProperties) -> GeneratedSummary:
"""Build the summary report from a Markdown file.
Args:
properties: The Markdown file, what to say about it, and the destination.
Returns:
The summary that was written.
Raises:
PipelineError: The Markdown cannot be read, or the destination cannot be written.
"""
try:
markdown = properties.markdown.read_text(encoding="utf-8")
except (OSError, UnicodeDecodeError) as error:
message = f"Cannot read the summary '{properties.markdown}': {error}"
raise PipelineError(message) from error
name, content = render_summary(
markdown,
scan_id=properties.scan_id,
project=properties.project,
manufacturer=properties.manufacturer,
)
destination = _destination(properties.path, name)
try:
destination.parent.mkdir(parents=True, exist_ok=True)
destination.write_bytes(content)
except OSError as error:
message = f"Cannot write the summary to '{destination}': {error}"
raise PipelineError(message) from error
context.logger.info(f"Summary written to {destination} ✔")
return GeneratedSummary(source=properties.markdown, path=destination, size=len(content))
+322 -258
View File
@@ -17,12 +17,18 @@ inside the web application that happens to import them today.
from __future__ import annotations from __future__ import annotations
import re
from collections import Counter
from datetime import UTC, datetime from datetime import UTC, datetime
from typing import TYPE_CHECKING, Any from typing import TYPE_CHECKING, Any
from .vendor.cipherverifier.core.services.sarif import summarize_sarif from .vendor.cipherverifier.core.services.sarif import _build_finding, _build_report, _rules_by_id
from .vendor.cipherverifier.core.services.vdr import summarize_vdr from .vendor.cipherverifier.core.services.vdr import summarize_vdr
from .cloc import main_languages
from .layout import TITLE
from .severity import finding_severity
if TYPE_CHECKING: if TYPE_CHECKING:
from .vendor.cipherverifier.core.services.sarif import Finding, SarifReport from .vendor.cipherverifier.core.services.sarif import Finding, SarifReport
from .vendor.cipherverifier.core.services.vdr import VdrReport, Vulnerability from .vendor.cipherverifier.core.services.vdr import VdrReport, Vulnerability
@@ -32,32 +38,40 @@ if TYPE_CHECKING:
# What stands in for a field the stored artifacts do not answer. # What stands in for a field the stored artifacts do not answer.
UNKNOWN = "<xxx>" UNKNOWN = "<xxx>"
# How many of each list the report carries. A report is read, not audited; the full set is in the # How many entries the summary lists carry. The sections themselves list every finding.
# artifacts it was built from.
MOST_CRITICAL = 6 MOST_CRITICAL = 6
CARDS_PER_GROUP = 8
BARS = 10 BARS = 10
# The severity a SARIF level is reported under. SARIF names four levels and the report bands five,
# so the mapping is written down rather than inferred from the spelling.
SEVERITY_OF_LEVEL = {
"error": "Critical",
"warning": "High",
"note": "Medium",
"none": "Low",
"info": "Low",
}
SEVERITY_ORDER = ("Critical", "High", "Medium", "Low") SEVERITY_ORDER = ("Critical", "High", "Medium", "Low")
# What the dependency check is called wherever the report names its source.
SCA = "CipherCell SCA"
def build_context(sources: ReportSources, documents: dict[str, Any]) -> dict[str, Any]: # The order the sources are reported in, everywhere: the dependency check first, then the SARIF
# tools in this order. A tool not named here follows them, in the order its run appears.
TOOL_ORDER = ("CipherCell Lynkeus", "CipherCell SAST")
# A full git commit SHA.
_COMMIT = re.compile(r"[0-9a-f]{40}")
def build_context(
sources: ReportSources,
documents: dict[str, Any],
*,
project: str | None = None,
manufacturer: str | None = None,
version: str | None = None,
) -> dict[str, Any]:
"""Build the report context from a scan's parsed artifacts. """Build the report context from a scan's parsed artifacts.
Args: Args:
sources: The scan and its stored files. sources: The scan and its stored files.
documents: The parsed artifacts by role -- ``metadata``, ``findings``, documents: The parsed artifacts by role -- ``metadata``, ``findings``,
``vulnerabilities`` -- each absent where the scan stored none. ``vulnerabilities`` -- each absent where the scan stored none.
project: The project name the caller supplied. Wins over the one in the metadata.
manufacturer: The manufacturer the caller supplied. The artifacts never name one.
version: The version the caller supplied. Wins over the one in the metadata.
Returns: Returns:
The context :func:`layout.render_pdf` The context :func:`layout.render_pdf`
@@ -65,31 +79,84 @@ def build_context(sources: ReportSources, documents: dict[str, Any]) -> dict[str
the template renders as a shorter report. the template renders as a shorter report.
""" """
metadata = documents.get("metadata") or {} metadata = documents.get("metadata") or {}
sast = summarize_sarif(documents["findings"]) if documents.get("findings") else None project = project or (metadata.get("configuration") or {}).get("repository_name")
version = version or _version(metadata)
tools = _tools(documents.get("findings"))
sca = summarize_vdr(documents["vulnerabilities"]) if documents.get("vulnerabilities") else None sca = summarize_vdr(documents["vulnerabilities"]) if documents.get("vulnerabilities") else None
context: dict[str, Any] = { context: dict[str, Any] = {
"cover": _cover(sources, metadata, sast, sca), "cover": _cover(sources, tools, sca, project=project, manufacturer=manufacturer, version=version),
"scan": _scan(sources, metadata), "summary": _summary(tools, sca, documents.get("metrics")),
"project": _project(metadata), "sections": ([_sca_section(sca)] if sca is not None else [])
"summary": _summary(sast, sca, documents.get("metrics")), + [_tool_section(name, report) for name, report in tools],
"sections": [],
} }
critical = _critical(sast, sca) # Without the pipeline's metadata or cloc's count this page would be nothing but gaps.
if metadata or documents.get("cloc"):
context["project"] = _project(project, version, documents.get("cloc"))
critical = _critical(tools, sca)
if critical["table"]["rows"]: if critical["table"]["rows"]:
context["critical"] = critical context["critical"] = critical
if sast is not None: set_aside = _set_aside_section(tools)
context["sections"].append(_sast_section(sast))
if sca is not None: if set_aside is not None:
context["sections"].append(_sca_section(sca)) context["sections"].append(set_aside)
context["contents"] = _contents(context)
return context return context
def _tools(sarif: Any) -> list[tuple[str, SarifReport]]: # noqa: ANN401
"""The SARIF document split by the tool that produced each run, in the order the runs appear."""
if not isinstance(sarif, dict):
return []
runs = [run for run in sarif.get("runs") or [] if isinstance(run, dict)]
tools = [(_tool_name(run), _summarize(run)) for run in runs]
return sorted(tools, key=lambda pair: TOOL_ORDER.index(pair[0]) if pair[0] in TOOL_ORDER else len(TOOL_ORDER))
def _summarize(run: dict[str, Any]) -> SarifReport:
"""Summarise one run, as :func:`summarize_sarif` would, keeping each result's own score.
The summariser reads ``security-severity`` off the rule only. Lynkeus scores each result instead,
and a result's own score is the more specific one, so it takes the rule's place.
"""
rules = _rules_by_id([run])
findings = []
for result in run.get("results") or []:
finding = _build_finding(result, run, rules)
stated = (result.get("properties") or {}).get("security-severity")
if stated is not None:
finding.rule_security_severity = str(stated)
findings.append(finding)
return _build_report(findings, ranked=any(finding.rank is not None for finding in findings))
def _tool_name(run: dict[str, Any]) -> str:
"""What the tool that produced a run calls itself."""
driver = (run.get("tool") or {}).get("driver") or {}
return str(driver.get("name") or driver.get("fullName") or UNKNOWN)
def _contents(context: dict[str, Any]) -> list[dict[str, str]]:
"""The table of contents: every top-level part of the report that is actually in it."""
parts = [context[key] for key in ("project", "summary", "critical") if key in context]
return [{"id": part["id"], "title": part["title"]} for part in [*parts, *context["sections"]]]
# --- the page ------------------------------------------------------------------------------ # --- the page ------------------------------------------------------------------------------
@@ -117,45 +184,44 @@ def _stated(value: Any) -> str: # noqa: ANN401
def _cover( def _cover(
sources: ReportSources, sources: ReportSources,
metadata: dict[str, Any], tools: list[tuple[str, SarifReport]],
sast: SarifReport | None,
sca: VdrReport | None, sca: VdrReport | None,
*,
project: str | None,
manufacturer: str | None,
version: str | None,
) -> dict[str, Any]: ) -> dict[str, Any]:
"""The first page.""" """The first page."""
configuration = metadata.get("configuration") or {} technologies = ([SCA] if sca is not None else []) + [name for name, _ in tools]
technologies = [name for name, present in (("SAST", sast is not None), ("SCA", sca is not None)) if present]
return { return {
"scan_id": str(sources.scan_id), "scan_id": str(sources.scan_id),
"subject": _stated(configuration.get("repository_name")), "subject": _stated(project),
"footer_date": datetime.now(UTC).strftime("%d %B %Y"), "footer_date": datetime.now(UTC).strftime("%d %B %Y"),
"logo": "cc_logo.png", "logo": "cc_logo.png",
"title": "SAST/SCA Report", "title": TITLE,
"identity": [ "identity": [
_fact("Scan Identifier", sources.scan_id, kind="mono"), _fact("Manufacturer", manufacturer),
# Neither exists in the store today: a scan has artifacts, and a report of it is not _fact("Project", project),
# itself recorded as one. _fact("Version", version, kind="mono"),
_fact("Report ID", None, kind="mono"), _fact("Technology", ", ".join(technologies) or None, gap=True),
_fact("Artifact ID", None, kind="mono"),
_fact("Technology", ", ".join(technologies) or None),
_fact("Project", configuration.get("repository_name"), gap=True),
_fact("Version", configuration.get("repository_version")),
_fact("Report Created", datetime.now(UTC).strftime("%A, %B %d, %Y")), _fact("Report Created", datetime.now(UTC).strftime("%A, %B %d, %Y")),
_fact("Manufacturer", None), _fact("Scan Identifier", sources.scan_id, kind="mono"),
], ],
"severity_ledger": _severity_ledger(sast, sca), "severity_ledger": _severity_ledger(tools, sca),
} }
def _severity_ledger(sast: SarifReport | None, sca: VdrReport | None) -> dict[str, Any] | None: def _severity_ledger(tools: list[tuple[str, SarifReport]], sca: VdrReport | None) -> dict[str, Any] | None:
"""How many issues of each severity survived filtering, over both halves of the pipeline.""" """How many issues of each severity survived filtering, over every tool and the dependencies."""
counts = dict.fromkeys(SEVERITY_ORDER, 0) counts = dict.fromkeys(SEVERITY_ORDER, 0)
for finding in _true_findings(sast): for _, report in tools:
counts[SEVERITY_OF_LEVEL.get(finding.level, "Low")] += 1 for finding in _kept(report):
counts[_finding_severity(finding)] += 1
for vulnerability in sca.vulnerabilities if sca else []: for vulnerability in sca.vulnerabilities if sca else []:
counts[_vdr_severity(vulnerability)] = counts.get(_vdr_severity(vulnerability), 0) + 1 counts[_vdr_severity(vulnerability)] += 1
if not any(counts.values()): if not any(counts.values()):
return None return None
@@ -168,84 +234,76 @@ def _severity_ledger(sast: SarifReport | None, sca: VdrReport | None) -> dict[st
return {"title": "True issues by severity", "rows": rows} return {"title": "True issues by severity", "rows": rows}
def _scan(sources: ReportSources, metadata: dict[str, Any]) -> dict[str, Any]: def _version(metadata: dict[str, Any]) -> str | None:
"""What the pipeline did, stage by stage.""" """What version of the project was scanned: the stated one, else the commit the SBOM was uploaded for.
stages = [stage for stage in metadata.get("stages") or [] if isinstance(stage, dict)]
failed = [stage["label"] for stage in stages if stage.get("status") not in ("success", None)]
return { The pipeline leaves ``repository_version`` empty but tags the Dependency-Track upload with the
"title": "CipherCell Scan Information", commit it scanned, which is the most precise version there is.
"overview": [
_fact("Scan started at", _moment(metadata.get("pipeline_start_iso"))),
_fact("Scan duration", _duration(metadata.get("pipeline_duration_seconds"))),
_fact("Scan completed at", _moment(metadata.get("pipeline_end_iso"))),
_fact(
"Pipeline Steps", ", ".join(_stated(stage.get("label") or stage.get("key")) for stage in stages) or None
),
_fact("Errors Detected", ", ".join(failed) if failed else "None" if stages else None),
],
"stages": [_stage(stage, sources) for stage in stages],
}
def _stage(stage: dict[str, Any], sources: ReportSources) -> dict[str, Any]:
"""One stage of the pipeline.
The tools a stage ran and the profile it ran them under are not recorded in the metadata, so
they are left unstated. What is recorded is when it ran, how it ended, and -- indirectly -- what
it produced, which is read off the names of the files the scan stored.
""" """
return {
"title": stage.get("label") or stage.get("key") or UNKNOWN,
"rows": [
_fact("Selected Tools", None),
_fact("Profile", None),
_fact("Duration", _duration(stage.get("duration_seconds"))),
_fact("Created Artifacts", ", ".join(sorted(sources.files)) or None, kind="mono"),
_fact("Errors Detected", "None" if stage.get("status") == "success" else stage.get("status")),
],
}
def _project(metadata: dict[str, Any]) -> dict[str, Any]:
"""What was scanned."""
configuration = metadata.get("configuration") or {} configuration = metadata.get("configuration") or {}
stated = configuration.get("repository_version")
if stated:
return str(stated)
commit = (configuration.get("deptrace_upload") or {}).get("project_version")
return commit if isinstance(commit, str) and _COMMIT.fullmatch(commit) else None
def _project(project: str | None, version: str | None, cloc: dict[str, Any] | None) -> dict[str, Any]:
"""What was scanned, and how much of it there is, as cloc counted it."""
return { return {
"id": "project",
"title": "Project Information", "title": "Project Information",
"overview": [ "overview": [
_fact("Project", configuration.get("repository_name")), _fact("Project", project),
_fact("Version", configuration.get("repository_version")), _fact("Version", version, kind="mono"),
# Nothing in the pipeline counts lines, so nothing here can report them. _fact("Analysed lines of code", f"{cloc['code']:,} in {cloc['files']:,} files" if cloc else None),
_fact("Analysed lines of code", None),
], ],
# Left as one unstated row rather than a table of zeroes: no stage reports a language "languages": {"title": "Main languages", "rows": _languages(cloc)},
# breakdown, and a share invented from the file extensions of the findings would be a
# measurement of where the bugs are, not of what the project is written in.
"languages": {"title": "Languages", "rows": [{"label": UNKNOWN, "share": UNKNOWN}]},
} }
def _summary(sast: SarifReport | None, sca: VdrReport | None, metrics: dict[str, Any] | None) -> dict[str, Any]: def _languages(cloc: dict[str, Any] | None) -> list[dict[str, Any]]:
"""The main languages with their lines of code and share of the whole project.
Left as one unstated row rather than a table of zeroes where cloc did not run: a share invented
from the file extensions of the findings would measure where the bugs are, not what the project
is written in.
"""
if not cloc:
return [{"label": UNKNOWN, "share": UNKNOWN}]
return [
{"label": one["name"], "share": f"{one['code']:,} lines ({one['code'] / cloc['code'] * 100:.1f} %)"}
for one in main_languages(cloc)
]
def _summary(
tools: list[tuple[str, SarifReport]], sca: VdrReport | None, metrics: dict[str, Any] | None
) -> dict[str, Any]:
"""The counts, and where they concentrate.""" """The counts, and where they concentrate."""
true_sast = _true_findings(sast)
sca_total = sca.total if sca else 0 sca_total = sca.total if sca else 0
raw = (sast.total if sast else 0) + sca_total raw = sum(report.total for _, report in tools) + sca_total
filtered = sast.false_positive_count if sast else 0 true_issues = ([f"{sca_total} {SCA}"] if sca else []) + [f"{len(_kept(report))} {name}" for name, report in tools]
summary = { summary = {
"id": "summary",
"title": "Summary", "title": "Summary",
"overview": [ "overview": [
_fact("Raw findings", raw or None), _fact("Raw findings", raw or None),
# Only SAST findings carry a filtering verdict. A vulnerability suppressed in # Only SARIF findings can be set aside. A vulnerability suppressed in Dependency-Track
# Dependency-Track never reaches the report at all, so the two cannot be added up. # never reaches the report at all, so the two cannot be added up.
_fact("Filtered", f"{filtered} (SAST)" if sast else None), _fact(
_fact("True issues", f"{len(true_sast)} SAST, {sca_total} SCA" if sast or sca else None), "Suppressed / False Positives",
# Each half stated only where its artifact was there. A scan that ran no SCA stage ", ".join(f"{len(_set_aside(report))} {name}" for name, report in tools) or None,
# reports its SAST rules and says nothing about components, rather than nothing at all. ),
_fact("Distinct issue types (true)", ", ".join(_issue_types(sast, sca)) or None), _fact("True issues", ", ".join(true_issues) or None),
_fact("Distinct issue types (true)", ", ".join(_issue_types(tools, sca)) or None),
], ],
"distributions": _distributions(true_sast, sca), "distributions": _distributions(tools, sca),
} }
dependencies = metrics.get("dependencies") if isinstance(metrics, dict) else None dependencies = metrics.get("dependencies") if isinstance(metrics, dict) else None
@@ -297,13 +355,11 @@ def _metric_cards(dependencies: dict[str, Any]) -> list[dict[str, Any]]:
}, },
{ {
"title": "Temporal", "title": "Temporal",
"note": "Published dates and vulnerability age are calculated from the advisory timestamps. Time-to-exploit and volume trends require historical exploitation or scan data that this export does not contain.", "note": "Published dates and vulnerability age are calculated from the advisory timestamps.",
"rows": [ "rows": [
_fact("Newest published", _published(published.get("newest"))), _fact("Newest published", _published(published.get("newest"))),
_fact("Mean vulnerability age", _days(published.get("mean_days"))), _fact("Mean vulnerability age", _days(published.get("mean_days"))),
_fact("Median vulnerability age", _days(published.get("median_days"))), _fact("Median vulnerability age", _days(published.get("median_days"))),
_fact("Time to exploit", temporal.get("time_to_exploit_days")),
_fact("CVE volume trends", temporal.get("cve_volume_trends")),
], ],
}, },
{ {
@@ -313,8 +369,6 @@ def _metric_cards(dependencies: dict[str, Any]) -> list[dict[str, Any]]:
_fact("Affected dependencies", surface.get("products")), _fact("Affected dependencies", surface.get("products")),
_fact("Potential remedies", _remedy(upgrades)), _fact("Potential remedies", _remedy(upgrades)),
_fact("CWE coverage", taxonomy.get("coverage")), _fact("CWE coverage", taxonomy.get("coverage")),
_fact("CWE abstraction level", taxonomy.get("abstraction_level")),
_fact("CWE relationships", taxonomy.get("relationships")),
], ],
}, },
] ]
@@ -340,35 +394,21 @@ def _remedy(value: dict[str, Any]) -> str | None:
return None return None
def _issue_types(sast: SarifReport | None, sca: VdrReport | None) -> list[str]: def _issue_types(tools: list[tuple[str, SarifReport]], sca: VdrReport | None) -> list[str]:
"""How many distinct kinds of issue each half of the pipeline found.""" """How many distinct kinds of true issue each tool, and the dependency check, found."""
counted = [] counted = [f"{sca.component_count} {SCA} affected components"] if sca is not None else []
if sast is not None: return counted + [f"{len({one.rule_id for one in _kept(report)})} {name} rules" for name, report in tools]
counted.append(f"{sast.rule_count} SAST rules")
if sca is not None:
counted.append(f"{sca.component_count} affected components")
return counted
def _distributions(findings: list[Finding], sca: VdrReport | None) -> list[dict[str, Any]]: def _distributions(tools: list[tuple[str, SarifReport]], sca: VdrReport | None) -> list[dict[str, Any]]:
"""Where the issues concentrate: by rule for SAST, by package for SCA.""" """Where the issues concentrate: by package for SCA, by rule for each SARIF tool."""
blocks = [] blocks = []
if findings:
blocks.append(
{
"title": f"True SAST issues by rule ({len(findings)})",
**_bars(_group(findings, lambda one: one.rule_id or UNKNOWN, _finding_severity)),
}
)
if sca and sca.vulnerabilities: if sca and sca.vulnerabilities:
blocks.append( blocks.append(
{ {
"title": f"Most affected components, true SCA issues ({sca.component_count})", "title": f"Most affected components, true {SCA} issues ({sca.component_count})",
**_bars( **_bars(
_group( _group(
[(one, name) for one in sca.vulnerabilities for name in one.component_names], [(one, name) for one in sca.vulnerabilities for name in one.component_names],
@@ -380,6 +420,17 @@ def _distributions(findings: list[Finding], sca: VdrReport | None) -> list[dict[
} }
) )
for name, report in tools:
findings = _kept(report)
if findings:
blocks.append(
{
"title": f"True {name} issues by rule ({len(findings)})",
**_bars(_group(findings, lambda one: one.rule_id or UNKNOWN, _finding_severity)),
}
)
return blocks return blocks
@@ -450,139 +501,153 @@ def _caveat(shown: list[Any], grouped: list[Any], largest: int) -> str:
return f"{len(shown)} of {len(grouped)} shown; {scale}" return f"{len(shown)} of {len(grouped)} shown; {scale}"
def _critical(sast: SarifReport | None, sca: VdrReport | None) -> dict[str, Any]: def _critical(tools: list[tuple[str, SarifReport]], sca: VdrReport | None) -> dict[str, Any]:
"""The handful worth reading first, whichever half of the pipeline found them.""" """The handful worth reading first, whichever tool found them."""
rows: list[dict[str, Any]] = [] rows: list[tuple[str, dict[str, Any]]] = []
for finding in sorted(_true_findings(sast), key=lambda one: SEVERITY_ORDER.index(_finding_severity(one))): for vulnerability in sca.vulnerabilities if sca else []:
severity = _vdr_severity(vulnerability)
rows.append( rows.append(
{ (
"cells": [ severity,
"SAST", {
{"value": _finding_severity(finding), "kind": "severity"}, "cells": [
_stated(finding.rule_id), SCA,
{"value": _stated(_where(finding)), "kind": "mono"}, {"value": severity, "kind": "severity"},
_stated(finding.message), _stated(vulnerability.id),
] {"value": _stated(", ".join(vulnerability.component_names)), "kind": "mono"},
} _stated(vulnerability.description),
]
},
)
) )
for vulnerability in sorted( for name, report in tools:
sca.vulnerabilities if sca else [], for finding in _kept(report):
key=lambda one: SEVERITY_ORDER.index(_vdr_severity(one)), severity = _finding_severity(finding)
): rows.append(
rows.append( (
{ severity,
"cells": [ {
"SCA", "cells": [
{"value": _vdr_severity(vulnerability), "kind": "severity"}, name,
_stated(vulnerability.id), {"value": severity, "kind": "severity"},
{"value": _stated(", ".join(vulnerability.component_names)), "kind": "mono"}, _stated(finding.rule_id),
_stated(vulnerability.description), {"value": _stated(_where(finding)), "kind": "mono"},
] _stated(_text(finding.message)),
} ]
) },
)
)
# Sorted is stable, so within a severity the sources keep the report's order, and each tool its own
# -- the rank, where it has one.
rows.sort(key=lambda pair: SEVERITY_ORDER.index(pair[0]))
return { return {
"id": "critical",
"title": "Most Critical Vulnerabilities", "title": "Most Critical Vulnerabilities",
"table": { "table": {
"columns": ["Type", "Severity", "CVE/CWE", "Affected File / Component", "Description"], "columns": ["Source", "Severity", "CVE/CWE", "Affected File / Component", "Description"],
"rows": rows[:MOST_CRITICAL], "rows": [row for _, row in rows[:MOST_CRITICAL]],
}, },
} }
def _sast_section(sast: SarifReport) -> dict[str, Any]: def _tool_section(name: str, report: SarifReport) -> dict[str, Any]:
"""The SAST findings, split into what was kept and what was set aside.""" """Every finding of one tool that was not set aside."""
true_findings = _true_findings(sast) kept = _kept(report)
filtered = [one for one in sast.findings if one.false_positive] group: dict[str, Any] = {"title": f"True Issues ({len(kept)})", "cards": [_finding_card(one) for one in kept]}
groups: list[dict[str, Any]] = []
if true_findings: if not kept:
groups.append( group["note"] = f"Every finding {name} reported is listed under False Positives and Suppressed Findings."
{
"title": f"True Issues ({len(true_findings)})",
"cards": [_finding_card(one) for one in true_findings[:CARDS_PER_GROUP]],
}
)
if filtered: return {"id": _anchor(name), "title": name, "new_page": True, "groups": [group]}
groups.append(
{
"title": f"Filtered Issues ({len(filtered)})",
"note": (
"The following findings were reported by the scanners but assessed as not exploitable in "
"this artifact. Each entry carries the verdict that filtered it."
),
"cards": [_finding_card(one, filtered_out=True) for one in filtered[:CARDS_PER_GROUP]],
"table": _filtered_table(filtered),
}
)
return {"title": "SAST Vulnerabilities", "groups": groups}
def _finding_card(finding: Finding, *, filtered_out: bool = False) -> dict[str, Any]: def _set_aside_section(tools: list[tuple[str, SarifReport]]) -> dict[str, Any] | None:
"""The false positives and the suppressed findings of every tool, at the end of the report."""
grouped = [(name, _set_aside(report)) for name, report in tools]
grouped = [(name, findings) for name, findings in grouped if findings]
if not grouped:
return None
groups: list[dict[str, Any]] = [
{
"title": "Overview",
"note": (
"The following findings were reported but set aside: classified as false positives, or "
"suppressed by the tool that reported them. Each entry carries the reason it was set aside."
),
"table": _set_aside_table(grouped),
}
]
groups += [
{"title": f"{name} ({len(findings)})", "cards": [_finding_card(one, set_aside=True) for one in findings]}
for name, findings in grouped
]
return {
"id": "set-aside",
"title": "False Positives and Suppressed Findings",
"new_page": True,
"groups": groups,
}
def _finding_card(finding: Finding, *, set_aside: bool = False) -> dict[str, Any]:
"""One SARIF finding.""" """One SARIF finding."""
rows = [ rows = []
_row("Severity", _finding_severity(finding), kind="severity"),
_row("Location", _where(finding), kind="mono"), # A finding a tool ruled out carries no severity worth stating.
_row("Description", finding.message), if finding.level != "none":
rows.append(_row("Severity", _finding_severity(finding), kind="severity"))
# Not every finding points at a line: an area an agent ruled out names it in its message instead.
if finding.location is not None:
rows.append(_row("Location", _where(finding), kind="mono"))
rows += [
_row("Description", _text(finding.message)),
_row("Reported By", ", ".join(finding.tools) or None), _row("Reported By", ", ".join(finding.tools) or None),
] ]
if filtered_out: if set_aside:
rows.append( reason = finding.polygraph_reasoning or "; ".join(
_row( one.justification for one in finding.suppressions if one.justification
"Filtering Reason",
finding.polygraph_reasoning,
aside=True,
token=finding.polygraph_verdict,
)
) )
else: rows.append(_row("Verdict", finding.verdict_label or "Suppressed"))
# Polygraph's verdict where it looked at this one, and nothing where it did not: silence is rows.append(_row("Reason", reason, aside=True))
# not a clean bill of health, and must not be written as one. elif finding.verdict_label:
rows.append(_row("Evidence", finding.verdict_label or None)) rows.append(_row("Evidence", finding.verdict_label))
return {"title": f"{_stated(finding.rule_id)} — {_stated(finding.rule_name)}", "rows": rows} return {"title": f"{_stated(finding.rule_id)} — {_stated(finding.rule_name)}", "rows": rows}
def _filtered_table(findings: list[Finding]) -> dict[str, Any]: def _set_aside_table(grouped: list[tuple[str, list[Finding]]]) -> dict[str, Any]:
"""How many of each rule were filtered, and on what grounds.""" """How many findings of each tool and rule were set aside."""
grouped: dict[str, list[Finding]] = {} rows: list[dict[str, Any]] = []
for finding in findings: for name, findings in grouped:
grouped.setdefault(finding.rule_id or UNKNOWN, []).append(finding) counts = Counter(one.rule_id or UNKNOWN for one in findings)
rows += [{"cells": [name, rule, count]} for rule, count in counts.most_common()]
rows: list[dict[str, Any]] = [ rows.append({"cells": ["Total", "", sum(len(findings) for _, findings in grouped)], "total": True})
{
"cells": [
rule,
len(group),
{"value": _stated(group[0].polygraph_verdict), "kind": "mono"},
]
}
for rule, group in sorted(grouped.items(), key=lambda pair: -len(pair[1]))
]
rows.append({"cells": ["Total", len(findings), ""], "total": True})
return { return {"columns": ["Tool", "Rule", "Set aside"], "rows": rows}
"title": f"Filtered SAST issues by rule ({len(findings)})",
"columns": ["Rule", "Filtered", "Dominant verdict"],
"rows": rows,
}
def _sca_section(sca: VdrReport) -> dict[str, Any]: def _sca_section(sca: VdrReport) -> dict[str, Any]:
"""The dependency findings.""" """The dependency findings."""
return { return {
"title": "SBOM / SCA Vulnerabilities", "id": "sca",
"title": f"{SCA} — Dependency Vulnerabilities",
"new_page": True, "new_page": True,
"groups": [ "groups": [
{ {
"title": f"True Issues by CVEs ({sca.total})", "title": f"True Issues by CVEs ({sca.total})",
"cards": [_vulnerability_card(one) for one in sca.vulnerabilities[:CARDS_PER_GROUP]], "cards": [_vulnerability_card(one) for one in sca.vulnerabilities],
} }
], ],
} }
@@ -622,7 +687,6 @@ def _vulnerability_card(vulnerability: Vulnerability) -> dict[str, Any]:
kind="mono", kind="mono",
), ),
_row("Known for", f"{metrics.published_days_ago} days" if metrics.published_days_ago else None), _row("Known for", f"{metrics.published_days_ago} days" if metrics.published_days_ago else None),
_row("Recommendation", vulnerability.recommendation),
], ],
} }
@@ -630,14 +694,34 @@ def _vulnerability_card(vulnerability: Vulnerability) -> dict[str, Any]:
# --- readings ------------------------------------------------------------------------------ # --- readings ------------------------------------------------------------------------------
def _true_findings(sast: SarifReport | None) -> list[Finding]: def _is_set_aside(finding: Finding) -> bool:
"""Whether a finding was classified as a false positive or suppressed by the tool."""
return finding.false_positive or bool(finding.suppressions)
def _kept(report: SarifReport) -> list[Finding]:
"""The findings that survived filtering.""" """The findings that survived filtering."""
return [one for one in sast.findings if not one.false_positive] if sast else [] return [one for one in report.findings if not _is_set_aside(one)]
def _set_aside(report: SarifReport) -> list[Finding]:
"""The findings that were filtered out or suppressed."""
return [one for one in report.findings if _is_set_aside(one)]
def _text(message: str) -> str:
"""A message on one line. Paragraph breaks -- a title above its description -- would otherwise run together."""
return re.sub(r"\s*\n\s*\n\s*", " — ", message.strip())
def _anchor(title: str) -> str:
"""An id the table of contents can link to."""
return re.sub(r"[^a-z0-9]+", "-", title.lower()).strip("-") or "section"
def _finding_severity(finding: Finding) -> str: def _finding_severity(finding: Finding) -> str:
"""The severity band a SARIF level falls in.""" """The severity band a SARIF finding falls in; see :mod:`severity`."""
return SEVERITY_OF_LEVEL.get(finding.level, "Low") return finding_severity(finding.rule_id, finding.level, finding.rule_security_severity)
def _vdr_severity(vulnerability: Vulnerability) -> str: def _vdr_severity(vulnerability: Vulnerability) -> str:
@@ -648,13 +732,14 @@ def _vdr_severity(vulnerability: Vulnerability) -> str:
def _where(finding: Finding) -> str | None: def _where(finding: Finding) -> str | None:
"""Where a finding points, as ``file:line``.""" """Where a finding points, as ``file:line``, every location of it."""
if finding.location is None: if finding.location is None:
return None return None
line = finding.location.start_line return ", ".join(
f"{one.file}:{one.start_line}" if one.start_line else one.file
return f"{finding.location.file}:{line}" if line else finding.location.file for one in [finding.location, *finding.extra_locations]
)
def _percent(share: float | None) -> str | None: def _percent(share: float | None) -> str | None:
@@ -663,24 +748,3 @@ def _percent(share: float | None) -> str | None:
return None return None
return f"{share * 100:.2f} %" if share < 0.01 else f"{share * 100:.1f} %" return f"{share * 100:.2f} %" if share < 0.01 else f"{share * 100:.1f} %"
def _moment(stated: Any) -> str | None: # noqa: ANN401
"""An ISO timestamp as the report writes it, or nothing where it cannot be read."""
if not isinstance(stated, str):
return None
try:
return datetime.fromisoformat(stated).strftime("%A, %B %d, %Y %H:%M:%S (%Z)").replace("()", "").strip()
except ValueError:
return None
def _duration(seconds: Any) -> str | None: # noqa: ANN401
"""A number of seconds as ``HH:MM:SS``."""
if not isinstance(seconds, (int, float)) or isinstance(seconds, bool):
return None
whole = int(seconds)
return f"{whole // 3600:02d}:{whole % 3600 // 60:02d}:{whole % 60:02d}"
+9 -7
View File
@@ -23,7 +23,7 @@ TEMPLATE_NAME = "report.html.j2"
STYLESHEET = TEMPLATE_DIR / "report.css" STYLESHEET = TEMPLATE_DIR / "report.css"
# What the document calls itself, in the PDF's own metadata rather than only on its first page. # What the document calls itself, in the PDF's own metadata rather than only on its first page.
TITLE = "SAST/SCA Report" TITLE = "Complete Security Findings Report"
AUTHOR = "CipherCell" AUTHOR = "CipherCell"
@@ -32,7 +32,6 @@ AUTHOR = "CipherCell"
# shorter report rather than one full of empty tables. # shorter report rather than one full of empty tables.
CONTEXT = { CONTEXT = {
"cover": "scan_id, footer_date, logo, title, identity[], severity_ledger{title, rows[]}", "cover": "scan_id, footer_date, logo, title, identity[], severity_ledger{title, rows[]}",
"scan": "title, overview[], stages[{title, rows[], table?}]",
"project": "title, overview[], languages{title, rows[]}", "project": "title, overview[], languages{title, rows[]}",
"summary": "title, overview[], distributions[{title, rows[], legend}]", "summary": "title, overview[], distributions[{title, rows[], legend}]",
"critical": "title, table{columns[], rows[]}", "critical": "title, table{columns[], rows[]}",
@@ -40,11 +39,12 @@ CONTEXT = {
} }
def render_html(context: dict[str, Any]) -> str: def render_html(context: dict[str, Any], template: str = TEMPLATE_NAME) -> str:
"""Render the report as an HTML document. """Render the report as an HTML document.
Args: Args:
context: The page-ready context; see :data:`CONTEXT`. context: The page-ready context; see :data:`CONTEXT`.
template: Which template in :data:`TEMPLATE_DIR` to draw it with.
Returns: Returns:
The rendered HTML. The rendered HTML.
@@ -56,14 +56,16 @@ def render_html(context: dict[str, Any]) -> str:
lstrip_blocks=True, lstrip_blocks=True,
) )
return environment.get_template(TEMPLATE_NAME).render(**context) return environment.get_template(template).render(**context)
def render_pdf(context: dict[str, Any]) -> bytes: def render_pdf(context: dict[str, Any], template: str = TEMPLATE_NAME, title: str = TITLE) -> bytes:
"""Render the report as a PDF. """Render the report as a PDF.
Args: Args:
context: The page-ready context; see :data:`CONTEXT`. context: The page-ready context; see :data:`CONTEXT`.
template: Which template in :data:`TEMPLATE_DIR` to draw it with.
title: What the document calls itself in its metadata.
Returns: Returns:
The PDF. The PDF.
@@ -79,12 +81,12 @@ def render_pdf(context: dict[str, Any]) -> bytes:
# Relative URLs in the template and the stylesheet -- the mark, the font files -- resolve # Relative URLs in the template and the stylesheet -- the mark, the font files -- resolve
# against the template directory. # against the template directory.
document = HTML(string=render_html(context), base_url=str(TEMPLATE_DIR)).render( document = HTML(string=render_html(context, template), base_url=str(TEMPLATE_DIR)).render(
stylesheets=[CSS(filename=str(STYLESHEET), font_config=fonts)], stylesheets=[CSS(filename=str(STYLESHEET), font_config=fonts)],
font_config=fonts, font_config=fonts,
) )
document.metadata.title = f"{TITLE} - {context.get('cover', {}).get('subject', '')}".strip(" -") document.metadata.title = f"{title} - {context.get('cover', {}).get('subject', '')}".strip(" -")
document.metadata.authors = [AUTHOR] document.metadata.authors = [AUTHOR]
return bytes(document.write_pdf()) return bytes(document.write_pdf())
+33 -14
View File
@@ -16,6 +16,7 @@ from typing import TYPE_CHECKING, Any
from .vendor.cc_common.api import PipelineError from .vendor.cc_common.api import PipelineError
from .vendor.cc_common.api import context as logging_context from .vendor.cc_common.api import context as logging_context
from .cloc import parse_cloc
from .context import build_context from .context import build_context
from .layout import render_pdf from .layout import render_pdf
@@ -27,40 +28,56 @@ if TYPE_CHECKING:
# which stage produced it; the first one a scan carries wins. # which stage produced it; the first one a scan carries wins.
INPUTS = { INPUTS = {
"metadata": ("metadata.json", "meta.json"), "metadata": ("metadata.json", "meta.json"),
"findings": ("sast_scanner_ranked.sarif", "polygraph.sarif", "sast_scanner_combined.sarif", "prism.sarif"), "findings": (
"merged.sarif",
"sast_scanner_ranked.sarif",
"polygraph.sarif",
"sast_scanner_combined.sarif",
"prism.sarif",
),
"vulnerabilities": ("vdr.json", "vdr-2.json", "vdr-3.json"), "vulnerabilities": ("vdr.json", "vdr-2.json", "vdr-3.json"),
} }
# What a report cannot be built without. The rest is reported as far as it was found: a scan that # A report needs at least one of these to have anything to say. The rest is reported as far as it
# ran only half the pipeline is a shorter report, not a refusal. # was found: a scan that ran only half the pipeline is a shorter report, not a refusal.
REQUIRED = ("metadata",) REQUIRED_ANY = ("findings", "vulnerabilities")
# What the file is called. The scan it is about is in the name, because a directory of these is # What the file is called. The scan it is about is in the name, because a directory of these is
# otherwise unreadable. # otherwise unreadable.
NAME = "report_{scan}.pdf" NAME = "report_{scan}.pdf"
def render(sources: ReportSources, metrics: dict[str, Any] | None = None) -> tuple[str, bytes]: def render(
sources: ReportSources,
metrics: dict[str, Any] | None = None,
*,
project: str | None = None,
manufacturer: str | None = None,
version: str | None = None,
) -> tuple[str, bytes]:
"""Render the report of one scan. """Render the report of one scan.
Args: Args:
sources: What :func:`sources.collect` found. sources: What :func:`sources.collect` found.
metrics: The metrics document, if one was built.
project: The project name the caller supplied; overrides the metadata's.
manufacturer: The manufacturer the caller supplied.
version: The version the caller supplied; overrides the commit the metadata names.
Returns: Returns:
The file name the report should be saved under, and its bytes. The file name the report should be saved under, and its bytes.
Raises: Raises:
PipelineError: The scan is missing an input the report cannot be built PipelineError: The scan carries neither findings nor vulnerabilities, or
without, or one of its documents cannot be parsed. Both are reported one of its documents cannot be parsed. Both are reported by name: "no
by name: "no PDF was produced" leaves whoever ran it guessing which PDF was produced" leaves whoever ran it guessing which stage of their
stage of their pipeline did not finish. pipeline did not finish.
""" """
documents = {name: _parsed(sources, name, candidates) for name, candidates in INPUTS.items()} documents = {name: _parsed(sources, name, candidates) for name, candidates in INPUTS.items()}
documents["metrics"] = metrics documents["metrics"] = metrics
missing = [name for name in REQUIRED if documents.get(name) is None] documents["cloc"] = parse_cloc(sources.first("cloc.txt"))
if all(documents.get(name) is None for name in REQUIRED_ANY):
if missing: message = f"Scan {sources.scan_id} carries no {' and no '.join(REQUIRED_ANY)}, so no report can be built."
message = f"Scan {sources.scan_id} carries no {' and no '.join(missing)}, so no report can be built."
raise PipelineError(message) raise PipelineError(message)
@@ -68,7 +85,9 @@ def render(sources: ReportSources, metrics: dict[str, Any] | None = None) -> tup
logging_context.logger.info(f"Rendering the report from {', '.join(found)}...") logging_context.logger.info(f"Rendering the report from {', '.join(found)}...")
return NAME.format(scan=sources.scan_id), render_pdf(build_context(sources, documents)) context = build_context(sources, documents, project=project, manufacturer=manufacturer, version=version)
return NAME.format(scan=sources.scan_id), render_pdf(context)
def _parsed(sources: ReportSources, name: str, candidates: tuple[str, ...]) -> Any | None: # noqa: ANN401 def _parsed(sources: ReportSources, name: str, candidates: tuple[str, ...]) -> Any | None: # noqa: ANN401
-103
View File
@@ -97,108 +97,6 @@ def _cover() -> dict[str, Any]:
} }
def _scan() -> dict[str, Any]:
"""What the pipeline did, stage by stage, and what each stage was configured with."""
return {
"title": "CipherCell Scan Information",
"overview": [
_fact("Scan started at", "Wednesday, September 16, 2026 05:43:14 (UTC)"),
_fact("Scan duration", "07:30:38"),
_fact("Scan completed at", "Wednesday, September 16, 2026 13:13:52 (UTC)"),
_fact(
"Pipeline Steps",
"Preanalysis, SAST, SBOM Dependency Tracking, SBOM Reachability, AI FP Detection, AI Ranking",
),
_fact("Errors Detected", "None (2 warnings, see SBOM Reachability)"),
],
"stages": [
{
"title": "Pre-Analysis Settings",
"rows": [
_fact("Selected Tools", "Propylon v0.9.4"),
_fact("Profile", "deterministic"),
_fact("Errors Detected", "None"),
],
},
{
"title": "SAST Settings",
"rows": [
_fact("Selected Tools", "ruff v0.14.4, bandit v1.8.6, cppcheck v2.18.0, semgrep v1.145.0"),
_fact("Language Coverage", "C (cppcheck, semgrep), Python (ruff, bandit, semgrep)"),
_fact("Rule Profile", "security — ruleset ciphercell-sec-2026.03, sha256 8f2b41c6…d97a"),
_fact("Deduplication Method", "Deterministic"),
_fact("Created Artifacts", "sast_combined.sarif", kind="mono"),
_fact("Errors Detected", "None"),
],
},
{
"title": "SBOM Dependency Tracking Settings",
"rows": [
_fact("Selected Tools", "syft v1.33.0, cdxgen v11.10.0, deptrace v0.7.2"),
_fact("Profile", "Standard SBOM"),
_fact("SBOM Format", "CycloneDX 1.6"),
_fact("Advisory Sources", "NVD, OSV.dev, GitHub Advisory Database, Sonatype OSS Index"),
_fact("Created Artifacts", "sbom.cdx.json, sbom.vdr.json", kind="mono"),
_fact("Errors Detected", "None"),
],
},
{
"title": "SBOM Reachability",
"rows": [
_fact("Selected Tools", "CodeQL v2.24.1, Joern v4.0.312"),
_fact("Profile", "Default Taint Analysis"),
_fact("Created Artifacts", "sbom_annotated.vdr.json", kind="mono"),
_fact("Errors Detected", "2 warnings"),
],
"table": {
"columns": ["ID", "Stage", "Warning"],
"rows": [
{
"cells": [
"W-01",
"Joern, JavaScript",
"2 of 148 entry points could not be resolved (dynamic require() in "
"server/plugins/loader.js:41). The 6 findings depending on these entry points "
"were escalated and are reported conservatively as true issues.",
]
},
{
"cells": [
"W-02",
"CodeQL, C",
"Database build skipped for vendored sources under third_party/; excluded by "
"scan scope, no reachability verdict is claimed for those paths.",
]
},
],
},
},
{
"title": "AI FP Detection",
"rows": [
_fact("Selected Tools", "Polygraph v1.6.2"),
_fact("Profile", "static and escalation agent"),
_fact("AI Model", "Qwen3-32B, digest sha256:4ab1…c70f"),
_fact("Inference", "On-premise, CipherCell inference cluster"),
_fact("Reasoning Effort", "Default (classification), Default (escalation)"),
_fact("Response Cache", "Enabled"),
_fact("Context Enrichment", "Static (50 lines before, 50 lines after)"),
_fact("Errors Detected", "None"),
],
},
{
"title": "Ranking",
"rows": [
_fact("Selected Tools", "Prism v0.4.1"),
_fact("Effort", "Standard"),
_fact("Context Size", "Minimal"),
_fact("Errors Detected", "None"),
],
},
],
}
def _project() -> dict[str, Any]: def _project() -> dict[str, Any]:
"""What was scanned, and what it is written in.""" """What was scanned, and what it is written in."""
return { return {
@@ -517,7 +415,6 @@ def sample_context() -> dict[str, Any]:
"""A complete context, covering every section the template can draw.""" """A complete context, covering every section the template can draw."""
return { return {
"cover": _cover(), "cover": _cover(),
"scan": _scan(),
"project": _project(), "project": _project(),
"summary": _summary(), "summary": _summary(),
"critical": _critical(), "critical": _critical(),
+129
View File
@@ -0,0 +1,129 @@
"""The severity a SARIF finding is reported under.
In order of preference:
1. The finding's own ``security-severity`` score, banded like CVSS. Where a tool scored the finding
itself -- Lynkeus does -- that is the most specific reading there is.
2. Its weakness class, from :data:`CWE_SEVERITY`. The tools do not agree on what their levels mean
-- Trivy's ``error`` for a container running as root and Bandit's ``note`` for a hard-coded
database password say more about the tool than about the risk -- so the CWE is preferred to them.
3. Its SARIF level, from :data:`SEVERITY_OF_LEVEL`, for a finding with neither.
Each entry of the table is the severity a confirmed instance of that weakness typically carries, before any
context about the application is known: roughly where the CVEs filed under it land on the CVSS
bands. Critical is kept for classes that usually hand an attacker code execution or the whole data
store; weaknesses that only matter together with another flaw, or that harden rather than defend,
sit at Medium or Low.
"""
from __future__ import annotations
import re
CWE_SEVERITY = {
# Injection that typically ends in code execution or full database access.
"CWE-77": "Critical", # Command injection
"CWE-78": "Critical", # OS command injection
"CWE-89": "Critical", # SQL injection
"CWE-94": "Critical", # Code injection
"CWE-95": "Critical", # Eval injection
"CWE-502": "Critical", # Deserialization of untrusted data
# Direct compromise of confidentiality or access control, usually needing some foothold.
"CWE-22": "High", # Path traversal
"CWE-96": "High", # Static code injection
"CWE-259": "High", # Hard-coded password
"CWE-287": "High", # Improper authentication
"CWE-306": "High", # Missing authentication for a critical function
"CWE-321": "High", # Hard-coded cryptographic key
"CWE-434": "High", # Unrestricted upload of dangerous file type
"CWE-611": "High", # XML external entities
"CWE-798": "High", # Hard-coded credentials
"CWE-862": "High", # Missing authorization
"CWE-863": "High", # Incorrect authorization
"CWE-918": "High", # Server-side request forgery
# Real weaknesses whose impact depends on context, or that need another flaw to be exploited.
"CWE-15": "Medium", # External control of a configuration setting
"CWE-20": "Medium", # Improper input validation
"CWE-79": "Medium", # Cross-site scripting
"CWE-200": "Medium", # Exposure of sensitive information
"CWE-250": "Medium", # Execution with unnecessary privileges
"CWE-269": "Medium", # Improper privilege management
"CWE-295": "Medium", # Improper certificate validation
"CWE-312": "Medium", # Cleartext storage of sensitive information
"CWE-319": "Medium", # Cleartext transmission of sensitive information
"CWE-326": "Medium", # Inadequate encryption strength
"CWE-327": "Medium", # Broken or risky cryptographic algorithm
"CWE-328": "Medium", # Weak hash
"CWE-330": "Medium", # Insufficiently random values
"CWE-338": "Medium", # Cryptographically weak PRNG
"CWE-352": "Medium", # Cross-site request forgery
"CWE-400": "Medium", # Uncontrolled resource consumption
"CWE-522": "Medium", # Insufficiently protected credentials
"CWE-532": "Medium", # Sensitive information in a log file
"CWE-601": "Medium", # Open redirect
"CWE-732": "Medium", # Incorrect permission assignment
"CWE-770": "Medium", # Allocation of resources without limits
"CWE-1333": "Medium", # Inefficient regular expression (ReDoS)
# Hygiene: worth fixing, rarely exploitable on its own.
"CWE-117": "Low", # Log injection
"CWE-209": "Low", # Information in an error message
"CWE-377": "Low", # Insecure temporary file
"CWE-390": "Low", # Error detected without action
"CWE-391": "Low", # Unchecked error condition
"CWE-614": "Low", # Sensitive cookie without the Secure flag
"CWE-703": "Low", # Improper handling of exceptional conditions
"CWE-754": "Low", # Improper check for unusual conditions
"CWE-755": "Low", # Improper handling of exceptional conditions
"CWE-1004": "Low", # Sensitive cookie without the HttpOnly flag
}
# For a finding whose CWE is not in the table, or that has none. One band below the old one-to-one
# reading of the level, because a tool's "error" is not a statement that the finding is critical.
SEVERITY_OF_LEVEL = {
"error": "High",
"warning": "Medium",
"note": "Low",
"none": "Low",
"info": "Low",
}
# The CVSS v3 qualitative bands, by their lower bounds. A score of 0 has no band of its own in the
# report and reads as Low.
SCORE_BANDS = ((9.0, "Critical"), (7.0, "High"), (4.0, "Medium"))
_CWE = re.compile(r"CWE-(\d+)", re.IGNORECASE)
def finding_severity(rule_id: str, level: str, security_severity: str = "") -> str:
"""The severity a SARIF finding is reported under.
Args:
rule_id: The finding's rule. The pipeline's tools name their rules by CWE.
level: The SARIF level.
security_severity: The finding's ``security-severity`` score, empty where it has none.
Returns:
One of Critical, High, Medium or Low.
"""
scored = _score_band(security_severity)
if scored is not None:
return scored
found = _CWE.search(rule_id)
known = CWE_SEVERITY.get(f"CWE-{found.group(1)}") if found else None
return known or SEVERITY_OF_LEVEL.get(level, "Low")
def _score_band(stated: str) -> str | None:
"""The band a ``security-severity`` score falls in, or nothing where it is not a score."""
try:
score = float(stated)
except ValueError:
return None
if not 0 <= score <= 10:
return None
return next((name for bound, name in SCORE_BANDS if score >= bound), "Low")
+239
View File
@@ -0,0 +1,239 @@
"""Turns a Markdown summary into the summary report.
The summary is written by hand, beside the scan's artifacts, and reads as a document of its own: a
title, a few lines saying what it covers, then one section per source. This module sets it in the
dress of the full report -- the same cover, running header and footers, table of contents, headings
and tables -- so the two sit together as one series.
No design of its own. The Markdown is rendered to HTML whose elements are the ones ``report.css``
already styles, by moving them onto the full report's heading levels and classes:
- The first level-one heading is the title. Written as ``<title>: <subject>``, the subject is the
project, unless one is given.
- What comes before the first section is the introduction, and goes on the cover.
- Every further level-one or level-two heading is a section: it starts a new page, as the full
report's sections do, is listed in the table of contents and keeps the anchor GitHub would give it, so the links a
summary makes to its own sections go on working in the PDF.
- A paragraph that is nothing but bold text is the subheading it is being used as.
- Tables are drawn as the full report's grids. A number in a severity or CVSS column carries its
severity colour, and a row that opens with a bold ``Total`` is set as a total.
"""
from __future__ import annotations
import re
from datetime import UTC, datetime
from typing import TYPE_CHECKING, Any
from markdown_it import MarkdownIt
from markupsafe import Markup
from .context import UNKNOWN, _fact, _stated
from .layout import render_pdf
from .severity import _score_band
if TYPE_CHECKING:
from uuid import UUID
from markdown_it.token import Token
TEMPLATE_NAME = "summary.html.j2"
# What the document calls itself when the Markdown does not say.
TITLE = "Security Scan Summary"
# The shape the summary template expects.
CONTEXT = {
"cover": "subject, footer, footer_date, logo, title, identity[], intro?",
"contents": "[{id, title}]",
"body": "the rendered sections",
}
# What the file is called: after the scan where there is one, else after the project.
NAME = "summary_{name}.pdf"
# A column whose numbers are CVSS scores, to be coloured by the band each falls in.
_SCORED_COLUMN = re.compile(r"severity|cvss|score", re.IGNORECASE)
_SCORE = re.compile(r"\d+(?:\.\d+)?")
# Where Markdown's heading levels land on the full report's: a section is its ``h1``, and what is
# under it one and two steps down.
_LEVELS = {"h1": "h1", "h2": "h1", "h3": "h2", "h4": "h3", "h5": "h3", "h6": "h3"}
def render_summary(
markdown: str,
*,
scan_id: UUID | None = None,
project: str | None = None,
manufacturer: str | None = None,
) -> tuple[str, bytes]:
"""Render a Markdown summary as a PDF.
Args:
markdown: The summary.
scan_id: The scan it is about, if it is to say so.
project: The project name; overrides the subject of the title.
manufacturer: Who makes the scanned product.
Returns:
The file name the summary should be saved under, and its bytes.
"""
context = build_summary_context(markdown, scan_id=scan_id, project=project, manufacturer=manufacturer)
name = str(scan_id) if scan_id else _slug(context["cover"]["subject"]) or "report"
return NAME.format(name=name), render_pdf(context, TEMPLATE_NAME, context["cover"]["title"])
def build_summary_context(
markdown: str,
*,
scan_id: UUID | None = None,
project: str | None = None,
manufacturer: str | None = None,
) -> dict[str, Any]:
"""Map a Markdown summary onto the page.
Args:
markdown: The summary.
scan_id: The scan it is about, if it is to say so.
project: The project name; overrides the subject of the title.
manufacturer: Who makes the scanned product.
Returns:
The page-ready context; see :data:`CONTEXT`.
"""
parser = MarkdownIt("commonmark").enable(["table", "strikethrough"])
tokens = parser.parse(markdown)
heading = _title(tokens)
title, _, subject = heading.partition(":") if heading else (TITLE, "", "")
project = project or subject.strip() or None
contents = _sections(tokens, parser)
_subheadings(tokens)
_tables(tokens)
start = next((index for index, token in enumerate(tokens) if token.meta.get("section")), len(tokens))
intro, body = tokens[:start], tokens[start:]
render = parser.renderer.render
identity = [
_fact("Manufacturer", manufacturer),
_fact("Project", project),
_fact("Report Created", datetime.now(UTC).strftime("%A, %B %d, %Y")),
]
if scan_id:
identity.append(_fact("Scan Identifier", scan_id, kind="mono"))
return {
"cover": {
"subject": _stated(project),
"footer": f"Scan: {scan_id}" if scan_id else _stated(project),
"footer_date": datetime.now(UTC).strftime("%d %B %Y"),
"logo": "cc_logo.png",
"title": title.strip(),
"identity": identity,
"intro": Markup(render(intro, parser.options, {})) if intro else None, # noqa: S704
},
"contents": contents,
"body": Markup(render(body, parser.options, {})), # noqa: S704
}
def _title(tokens: list[Token]) -> str | None:
"""Take the first level-one heading out of the document, and say what it read."""
for index, token in enumerate(tokens):
if token.type == "heading_open" and token.tag == "h1":
text = tokens[index + 1].content
del tokens[index : index + 3]
return text
return None
def _sections(tokens: list[Token], parser: MarkdownIt) -> list[dict[str, Any]]:
"""Move every heading onto the full report's levels, and list the sections for the contents."""
contents: list[dict[str, Any]] = []
taken: set[str] = set()
for index, token in enumerate(tokens):
if token.type not in ("heading_open", "heading_close"):
continue
section = token.tag in ("h1", "h2")
token.tag = _LEVELS[token.tag]
if token.type == "heading_close" or not section:
continue
inline = tokens[index + 1]
anchor = _unique(_github_anchor(inline.content), taken)
token.attrSet("id", anchor)
token.meta["section"] = True
# As in the full report, every section opens a page of its own.
token.attrSet("style", "break-before: page")
title = Markup(parser.renderer.render(inline.children or [], parser.options, {})) # noqa: S704
contents.append({"id": anchor, "title": title})
return contents
def _subheadings(tokens: list[Token]) -> None:
"""Set a paragraph that is nothing but bold text as the subheading it stands in for."""
for index, token in enumerate(tokens):
if token.type != "paragraph_open" or token.hidden:
continue
children = tokens[index + 1].children or []
kinds = [child.type for child in children]
if kinds == ["strong_open", "text", "strong_close"]:
token.tag = tokens[index + 2].tag = "h2"
tokens[index + 1].children = [children[1]]
def _tables(tokens: list[Token]) -> None:
"""Draw tables as the full report's grids: scores coloured, totals set as totals."""
columns: list[str] = []
column = 0
for index, token in enumerate(tokens):
if token.type == "table_open":
token.attrSet("class", "grid")
columns = []
elif token.type == "th_open":
columns.append(tokens[index + 1].content)
elif token.type == "tr_open":
column = 0
cell = tokens[index + 2] if tokens[index + 1].type == "td_open" else None
if cell is not None and cell.content.strip() == "**Total**":
token.attrSet("class", "total")
elif token.type == "td_open":
header = columns[column] if column < len(columns) else ""
content = tokens[index + 1].content.strip()
band = _score_band(content) if _SCORED_COLUMN.search(header) and _SCORE.fullmatch(content) else None
if band:
token.attrSet("class", f"sev sev-{band.lower()}")
column += 1
def _github_anchor(text: str) -> str:
"""The anchor GitHub gives a heading, so the links a summary makes to itself still land."""
return re.sub(r"[^\w\- ]", "", text.strip().lower()).replace(" ", "-")
def _unique(anchor: str, taken: set[str]) -> str:
"""The anchor, numbered as GitHub numbers a second heading of the same name."""
candidate, number = anchor or "section", 0
while candidate in taken:
number += 1
candidate = f"{anchor}-{number}"
taken.add(candidate)
return candidate
def _slug(text: str) -> str:
"""A name fit for a file."""
return "" if text == UNKNOWN else re.sub(r"[^a-z0-9]+", "-", text.lower()).strip("-")
+92
View File
@@ -0,0 +1,92 @@
{#
The pieces the report templates share: `report.html.j2`, the full report, and `summary.html.j2`,
the summary set from Markdown. Kept apart so that the two documents draw a label, a value or a
table the same way, and a change to one is a change to both.
#}
{# A value is drawn by what it is, not by where it sits: an identifier, a path or a CVSS vector is
set in the monospace face wherever it appears, and a severity carries its colour. #}
{% macro value(row) %}
{% if row.token %}<span class="token">{{ row.token }}</span> — {% endif %}
{% if row.kind == "severity" %}
<span class="sev sev-{{ row.value|lower }}">{{ row.value }}</span>
{% elif row.kind == "mono" %}
<span class="mono">{{ row.value }}</span>
{% else %}
{{ row.value }}
{% endif %}
{% endmacro %}
{% macro facts(rows, wide=false) %}
<table class="facts{% if wide %} wide{% endif %}">
{% for row in rows %}
<tr{% if row.gap %} class="gap"{% endif %}>
<td>{{ row.label }}:</td>
<td>{{ value(row) }}</td>
</tr>
{% endfor %}
</table>
{% endmacro %}
{% macro card(title, rows, note="") %}
{# Wrapped, because break-inside on the table does not hold its caption: the title would stay on
one page while the rows it titles moved to the next. #}
<div class="block">
<table class="card">
<caption>{{ title }}</caption>
{% for row in rows %}
<tr{% if row.aside %} class="aside"{% endif %}>
<th>{{ row.label }}</th>
<td>{{ value(row) }}</td>
</tr>
{% endfor %}
</table>
{% if note %}<p class="note card-note">{{ note }}</p>{% endif %}
</div>
{% endmacro %}
{# Scaled against the busiest row, not against the total: what is read off a row of these is which
one is the outlier, and a scale set by the sum flattens all of them. #}
{% macro distribution(rows, legend) %}
{# The bars and the legend that decodes them are one thing; a break between them leaves a reader
with coloured segments and no key. #}
<div class="block">
<table class="bars">
{% for row in rows %}
<tr>
<td{% if row.mono %} class="mono"{% endif %}>{{ row.label }}</td>
<td class="count">{{ row.count }}</td>
<td class="track">
<div class="bar">
{% for part in row.parts %}
<span class="seg-{{ part.severity|lower }}" style="width: {{ part.share }}%"></span>
{% endfor %}
</div>
</td>
</tr>
{% endfor %}
</table>
<div class="legend">
{% for item in legend.entries %}
<span class="item"><span class="key seg-{{ item.severity|lower }}"></span>{{ item.label }}</span>
{% endfor %}
{% if legend.caveat %}<span class="caveat">— {{ legend.caveat }}</span>{% endif %}
</div>
</div>
{% endmacro %}
{% macro grid(table) %}
<table class="grid">
<thead>
<tr>{% for column in table.columns %}<th>{{ column }}</th>{% endfor %}</tr>
</thead>
<tbody>
{% for row in table.rows %}
<tr{% if row.total %} class="total"{% endif %}>
{% for cell in row.cells %}
<td>{% if cell is mapping %}{{ value(cell) }}{% else %}{{ cell }}{% endif %}</td>
{% endfor %}
</tr>
{% endfor %}
</tbody>
</table>
{% endmacro %}
+117
View File
@@ -62,6 +62,13 @@
size: A4; size: A4;
margin: 22mm 18mm 18mm 18mm; margin: 22mm 18mm 18mm 18mm;
@top-center {
content: element(page-header);
width: 100%;
vertical-align: bottom;
padding-bottom: 4mm;
}
@bottom-left { @bottom-left {
content: string(scan-footer); content: string(scan-footer);
font-family: "Space Grotesk", sans-serif; font-family: "Space Grotesk", sans-serif;
@@ -101,6 +108,26 @@ code,
font-size: 0.93em; font-size: 0.93em;
} }
/* The cover carries the mark large already; a second, small one above it would only repeat it. */
@page :first {
@top-center {
content: none;
}
}
/* The mark -- the logo and the company name, which the image carries together -- above every page,
ruled off from the content. */
.page-header {
position: running(page-header);
padding-bottom: 1.6mm;
border-bottom: 0.4pt solid var(--border);
}
.page-header img {
height: 5mm;
display: block;
}
/* Carried into the running footer on every page. Set once, on the cover. */ /* Carried into the running footer on every page. Set once, on the cover. */
.footer-scan { .footer-scan {
string-set: scan-footer content(); string-set: scan-footer content();
@@ -420,3 +447,93 @@ p.note {
.legend .caveat { .legend .caveat {
color: var(--muted); color: var(--muted);
} }
/* --- Table of contents -------------------------------------------------------------------- */
/* Page numbers are read off where each link lands, so they stay right however long a section runs. */
.toc {
list-style: none;
padding: 0;
margin: 0;
}
.toc li {
padding: 1.4mm 0;
border-bottom: 0.4pt solid var(--border);
}
.toc a {
color: var(--ink);
text-decoration: none;
}
.toc a::after {
content: leader(".") target-counter(attr(href), page);
}
/* --- Prose (the summary report) ----------------------------------------------------------- */
/* The summary is written in Markdown, so it brings what the full report never has: running text,
lists and links. Set to the full report's measure and rhythm, so the two read as one series. */
.prose p,
.prose ul,
.prose ol,
.prose blockquote {
max-width: 165mm;
}
/* Every section of the summary opens a page, so the space a heading keeps above it would only push
the page down. */
.prose h1 {
margin-top: 0;
}
.prose ul,
.prose ol {
margin: 0 0 3mm 0;
padding-left: 5mm;
}
.prose li {
margin-bottom: 1.4mm;
}
.prose li > p {
margin: 0;
}
.prose a {
color: var(--ink);
text-decoration: underline;
text-decoration-color: var(--border);
}
.prose blockquote {
margin: 0 0 3mm 0;
padding-left: 3mm;
border-left: 0.8pt solid var(--border);
color: var(--muted);
}
.prose pre {
background: var(--label-fill);
border: 0.4pt solid var(--border);
padding: 2mm;
white-space: pre-wrap;
margin: 0 0 3mm 0;
}
.prose hr {
border: none;
border-top: 0.4pt solid var(--border);
margin: 5mm 0;
}
/* The severity colour, where it marks a score in a table, is on the cell itself. */
.grid td.sev {
white-space: nowrap;
}
.cover-intro {
margin-top: 9mm;
}
+13 -100
View File
@@ -8,96 +8,11 @@
The context this expects is described by `layout.CONTEXT`, and `sample.sample_context()` builds a The context this expects is described by `layout.CONTEXT`, and `sample.sample_context()` builds a
complete one. complete one.
#} #}
{# A value is drawn by what it is, not by where it sits: an identifier, a path or a CVSS vector is {% from "macros.html.j2" import value, facts, card, distribution, grid %}
set in the monospace face wherever it appears, and a severity carries its colour. #}
{% macro value(row) %}
{% if row.token %}<span class="token">{{ row.token }}</span> — {% endif %}
{% if row.kind == "severity" %}
<span class="sev sev-{{ row.value|lower }}">{{ row.value }}</span>
{% elif row.kind == "mono" %}
<span class="mono">{{ row.value }}</span>
{% else %}
{{ row.value }}
{% endif %}
{% endmacro %}
{% macro facts(rows, wide=false) %}
<table class="facts{% if wide %} wide{% endif %}">
{% for row in rows %}
<tr{% if row.gap %} class="gap"{% endif %}>
<td>{{ row.label }}:</td>
<td>{{ value(row) }}</td>
</tr>
{% endfor %}
</table>
{% endmacro %}
{% macro card(title, rows, note="") %}
{# Wrapped, because break-inside on the table does not hold its caption: the title would stay on
one page while the rows it titles moved to the next. #}
<div class="block">
<table class="card">
<caption>{{ title }}</caption>
{% for row in rows %}
<tr{% if row.aside %} class="aside"{% endif %}>
<th>{{ row.label }}</th>
<td>{{ value(row) }}</td>
</tr>
{% endfor %}
</table>
{% if note %}<p class="note card-note">{{ note }}</p>{% endif %}
</div>
{% endmacro %}
{# Scaled against the busiest row, not against the total: what is read off a row of these is which
one is the outlier, and a scale set by the sum flattens all of them. #}
{% macro distribution(rows, legend) %}
{# The bars and the legend that decodes them are one thing; a break between them leaves a reader
with coloured segments and no key. #}
<div class="block">
<table class="bars">
{% for row in rows %}
<tr>
<td{% if row.mono %} class="mono"{% endif %}>{{ row.label }}</td>
<td class="count">{{ row.count }}</td>
<td class="track">
<div class="bar">
{% for part in row.parts %}
<span class="seg-{{ part.severity|lower }}" style="width: {{ part.share }}%"></span>
{% endfor %}
</div>
</td>
</tr>
{% endfor %}
</table>
<div class="legend">
{% for item in legend.entries %}
<span class="item"><span class="key seg-{{ item.severity|lower }}"></span>{{ item.label }}</span>
{% endfor %}
{% if legend.caveat %}<span class="caveat">— {{ legend.caveat }}</span>{% endif %}
</div>
</div>
{% endmacro %}
{% macro grid(table) %}
<table class="grid">
<thead>
<tr>{% for column in table.columns %}<th>{{ column }}</th>{% endfor %}</tr>
</thead>
<tbody>
{% for row in table.rows %}
<tr{% if row.total %} class="total"{% endif %}>
{% for cell in row.cells %}
<td>{% if cell is mapping %}{{ value(cell) }}{% else %}{{ cell }}{% endif %}</td>
{% endfor %}
</tr>
{% endfor %}
</tbody>
</table>
{% endmacro %}
{# ------------------------------------------------------------------------------------------ #} {# ------------------------------------------------------------------------------------------ #}
<div class="page-header"><img src="{{ cover.logo }}" alt="CipherCell" /></div>
<span class="footer-scan">Scan: {{ cover.scan_id }}</span> <span class="footer-scan">Scan: {{ cover.scan_id }}</span>
<span class="footer-date">{{ cover.footer_date }}</span> <span class="footer-date">{{ cover.footer_date }}</span>
@@ -121,19 +36,17 @@
</table> </table>
{% endif %} {% endif %}
{% if scan %} {% if contents %}
<h1 style="break-before: page">{{ scan.title }}</h1> <h1 style="break-before: page">Table of Contents</h1>
{{ facts(scan.overview, wide=true) }} <ol class="toc">
{% for entry in contents %}
{% for stage in scan.stages %} <li><a href="#{{ entry.id }}">{{ entry.title }}</a></li>
<h3>{{ stage.title }}</h3> {% endfor %}
{{ facts(stage.rows, wide=true) }} </ol>
{% if stage.table %}{{ grid(stage.table) }}{% endif %}
{% endfor %}
{% endif %} {% endif %}
{% if project %} {% if project %}
<h1 style="break-before: page">{{ project.title }}</h1> <h1 id="{{ project.id }}" style="break-before: page">{{ project.title }}</h1>
{{ facts(project.overview, wide=true) }} {{ facts(project.overview, wide=true) }}
{% if project.languages %} {% if project.languages %}
@@ -150,7 +63,7 @@
{% endif %} {% endif %}
{% if summary %} {% if summary %}
<h1>{{ summary.title }}</h1> <h1 id="{{ summary.id }}"{% if not project %} style="break-before: page"{% endif %}>{{ summary.title }}</h1>
{{ facts(summary.overview, wide=true) }} {{ facts(summary.overview, wide=true) }}
{% for block in summary.distributions %} {% for block in summary.distributions %}
@@ -164,12 +77,12 @@
{% endif %} {% endif %}
{% if critical %} {% if critical %}
<h2 style="break-before: page">{{ critical.title }}</h2> <h2 id="{{ critical.id }}" style="break-before: page">{{ critical.title }}</h2>
{{ grid(critical.table) }} {{ grid(critical.table) }}
{% endif %} {% endif %}
{% for section in sections %} {% for section in sections %}
<h1{% if section.new_page %} style="break-before: page"{% endif %}>{{ section.title }}</h1> <h1{% if section.id %} id="{{ section.id }}"{% endif %}{% if section.new_page %} style="break-before: page"{% endif %}>{{ section.title }}</h1>
{% for group in section.groups %} {% for group in section.groups %}
<h2>{{ group.title }}</h2> <h2>{{ group.title }}</h2>
+36
View File
@@ -0,0 +1,36 @@
{#
The summary report, set from a Markdown file in the dress of the full report: the same cover,
running header and footers, table of contents and headings, with the body in between.
The body arrives already rendered -- :mod:`summary` turns the Markdown into HTML whose elements
are the ones `report.css` already styles -- so this file only places it. The context this expects
is described by `summary.CONTEXT`.
#}
{% from "macros.html.j2" import facts %}
<div class="page-header"><img src="{{ cover.logo }}" alt="CipherCell" /></div>
<span class="footer-scan">{{ cover.footer }}</span>
<span class="footer-date">{{ cover.footer_date }}</span>
<div class="cover-mark">
<img src="{{ cover.logo }}" alt="CipherCell" />
</div>
<div class="cover-title">{{ cover.title }}</div>
{{ facts(cover.identity) }}
{% if cover.intro %}
<div class="prose cover-intro">{{ cover.intro }}</div>
{% endif %}
{% if contents %}
<h1 style="break-before: page">Table of Contents</h1>
<ol class="toc">
{% for entry in contents %}
<li><a href="#{{ entry.id }}">{{ entry.title }}</a></li>
{% endfor %}
</ol>
{% endif %}
<div class="prose">{{ body }}</div>