Lesson 16: A CI assembly line¶
This lesson assembles the course into a pipeline. The shape below runs the expensive part once, derives everything else from artefacts, gates on drift, and publishes for both humans and tools. Every command in it is one you have already run by hand.
The shape¶
push / pull_request
|
v
+-------------------+ reachables.json (chunks) +------------------+
| engine run | --------------------------------> | merge-slices |
| (atom or other) | usages.slices.json +------------------+
+-------------------+ | |
| v
| merged.json
| |
+-----------------------------+-----------------------------+----------+
| | | | |
v v v v v
stats --json validate-lines convert sarif visualize ingest
| | | -b sbom.json |
v v v v v
stats artifact anchor report sarif upload html page unified.json
(artifact) |
+-------+-------+
v v
attack-surface drift --fail-on
explain -f markdown (exit 2 = gate)
The engine run is the only expensive box. Everything below it is seconds, which is the whole point of the reports-on-disk design from lesson 1: rerun any downstream box whenever you want, without paying for the analysis again.
A GitHub Actions workflow¶
This workflow is written for a JavaScript project analysed with atom, exactly the course's cdxgen setup. It gates on drift, uploads SARIF to code scanning, and attaches the human artefacts to the workflow run.
name: flows
on:
pull_request:
push:
branches: [main]
permissions:
contents: read
security-events: write
jobs:
analyse:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 22
- name: Install the tools
run: |
npm install -g @appthreat/atom
python -m pip install atom-tools
# The engine run. Cache the .chen AST directory between runs; it is the
# difference between a warm and a cold frontend (lesson 2).
- uses: actions/cache@v4
with:
path: .chen
key: astcache-${{ runner.os }}-${{ hashFiles('**/*.js', '**/*.ts') }}
- name: Slice the tree
run: |
atom reachables -l jssrc -o reports/app.atom -s reports/app.reachables.json .
atom usages -l jssrc -o reports/app.atom -s reports/app.usages.json . --reuse-atom
- name: Merge and summarise
run: |
atom-tools merge-slices -i 'reports/app.reachables*.json' -o merged.reachables.json
atom-tools stats -i merged.reachables.json --json -o stats.json
- name: Validate line anchors
run: |
atom-tools validate-lines -t js -i reports/app.usages.json \
-d . -l 5 -r anchors.txt -j anchors.json || true
- name: Convert to SARIF
run: atom-tools convert -i merged.reachables.json -f sarif -o flows.sarif
- uses: github/codeql-action/upload-sarif@v3
with:
sarif_file: flows.sarif
- name: Drift gate against the baseline branch artefact
run: |
atom-tools drift --old baseline/merged.reachables.json \
--new merged.reachables.json \
--fail-on new-high,new-anonymous-endpoint,new-package
- name: Human artefacts
run: |
atom-tools visualize -i merged.reachables.json -b sbom.json -o viz
atom-tools explain -i merged.reachables.json -f markdown -o narrative.md --max-flows 20
- uses: actions/upload-artifact@v4
with:
name: flow-reports
path: |
stats.json
anchors.txt
viz.html
narrative.md
Three details in that file encode lessons from the course. The cache entry for
.chen is lesson 2's AST cache. The validate-lines step carries || true
because an anchor drift report is information for the reviewer, not
necessarily a build failure; drop the suffix if you want it strict. And the
drift step carries no || true at all, on purpose: its exit code 2 is the
gate.
How the baseline reaches the drift step¶
Drift compares two files, so the workflow needs yesterday's merged slice. Two common shapes. The first stores it as a workflow artefact on the default branch and downloads it before the gate. The second commits it to a branch or a store (an S3 object, an OCI artifact) keyed by branch name. Either way, the baseline must come from the same engine version and the same analysis options as the current run, because identity keys include the taxonomy-derived severity: mixed versions produce noisy diffs that teach reviewers to ignore them.
Choosing gates¶
The six gate keys from lesson 12 are a menu, not a checklist. A library with
no routes gets new-high and new-package. A service behind an authenticating
gateway adds new-anonymous-endpoint. new-sink-category is the early-warning
one: the day the codebase grows its first reachable code-execution category
is the day you want the build to say so, whatever the count. Start strict on
pull requests, and once the deltas are routinely human-sized, keep them strict;
the moment a gate fires on every run and gets ignored every time, it is no
longer a gate.
Exit codes are the contract: 0 clean, 1 the tool failed, 2 a gate tripped. Script the difference explicitly, because "the diff tool broke" and "the diff found risk" deserve different responses.
Exit codes and scripting shapes¶
check-reachable prints True or False and always exits 0, so gate on
stdout when you use it in a script:
if [ "$(atom-tools check-reachable -i merged.reachables.json -p 'left-pad:1.3.0')" = "True" ]; then
echo "left-pad 1.3.0 is on a reachable path; the upgrade needs a flow review"
fi
The drift gate, by contrast, is pure exit code, which is why the workflow step needs nothing but the command line.
Where the docs site itself fits¶
This course is built with MkDocs from the repository's docs/ directory, and
the same repository CI builds it with mkdocs build --strict on every pull
request. Strict mode turns broken links and missing pages into build failures,
which is the same discipline the drift gate applies to analysis: artefacts that
claim to describe something must actually reference it. If you adopt one idea
from this lesson, adopt that one: every artefact in the pipeline, from SARIF to
narratives to documentation, should fail loudly when it drifts from what it
describes.
Exercises¶
Turn the workflow into a reusable workflow file parameterised by language, so
a polyglot monorepo can call it once per service with -l as the input. Add an
explaining step that runs atom-tools explain --query "flows[sink_category=sql]"
and comments the output on the pull request when it is non-empty. Then break
the gate deliberately: introduce a reachable file-write from a request
parameter in a scratch branch and watch which gate fires.