Skip to content

feat!(detector): route NVD to vuls2#2575

Merged
shino merged 109 commits into
masterfrom
shino/cpe-detect-nvd
Jun 24, 2026
Merged

feat!(detector): route NVD to vuls2#2575
shino merged 109 commits into
masterfrom
shino/cpe-detect-nvd

Conversation

@shino

@shino shino commented Jun 10, 2026

Copy link
Copy Markdown
Collaborator

What

Route all NVD-derived data through the vuls2 library — both CPE detection and enrichment (CveContent, exploits, mitigations, US-CERT alerts) — while keeping go-cve-dictionary for the sources vuls2 does not carry (JVN / JP-CERT, Cisco, Paloalto, Fortinet, VulnCheck, EUVD, MITRE).

Companion to the vuls-data-update / vuls2 work this pins via go.mod (latest: MaineK00n/vuls-data-update#850, MaineK00n/vuls2#388).

Why

  • go-cve-dictionary's CPE path matches NVD by version-range comparison only, which cannot decide membership for non-semver version formats (cisco ios 15.1(4)m3, juniper junos 21.4r3) and suffers a go-cpe numeric-prefix over-match (5.15.10 matching a 5.15.103 query). The vuls2 DB carries the NVD feed with cpematch-expanded criteria, fixing both.
  • Consolidating NVD in vuls2 means detection and enrichment share one NVD source and emit a single canonical nvd CveContent, instead of go-cve-dictionary's per-CVSS-source multi-entry set.

What moves where

Data Path
NVD CPE detection vuls2 (DetectCPEs, cpematch-expanded criteria)
NVD CveContent / exploits / mitigations / US-CERT alerts vuls2 (EnrichVulnInfosenrichNVD)
JVN / JP-CERT, Cisco, Paloalto, Fortinet, VulnCheck, EUVD, MITRE go-cve-dictionary

How

Match quality is decided upstream. vuls-data-update's cpecriterion.Accept returns a MatchQuality (Exact / VersionUnconfirmed), and accept indexes are bucketed by quality in the extracted criterion. vuls0 no longer re-derives CPE semantics — walkCPECriteria is a pure projection of Accepts.CPE.{Exact,VersionUnconfirmed} onto two confidence tiers: ExactVersionMatch (100) and VendorProductMatch (10). go-cve-dictionary's middle RoughVersionMatch (80) is retired, and the old version-range vendorProductEligible fallback is gone (the version-unconfirmed tier replaces it).

NVD enrichment (enrichNVD) builds the single nvd CveContent (CVSS / summary / CWE / references), plus NVD exploits/mitigations and US-CERT alerts (NVD references whose URL contains us-cert), mirroring the detection path. It is gated so it never duplicates a CVE the CPE detection already filled. go-cve-dictionary's NVD contribution (detection, CveContent, US-CERT) is stripped; JP-CERT (from JVN) and the JVNDB advisory gating stay on the dictionary path.

detector API stays master-shaped (DetectPkgCvesDetectCpeURIsCves) so library consumers are unaffected. EnrichVulnInfos runs between them and sets AlertDict.USCERT; the dictionary fill that follows now sets only .JPCERT, preserving it.

deps: vuls2 98ac8e5 (nightly, #388), vuls-data-update bfedbc24 (#850). Builds standalone — no go.work overrides.

Testing

  • go test ./... green; gofmt clean.
  • vuls-compare over CPE-bearing fixtures (apache / cisco non-semver / linux_kernel over-match / juniper): detected CVE sets match the classic dictionary path (0 over-detect, 0 missed). NVD CveContent intentionally changes shape — vuls2's single canonical entry vs the dictionary's per-CVSS-source set.
  • NVD detection + enrich verified against a locally-built NVD vuls2 DB (vuls db add nvd-feed-cve-v2) with an empty go-cve-dictionary, confirming NVD flows entirely through vuls2:
    • CPE detection (openssl 1.0.1f): 74 CVEs via vuls2, all carrying nvd content (NvdExactVersionMatch).
    • enrich (rhel_90): nvd CveContents go 0 → 7018 with the NVD DB, unchanged with empty gocve (vs 5194 multi-entry on the dictionary path).
    • US-CERT: Heartbleed (CVE-2014-0160) → US-CERT-TA14-098A, surviving the dictionary fill step.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR reroutes NVD-based CPE detection to the vuls2 DB (to leverage cpematch-expanded criteria and avoid version-format pitfalls) while keeping go-cve-dictionary CPE detection for non-NVD sources (JVN, Cisco, Paloalto, Fortinet, Vulncheck, EUVD, MITRE, etc.). It also updates vuls2-path data conversion to preserve user-supplied CPE forms and to carry exploit/mitigation metadata through merges.

Changes:

  • Update the vuls2 detector to run CPE detection (NVD) alongside OS-package detection in one DB session and convert CPEs to/from CPE 2.3 FS.
  • Update the main detector flow to run go-cve-dictionary CPE detection first (stripping NVD) and then run vuls2 detection.
  • Bump dependencies (vuls2, vuls-data-update, and related indirect modules) and update tests/source ID handling.

Reviewed changes

Copilot reviewed 5 out of 6 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
detector/detector.go Collect CPEs from scan result snapshot + OWASP + synthesized Apple CPEs; strip NVD results from go-cve-dict CPE detection; run unified vuls2 detection afterwards.
detector/vuls2/vuls2.go Add vuls2 CPE detection path, CPE FS conversion/restoration, CPE-AND relax pruning, exploit/mitigation conversion & merge behavior.
detector/vuls2/vendor.go Update Red Hat VEX source ID handling and sorting/tag comparison logic.
detector/vuls2/vuls2_test.go Update tests for new exported helper signatures and updated criterion/source types.
go.mod Pin updated vuls2 and vuls-data-update versions and bump several direct/indirect deps.
go.sum Module sum updates corresponding to dependency bumps.
Comments suppressed due to low confidence (1)

detector/vuls2/vuls2.go:101

  • When merging vuls2-detected VulnInfo into an existing r.ScannedCves entry (e.g., after DetectCpeURIsCves ran first), the merge currently ignores CpeURIs, Exploits, and Mitigations, so those fields can be silently dropped for CVEs present in both paths. Also, viBase.CveContents may be nil for existing entries, which would panic on assignment.
	vulnInfos, err := postConvert(vuls2Scanned, vuls2Detected, fsToOriginalCPE)
	if err != nil {
		return xerrors.Errorf("Failed to post convert. err: %w", err)
	}


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread detector/vuls2/vuls2.go Outdated
Comment thread detector/detector.go Outdated
shino added a commit that referenced this pull request Jun 10, 2026
…hs run

Per Copilot review on #2575:

- detect(): when the same RootID was flagged by both the OS-package and
  CPE paths, addDetection fetched Vulnerability/Advisory data only for
  the FIRST detection's ecosystem/datasources and just appended the
  later detection — dropping the content the other path needs. Collect
  detections first, then fetch once per RootID without
  ecosystem/datasource narrowing (mirrors vuls2's pkg/detect.detect).
- detector.go: format the cpeURIs slice with %v instead of %s in the
  DetectCpeURIsCves error message.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@shino shino requested a review from Copilot June 10, 2026 07:13

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 6 changed files in this pull request and generated 1 comment.

Comment thread detector/detector.go Outdated
shino added a commit that referenced this pull request Jun 10, 2026
…mode

Per Copilot review on #2575: DetectPkgCves had become post-processing
only, with the actual vuls2 detection moved to the main Detect flow.
But server mode (server/server.go) calls DetectPkgCves directly and
never the main flow — OS-package detection silently disappeared there.

Restore the master-era division of responsibility:

- DetectPkgCves runs vuls2.Detect (OS packages / Microsoft KB) again,
  so every caller — report flow and server mode alike — gets package
  detection, and the FixState / ListenPortStats post-processing inside
  DetectPkgCves once again runs AFTER detection results exist.
- vuls2.Detect drops the cpeURIs parameter; CPE detection moves to a
  new vuls2.DetectCPEs, called from the main Detect flow after the CPE
  list is collected. DetectCPEs suppresses the OS-package /
  Microsoft-KB inputs in the converted scan result so the two entry
  points cannot double-detect packages (vuls2's merge appends
  AffectedPackages, so a double run would duplicate them).
- Both share the session/convert plumbing via detectWith.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@shino shino requested a review from Copilot June 10, 2026 07:27

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 6 changed files in this pull request and generated 2 comments.

Comment thread detector/vuls2/vuls2.go Outdated
Comment thread detector/vuls2/vuls2.go Outdated
shino added a commit that referenced this pull request Jun 10, 2026
Per Copilot review on #2575:
- isVulnerableTrue's comment claimed CPE criteria have no Vulnerable
  concept, but the implementation does consult CPE.Vulnerable; restate
  the doc to match (Version and CPE check their flag, the rest default
  to true).
- "vuls/ normalises" → "vuls normalises".

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@shino shino requested a review from Copilot June 10, 2026 07:34

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 6 changed files in this pull request and generated 3 comments.

Comment thread detector/vuls2/vuls2.go Outdated
Comment thread detector/vuls2/vuls2.go Outdated
Comment thread detector/detector.go Outdated

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 6 changed files in this pull request and generated 3 comments.

Comment thread detector/detector.go Outdated
Comment thread detector/vuls2/vuls2.go
Comment thread detector/vuls2/vuls2.go Outdated
shino added a commit that referenced this pull request Jun 10, 2026
…os + dedup FS CPEs

Per Copilot review on #2575:

- Detect's ScannedCves merge loop now dedup-appends Exploits and
  Mitigations when the CVE already exists (e.g. registered first by the
  go-cve-dictionary non-NVD path); previously the vuls2-derived entries
  were silently dropped for mixed-source CVEs.
- toFSCPEs dedups by FS form: the first user-supplied form wins in both
  the detection list and the reverse map, instead of re-detecting the
  same FS string for each duplicate input.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@shino shino requested a review from Copilot June 10, 2026 07:56

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 6 changed files in this pull request and generated 2 comments.

Comment thread detector/vuls2/vuls2.go
Comment thread detector/vuls2/vuls2.go Outdated
shino added a commit that referenced this pull request Jun 10, 2026
Per Copilot review on #2575:

- The content-level Exploit/Mitigations mapping ran for every
  CveContentType while hard-coding ExploitTypeNVD — non-NVD sources
  (e.g. Red Hat) that populate the same content slots would have their
  entries mis-labelled. Gate the mapping on cctype == models.Nvd and
  set the Mitigation CveContentType to models.Nvd explicitly, mirroring
  ConvertNvdToModel exactly.
- Correct the §2 comment: AffectedPackages carries omitempty, so nil
  vs empty slice is invisible in JSON; the real reasons to keep ps nil
  are skipping the allocation and the nil-means-not-detected in-memory
  convention.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@shino shino requested a review from Copilot June 10, 2026 08:05

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 6 changed files in this pull request and generated 4 comments.

Comment thread detector/vuls2/vuls2.go Outdated
Comment thread detector/vuls2/vuls2.go
Comment thread detector/vuls2/vuls2_test.go Outdated
Comment thread detector/detector.go Outdated
shino added a commit that referenced this pull request Jun 10, 2026
Rename DetectPkgCves to DetectCves and fold the CPE pipeline into it,
so every caller shares one detection entry point:

- OS packages / Microsoft KB: vuls2.Detect, gated by family support
  (unchanged).
- CPE URIs: DetectCpeURIsCves (go-cve-dictionary, non-NVD sources,
  NVD stripped) followed by vuls2.DetectCPEs — moved here from the
  main Detect flow. No family gate, so families the package path
  skips (pseudo / macOS) still get their CPE lists checked.
- FixState / ListenPortStats post-processing keeps running last,
  after all detection results exist.

The vuls2 library keeps Detect (OS pkg) and DetectCPEs (CPE) as
separate entry points on purpose: when CPE-capable sources beyond NVD
(JVN, Vulncheck, ...) move into the vuls2 DB, per-source confidence
aggregation can then live entirely inside the CPE path.

server mode passes nil cpes/cpeURIs — the CPE stages are no-ops and
today's behaviour (package detection only) is preserved, while the
plumbing to enable CPE detection there later is now a parameter
change away. Also resolves the stale main-flow comment that still
referred to vuls2.Detect for CPE detection (Copilot, #2575).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
shino added a commit that referenced this pull request Jun 10, 2026
…tion passes

Per Copilot review on #2575: with Detect (OS packages) and DetectCPEs
(CPE URIs) running as separate passes over the same ScannedCves map,
the merge into an existing VulnInfo

- appended CveContents wholesale, duplicating identical entries (same
  SourceLink) when both passes detected the same CVE — now deduped by
  SourceLink per content type;
- never merged CpeURIs, leaving them empty for CVEs first registered
  by the package pass — now dedup-appended.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
shino added a commit that referenced this pull request Jun 10, 2026
Per Copilot review on #2575:

- Test_pruneCriteria gains an ecosystem arg and two cases: under
  ecosystem "cpe" an AND with an env-only vulnerable=false guard
  subtree keeps the vulnerable=true product criterion (relax skips the
  guard); under a non-CPE ecosystem the same shape fails the AND.
- Test_postConvert now threads fsToOriginalCPE; the
  "redhat vex + epel + cpe" case maps the detected FS string back to a
  user-supplied CPE 2.2 URI and asserts CpeURIs carries the original
  form.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@shino shino requested a review from Copilot June 10, 2026 08:21
shino and others added 21 commits June 19, 2026 22:13
detectWith runs once for packages and once for CPEs in a report run;
two identical "N CVEs are detected with vuls2" lines were
indistinguishable.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…points

DetectPkgCves/DetectCpeURIsCves advertise themselves as library entry
points, but a zero-value ScanResult (ScannedCves == nil) panicked on
the first map assignment in mergeIntoScannedCves or the
go-cve-dictionary helper. The report and server flows always
initialize the map, so this only bit direct library consumers — guard
both entry paths and pin it with a merge test case.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The skip gate listed every Has* accessor, but go-cve-dictionary's
GetByCpeURI admits a detail on Nvd/Vulncheck/Jvn/Fortinet/Paloalto/
Cisco matches only — EUVD and MITRE are enrichment contents that ride
along and are no detection basis (getMaxConfidence has no tier for
them either). An NVD-only detection carrying EUVD/MITRE content
therefore survived the NVD strip and registered with a zero-value
confidence. Align the gate with the dictionary's admission sources.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A criterion with version=* and no Range / no CPEMatches states that
every version of the product is affected, so an accepted match is
exact regardless of the scanned version — not vendor:product. The
previous code demoted all version-unrestricted accepts to
VendorProductMatch, which was wrong for NVD (where a bare version=*
is a deliberate "all versions" statement).

Split the accept classification:
- version=* (no Range/CPEMatches): all versions affected -> exact
- version=NA: no version concept -> vendor:product (matches gocve)
- version-restricted + version-less query: cannot compare -> vendor:product
- version-restricted + concrete query: -> exact

JVN, whose every criterion is version=*, must not be inflated to
exact: it carries no version data at all. That demotion already lives
in toVuls0Confidence (JVN maps to JvnVendorProductMatch from whichever
tier it lands in); a comment now states why, and a postConvert case
pins JVN version=* -> JvnVendorProductMatch alongside NVD version=* ->
NvdExactVersionMatch.

cpecriterion.Accept already accepts version=* (no narrowing -> accept
on attribute match), and the NVD extractor emits such criteria, so no
vuls-data-update change is needed.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
JVN is not a vuls2 CPE source yet (go-cve-dictionary serves it), so
the exact-tier JVN case in toVuls0Confidence is dead code today.
Reframe the comment from "JVN never reaches exact" (an active-demotion
claim) to "currently unreachable; when JVN migrates a version=* match
is legitimately exact — revisit then (no JvnExactVersionMatch exists)".
Drop the postConvert JVN case that forced this unreachable path and
pinned the placeholder VP behaviour.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
walkCPECriteria's recursive walk propagated the child error bare while
its sibling walkPkgCriteria wraps the same recursion with
xerrors.Errorf("Failed to walk criteria. err: %w", err). Match the
convention so a deep CPE-tree error keeps its locality.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The previous commit wrapped walkCPECriteria's recursion with "Failed
to walk cpe criteria", the same message the walkVulnerabilityDetections
closure already adds — a literal double wrap. walkPkgCriteria's
recursion uses the generic "Failed to walk criteria" with the closure
supplying the "pkg criteria" specificity; mirror that for cpe so the
two paths are symmetric and the dup is gone.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
master wraps the walkCriteria result with "Failed to walk criteria" at
the walkVulnerabilityDetections call site; the refactor's IIFE
dispatcher left a bare `return err` there, dropping that wrap. Replace
the IIFE with a plain if/else so each branch wraps the walk error
("Failed to walk cpe/pkg criteria") and returns directly — no bare
propagation, no double wrap, and no tuple-returning closure.

The walkCPECriteria / walkPkgCriteria top-level bare returns stay as
is: they propagate an already recursion-wrapped error up to this
caller wrap, which is net-identical to master's recursion-wrap +
caller-wrap.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
walkCPECriteria / walkPkgCriteria propagated walk(ca)/walk(pruned)
errors bare at their top-level. Wrap them too so no error return in
the CPE detection path is bare — a reader never has to ask "why isn't
this one wrapped?". The message repeats the recursion's "Failed to
walk criteria" by design; consistency beats avoiding the minor
duplication.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Drop the verified-replaces-unverified special case on Exploits and
match the AppendIfMissing convention of Confidences / DistroAdvisories
/ Mitigations. A lone exploit whose only difference across passes is
the Verified flag would mean the upstream data disagrees with itself —
not something this merge should arbitrate — and a one-off "replace"
rule here was the odd one out that invited "why is this different?".

The merge test now asserts first-wins for a same-key exploit.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Exploits.AppendIfMissing got a test but its sibling Mitigations did
not. Mirror it to lock in the dedup key (CveContentType, URL,
Mitigation): append when missing, first-wins on the same key, and a
differing Mitigation text being a distinct entry.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…t test

The vuls2 CPE path now emits NVD CveContents / Exploits / Mitigations,
but FillCvesWithGoCVEDictionary later appended its own NVD-derived
copies with no dedup, so vuls2-detected NVD CVEs ended up with
duplicated cveContents[nvd], exploit, and mitigation entries.

- Exploits / Mitigations: switch the plain append to AppendIfMissing.
- nvd CveContents: drop any pre-filled nvd entries before appending
  go-cve-dictionary's, since gocve is the NVD content authority during
  the transition. A SourceLink dedup cannot be used here — gocve emits
  one nvd CveContent per CVSS source, all sharing the NVD SourceLink,
  so it would collapse those legitimate per-source entries.

Also gofmt the Mitigations test added in the previous commit (the
goimports gap that failed lint).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
vendorProductEligible mirrored go-cve-dictionary's match() by re-evaluating
a range with RPM-style comparison when the semver comparator could not parse
the query (e.g. juniper "21.4r3", safari "1.0.0b1"), reporting in-range hits
at VendorProductMatch. Empirically that fallback is neither necessary nor
sufficient:

- Not necessary: with a well-formed query the matcher already reaches every
  affected version at ExactVersionMatch. Normalising juniper's joined form to
  version=21.4 / update=r3 makes the same wildcard range ("< 22.2") evaluate
  as plain semver; detection is byte-identical with the fallback on or off
  (199 CVEs either way). The fallback only compensates for a malformed query
  representation, which is a detect-side normalizer's responsibility.

- Not sufficient: RPM comparison gives no consistent order for NVD's messy
  pre-release strings. "4.0_beta" > "4.0" but "4beta" < "4.0" (the same
  "4 beta" ordered oppositely by NVD spelling), and "1.0.0b1" > "1.0.0". It
  only lands correct when the leading version digits already dominate; near a
  boundary it mis-orders. Vendors like safari, whose NVD version formats are
  inconsistent, cannot be served by any version-comparison heuristic here.

The fallback only ever produced the retired RoughVersionMatch tier (folded
to VendorProductMatch), so removing it leaves ExactVersionMatch untouched;
it drops only the fuzzy in-range VP guesses for non-semver query versions.
Splitting such version strings belongs in a future detect-side query
normalizer (tractable for regular forms like juniper, a known gap for
irregular ones like safari).

Simplify vendorProductEligible to the version-less cases (query ANY/NA, or
criterion NA), delete rangeVendorProductEligible, and drop the now-unused
go-rpm-version and cpecriterion range/criterion imports.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…iving it

CPE match-quality determination moved upstream into vuls-data-update's
cpecriterion.Match (exact / version-unconfirmed). walkCPECriteria is now a
pure projection: it reads FilteredCriterion.Accepts.CPE.{Exact,
VersionUnconfirmed} and folds the supporting scanned CPEs up the AND/OR tree
into vuls0's exact / vendor:product tiers — no more raw CPE re-judgement.

Removed from vuls0: vendorProductEligible, the inline pvpEqual, the
accept-empty vendor:product fallback, and the scanned-version re-derivation
(go-cpe WFN matching, range compare, and the retired RPM fallback are gone
from this layer). The version=NA "all versions" detection the fallback used
to supply now comes through cpecriterion.Match at VersionUnconfirmed, so
NVD/cisco/linux detection is byte-identical (vuls-compare gate).

walkCPECriteria takes a sourceID and demotes JVN matches (JVNFeedRSS /
JVNFeedDetail) from exact to vendor:product — JVN carries no version data, a
source-semantics call kept in vuls0 rather than the source-agnostic matcher.
toVuls0Confidence's JVN exact branch is now documented as unreachable for that
reason. RoughVersionMatch stays retired.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Follow-up cleanup now that exact/vendor:product categorization moved
into vuls-data-update and the RPM fallback is gone:

- unify the three dedup shapes (walkCPECriteria closure, the manual
  slices.Contains loop in walkVulnerabilityDetections, and the three
  near-identical blocks in postConvert) into one appendMissing helper;
  let postConvert own accumulation-dedup so walkVulnerabilityDetections
  appends uniformly
- rename the sourceData/affected cpes field to exactCpes for symmetry
  with vpCpes, making the two-tier model self-documenting
- refresh stale comments that still described the removed
  vendorProductEligible / RPM "fallback"; vpCpes is now simply the
  VersionUnconfirmed accepts

Behavior-preserving: detector/models tests pass, gofmt clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
mergeIntoScannedCves dedups every merged field (WindowsKBFixedIns,
DistroAdvisories, Confidences, CpeURIs, Exploits, Mitigations) except
CveContents, which plain-appended. The two vuls2 passes can never
produce the same content type, but the function also merges across a
go-cve-dictionary pass and caller-provided ScannedCves — now that vuls2
emits Nvd/Jvn-typed contents, such a base can already carry the same
type for a CVE, duplicating it into repeated sources/links in reports.

Dedup identical entries on append (full-content equality via
reflect.DeepEqual; SourceLink is not an identity). Genuinely different
contents of the same type are still both kept — picking a winner
between conflicting sources stays out of scope here.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…qual

Avoid reflect in this production merge path. Compare on the
(type, CVE, source link) identity instead of full-content
reflect.DeepEqual, mirroring the key-based dedup the sibling merges
(Exploits, Mitigations, Confidences) already use.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
mergeIntoScannedCves blindly appended vi.AffectedPackages into the base
VulnInfo, so a base already carrying package statuses — a caller-
provided ScannedCves, or a prior detection pass — accumulated duplicate
or contradictory rows for the same package.

Use PackageFixStatuses.Store(), the type's own insert-or-update-by-name
helper (last write wins), matching the key-based dedup the other merged
fields use.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Point at the merged dependency chain now that the upstream PRs landed:
- vuls2 -> v0.0.1-alpha.0.20260619063813-98ac8e5258eb (nightly, #388)
- vuls-data-update -> v0.0.0-20260618100055-bfedbc246360 (#850),
  matching the version vuls2's nightly pins

Standalone (GOWORK=off) build and detector/models tests pass.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ve-dictionary

Move NVD enrichment to the vuls2 path so NVD content comes from one
place (the vuls2 DB), matching how NVD CPE detection already works:

- vuls2 enrich(): add NVDFeedCVEv2 to the enrichment DataSources.
- enrichNVD(): build CveContent[nvd] (+ NVD exploits/mitigations),
  mirroring the NVD branch of the CPE detection path; guarded so it does
  not duplicate an NVD-detected CVE's existing entry.
- FillCvesWithGoCVEDictionary: drop NVD CveContent and NVD-derived
  exploits/mitigations (kept VulnCheck/JVN/EUVD/Fortinet/MITRE/Paloalto/
  Cisco, and NVD cert alerts which vuls2 does not provide).

Verified against a locally-built NVD vuls2 DB (db add nvd-feed-cve-v2):
with an empty go-cve-dictionary, NVD CPE detection still finds CVEs and
both detected and cross-source-enriched CVEs carry a single nvd
CveContent — confirming NVD now flows entirely through vuls2.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…-dictionary

US-CERT alerts are just NVD references whose URL contains "us-cert", so
they belong with the migrated NVD content. enrichNVD now derives
AlertDict.USCERT from those references (Title "US-CERT-<id>", matching
go-cve-dictionary's NvdCert), done regardless of whether the CPE
detection path already filled the nvd CveContent (that path does not
populate AlertDict).

FillCvesWithGoCVEDictionary stops filling US-CERT and now sets only
AlertDict.JPCERT, preserving the USCERT the vuls2 enrich pass set
earlier. JP-CERT stays on the go-cve-dictionary path because it derives
from JVN, which is not migrated.

Verified against the locally-built NVD vuls2 DB: enrich of
CVE-2021-32951 yields USCERT "US-CERT-icsa-21-229-02" with empty
go-cve-dictionary.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@MaineK00n

Copy link
Copy Markdown
Collaborator

enrichNVD has no unit test coverage in the enrich path.

Test_enrich covers the other enrich sources (redhat-cve, cisa-kev, vulncheck-kev, enisa-kev, metasploit, exploit-*, nuclei-repository), but there's no NVD case, and testdata/fixtures/enrich/ has no nvd-feed-cve-v2 fixture. As a result, enrichNVD — the core of this refactor (851aa34, 8c59083) — is never exercised through the enrich entry point.

The NVD content assembly (toReference, exploit/mitigation lifting, CVSS mapping) is tested via Test_postConvert, but that's the detection-time postConvert path, not enrichNVD. Specifically, these branches are currently untested:

  1. NVDFeedCVEv2 -> enrichNVD dispatch and CveContent/Exploit/Mitigation generation when there's no pre-existing NVD content (hasContent == false).
  2. hasContent == true skip path — NVD CveContent already filled by the CPE detection path, so content/exploits/mitigations must be left in place.
  3. US-CERT alert derivation — no fixture anywhere contains a us-cert reference URL, so the single place US-CERT is now produced has zero coverage.

Could you add an nvd-feed-cve-v2 fixture to testdata/fixtures/enrich/ and a Test_enrich case (or two) covering:

  • hasContent == false: NVD CveContent + Exploit + Mitigation are generated, and US-CERT alerts are appended from a us-cert reference URL.
  • hasContent == true: existing NVD CveContent is preserved (not duplicated/overwritten), while US-CERT is still derived.

This would lock in both behaviors the refactor introduced and guard against regressions in the go-cve-dictionary -> vuls2 migration.

Add an nvd-feed-cve-v2 enrich fixture (CVE-2014-0160) and two Test_enrich
cases, exercising enrichNVD through the enrich entry point — previously
only the detection-time postConvert path touched NVD content assembly:

- hasContent == false: NVDFeedCVEv2 -> enrichNVD dispatch generates the
  nvd CveContent, NVD Exploit and Mitigation, and appends a US-CERT
  alert from a us-cert reference URL.
- hasContent == true: a pre-existing nvd CveContent (as the CPE
  detection path leaves it) is preserved — no duplicate/overwrite, no
  exploits/mitigations re-added — while US-CERT is still derived.

Locks in both behaviors of the go-cve-dictionary -> vuls2 NVD migration.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@shino

shino commented Jun 22, 2026

Copy link
Copy Markdown
Collaborator Author

Thanks — added in 6fc364b.

testdata/fixtures/enrich/nvd-feed-cve-v2/ (CVE-2014-0160) + two Test_enrich cases now exercise enrichNVD through the enrich entry point:

  • hasContent == false: NVDFeedCVEv2 -> enrichNVD dispatch generates the nvd CveContent, the NVD Exploit and Mitigation, and appends a US-CERT alert derived from a us-cert reference URL (US-CERT-TA14-098A).
  • hasContent == true: a pre-existing nvd CveContent (as the CPE detection path leaves it) is preserved — not duplicated/overwritten, exploits/mitigations not re-added — while US-CERT is still derived.

The fixture carries a us-cert reference so the US-CERT derivation path has coverage for the first time.

@MaineK00n MaineK00n left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🪦

CI did not fire for the previous pushes (Actions stopped registering
pull_request events repo-wide after 2026-06-22T00:11). Re-trigger.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@shino

shino commented Jun 22, 2026

Copy link
Copy Markdown
Collaborator Author

CI does not run, close temporally with wishes

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants