vulnerability-operator

Scans SBOMs and Images for vulnerabilities

Overview

This operator scans all SBOMs from a git-repository for vulnerabilities using Grype. The result-list can be emitted as JSON-file served via an endpoint and/or as Prometheus metrics. There may be more targets in the future. The scans are done periodically.

Kubernetes Compatibility

The image contains versions of k8s.io/client-go. Kubernetes aims to provide forwards & backwards compatibility of one minor version between client and server:

vulnerability-operator	k8s.io/{api,apimachinery,client-go}	expected kubernetes compatibility
main	v0.30.0	1.29.x, 1.30.x, 1.31.x
0.24.0	v0.30.0	1.29.x, 1.30.x, 1.31.x
0.23.0	v0.29.3	1.28.x, 1.29.x, 1.30.x
0.22.0	v0.28.4	1.27.x, 1.28.x, 1.29.x
0.19.0	v0.27.4	1.26.x, 1.27.x, 1.28.x
0.17.0	v0.26.3	1.25.x, 1.26.x, 1.27.x
0.13.0	v0.25.4	1.24.x, 1.25.x, 1.26.x
0.8.0	v0.24.3	1.23.x, 1.24.x, 1.25.x
0.5.0	v0.23.5	1.22.x, 1.23.x, 1.24.x

However, the operator will work with more versions of Kubernetes in general.

Installation

Manifests

kubectl apply -f deploy/

Helm-Chart

Create a YAML file first with the required configurations or use helm-flags instead.

helm repo add ckotzbauer https://ckotzbauer.github.io/helm-charts
helm install ckotzbauer/vulnerability-operator -f your-values.yaml

Configuration

All parameters are cli-flags.

Parameter	Required	Default	Description
`verbosity`	`false`	`info`	Log-level (debug, info, warn, error, fatal, panic)
`cron`	`false`	`@hourly`	Backround-Service interval (CRON). All options from github.com/robfig/cron are allowed
`sources`	`false`	`git`	Comma-delimited list of sources to gather SBOMs from. Possible source currently only `git`
`targets`	`false`	`json`	Comma-delimited list of targets to sent vulnerability-data to. Possible targets `json`, `metrics`, `policyreport`
`grype-config-file`	`false`	`""`	Path to grype-config-file to specify ignore-rules.
`filter-config-file`	`false`	`""`	Path to filter-config-file to specify ignore- and audit-rules. (yaml formatted)
`only-fixed`	`false`	`false`	Only report CVEs where a fix is available.
`min-severity`	`false`	`medium`	Only report CVEs with a severity greater or equal (negligible, low, medium, high, critical).
`git-workingtree`	`false`	`/work`	Directory to place the git-repo.
`git-repository`	`true` when `git` target is used.	`""`	Git-Repository-URL (HTTPS).
`git-branch`	`false`	`main`	Git-Branch to checkout.
`git-path`	`false`	`""`	Folder-Path inside the Git-Repository.
`git-access-token`	`false`	`""`	Git-Personal-Access-Token with read-permissions.
`git-username`	`false`	`""`	Git-Username
`git-password`	`false`	`""`	Git-Password
`github-app-id`	`false`	`""`	GitHub App-ID.
`github-app-installation-id`	`false`	`""`	GitHub App-Installation-ID.
`reports-dir`	`false`	`/reports`	Directory to place the reports.

The flags can be configured as args or as environment-variables prefixed with VULN_ to inject sensitive configs as secret values.

Example Helm-Config

args:
  targets: metrics
  min-severity: low
  git-repository: https://github.com/XXX/XXX
  git-path: dev-cluster/sboms
  verbosity: debug
  cron: "0 0 * * * *"

envVars:
  - name: VULN_GIT_ACCESS_TOKEN
    valueFrom:
      secretKeyRef:
        name: "vulnerability-operator"
        key: "accessToken"

servicemonitor:
  enabled: true

Sources

Git

The contents of this git-repository are typically generated from the sbom-generator. All files named sbom.json, sbom.txt, sbom.xml or sbom.spdx are gathered regarding the git-* config-flags. You can use a token-based authentication (e.g. a PAT for GitHub) with --git-access-token, BasicAuth with username and password (--git-username, --git-password) or Github App Authentication (--github-app-id, --github-app-installation-id, env: VULN_GITHUB_APP_PRIVATE_KEY) The private-key has to be Base64 encoded.

Targets

JSON

All found vulnerabilities can be requested as file from the /reports/report.json endpoint. The data is structured like this:

Example JSON

[
  {
    "ID": "CVE-2019-19924",
    "Severity": "Medium",
    "Type": "rpm",
    "Package": "sqlite",
    "Installed": "3.7.17-8.el7_7.1",
    "FixedIn": [],
    "FixState": "wont-fix",
    "URLs": [
      "https://access.redhat.com/security/cve/CVE-2019-19924"
    ],
    "ImageID": "docker.elastic.co/beats/filebeat@sha256:e418d12e08a1b74140c9edc6bdc773110b0f802340e25e2716950bac86ae14ce",
    "Containers": [
      {
        "PodNamespace": "elastic-system",
        "PodName": "filebeat-filebeat-6xkf4",
        "ContainerName": "filebeat"
      },
      {
        "PodNamespace": "elastic-system",
        "PodName": "filebeat-filebeat-g6zbh",
        "ContainerName": "filebeat"
      },
      {
        "PodNamespace": "elastic-system",
        "PodName": "filebeat-filebeat-jkgnh",
        "ContainerName": "filebeat"
      }
    ]
  },
  {
    "ID": "CVE-2020-16250",
    "Severity": "Critical",
    "Type": "go-module",
    "Package": "github.com/hashicorp/vault/api",
    "Installed": "v1.3.1",
    "FixedIn": [],
    "FixState": "unknown",
    "URLs": [
      "https://www.hashicorp.com/blog/category/vault/",
      "https://github.com/hashicorp/vault/blob/master/CHANGELOG.md#151",
      "http://packetstormsecurity.com/files/159478/Hashicorp-Vault-AWS-IAM-Integration-Authentication-Bypass.html"
    ],
    "ImageID": "ghcr.io/kyverno/kyverno@sha256:4fc715e9287446222bf12b1245899b195ecea8beda54c6f6a3587373c376cad1",
    "Containers": [
      {
        "PodNamespace": "kyverno",
        "PodName": "kyverno-555dcf9f66-csmq5",
        "ContainerName": "kyverno"
      },
      {
        "PodNamespace": "kyverno",
        "PodName": "kyverno-555dcf9f66-gsphr",
        "ContainerName": "kyverno"
      }
    ]
  }
]

Metrics

Every CVE is exported with a Prometheus vuln_operator_cves gauge-metric for each container it appears in.

vuln_operator_cves{container_name="kyverno", cve="CVE-2020-16250", fix_state="unknown", image_id="ghcr.io/kyverno/kyverno@sha256:4fc715e9287446222bf12b1245899b195ecea8beda54c6f6a3587373c376cad1", package="github.com/hashicorp/vault/api", k8s_name="kyverno", k8s_namespace="kyverno", k8s_kind="Deployment", severity="Critical", type="go-module", version="v1.3.1"}

Note: The operator removes all metrics from the vector before re-populating it. In the meanwhile the data is not expressive.

Grafana Dashboard

There's a dashboard for Grafana to view the collected vulnerability metrics.

PolicyReport

With the policyreport target set, the operator stores the scan-results as PolicyReport CRs inside the cluster. Each PolicyReport object is owned by the corresponding pod to enable autocleanup by Kubernetes.

Example PolicyReport

apiVersion: wgpolicyk8s.io/v1alpha2
kind: PolicyReport
metadata:
  creationTimestamp: "2022-06-18T14:57:27Z"
  generation: 3
  labels:
    kubernetes.io/created-by: vulnerability-operator
  name: vuln-kyverno-688464bd95-55drc
  namespace: kyverno
  ownerReferences:
  - apiVersion: v1
    kind: Pod
    name: kyverno-688464bd95-55drc
    uid: 24bdccbb-653a-4005-8463-cf511a919037
  resourceVersion: "160903586"
  uid: e3cd1447-49ef-481d-b44b-cc38239ea870
results:
- category: github.com/theupdateframework/go-tuf
  message: 'github.com/theupdateframework/go-tuf: GHSA-66x3-6cw3-v5gj'
  policy: GHSA-66x3-6cw3-v5gj
  properties:
    FixedVersion: 0.3.0
    InstalledVersion: v0.0.0-20220211205608-f0c3294f63b9
    URL: https://github.com/advisories/GHSA-66x3-6cw3-v5gj
  resources:
  - kind: Pod
    name: kyverno-688464bd95-55drc
    namespace: kyverno
    uid: 24bdccbb-653a-4005-8463-cf511a919037
  result: fail
  severity: high
  source: vulnerability-operator
  timestamp:
    nanos: 728434599
    seconds: 1655564847
- category: google.golang.org/protobuf
  message: 'google.golang.org/protobuf: CVE-2015-5237'
  policy: CVE-2015-5237
  properties:
    FixedVersion: ""
    InstalledVersion: v1.28.0
    URL: https://github.com/google/protobuf/issues/760
  resources:
  - kind: Pod
    name: kyverno-688464bd95-55drc
    namespace: kyverno
    uid: 24bdccbb-653a-4005-8463-cf511a919037
  result: fail
  severity: high
  source: vulnerability-operator
  timestamp:
    nanos: 728434599
    seconds: 1655564847
- category: google.golang.org/protobuf
  message: 'google.golang.org/protobuf: CVE-2021-22570'
  policy: CVE-2021-22570
  properties:
    FixedVersion: ""
    InstalledVersion: v1.28.0
    URL: https://github.com/protocolbuffers/protobuf/releases/tag/v3.15.0
  resources:
  - kind: Pod
    name: kyverno-688464bd95-55drc
    namespace: kyverno
    uid: 24bdccbb-653a-4005-8463-cf511a919037
  result: fail
  severity: high
  source: vulnerability-operator
  timestamp:
    nanos: 728434599
    seconds: 1655564847
- category: google.golang.org/protobuf
  message: 'google.golang.org/protobuf: CVE-2015-5237'
  policy: CVE-2015-5237
  properties:
    FixedVersion: ""
    InstalledVersion: v1.28.0
    URL: https://github.com/google/protobuf/issues/760
  resources:
  - kind: Pod
    name: kyverno-688464bd95-55drc
    namespace: kyverno
    uid: 24bdccbb-653a-4005-8463-cf511a919037
  result: fail
  severity: high
  source: vulnerability-operator
  timestamp:
    nanos: 728434599
    seconds: 1655564847
- category: google.golang.org/protobuf
  message: 'google.golang.org/protobuf: CVE-2021-22570'
  policy: CVE-2021-22570
  properties:
    FixedVersion: ""
    InstalledVersion: v1.28.0
    URL: https://github.com/protocolbuffers/protobuf/releases/tag/v3.15.0
  resources:
  - kind: Pod
    name: kyverno-688464bd95-55drc
    namespace: kyverno
    uid: 24bdccbb-653a-4005-8463-cf511a919037
  result: fail
  severity: high
  source: vulnerability-operator
  timestamp:
    nanos: 728434599
    seconds: 1655564847
- category: github.com/theupdateframework/go-tuf
  message: 'github.com/theupdateframework/go-tuf: GHSA-66x3-6cw3-v5gj'
  policy: GHSA-66x3-6cw3-v5gj
  properties:
    FixedVersion: 0.3.0
    InstalledVersion: v0.0.0-20220211205608-f0c3294f63b9
    URL: https://github.com/advisories/GHSA-66x3-6cw3-v5gj
  resources:
  - kind: Pod
    name: kyverno-688464bd95-55drc
    namespace: kyverno
    uid: 24bdccbb-653a-4005-8463-cf511a919037
  result: fail
  severity: high
  source: vulnerability-operator
  timestamp:
    nanos: 728434599
    seconds: 1655564847
scope:
  kind: Pod
  name: kyverno-688464bd95-55drc
  namespace: kyverno
  uid: 24bdccbb-653a-4005-8463-cf511a919037
summary:
  error: 0
  fail: 6
  pass: 0
  skip: 0
  warn: 0

Context-based vulnerability filtering

You can apply filter rules to either ignore certain vulnerabilities or to move them into a separate metric. This is useful if you want to exclude vulnerabilities from your metric, for example false positives or issues that are outside of the applications execution path.

Filter rules support the following properties:

CVE
package name
container context (with the following subset)
- image
- namespace
- kind
- name

Container context properties support glob patterns * and ?, while CVE and package only match its exact value. The container context only applies, if all of its properties are matching (AND condition).

The filter rules are provided through a yaml formatted file. Its path is configured with --filter-config-file.

Example filter configuration

ignore:
  # Ignore any vulnerabilities in ruby gem rdoc
  - package: rdoc
  # Ignore CVE GHSA-8cr8-4vfw-mr7h
  - vulnerability: GHSA-8cr8-4vfw-mr7h
audit:
  # If GHSA-fp4w-jxhp-m23p was found in gitlab-ce images, move it the "audit" metric
  - vulnerability: GHSA-fp4w-jxhp-m23p
    context:
    - image: gitlab/gitlab-ce*
      namespace: "*"
      kind: "*"
      name: "*"
  # Move any CVE for the git package to the "audit" metric, if it was found in a gitlab-*-redis deployment
  - package: git
    context:
    - image: "*"
      namespace: "*"
      kind: Deployment
      name: gitlab-*-redis

Targets

For the Prometheus target, the separate metric vuln_operator_cves_audit contains matches from the audit section.

For the json target, a separate file audited.json is provided.

Security

The docker-image is based on a scratch-image to reduce the attack-surface and keep the image small. Furthermore the image and release-artifacts are signed with cosign and attested with provenance-files. The release-process satisfies SLSA Level 2. All of those "metadata files" are also stored in a dedicated repository ghcr.io/ckotzbauer/vulnerability-operator-metadata. Both, SLSA and the signatures are still experimental for this project. When discovering security issues please refer to the Security process.

# README

vulnerability-operator

Overview

Kubernetes Compatibility

Installation

Manifests

Helm-Chart

Configuration

Example Helm-Config

Sources

Git

Targets

JSON

Metrics

Grafana Dashboard

PolicyReport

Context-based vulnerability filtering

Targets

Security

License

Changelog

Contributing

Code of conduct

# Variables