internal/pkg/runtime/launcher/oci

This package contains routines that configure and launch a container in an OCI bundle format, using a low-level OCI runtime, either crun or runc at this time. crun is currently preferred. runc is used where crun is not available.

Note - at present, all functionality works with either crun or runc. However, in future crun may be required for all functionality, as runc does not support some limited ID mappings etc. that may be beneficial in an HPC scenario.

The package contrasts with internal/pkg/runtime/launcher/native which executes Singularity format containers (SIF/Sandbox/squashfs/ext3), using one of our own runtime engines (internal/pkg/runtime/engine/*).

There are two flows that are implemented here.

• Basic OCI runtime operations against an existing bundle, which will be executed via the singularity oci command group. These are not widely used by end-users of singularity.
• A Launcher, that implements an Exec function that will be called by 'actions' (run/shell/exec) in --oci mode, and will:
  • Prepare an OCI bundle according to launcher.Options passed through from the CLI layer.
  • Execute the bundle, interactively, via the OCI Run operation.

Note - this area of code is under heavy development for experimental introduction in CE 3.11. It is likely that it will be heavily refactored, and split, in future.

Basic OCI Operations

The following files implement basic OCI operations on a runtime bundle:

oci_linux.go

Defines constants, path resolution, and minimal bundle locking functions.

oci_runc_linux.go

Holds implementations of the Run / Start / Exec / Kill / Delete / Pause / Resume / State OCI runtime operations.

See https://github.com/opencontainers/runtime-spec/blob/main/runtime.md#operations

These functions are thin wrappers around the runc/crun operations of the same name.

oci_conmon_linux.go

Hold an implementation of the Create OCI runtime operation. This calls out to conmon, which in turn calls crun or runc.

conmon is used to manage logging and console streams for containers that are started backgrounded, so we don't have to do that ourselves.

oci_attach_linux.go

Implements an Attach function, which can attach the user's console to the streams of a container running in the background, which is being monitored by conmon.

Testing

End-to-end flows of basic OCI operations on an existing bundle are tested in the OCI group of the e2e suite, e2e/oci.

Launcher Flow

The Launcher type connects the standard singularity CLI actions (run/shell/exec), to execution of an OCI container in a native bundle. Invoked with the --oci flag, this is in contrast to running a Singularity format container, with Singularity's own runtime engine.

spec_linux.go

Provides a minimal OCI runtime spec, that will form the basis of container execution that is roughly comparable to running a native singularity container with --compat (--containall).

mounts_linux.go

Provides code handling the addition of required mounts to the OCI runtime spec.

process_linux.go

Provides code handling configuration of container process execution, including user mapping.

launcher_linux.go

Implements Launcher.Exec, which is called from the CLI layer. It will:

• Create a temporary bundle directory.
• Use pkg/ocibundle/native to retrieve the specified image, and extract it in the temporary bundle.
• Configure the container by creating an appropriate runtime spec.
• Call the interactive OCI Run function to execute the container with crun or runc.

Namespace Considerations

An OCI container started via Launch.Exec as a non-root user always uses at least one user namespace.

The user namespace is created prior to calling runc or crun, so we'll call it an outer user namespace.

Creation of this outer user namespace is via using the RunNS function, instead of Run. The RunNS function executes the Singularity starter binary, with a minimal configuration of the fakeroot engine ( internal/pkg/runtime/engine/fakeroot/config).

The starter will create a user namespace and ID mapping, and will then execute singularity oci run to perform the basic OCI Run operation against the bundle that the Launcher.Exec function has prepared.

The outer user namespace from which runc or crun is called always maps the host user id to root inside the userns.

When a container is run in --fakeroot mode, the outer user namespace is the only user namespace. The OCI runtime config does not request any additional userns or ID mapping be performed by crun / runc.

When a container is not run in --fakeroot mode, the OCI runtime config for the bundle requests that crun / runc:

• Create another, inner, user namespace for the container.
• Apply an ID mapping which reverses the 'fakeroot' outer ID mapping.

I.E. when a container runs without --fakeroot, the ID mapping is:

• User ID on host (1001)
• Root in outer user namespace (0)
• User ID in container (1001)

Testing

End-to-end testing of ßthe launcher flow is via the e2e/actions suite. Tests prefixed oci.