Categorygithub.com/codeready-toolchain/toolchain-e2e
module
0.0.0-20241005003450-2e596219bdd7
Repository: https://github.com/codeready-toolchain/toolchain-e2e.git
Documentation: pkg.go.dev
Overview
Versions
1
Dependencies
0
Dependents
0
Files
0

# README

= CodeReady Toolchain E2E Tests

image:https://goreportcard.com/badge/github.com/codeready-toolchain/toolchain-e2e[Go Report Card, link="https://goreportcard.com/report/github.com/codeready-toolchain/toolchain-e2e"] image:https://godoc.org/github.com/codeready-toolchain/toolchain-e2e?status.png[GoDoc,link="https://godoc.org/github.com/codeready-toolchain/toolchain-e2e"]

This repo contains e2e tests for https://github.com/codeready-toolchain/host-operator[host] and https://github.com/codeready-toolchain/member-operator[member] operators of CodeReady Toolchain.

== Build

Requires Go version 1.20.x (1.20.11 or higher) - download for your development environment https://golang.org/dl/[here].

This repository uses https://github.com/golang/go/wiki/Modules[Go modules].

== Step by step guide - running in CodeReady Containers

Refer to link:CRC.adoc[this guide] for detailed instructions on running the e2e tests in a local CodeReady Containers cluster.

== End-to-End Tests

The e2e tests are executed against host and member operators running in OpenShift. The operators are built from the https://github.com/codeready-toolchain/host-operator[host-operator] and https://github.com/codeready-toolchain/member-operator[member-operator] repositories.

Since the changes in the e2e repo sometimes require changes in some of the operator repositories at the same time, the logic that executes the e2e tests supports a feature of pairing the e2e PR with one other PR based on branch names. Before the e2e tests are executed in openshift-ci, the logic automatically tries to pair a PR opened for this (toolchain-e2e) repository with a branch of the same name that potentially could exist in any of the developer's fork of the operator repositories.

For example, if a developer with GH account cooljohn opens a PR (for toolchain-e2e repo) from a branch fix-reconcile, then the logic checks if there is a branch fix-reconcile also in the cooljohn/host-operator and cooljohn/member-operator forks. Let's say that cooljohn/host-operator contains such a branch but cooljohn/member-operator doesn't, then the logic:

  1. clones latest changes of both repos https://github.com/codeready-toolchain/host-operator[codeready-toolchain/host-operator] and https://github.com/codeready-toolchain/member-operator[codeready-toolchain/member-operator]
  2. fetches the fix-reconcile branch from cooljohn/host-operator fork
  3. merges master branch with the changes from fix-reconcile branch inside of host-operator repo
  4. builds images from the merge branch of host-operator repo and from master branch of member-operator repo & deploys them to OpenShift
  5. runs e2e tests taken from the opened PR

It would work analogically also for the case when none of the repositories contain the branch name. However, if both repositories contain the branch name then it will result in an error. This is by design because OLM does not ensure that both the host-operator and member-operator will be updated at the same time in production. Prohibiting PR pairing with more than one repo helps ensure they do not depend on one another and can be updated separately.

If you still don't know what to do with e2e tests in some use-cases, go to <> section where all use-cases are covered.

NOTE: You can also pair a toolchain-e2e PR with a PR from the https://github.com/kubesaw/ksctl repository, but this is one-directional only (e2e tests are not yet executed in the ksctl repo). To rely on a locally installed version of the ksctl binary, set USE_INSTALLED_KSCTL=true. If you want to build and use the ksctl binary from your local kubesaw/ksctl repository, set the KSCTL_REPO_PATH variable to point to your kubesaw/ksctl folder.

=== Running Locally

Prerequisites:

  • Install the link:required_tools.adoc[required tools].
  • Configure link:quay.adoc[your quay account for dev deployment].

==== Running in a Development Environment

See the procedure to install the Dev Sandbox in a development environment link:dev_install.adoc[here].

=== Running End-to-End Tests

. link:quay.adoc[Configure your quay account for dev deployment] Although the e2e tests are in the separated repository than the actual operators are, it's still possible to run them against the current code that is at HEAD of the operator repositories. There are multiple Makefile targets that will execute the e2e tests, they just differ in where the operators' code is taken from:

The e2e tests will take care of creating all needed namespaces with random names (or see below for enforcing some specific namespace names). It will also create all required CRDs, role and role bindings for the service accounts, build the Docker images for both operators and push them to the OpenShift container registry. Finally, it will deploy the operators and run the tests using the operator-sdk.

NOTE: you can override the default namespace names where the end-to-end tests are going to be executed - eg.: make test-e2e HOST_NS=my-host MEMBER_NS=my-member file.

NOTE: you can disable SSL/TLS certificate verification in tests setting the DISABLE_KUBE_CLIENT_TLS_VERIFY variable to true - eg.: make test-e2e DISABLE_KUBE_CLIENT_TLS_VERIFY=true. This flag helps when you test in clusters using Self-Signed Certificates.

NOTE: you can specify a regular expression to selectively run particular test cases by setting the TESTS_RUN_FILTER_REGEXP variable. eg.: make test-e2e TESTS_RUN_FILTER_REGEXP="TestSetupMigration". For more information see the https://pkg.go.dev/cmd/go#hdr-Testing_flags[go test -run documentation].

=== Running/Debugging e2e tests from your IDE

In order to run/debug tests from your IDE you'll need to export some required env variables, those will be used by the test framework to interact with the operator namespaces and the other toolchain resources in you cluster. Following snippet of code should be TEMPORARILY added at the top of the test you want to run/debug from your IDE:

os.Setenv("MEMBER_NS","toolchain-member-18161051")
os.Setenv("MEMBER_NS_2","toolchain-member2-18161051")
os.Setenv("HOST_NS","toolchain-host-18161051")
os.Setenv("REGISTRATION_SERVICE_NS","toolchain-host-18161051")
os.Setenv("KUBECONFIG", "~/aws-cluster-test/my-devsandbox/auth/kubeconfig")

example of Test case code containing the debugging env variables:

package parallel

import (
	"context"
	"os"
	"testing"
)

func TestCreateSpaceRequest(t *testing.T) {
	os.Setenv("MEMBER_NS","toolchain-member-18161051")
	os.Setenv("MEMBER_NS_2","toolchain-member2-18161051")
	os.Setenv("HOST_NS","toolchain-host-18161051")
	os.Setenv("REGISTRATION_SERVICE_NS","toolchain-host-18161051")
	os.Setenv("KUBECONFIG", "~/aws-cluster-test/my-devsandbox/auth/kubeconfig")
	// some more code here ...

	t.Run("create space request", func(t *testing.T) {
        // test case implementation here ...
....

NOTE: replace the values with the ones from your dev/test environment and REMEMBER TO REMOVE THE SNIPPET BEFORE COMMITTING THE CODE OR OPENING A PR IN GH :)

===== What To Do

If you are still confused by the different e2e/operator location, execution and branch pairing, see the following cases and needed steps:

== Deploying End-to-End Resources Without Running Tests

All e2e resources (host operator, member operator, registration-service, CRDs, etc) can be deployed without running tests:

  • make dev-deploy-e2e-local - deploys the same resources as make test-e2e-local but doesn't run tests.

  • make dev-deploy-e2e - deploys the same resources as make test-e2e but doesn't run tests.

By default these targets deploy resources to toolchain-host-operator and toolchain-member-operator namespaces.

NOTE: If running in CodeReady Containers eval $(crc oc-env) is required.

== How to Test Mailgun/Twilio Notifications in a Dev Environment

# Packages

No description provided by the author
No description provided by the author
No description provided by the author