README.md

<!-- This document is generated by `make docs` from `.gitlab/README.md` -->

# OpenTofu CI/CD Component

This project is home to the **OpenTofu CI/CD component** and it's related assets,
like the `gitlab-tofu` wrapper script and OCI images containing that script
together with an OpenTofu version.

Read more:

- [CI/CD components](https://docs.gitlab.com/ee/ci/components)
- [Development guide for GitLab CI/CD components](https://docs.gitlab.com/ee/development/cicd/components)
- [CI/CD Catalog](https://docs.gitlab.com/ee/ci/components/index.html#cicd-catalog)

**Note**: Please make sure to use a released version of this CI/CD component.
You find all releases on the [Releases Overview Page](https://gitlab.com/components/opentofu/-/releases).

♻️ **Migrating from the Terraform CI/CD templates?** Check **[this](#migrating-from-the-terraform-cicd-templates)** out.

[[_TOC_]]

## Usage

```yaml
include:
  - component: $CI_SERVER_FQDN/components/opentofu/full-pipeline@<VERSION>
    inputs:
      # The version must currently be specified explicitly as an input,
      # to find the correctly associated images. # This can be removed
      # once https://gitlab.com/gitlab-org/gitlab/-/issues/438275 is solved.
      version: <VERSION> # component version
      opentofu_version: <OPENTOFU_VERSION>

stages: [validate, test, build, deploy, cleanup]

---

# ... or without the destroy jobs:
include:
  - component: $CI_SERVER_FQDN/components/opentofu/validate-plan-apply@<VERSION>
    inputs:
      # The version must currently be specified explicitly as an input,
      # to find the correctly associated images. # This can be removed
      # once https://gitlab.com/gitlab-org/gitlab/-/issues/438275 is solved.
      version: <VERSION> # component version
      opentofu_version: <OPENTOFU_VERSION>

stages: [validate, build, deploy]

# ... or in a child pipeline:
include:
  - component: $CI_SERVER_FQDN/components/opentofu/validate-plan-apply@<VERSION>
    inputs:
      # The version must currently be specified explicitly as an input,
      # to find the correctly associated images. # This can be removed
      # once https://gitlab.com/gitlab-org/gitlab/-/issues/438275 is solved.
      version: <VERSION> # component version
      opentofu_version: <OPENTOFU_VERSION>
      trigger_in_child_pipeline: true
```

A concrete example may look like this:

```yaml
# Using version `0.10.0`:
include:
  - component: $CI_SERVER_FQDN/components/opentofu/full-pipeline@0.10.0
    inputs:
      # The version must currently be specified explicitly as an input,
      # to find the correctly associated images. # This can be removed
      # once https://gitlab.com/gitlab-org/gitlab/-/issues/438275 is solved.
      version: 0.10.0 # component version
      opentofu_version: 1.6.1

stages: [validate, test, build, deploy, cleanup]

---

# ... in case you absolutely know what you are doing and are
# aware that this may introduce breaking changes, you may use the latest release:
include:
  - component: $CI_SERVER_FQDN/components/opentofu/full-pipeline@~latest
    inputs:
      # The version must currently be specified explicitly as an input,
      # to find the correctly associated images. # This can be removed
      # once https://gitlab.com/gitlab-org/gitlab/-/issues/438275 is solved.
      version: latest # component version
      opentofu_version: 1.6.1

stages: [validate, test, build, deploy, cleanup]
```

Or import all jobs as hidden templates ready to be extended:

```yaml
include:
  - component: $CI_SERVER_FQDN/components/opentofu/job-templates@<VERSION>
    inputs:
      # The version must currently be specified explicitly as an input,
      # to find the correctly associated images. # This can be removed
      # once https://gitlab.com/gitlab-org/gitlab/-/issues/438275 is solved.
      version: <VERSION> # component version
      opentofu_version: <OPENTOFU_VERSION>

stages: [...]

fmt:
  extends: [.opentofu:fmt]

...
```

### OpenTofu Version

The OpenTofu version can be specified with the `opentofu_version` input.
More details can be found [here](#available-opentofu-versions).

### Base Image OS

The GitLab OpenTofu images come in multiple base image variants:

- `alpine` (default)
- `debian`

The base image OS can be specified with the `base_os` input.

### GitLab-managed Terraform state backend

This component - by leveraging the [`gitlab-tofu`](src/gitlab-tofu.sh) CLI internally -
automatically configures the
[GitLab-managed Terraform state backend](https://docs.gitlab.com/ee/user/infrastructure/iac/terraform_state.html).
The only thing required is that the Terraform configuration must specify an empty `http` backend block, like this:

```hcl
terraform {
  backend "http" {}
}
```

We recommend having a dedicated `backend.tf` file inside your `root_dir`
with the aforementioned block.

### Access to Terraform Module Registry

Similar to automatically configuring the [GitLab-managed Terraform state backend]
the component also sets up credentials to authenticate with the
[Terraform Module Registry](https://docs.gitlab.com/ee/user/packages/terraform_module_registry/)
of the project the pipeline runs in.
It basically sets the `TF_TOKEN_<domain>` variable to the `$CI_JOB_TOKEN`, where `<domain>` is
the GitLab instance domain, for example for GitLab.com this would set `TF_TOKEN_gitlab_com` to
the `$CI_JOB_TOKEN`. However, it'll only do so if the variable is not already provided.
Thus, if you want to authenticate differently or to another Terraform Module Registry,
you may just provide the `TF_TOKEN_<domain>` variable yourself, e.g. via CI/CD variables.

### Access to GitLab via `glab` or GitLab Terraform Provider

The GitLab CLI `glab` is pre-installed in all the images.
If you want to use `glab` or the GitLab Terraform Provider we recommend configuring
a CI/CD variable called `GITLAB_TOKEN`.
This will automatically authenticate both tools.
For `glab` you can just start using it, for the Terraform Provider you just need to define
the provider requirement. Tofu will do the rest.

### Opinionated Templates

This component repository also provides some templates that may often be used,
for example one that only runs validation (`fmt` and `validate`), plan and an apply,
but no destructive actions.

- [`validate-plan`](templates/validate-plan.yml)
- [`validate-plan-apply`](templates/validate-plan-apply.yml)
- [`validate-plan-destroy`](templates/validate-plan-destroy.yml)

These templates support the `trigger_in_child_pipeline` input which will
include the component but run all its job in a child pipeline. This may be useful in cases
where you want to run dedicated child pipeline for each of your environments.

### Job Templates

Instead of including the `full-pipeline` or another opinionated template,
it's also possible to include individual jobs
and compose your own pipeline, for example, to just run the `fmt` job you can do:

```yaml
include:
  - component: $CI_SERVER_FQDN/components/opentofu/fmt@<VERSION>
    inputs:
      # The version must currently be specified explicitly as an input,
      # to find the correctly associated images. # This can be removed
      # once https://gitlab.com/gitlab-org/gitlab/-/issues/438275 is solved.
      version: <VERSION>
      opentofu_version: 1.6.1
      root_dir: tofu/
```

Or you can also include the `job-templates` template, that will include
all available OpenTofu jobs as hidden job templates prefixed with `.opentofu:`.
Those are especially useful when you want to minimize your includes and
you want to extend the jobs:

```yaml
include:
  - component: $CI_SERVER_FQDN/components/opentofu/job-templates@<VERSION>
    inputs:
      # The version must currently be specified explicitly as an input,
      # to find the correctly associated images. # This can be removed
      # once https://gitlab.com/gitlab-org/gitlab/-/issues/438275 is solved.
      version: <VERSION>
      opentofu_version: 1.6.1

plan:
  extends: [.opentofu:plan]
  parallel:
    matrix:
      - TF_ROOT: test/
      - TF_ROOT: prod/
```

Have a look at the [`full-pipeline`](templates/job-templates.yml) for how it's constructed.

The following job components exist:

- [`fmt`](templates/fmt.yml): Check formatting of configuration files.
- [`validate`](templates/validate.yml): Validate configuration.
- [`test`](templates/test.yml): Test configuration.
- [`plan`](templates/plan.yml): Plan an apply or destroy.
- [`apply`](templates/apply.yml): Apply a configuration.
- [`destroy`](templates/destroy.yml): Destroy a configuration.
- [`delete-state`](templates/delete-state.yml): Delete the GitLab-managed Terraform state.
- [`custom-command`](templates/custom-command.yml): Run a custom OpenTofu command.
- [`module-release`](templates/module-release.yml): Release an OpenTofu module to the GitLab Terraform Module Registry.

Have a look at the individual template spec to learn about the available inputs.

### Inputs

Please checkout the individual templates for the input definitions.
The [catalog page](https://gitlab.com/explore/catalog/components/opentofu)
beautifully renders the inputs for each templates - check it out!


### Available OpenTofu Versions

The following OpenTofu versions are available with this component via the `opentofu_version` input:

- [`1.8.5`](https://github.com/opentofu/opentofu/releases/tag/v1.8.5)
- [`1.9.0-alpha1`](https://github.com/opentofu/opentofu/releases/tag/v1.9.0-alpha1)
- [`1.8.4`](https://github.com/opentofu/opentofu/releases/tag/v1.8.4)
- [`1.8.3`](https://github.com/opentofu/opentofu/releases/tag/v1.8.3)
- [`1.8.2`](https://github.com/opentofu/opentofu/releases/tag/v1.8.2)
- [`1.8.1`](https://github.com/opentofu/opentofu/releases/tag/v1.8.1)
- [`1.8.0`](https://github.com/opentofu/opentofu/releases/tag/v1.8.0)
- [`1.7.3`](https://github.com/opentofu/opentofu/releases/tag/v1.7.3)
- [`1.6.2`](https://github.com/opentofu/opentofu/releases/tag/v1.6.2)

### Variables

(🚧 *This section is work in progress*)

Have a look at the [`src/gitlab-tofu.sh`](src/gitlab-tofu.sh) script and how the `TF_`-prefixed
variables are being used. You may set them according to your needs.

### Auto-forwarded predefined CI variables

The `gitlab-tofu` script auto-forwards some "popular"
[predefined CI/CD variables](https://docs.gitlab.com/ee/ci/variables/predefined_variables.html)
as OpenTofu variables.

The forwarded variables are:

- `CI_JOB_ID`
- `CI_COMMIT_SHA`
- `CI_JOB_STAGE`
- `CI_PROJECT_ID`
- `CI_PROJECT_NAME`
- `CI_PROJECT_NAMESPACE`
- `CI_PROJECT_PATH`
- `CI_PROJECT_URL`

To use them in your OpenTofu configuration you can define a string variable with the same name
but in *lower snake_case*. For example the `CI_PROJECT_NAME` CI/CD variable can be accessed in
the OpenTofu configuration like this:

```hcl
variable "ci_project_name" {
  type        = string
  description = "The name of the directory for the project."
}
```

### Install additional tools

The `gitlab-opentofu` container image deliberately comes with minimal tooling
to keep the image size small and be the least common denominator for our users.

However, it is sometimes necessary to install additional tools. To do that you
can overwrite the included jobs with a `before_script` entry. The `gitlab-opentofu`
image uses `alpine` as its base image and therefore `apk` can be used to install
the tools. For example to install `jq`:

```yaml
include:
  - component: $CI_SERVER_FQDN/components/opentofu/validate-plan@<VERSION>
    inputs:
      version: <VERSION>
      opentofu_version: 1.6.1

plan:
  before_script:
    - apk add jq
```

### Source `gitlab-tofu` script to run custom commands later

The `gitlab-tofu` script can be sourced instead of executed.
This allows you to setup the shell and run your own `tofu` or
`gitlab-tofu` commands.

You can use the following snipped either in your script,
directly in the shell and also in `script` keywords of your
pipeline job:

```shell
. $(which gitlab-tofu)
```

There is a slight chance when doing this in a more exotic environment
or shell that `gitlab-tofu` is not able to detect that it is sourced
and will try to execute a `tofu` command.
In this case you can set the `GITLAB_TOFU_SOURCE` environment variable
to `true` before sourcing `gitlab-tofu`.

When the `gitlab-tofu` script is sourced it'll set the `GITLAB_TOFU_SOURCED`
variable to `true`. This variable is not exported by the script itself.

### Best Practices

This section is a collection of *some* best practices.
Feel free to contribute more that generally apply.
If a best practice really becomes the de-facto standard
we may make it the default behavior if possible.

#### Lockfile Handling

If you commit the Lockfile (`.terraform.lock.hcl`) to your repository
we recommend setting either the `TF_INIT_FLAGS` (handled by this component)
or `TF_CLI_ARGS_init` (handled by OpenTofu directly) to `-lockfile=readonly`
to prevent any changes to the lockfile during the pipeline job and with
that ensuring that OpenTofu really uses the locked dependencies.

### Examples

Here are some example repositories to demonstrate how this component maybe used:

- [timofurrer/opentofu-test](https://gitlab.com/timofurrer/opentofu-test): uses multiple environments configured for different kinds of pipelines with a single branch.

Please contribute your own examples!

## Releases & Versioning

This project currently releases tagged commits.
An overview of releases can be found on the [Releases page](https://gitlab.com/components/opentofu/-/releases)
and a Changelog can be found [here](CHANGELOG.md).

Each release is accessible in the [CI/CD Catalog](https://gitlab.com/explore/catalog).

### Component Versions

The component release versions follow [Semantic Versioning 2.0.0](https://semver.org/).

### Image Versions

This project releases multiple OCI image variants that can be used with the component.
The intention is that the images used in a component have the same version and or not mixed.
Due to the limitations described in https://gitlab.com/gitlab-org/gitlab/-/issues/438275
it's currently required to provide the component version in the `component` include field
and as the `version` input. Check out the [Usage](#Usage) section for examples.

There are `alpine` and `debian` variants available.

Each component release deploys the following images:

- `$CI_TEMPLATE_REGISTRY_HOST/components/opentofu/gitlab-opentofu:<VERSION>-opentofu<OPENTOFU_VERSION>-<OS_VARIANT>`
- `$CI_TEMPLATE_REGISTRY_HOST/components/opentofu/gitlab-opentofu:<VERSION>-opentofu-<OS_VARIANT>`
  - Includes the latest stable OpenTofu version at the time of releasing the component
- `$CI_TEMPLATE_REGISTRY_HOST/components/opentofu/gitlab-opentofu:<VERSION>-<OS_VARIANT>`
  - Includes the latest stable OpenTofu version at the time of releasing the component

In the above examples `<VERSION>` references the component version, `<OPENTOFU_VERSION>`
an OpenTofu release, from [here](https://github.com/opentofu/opentofu/releases) and
`OS_VARIANT` either `alpine` or `debian`.

The release notes contain a full list of images deployed to the registry.

*Note: unfortunately, these image versions are not SemVer compatible,
because `-` indicates a prerelease (which they are not in this case).
However, we cannot use the alternative `+` which would indicate build metadata
as we'd like.
See https://github.com/distribution/distribution/issues/1201*

### Image Signing

Every released image is [signed](https://docs.gitlab.com/ee/ci/yaml/signing_examples.html)
using [`sigstore/cosign`](https://github.com/sigstore/cosign).

You can use the following command to verify the signatures:

```shell
VERSION=X.Y.Z # put a released components/opentofu version here
IMAGE_REF=... # put a released components/opentofu image reference here
cosign verify "${IMAGE_REF}" --certificate-identity="https://gitlab.com/components/opentofu//.gitlab-ci.yml@refs/tags/${VERSION}" --certificate-oidc-issuer="https://gitlab.com"
```

For example, for image ref `registry.gitlab.com/components/opentofu/gitlab-opentofu:0.34.0-opentofu1.6.0-alpine` and version `0.34.0`:

```shell
cosign verify "registry.gitlab.com/components/opentofu/gitlab-opentofu:0.34.0-opentofu1.6.0-alpine" \
    --certificate-identity "https://gitlab.com/components/opentofu//.gitlab-ci.yml@refs/tags/0.34.0" \
    --certificate-oidc-issuer "https://gitlab.com"
```

For self-managed mirrors the OIDC issuer must be changed, too.

### Using with Renovate

To keep the component versions up to date you could use [Renovate](https://docs.renovatebot.com/).

Renovate users who use the component input `opentofu_version` should include the following `extends`
so that the OpenTofu version is raised to a maximum of the version suitable for the component:

```json
{
  "$schema": "https://docs.renovatebot.com/renovate-schema.json",
  "extends": ["local>components/opentofu"],
  ...
}
```

The above renovate config allows to update the `version` input together with the component include
version if the `version` input has a `# component version` comment suffix, like so:

```yaml
include:
  - component: $CI_SERVER_FQDN/components/opentofu/validate-plan-apply@<VERSION>
    inputs:
      # The version must currently be specified explicitly as an input,
      # to find the correctly associated images. # This can be removed
      # once https://gitlab.com/gitlab-org/gitlab/-/issues/438275 is solved.
      version: <VERSION> # component version
      opentofu_version: <OPENTOFU_VERSION>

stages: [validate, test, build, deploy, cleanup]
```

(You may need to adjust the path to the `components/opentofu` to match your mirror.)
Fore more details refer to the [Renovate documentation](https://docs.renovatebot.com/config-presets/).

Some more example configurations for your `renovate.json`:

- Package Rule to update all CI-Components
  ```json
  {
			"matchFileNames": [
				".gitlab-ci.yaml",
        ".gitlab-ci.yml",
				"templates/**/*.yaml",
        "templates/**/*.yml"
			],
			"groupName": "Pipeline",
			"semanticCommitType": "ci",
			"automerge": true
	},
  ```
- Package rule to pin only `major.minor` versions:
  ```json
  {
		  "matchManagers": ["gitlabci"],
		  "extractVersion": "^(?<version>\\d+\\.\\d+)"
	},
  ```
- Package rule to target a specific component:
  ```json
  {
      "matchPackageNames": ["components/opentofu"],
      "matchManagers": ["gitlabci"]
  },
  ```

**Example Repositories**:

- [timofurrer/opentofu-test](https://gitlab.com/timofurrer/opentofu-test)

## Usage on self-managed

GitLab CI/CD components are not yet distributed and available on self-managed GitLab instances.
(see details [here](https://gitlab.com/gitlab-org/gitlab/-/issues/415638)).
It's also not possible to just include CI/CD components across instance, thus an include like
`- component: gitlab.com/components/opentofu/full-pipeline@~latest` won't work from a
self-managed instance.
However, you could mirror this project from GitLab.com onto any self-managed instance using
a [repository pull mirror](https://docs.gitlab.com/ee/user/project/repository/mirror/pull.html).

If the component is being mirrored to another path than `components/opentofu`, then you also
need to change that path in the `include:component` and additionally provide the correct
`image_registry_base` input.

See also the official GitLab documentation for it
[here](https://docs.gitlab.com/ee/ci/components/#use-a-gitlabcom-component-in-a-self-managed-instance).

If you want to save runner resources you may disable the unit and integration tests
by setting the `SKIP_TESTS` CI/CD variable to `true`.

You can set the `OPENTOFU_COMPONENT_IMAGE_BUILD_RUNNER_TAG` CI/CD variable to a custom runner tag
to use for the image build job. This may be useful if you require a dedicated privileged runner.

The pipeline of this component respects the
[GitLab Dependency Proxy](https://docs.gitlab.com/ee/user/packages/dependency_proxy/) configuration
by detecting the `CI_DEPENDENCY_PROXY_DIRECT_GROUP_IMAGE_PREFIX` environment variable
and configuring `buildah` to use it when building the container images.

## Migrating from the Terraform CI/CD templates

When migrating from the GitLab Terraform CI/CD templates you can use the following migration rules:

- Used `Terraform.gitlab-ci.yml` -> Migrate to `validate-plan-apply`.
- Used `Terraform/Base.gitlab-ci.yml` -> Migrate to `job-templates`.
    - Migrate the `.terraform:` job prefix to `.opentofu:`.
- Used the `kics-iac-sast` job -> Additionally include the `Jobs/SAST-IaC.latest.gitlab-ci.yml` template.
- Migrate the following job names:
    - `build` -> `plan`
    - `deploy` -> `apply`
- Migrate the `TF_ROOT` variable to the `root_dir` input.
    - Although the `TF_ROOT` variable is still used and maybe overwritten after the import on individual jobs.
- Migrate the `TF_STATE_NAME` variable to the `state_name` input.
    - Although the `TF_STATE_NAME` variable is still used and maybe overwritten after the import on individual jobs.
- Migrate the `TF_AUTO_DEPLOY` variable to custom `rules` inputs.
- Used other variables -> Use the same variables with this component.

The same rules apply for the `latest` templates.
We also recommend to check out the [Usage](#Usage) section for more details about the available templates and inputs.

### OpenTofu component `inputs` vs. Terraform template `variables`

This OpenTofu CI/CD component makes use of [`inputs`](https://docs.gitlab.com/ee/ci/yaml/#specinputs)
whereas the Terraform CI/CD templates used [`variables`](https://docs.gitlab.com/ee/ci/yaml/#variables).
We recommend that you use the `inputs` with the OpenTofu component where available and required.
However, if needed you may overwrite the jobs and set the `variables` you like.

## Can I use this component with Terraform?

Probably. Although, we don't officially support it or maintain any compatibility layer where necessary.

The OpenTofu CI/CD component job mainly interface with the [`gitlab-tofu`](src/gitlab-tofu.sh) script
that is distributed with the `gitlab-opentofu` container image used as the base image for the jobs.
This base image also contains the `tofu` binary.

If you'd want to use Terraform instead you may provide your own container image
that contains at least a script called `gitlab-tofu` so that it's compatible with the component jobs.
Everything else in the job can be custom, like replacing `tofu` with `terraform`.

You may provide the `image_registry_base` input to any of the component includes, pointing to the
container registry URI hosting the container image. The container image name can be configured in
the `image_name` input. The image has be versioned so that it is compatible
with [the image versioning of this project](#image-versions).

## Contributing

See the [CONTRIBUTING.md](CONTRIBUTING.md) guide.