OpenTofu
- Reference
- Structure folders and files
- .kube files
- Roles and access
- Secret (temporary state)
- Code Structure
Reference
Structure folders and files
terraform
| -- /.kube → our cluster Kubeconfig files
| -- /templates
| -- | -- apply.yml → job that apply plans
| -- | -- delete-state.yml → contains delete-state job (currently not used)
| -- | -- destroy.yml → contains destroy job (currently not used)
| -- | -- fmt.yml → jop to check formation
| -- | -- plan.yml → job, that make planes
| -- | -- test.yml → job for tests (currently in pipe, does nothing)
| -- | -- validate.yml → job to validate all *.tf-files
| -- /modules → our templates
| -- | -- /gitrepo → template modul
| -- | -- | -- gitrepo.tf → template
| -- | -- | -- variables → variables
| -- | -- /namespace → template modul
| -- | -- | -- namespace.tf → template
| -- | -- | -- values → variables
| -- | -- /project → template modul
| -- | -- | -- project.tf → template
| -- | -- | -- values → variables
| -- | -- /roles-and_bindings
| -- | -- | -- | -- /global_role_binding → template modul
| -- | -- | -- | -- | -- global_role_binding.tf → template
| -- | -- | -- | -- | -- variables → variables
| -- | -- | -- | -- /project_role_binding → template modul
| -- | -- | -- | -- | -- project_role_binding.tf → template
| -- | -- | -- | -- | -- variables → variables
| -- | -- /templates → template for a new player
| -- | -- | -- player.CHANGEME.tf
| -- | -- /user → template modul for rancher2
| -- | -- | -- user.tf → template
| -- | -- | -- variables → variables
| -- repos_infrastructure.tf → contains module for the Fleet-System GitRepo Ressource
| -- repos_core_services.tf → contains module for the core-services
| -- main.tf → empty file, not needed
| -- providers.tf → contians terraform providers like kubernetes, and rancher2
| -- users-and-roles.tf → contains modules for user and roles
.kube files
There are two important files in .kube:
local.yaml runner.yaml (will likely not be needed in the future, but currently still needs to be updated) Both files need to regularly receive a new token, as they are only valid for 48 hours.
They are both used to access the cluster and environments.
Roles and access
Currently, we have two roles:
- admin / The admins are managed in
users-and-roles.tf
. - user-base / The regular users each have their own file.
The admin role can access all projects and namespaces, whereas the user-base role can only view the projects assigned to them.
Deploy a new player onto the cluster
The player must provide the following information for the DevOps team:
- Name of the player
- Repo link of the player
- GitLab Runner, if desired
To deploy a player onto the cluster in Rancher, a new file needs to be created.
The new file should be named player_CHANGEME.tf
and placed in the
The-Microservice-Dungeon/DevOps-Team/terraform
folder.
This file essentially contains two pieces of information: a profile for the player
and the target Git repository. Additionally, there are several settings that can be
configured to allow the player to access different namespaces or projects.
Here are the steps to follow:
- Create a new branch named player-CHANGEME.
- Open this branch and copy the file player_changeme.tf into the top-level directory of Terraform.
- Replace all occurrences of ‘CHANGEME’ in the file, including the filename, with the appropriate content.
- Review your changes, ensuring that all names (e.g., repo-link) and repository links are correct.
- Create a merge request to merge your branch into the main branch.
The code, which is explained in detail below, can be found as a complete file at the following location:
The file should be placed in the top directory of Terraform, similar to the existing players. Afterward, both the filename part ‘CHANGEME’ and every instance of the word ‘CHANGEME’ within the file must be replaced with the desired content. You’re done once you can no longer find the word ‘CHANGEME’ through a search.
OpenTofu generates a Kubernetes manifest of the type GitRepo. This is registered with the Fleet system. When changes are made to the resource, the Fleet system can detect the updates in the PlayerRepo and redeploy them accordingly. Additionally and a new profile is created, which can be used to log in to the cluster. With the new profile, you will see your own project. When logging in for the first time, a default password is required, which must be changed immediately afterwards!
https://the-microservice-dungeon.gitlab.io/docs/docs/getting-started/basics_player_on_prod/
Standard users need at least the following code
The following code snippets are contents of the file player_changeme.tf
located in the
terraform/modules/templates
directory.
- The word CHANGEME must be replaced with your player name.
new User
# User: player_CHANGEME / Rolle: user
module "user_player_CHANGEME" {
source = "./modules/user"
name = "player-CHANGEME"
username = "player-CHANGEME"
enabled = true
}
user-role
module "global_role_binding_player_CHANGEME" {
source = "./modules/roles_and_bindings/global_role_binding"
name = "player-CHANGEME"
global_role_id = "user-base"
user_id = module.user_player_CHANGEME.user_id
}
You can specify which access rights this player should have.
For a normal player, this is currently limited to one role: user-base
.
new project
module "project_player_CHANGEME" {
source = "./modules/project"
name = "player-CHANGEME"
description = "Project of user player_CHANGEME"
}
new namespace
module "namespace_player_CHANGEME" {
source = "./modules/namespace"
name = "player-CHANGEME"
project_id = module.project_player_CHANGEME.projectid
}
add user to the project
module "player_CHANGEME_user_project_binding" {
source = "./modules/roles_and_bindings/project_role_template_binding"
name = "player-CHANGEME-user-binding"
project_id = module.project_player_CHANGEME.projectid
user_id = module.user_player_CHANGEME.user_id
}
add the gitrepo to the cluster
# Repo for player_CHANGEME on Prod Cluster msd
module "msd_player_CHANGEME" {
source = "./modules/gitrepo"
repo_name = "msd-player-CHANGEME"
repo_source_link = "CHANGEME"
repo_paths = ["./helm-chart"]
repo_clientSecretName = "basic-auth-secret"
repo_targets = [
{
clusterName = "msd"
}
]
}
- The
repo_source_link
must be set to your repository URL.
After adding the new file for the player and successfully running the pipelines, you can log in with the new player.
rancher.dungeon-space.de
Deleting a player
To delete a player, you only need to delete the corresponding
file at the following folder The-Microservice-Dungeon/DevOps-Team/terraform
.
Secret (temporary state)
Secret for fleet-system repo (Modul gitrepo)
kubectl -n NAMESPACE create secret generic NAME --type=kubernetes.io/basic-auth --from-literal=username=YOURUSERNAME --from-literal=password=YOURTOKEN
- NAMESPACE: namespace where the secret will be installed / default:
fleet-default
- NAME: name of the secret and entered in
repo_clientSecretName
onModul: gitrepo
- YOURUSERNAME: name of your account (gitlab / github / etc.)
- YOURTOKEN: Auth Token of your account (gitlab / github / etc. )
Code Structure
Root Directory
providers.tf
- required_providers specifies the necessary providers (external services) required for this Terraform configuration.
- backend “http” {} placeholder for configuring the backend where Terraform stores its state remotely (currently empty but meant for HTTP-based storage).
- Multiple Providers: This configuration uses Rancher, Kubernetes, Helm, and GitLab providers, allowing management of Kubernetes clusters, Helm charts, and GitLab resources.
- Security: The insecure = true flag in both the Kubernetes and Helm providers suggests that SSL verification is disabled, which could be used in development environments.
- File Dependencies: Kubernetes provider configuration relies on two local files: .kube/local.yaml and .kube/runner.yaml, which must be present and updated regularly (as mentioned earlier, for tokens).
terraform {
required_providers {
rancher2 = {
source = "rancher/rancher2"
version = "4.1.0"
}
kubernetes = {
source = "hashicorp/kubernetes"
version = "2.27.0"
}
helm = {
source = "hashicorp/helm"
version = "2.12.1"
}
gitlab = {
source = "gitlabhq/gitlab"
version = "16.11.0"
}
}
backend "http" {}
}
# Configure the Rancher2 provider to admin
provider "rancher2" {}
provider "kubernetes" {
config_path = ".kube/local.yaml"
config_context = "local"
insecure = true
}
provider "helm" {
kubernetes {
config_path = ".kube/runner.yaml"
insecure = true
}
}
provider "gitlab" {}
user-and-roles.tf
- User Creation: Defines two users, “foo” (Administrator) and “bar” (Cluster User), using modules for user management.
- Role Assignment: Binds each user to specific global roles, foo as “admin” and bar as “user-base.”
- Modular Structure: Uses separate modules for creating users and assigning global roles, ensuring a modular and reusable setup.
# User: foo / Rolle: Administrator
module "user_foo" {
source = "./modules/user"
name = "foo"
username = "foo"
enabled = true
}
module "global_role_binding_foo" {
source = "./modules/roles/global_role_binding"
name = "foo"
global_role_id = "admin"
user_id = module.user_foo.user_id
}
# User: bar / Rolle: Cluster User
module "user_bar" {
source = "./modules/user"
name = "bar"
username = "bar"
enabled = true
}
module "global_role_binding_bar" {
source = "./modules/roles/global_role_binding"
name = "bar"
global_role_id = "user-base"
user_id = module.global_role_binding_bar.user_id
}
Modules
gitrepo
gitrepo.tf
-
Manifest Definition: Creates a Kubernetes resource with dynamically set values for apiVersion, kind, metadata, and spec, all controlled by variables (var.*).
-
Git Repository Configuration: In the spec, it configures the repository with:
- repo (source link), branch, and paths for Git repo management.
- insecureSkipTLSVerify to skip TLS verification if needed.
- targets (likely clusters or environments) and clientSecretName for authentication.
-
Flexible Variables: The use of var.* allows dynamic customization of the resource, making it adaptable to different environments and repositories.
resource "kubernetes_manifest" "fleet_gitrepo" {
manifest = {
apiVersion = var.repo_api
kind = var.repo_kind
metadata = {
name = var.repo_name
namespace = var.repo_namespace
}
spec = {
repo = var.repo_source_link
branch = var.repo_branch
insecureSkipTLSVerify = var.repo_insecure
paths = var.repo_paths
targets = var.repo_targets
clientSecretName = var.repo_clientSecretName
}
}
}
variables.tf
This Terraform configuration uses Basic Repository Configuration for Rancher Fleet, sets Git Settings like branch and paths, defines Cluster and Authentication with cluster targets and secrets, and specifies the Repository Link for managing Git resources in Kubernetes.
variable "repo_api" {
description = "The api version"
default = "fleet.cattle.io/v1alpha1"
}
variable "repo_kind" {
description = "kind of ressource"
default = "GitRepo"
}
variable "repo_name" {
description = "The name of the repo in Rancher"
}
variable "repo_namespace" {
description = "The namespace for the Git repository"
default = "fleet-default"
}
variable "repo_branch" {
description = "The branch of the Git repository"
default = "main"
}
variable "repo_insecure" {
description = "set insecure"
default = true
}
variable "repo_paths" {
description = "list of the paths of the Git repository"
type = list(string)
default = [""]
}
variable "repo_targets" {
description = "cluster targets of Git repository"
type = list(object({
clusterName = string
}))
default = [
{
clusterName = ""
}
]
}
variable "repo_source_link" {
description = "link of the Git repository"
}
variable "repo_clientSecretName" {
description = "The name of the repo clientSecretName in Rancher"
default = ""
}
namespace
This Terraform configuration creates a namespace in a Rancher-managed Kubernetes cluster. A namespace acts as an isolation zone within a Kubernetes cluster, allowing resources to be organized and secured separately. This isolation helps manage resources, control access, and prevent conflicts between applications or teams sharing the same cluster.
namespace.tf
resource "rancher2_namespace" "the_ns" {
name = var.name
project_id = var.project_id
}
values.tf
variable "name" {
type = string
}
variable "project_id" {
type = string
}
project
This configuration enables the creation of a project in a Rancher cluster, including essential details like the project name, description, and monitoring settings. It allows for organized management of resources within the cluster, providing a dedicated space for application development and deployment.
project.tf
resource "rancher2_project" "project" {
cluster_id = var.cluster_id
name = var.name
description = var.description
enable_project_monitoring = var.enable_project_monitoring
}
output "projectid" {
value = rancher2_project.project.id
}
values.tf
variable "name" {
type = string
}
variable "cluster_id" {
type = string
default = "c-m-fcvfr6nf"
}
variable "description" {
type = string
}
variable "enable_project_monitoring" {
type = bool
default = true
}
user
user.tf
This file helps automate the creation and management of Rancher users.
# Create a new rancher2 User
resource "rancher2_user" "basic_user" {
name = var.name
username = var.username
password = var.password
enabled = var.enabled
}
output "user_id" {
value = rancher2_user.basic_user.id
}
variables.tf
variable "name" {
type = string
default = "changeme"
}
variable "username" {
type = string
default = "changeme"
}
variable "password" {
type = string
default = "changeme123456"
}
variable "enabled" {
type = bool
default = true
}
Module / Roles
cluster role bindings
This configuration creates a global role binding for a specified user in Rancher, allowing the user to be assigned roles, thereby controlling access and permissions. The use of variables enhances flexibility, enabling easy customization of role bindings.
global_role_binding.tf
# Create a new rancher2 global_role_binding for User
resource "rancher2_global_role_binding" "changeme" {
name = var.name
global_role_id = var.global_role_id
user_id = var.user_id
}
variables.tf
variable "name" {
type = string
}
variable "global_role_id" {
type = string
default = "user-base"
}
variable "user_id" {
type = string
}
project role binding
This configuration allows the assignment of a role template to a user within a specific project in Rancher.
project_role_template_binding.tf
resource "rancher2_project_role_template_binding" "project_role_template_binding" {
name = var.name
project_id = var.project_id
user_id = var.user_id
role_template_id = var.role_template_id
}
variables.tf
variable "name" {
type = string
}
variable "role_template_id" {
type = string
default = "project-member"
}
variable "user_id" {
type = string
}
variable "project_id" {
type = string
}