Minikube DEPRECATED

• 1: Minikube Overview
• 2: Minikube install Cluster
• 3: Minikube on Windows WSL
• 4: Minikube Core Service Interaction
• 5: Minikube access to services
• 6: Minikube Local Testing

1 - Minikube Overview

Local Environment Minikube Overview

This site covers everything for our Repo [local-environment-minikube]

Goal of this Project is, to run a single script and everything is ready to use.

Infrastructure

Using the installInfrastructure.sh script.

1. add needed repos
2. install
   1. cert-manager (certificate controller)
   2. prometheus-stack (monitoring system)
   3. redpanda-operator (streaming data platform, our kafka replace)
   4. rabbitmq (message broker)
   5. kafka-rabbitmq-connector
3. install one-node-cluster.yml on redpanda (config for redpanda)

THIS SHOULD ONLY EDIT BY DEVOPS TEAM

First Build Cluster

Using the firstBuildCluster.sh script.

1. install automate installInfrastructure.sh
2. install chosen core service
   1. game
   2. map
   3. robot
   4. trading
   5. gamelog
3. allow config for private (insecure) registry

THIS SHOULD ONLY EDIT BY DEVOPS TEAM

Refer our [First Build Cluster]

Core Service Interaction

Using the core-service-interaction.sh script.

The interaction for install, update and delete the core-services

1. game
2. map
3. robot
4. trading
5. gamelog

Refer our [Core Service Interaction]

Delete Cluster

Using the deleteCluster.sh script.

This stop the Custer and delete everything

Run ./deleteCluster.sh

2 - Minikube install Cluster

Local Environment Minikube

First Build Cluster

This site covers everything for the firstBuildCluster.sh script in our Repo [local-environment-minikube]

Preparation

On Windows: Refer to our Windows WSL

When you using linux, or after prepare your Windows WSL machine, you can do the following steps (Windows users run this in WSL Linux).

Needed programms should be in system package repos, if not, see following links. When one of these is missing the script notice you about it, and didn’t execute.

1. Install needed programms

   • git
   • [minikube]
   • [kubectl]
   • [helm]

2. Config minikube

   • minikube config set memory 8192
   • minikube config set cpus 6
   • Driver
     • on Windows, minikube chose automated Docker Desktop from Windows Host. Do not specify a driver.
     • on Linux, the driver for Docker is automated chosen

3. Make sure, thar the following files are executable

   • firstBuildCluster.sh
   • installInfrastructure.sh
   • core-service-interaction.sh
   • deleteCluster.sh

You can run chmod +x *.sh, after this, all .sh files in the directory are executable

Running firstBuildCluster.sh script

Run ./firstBuildCluster.sh that install full infrastructure, and all core services. After this, you can use the cluster.

Only Infrastructure

Run ./firstBuildCluster.sh --noservice that install full infrastructure, and NO core services. Use core-service-interaction.sh for install needed core-services later.

Available Commands

Run ./firstBuildCluster.sh --help or ./firstBuildCluster.sh -h

Refer our [Install for local testing]

Local Testing / local (insecure) registry

Start and Stop

Run minikube stop to stop the cluster Run minikube start to start the cluster This start the existing cluster with all existing infrastructure and core-services

Ports for Services

Refer our [Service external access]

Interaction with Core Services

Refer our [Core Service Interaction]

3 - Minikube on Windows WSL

For WSL2
WSL Overview
WSL in MS-Store Info

• Preparation
  • In Windows Features
  • For WSL
  • For Docker
• WSL config in windows (global config)
  • first time installation of Ubuntu
  • config in Ubuntu
• Nice to Have

Preparation

In Windows Features

1. activate feature „WSL“ in Windows features
2. activate feature „Hypervisor“ in Windows features

For WSL

1. load WSL from MS store
2. load Ubuntu 22.04.1 LTS from MS store

For Docker

1. install Docker Desktop
2. activate in Docker Desktop under Settings -> Resources the Ubuntu Distro

WSL config in windows (global config)

Hint: Please enable show hidden files and file extension, so you can create .wslconfig and not create mistakenly .wslconfig.txt

edit / create C:\Users\<UserName>\.wslconfig insert following

# settings for all Linux distros
[wsl2]

# Limit for VM memory un GB or MB
memory=10GB

# Set VM virtual processors
processors=7

# allow nested virtualization
nestedVirtualization=true

run in powershell wsl.exe --shutdown as admin (shutdown wsl)

run in powershell wsl.exe -l -v as admin (see stopped distros)

this above give your wsl access to 10gb ram, 7 cpu cores

first time installation of Ubuntu

During the initial installation or first startup of WSL Ubuntu, a new user will be created. Enter a new username, then set a password. Note that, as is typical with Unix systems, the password input will not be visible. After this, this user will be used as the default user for Ubuntu.

config in Ubuntu

1. run Ubuntu installation
   1. run following for better network performance
      1. sudo rm /etc/resolv.conf
      2. sudo bash -c 'echo "nameserver 8.8.8.8" > /etc/resolv.conf'
      3. sudo bash -c 'echo "[network]" > /etc/wsl.conf'
      4. sudo bash -c 'echo "generateResolvConf = false" >> /etc/wsl.conf'
      5. sudo chattr +i /etc/resolv.conf
   2. edit config file sudo nano /etc/wsl.conf (when needed)

      [network]
      generateResolvConf=false
      

2. run in powershell wsl.exe --shutdown as admin (shutdown wsl)
3. run in powershell wsl.exe -l -v as admin (see stopped distros)
4. config Docker Desktop (settings –> Resources –> WSL integration)
5. restart wsl2
6. config minikube (see Readme) when needed
7. get local-environment-minikube
8. run ./firstBuildCluster.sh [First Build Cluster]

Nice to Have

(kubectl and helm must be installed on the host; see: here)

After installing the cluster, you can interact with it using kubectl and helm in your WSL Ubuntu.
You can also interact with kubectl and helm on your host system.
Open File Explorer and click on Linux on the left, then on your Ubuntu installation.
Navigate to the /home folder and copy the .kube folder to the /Users/<Your Username> directory.
Next, go to the .kube folder and open the config file.
There are three things that need to be adjusted there:

1. certificate-authority: \\wsl.localhost\Ubuntu-24.04\home\ubuntu\.minikube\ca.crt
2. client-certificate: \\wsl.localhost\Ubuntu-24.04\home\ubuntu\.minikube\profiles\minikube\client.crt
3. client-key: \\wsl.localhost\Ubuntu-24.04\home\ubuntu\.minikube\profiles\minikube\client.key

The path \\wsl.localhost\Ubuntu-24.04 is displayed in Windows Explorer when you click on Linux on the left and then on your Ubuntu installation. (The path might be different.)

Result for /User/your-user/.kube/conf

apiVersion: v1
clusters:
  - cluster:
      certificate-authority: \\wsl.localhost\Ubuntu-24.04\home\ubuntu\.minikube\ca.crt
      extensions:
        - extension:
            last-update: Thu, 30 May 2024 14:10:03 CEST
            provider: minikube.sigs.k8s.io
            version: v1.33.1
          name: cluster_info
      server: https://127.0.0.1:62920
    name: minikube
contexts:
  - context:
      cluster: minikube
      extensions:
        - extension:
            last-update: Thu, 30 May 2024 14:10:03 CEST
            provider: minikube.sigs.k8s.io
            version: v1.33.1
          name: context_info
      namespace: default
      user: minikube
    name: minikube
current-context: minikube
kind: Config
preferences: {}
users:
  - name: minikube
    user:
      client-certificate: \\wsl.localhost\Ubuntu-24.04\home\ubuntu\.minikube\profiles\minikube\client.crt
      client-key: \\wsl.localhost\Ubuntu-24.04\home\ubuntu\.minikube\profiles\minikube\client.key

4 - Minikube Core Service Interaction

local-environment-minikube Core Service Interaction

The ./core-service-interaction.sh is your way to install, delete or update core-services
enter ./core-service-interaction.sh -h or ./core-service-interaction.sh --help to see help with available commands \

If you like to interact more with your cluster, please check a look at kubectl documentation
(tip kubectl [command] for interaction that is not covered by this site )

current core-services for interaction

1. robot
2. map
3. trading
4. game
5. gamelog

Commands

For every command is a short, and a long version available

You can combine every command (except ALL and the non core-service commend) for multiple actions at same time

RUN ./core-service-interaction.sh and add a command

Examples

The ./core-service-interaction.sh -dga -ga remove the game service from cluster and install game after this

5 - Minikube access to services

Local Environment Minikube service access

Ports for Services

On Windows with WSL2

Enter minikube tunnel in an new WSL2 Linux (Ubuntu) Terminal.

The ports will be exposed for IP-address: 127.0.0.1 on Windows host system.

The access is 127.0.0.1:port

On Linux

The firstBuildCluster.sh will chose (automated) docker as standard driver.

Minikube is configured to export Ports to static IP-address 10.10.10.10

after entering minikube service list you see 10.10.10.10:nodePort

no minikube tunnel needed

Be aware, that you can still use minikube tunnel on Linux, but the IP-addresses for the services will be a random IP-address from the 10th-net

For more information please refer to Minikube accessing apps

For rabbitmq

External access for rabbitmq management (access over webbrowser) is Port 15672, but you can’t access this port on Windows WSL2, so a loadbalancer is in use to give you access on this port 15672

The same goes for the internal use of Rabbitmq over port 5672 this will be also exposed for your player service over a loadbalancer at port 5672.

On Windows WSL

As mentioned above this is the only way to access the rabbitmq management on your windows host webbrowser And the only way to interact internally with your player outside on the WSL

On Linux

On Linux as mentioned before, we use nodePorts

But you can still use the rabbitmq-service to access the internal and management

You can run kubectl get service --namespace rabbitmq

Choose ClusterIP from rabbitmq-service (not rabbitmq-*-lb).

After minikube tunnel you can access the management over this ClusterIP:15672. This is the same as 10.10.10.10:30007

For internal Player testing you can use the same ClusterIP, with internal Port ClusterIP:5672. This is the same as 10.10.10.10:30006

Be aware that the ClusterIP will change every time your build the cluster from bottom, but the minikube IP will be 10.10.10.10 (minikube stop and minikube start, the ClusterIP will not change)

6 - Minikube Local Testing

Local Environment Minikube Local Testing

This site covers everything for our Repo local-environment-minikube

Goal of this Project is, to run a single script and everything is ready to use.

Do this if you like to test your deployment locally before upload it on MSD gitlab

Normally your run your code from the IDE

Prepare Docker

on Linux

Open a Terminal and find your local IP-Address with ip a NOT localhost, recommended your IP-Address over LAN (WLAN is possible)

create or edit the /etc/docker/daemon.json

add the following to the daemon.json: ===> The UP-Address is your IP-Address from above

{
        "insecure-registries": ["192.168.188.121:5000"]
}

now restart your docker

on Windows

Open Powershell and find your WSL IP-Address with ipconfig NOT localhost, important it must the WSL Client IP Address

In Docker Desktop Settings -> Docker Engine

past the following into the text field: ===> The UP-Address is your IP-Address from above

{
        "insecure-registries": ["192.168.188.121:5000"]
}

now restart your docker

Install Cluster

(Windows User in WSL Linux)

Run ./firstBuildCluster.sh -ins or ./firstBuildCluster.sh -ins --allservice if your like to install all core services at same time

now the skript ask you to enter your IP-Address (see above this IP-Address) and the chosen Port enter both, and the automated installation will start.

Build Docker image

on Windows

The chosen service must be placed that you can run it into WSL

Example: player-hackschnitzel

on Both

Enter the Directory of the service with a terminal (for windows, in the WSL Linux)

make sure you have a Dockerfile int is needed for this Guide

change the minikube Environment to docker environment: eval $(minikube docker-env)

now install the local registry: sudo docker run -d -p 5000:5000 --restart=always --name registry registry:2 the port is the same as your port specified by running cluster build see above

now build your Docker Image: sudo docker build --no-cache -t player-hackschnitzel:v0.1 .

aft build, tag your Image to the registry: sudo docker tag player-hackschnitzel:v0.1 192.168.188.121:5000/player-hackschnitzel:v0.1 the IP-Address must be the same as that what your entered before

now push your Image into your registry: sudo docker push 192.168.188.121:5000/player-hackschnitzel:v0.1

Now we have our Image in our private registry

very basic helm chart

create basic helm charts

The following YAML-Files didn’t follow the way, how helm-charts should be built normally you have a values file with references into the yaml-files

1. deployment

apiVersion: apps/v1
kind: Deployment
metadata:
  name: player-hackschnitzel-deployment
  namespace: player-hackschnitzel
  labels:
    app: player-hackschnitzel
spec:
  replicas: 1
  selector:
    matchLabels:
      app: player-hackschnitzel
  template:
    metadata:
      labels:
        app: player-hackschnitzel
    spec:
      containers:
      - name: player-hackschnitzel
        image: 192.168.188.121:5000/player-hackschnitzel:v0.1
        imagePullPolicy: IfNotPresent
        resources:
          requests:
            memory: "256Mi"
            cpu: "500m"
          limits:
            memory: "1G"
            cpu: "1000m"
      volumes:
      - name: logs-volume
        emptyDir:
          sizeLimit: 500Mi
...

2. service

apiVersion: v1
kind: Service
metadata:
  name: player-hackschnitzel-service
spec:
  selector:
    app: player-hackschnitzel
  ports:
    - protocol: TCP
      port: 8000
      targetPort: 8000


---

apiVersion: v1
kind: Service
metadata:
  name: player-hackschnitzel-service-lb
spec:
  type: LoadBalancer
  selector:
    app: player-hackschnitzel
  ports:
    - protocol: TCP
      port: 38888
      targetPort: 8000

...

interaction

open a new terminal (Windows in WSL)

run helm install player-hackschnitzel helm-chart --namespace player-hackschnitzel --create-namespace for install deployment

run helm uninstall -n player-hackschnitzel player-hackschnitzel to remove deployment

run helm upgrade player-hackschnitzel helm-chart --namespace player-hackschnitzel for update the deployment

When your have another local image, change image in deployment and run helm update

When your change something inside the helmchart, e.g. the loadbalancer port run helm update