MSD Watcher

• 1: MSD Watcher Backend
• 2: MSD Watcher Frontend

1 - MSD Watcher Backend

Note: All paths in this section are relative to the package root, i.e. /backend/

The backend project exposes a few simple api endpoints for the frontend which interface with the kubernetes api (and in the future kafka) and clean up that data to a more workable format. It also acts as the webserver that serves the frontend.

Important Dependencies

• Quarkus as the main framework
• Quarkus Kubernetes Api to interface with kubernetes
• JSON in Java to dynamically build the Json responses for the frontend api

Functionality

Serving the frontend

This is simply done by using /src/main/resources/META-INF/resources as the build target for the frontend. Quarkus already serves everything that is in there as static html which is enough for our usecase.
To ensure that every url that isn’t defined as an endpoint by quarkus we use the NotFoundExceptionMapper.java found in the core package. Which tries to check if the index.html file generated by the frontend build exists and if so, redirects all traffic to it. Since frontend is a SPA the rest of the routing is handled by it and everything works as it should.
One additional note: If no index.html file is found (e.g. if the frontend build failed or is in progress) we redirect to a generic 404 page.

The request flow is now as follows:

graph TD
    A[Quarkus gets a request] --> B[Quarkus checks if this url matches an endpoint defined by itself]
    B -->|Yes| C[Quarkus answers]
    B -->|No| D[Check if an index.html for the frontend exists]
    D -->|Yes| E[Serve the frontend at this URL, let it handle the routing]
    D -->|No| F[Serve a generic 404 html page]

Interfacing with the kubernetes api

Each kubernetes resource that was deemed important gets its own class in the kubernetes package. Each one of these classes contains a getRaw*** method that queries the kubernetes api for a list of the desired resource. This raw list gets exposed under /<endpoint_identifier>/raw. Most of those classes also expose a parsed list which provides the JSON schemas that the frontend needs.

Currently, the following kubernetes resources are implemented:

[
  {
    “id”: UUID,
    “name”: Name,
    “namespace”: Namespace
  },
  …
]

[
  {
    “id”: UUID,
    “name”: Name,
    “namespace”: Namespace
  },
  …
]

[
   {
    “id”: UUID,
    “name”: Name,
    “namespace”: Namespace
    “targetIds”: [
        targetUUid,
        …
    ]
   },
   …
]

[
   {
    “id”: UUID,
    “name”: Name,
   },
   …
]

[
   {
    “id”: UUID,
    “name”: Name,
    “namespace”: Namespace
    “node”: NodeName
    “appName”: appNameFromLabels
    “status”: {
        “phase”: podStatusPhase,
        “startTime”: podStartTime
    },
    “ownerId”: ownerUUID
   },
   …
]

[
   {
    “id”: UUID,
    “name”: Name,
    “namespace”: Namespace
    “ownerId”: ownerUUID
   },
   …
]

Future Goals

• Find some way to know what is happening in the MSD
  • e.g. Kafka listener that captures all events, matches them to a pod and sends that information to the frontend via websocket
• Properly build the services endpoint
  • Goal: Match each pod to a service (e.g. the core MSD services game, trading, map, etc.)
• Test the endpoints
  • Difficulties: The kubernetes stuff needs a kubectl environment to work. Maybe mock all kubernetes api results?
• Role based auth system using RBAC and JWT.
  This is implemented, but not used on any of the endpoints. Currently only one admin user gets created on startup
  with credentials from the .env (see .env.example & the classes in the auth package).
  • Todos:
    • Define needed roles
      • Which endpoint can what role access?
      • Maybe create an endpoint to register new users (default GUEST role?)
    • Implement in the endpoints using annotations (see e.g. AuthController:72)

2 - MSD Watcher Frontend

Note: All paths in this section are relative to the package root, i.e. /frontend/

The frontend project acts as the main UI for the msd watcher. It visualizes the deployed kubernetes infrastructure in a graph, and is supposed to show activity (events, internal rest calls, etc.) too in the future.

Important dependencies

• Vue.js 3 acts as the main framework
• G6 is the graph visualization framework used
• Typescript as the language used
• Vite.js as the build tool

Functionality

At the moment the frontend only contains a (very rough and, at least for me, not satisfactory) visualization of some kubernetes resources as they are deployed in a cluster. It provides some helpers for view manipulation but not much else.

The problems with visualizing a kubernetes landscape

Kubernetes Landscapes consist of some number of nodes that are related in some kind of way. This kind of forms a DAG which gives us a good start to visualize it. But, there is a (relatively) big problem: It isn’t really a DAG.
First of, I wanted to show every namespace at once. This means we need multiple DAG like graphs at the same time, all with a mostly non overlapping layout somehow arranged on the background to make sense and not take up a whole ton of useless space. This was/is not easy to achieve, mainly because there aren’t any libraries or tools that aim to do such a thing.
There was a lot of research done to land at the current solution described in the section below. I will outline the important milestones here and why they worked/didn’t.

Kubeview

Kubeview is a tool developed to almost do what this aims to achieve.
It provides a kubernetes visualization and is even build using a similar architecture. But there are some problems that prevented this monitor simply being a fork of kubeview.

1. It (obviously) doesn’t contain any “activity monitoring” e.g. blinking nodes when a kafka event was emitted.
2. It only visualizes one namespace at a time.
   Since the goal is to visualize the entire MSD at a glance, we need to visualize everything at once. But maybe we can relatively easily adapt the code to work for our usecase?
3. The stack used and the state of the project.
   1. The code is a bit of a mess. At least from my perspective it didn’t seem like something easily expanded/improved
      on. Mostly because:
   2. It uses Go for the backend and Vue 2 for the frontend. I have no prior experience with Go, and the current code didn’t look very friendly / workable to me.
      Vue 2 would be workable (and early versions of this very project where heavily inspired by the way things where done in kubeview) but there was another roadblock that only showed its head after a lot of work already went into trying to make things work: kubeview uses cytoscape as its main visualization library

Cytoscape

Cytoscape was the first library I used to visualize the “kubernetes graph”. It worked great, had a nice api and ok docs. But once it became time to render multiple, separate graphs, i.e. the multiple namespaces, it became clear that cytoscape wasn’t the way forward.
The main problem was that cytoscape can only properly support one layout type per instance. This meant that every aspect of the visualization (the positioning of the namespaces, the positioning of the graphs in the namespace boxes and the graphs itself) all needed to use the same layouting algorithm. It was simply impossible to achieve any kind of good-looking layout with this constraint and thus, a new library was needed.
Trust me, this decision was not easy to make as cytoscape was up until this point pretty great to work with, and I’ve had it integrated into the project almost perfectly. But after ripping my hair out because I was just getting messy / useless / unreadable graphs left and right, something had to change.

The current state of the visualization

Currently, the graph gets rendered using G6. This provides an at least acceptable visualization after a ton of experimenting, but is still not perfect.

The following sections are best read with the code open on the side!

Terminology

Structure

The project consists of 2 components. App.vue which just provides an entrypoint and GraphViewer.vue which contains everything else, so it’s concerned with rendering, data fetching and even some layout/styling.

This monolithic architecture is NOT ideal and should be changed in the future!
Data handling, layout/styling and actually rendering the graph should probably all be handled in seperate components.

Lifecycle of GraphViewer.vue

1. On mount of the vue component we initialize the G6 Graph.
   We pass it a lot of options, some important ones to note:
   • layout: We use the dagre layout but with sortByCombo: true because our nodes are grouped by combos (namespaces)
   • modes.default: Enable zooming and dragging of the viewport/canvas and disable dragging nodes out of a combo
   • defaultNode / defaultCombo: some dimension and appearance settings for nodes and combos. e.g. label positioning & size
2. Now we fetch data from our backend. This currently includes:

   - Namespaces
   - DeamonSets
   - Deployments
   - ReplicaSets
   - Pods
   - Endpoints
   

   Each of them gets loaded using the native fetch api with some wrappers around it to make life easier. We also utilize promise await to avoid big promise chains but still maintain the order of fetches and ensure data fetching finished before trying to use any of it.
3. After we got the graph initialized and fetched the data, we build up our graph data.
   There are a few simple helper functions that do some very simple parsing and add the resources to the graph data array.
   Once everything was processed we simply pass that data to G6 and let it render.

Due to the reactivity of vue there are some DOM changes happening that don’t correspond to anything explicitly happening in the code like e.g. rendering the graph with G6.
Currently this only includes generating a bunch of buttons for each namespace as soon as the namespace data gets received. These buttons enable ‘focusing’ into a namespace.
This focusing is one of the things cytoscape simply did better: call cy.fit() with the name of the namespace we want to be as big as possible, in the middle of the screen and done!
In G6 we have to perform this focusing in 2 steps with a bit of manual math in between.

1. We get the actual namespace element from G6 in order to get its dimensions and calculate the needed zoom ratio we need to set, so it fits the screen.
2. We use the inbuilt focusItem function of G6 which (unfortunately) just centers the combo in the viewport
3. After that animation is done we apply the calculated zoom ratio to make the combo/namespace fill the entire screen.

Note for future development: This “focusing” method is obviously not ideal. The whole focusing should be done in one animation.
I’ve messed around with doing the transitions (mainly focusItem) without animations, storing the coordinates where we end up, resetting the view to before and then using these coordinates along with the zoom ratio with some method found in the “viewport operation” section of the G6 docs to create one smooth animation. I didn’t have any luck with this, mostly because I couldn’t get the viewport to properly reset after doing focusItem. I was not able to figure out which coordinates to use and how to manipulate them. I know this should be possible but due to time constraints and other factors I left it as is after a lot of experimenting.

The future

Maybe, just maybe X6 which is developed by the same people as G6 would provide a better experience and visualization since it is specifically build for diagrams and not graphs. But i’ve not tested it, I just stumbled across it during a frustrated session of research on how to just make this look good…