Experience Sitecore! | All posts tagged 'Kubernetes'

Experience Sitecore!

Martin Miles on Sitecore

Things beginners get incorrect about Kubernetes

On start playing with Kubernetes, one may face with one of the biggest delusions considering the K8S will work in the same way for both the development or testing environment. 

But It won't!

When it comes to containers in general and Kubernetes specifically, there is a big difference between occasional runs in a labs-alike conditions and in full production lifecycle. That is similar to a difference between just starting an app and long term running it full security and reliability enabled.

Not a Kubernetes exclusive problem, but is true for the entire variety of containers and microservices. Spin-up a container comes as relative simple task, while scaling containers as containerized microservices in the production turns to be more complicated.

Although Kubernetes has alternatives, it has quickly become a de-facto standard for orchestration. However there is a difference between launching K8S in a sandbox compared to a full production environment.



Delusion #1. Running containers with Kubernetes in the development or testing environment ensures that your operational needs will be satisfied.

The truth: the launch of Kubernetes in the development or testing environment allows cutting the corners, simplify things and not to bother with the operational load, which one faces when going live to Prod. Ops and safety considerations will become major areas of differences between K8S running in prod and in the development / testing environments. Failing a cluster in the labs conditions does not bring any losses.

For me it looks like a compromise between an agility and reliability: devs use containers to achieve flexibility while working with apps when developing and testing the code does its purpose. While the ops need to provide reliability, scaling, performance and safety provided by a sustainable, industry-proven platform. They are looking for a deployment automation for the clusters to ensure the repeatability and consistency. It also helps when restoring the system.

Versioning is also critical for operations. As far as possible, you need enabling versioning everywhere, including services deployment configuration, policies and infrastructure (applying the infrastructure-as-a-code approach). That results in environments becoming repeatable. As a good practice, avoid "latest" image versions, in order to avoid configuration drift effect.


Delusion #2. Both reliability and security got provided with Kubernetes

In reality: when using Kubernetes at non-production environments only, most unlikely reliability and security got provided, at least initially. Do not get discouraged, you will be there: it's a matter of designing an architecture before switching to the Prod.

Obviously, performance, scaling, availability and safety requirements are much higher in prod environments. This It is important to plan these requirements for the deployment of K8S into architecture, as well as build scaling and security plans into Helm-charts, etc.

But how could running a cluster in dev/testing environments lead to a false confidence?

This is common for non-production environments having all network connections open. It is acceptable that any service can refer to any other service: open connections are the defaults for Kubernetes. However such an approach is an evil practice for production environments and can lead to downtime. It also exposes larger areas for potential attack and increases threats to business.

When it comes to containers / microservices, one needs spending bigger effort for creating a highly available and reliable system. Orchestration itself helps a lot but isn't a "silver bullet", same applies to security. We will have to work hard to protect Kubernetes and reduce the surface of the attack. It is very important using RBAC with minimal privileges and enforce network policies, leaving only those channels services indeed use.

Also vulnerabilities of container images can rapidly turn ops into a critical state, while on development / testing environments this danger may absent at all. Pay attention to the base images used for building your containers: as far as possible, use trusted official images, or build your own. The last thing you want happening for your Kubernetes cluster is helping someone mining crypto coins.

It is recommended to refer to the security of containers as a ten-level system covering the container stack (host and registries), as well as questions related to the life cycle of containers (for example, API management). 


Delusion #3. Orchestration makes scaling a formality

Although Kubernetes considered being a completely necessary tool for scaling containers, it will be delusted to think that orchestration immediately sorts out scaling needs for the production environment. The volume of data at live environments is times more, please also keep in mind that monitoring may also need scaling. With increasing volumes, everything changes. 

It is impossible to ensure all K8S components implementing the interfaces correctly until you spin-up the prod: determining Kubernetes "working normally", and the API server and other controlled components get scaled according to your needs.

As I say, the development and testing environments go much easier. In local environments it is easy skipping basics like defining the right resources and restrictions for requests. Avoiding that can collapse you prod once later. 

Scaling the cluster both directions is a good example when the task goes easy locally, being clearly complicated at production: scaling prod clusters is more difficult than clusters for development/testing.

While Kubernetes makes it relatively simple scaling horizontally, DevOps still need keeping in mind some nuances, especially when it comes to maintaining services live when scaling an infrastructure. It is crucial to ensure that the main services, as well as a system monitoring and security alerts, were distributed across the cluster nodes and do work with stateful volumes so that data not being lost on scaling down.

Again, it all comes to proper planning and resources available. You need not just understand your needs for scaling when planning but most importantly - test them. Your production environment must be capable for handling much higher loads.


Delusion #4. Kubernetes works everywhere equally that same

In reality: differences in work in another environment may vary similar to those differences between running Kubernetes on the developer's laptop and prod server. The reality is that there may be serious differences depending on the vendor .Many believe that if the K8S works locally, it will work in any operational environment. 

Local environments commonly miss important components required by prod environments: monitoring, logging, certificate management and credentials. You need to keep that in mind, as that is another problem raised from a difference between prod  and development/testing environments.

However, that isn't Kubernetes exclusively, but applies to containers/microservices in general, especially in multicloud and hybrid cloud setups. Those Kubernetes implementations are more complicated than it seems initially, as many of the mandatory services are proprietary, like load balancing and firewalls. A container that works well locally may work unprotected (may not start at all) in the cloud with another setup of tools. Therefore, SERVICE MESH technologies like Istio attract so much attention. They guarantee the availability wherever your container works, so you do not need to think about infrastructure - which is the main reason for using containers.

I hope you can reach safer and more reliable production environments with Kubernetes keeping the above in mind!

Why CNAB could be a game-changer for Docker containers and how Sitecore can benefit from that?

Introduction

Currently, in order to run Sitecore in docker locally, one has to pull the code from a GitHub repository, build it (if not done yet, or pull already built images from a Docker registry), set the license file, and say "up" to docker-compose. Not to say the prerequisites required. If dealing with a cloud, then the deployment into Kubernetes is required, which also demands adequate skills.

Imagine an ideal situation where you don't need to do all the things, but just pull a "managing" image from a remote registry, and this image itself will care about running "all the things" internally, such as prerequisites, pulling the dependent images, preparing the environments, network and dependencies, doing alternative steps to docker-compose and much more.


Or going far beyond that: shipping is as a traditional GUI installer for non-technical people or deploying that same image into a cloud, any cloud, ready to use, does not that look as such desirable? What if I tell you this technology is already available and you can use it? Please welcome the new universal spec for packaging distributed apps created by Microsoft and Docker:

Cloud-Native Application Bundles or simply CNAB

Firstly, what the heck is CNAB at all? Using Cloud Native Application Bundle, a developer has an option of deploying the app either locally at his dev. machine, or in a public cloud as it bolts together containers and services into a seamless whole, placing a strong focus on standardization. It is an open-source specification that aims to facilitate the bundling, installing, and managing of containerized apps. With this bundle, users can define resources that can then be deployed to a number of run-time environments, such as Docker, Azure, Kubernetes, Helm, automation services (such as those used by GitOps), and more.

At a first glance, that task is supposed to be solved by Docker itself. However, when dealing with largely scaled hybrid infrastructures, its standard features become insufficient. Thus, CNAB is an attempt of standardizing the process of packaging, deployment, and lifecycle management of distributed apps on a basis of Kubernetes, Helm, Swarm, and others by using a unified JSON-based package format. 

Recently the CNAB spec reached version 1.0, which means it is ready for production deployment. The spec itself is now broken down into several chapters:

  • CNAB explains the fundamentals of the CNAB core 1.0.
  • CNAB Registry will describe how CNAB bundles can be stored inside of OCI Registries (this section is not yet complete).
  • CNAB Security explains the mechanisms for signing, verifying, and attesting CNAB packages.
  • CNAB Claims describes the CNAB Claims system, shows how records of CNAB installations formatted for storage
  • CNAB Dependencies describes how bundles can define dependencies on other bundles.

Tooling

Each of the organizations has provided its own tools that demonstrate CNAP capabilities: Microsoft released Duffle, while Docker shipped Docker app. Docker Desktop application is fully compatible with CNAB from May 2019.

CNAB is not the only solution for managing the cloud applications lifecycle. For example, Kubernetes has Сrossplane manager as well as package manager Helm. However, CNAB is the first ever solution that supports several most popular tools and is platform-agnostic. By the way, CNAB can also work with Helm and I came across a sample of it at GitHub.

Duffle is a simple command line tool that interacts with Cloud-Native Application Bundles - helping you package and unpackage distributed apps for deployment on whatever cloud platforms and services you use. Its goal is to exercise all parts of the specification and this tool also comes with very handy VS Code extensions, one of which named Duffle Coat allows you to create native executable installer (*.exe) of your bundle:


This results in the proper installer that will install and configure you Sitecore 9.3 locally from bundle image stored at docker registry:



Once again, instead of local Sitecore installations (like SIA does), we are having this CNAB installer that installs the platform from a Docker registry and with no prerequisites required at all! And CNAB bundle cares about all the dependencies and parameters. What magic!

Another tool, Porter, is Microsoft’s CNAB builder that gives you building blocks to create a cloud installer for your application, handling all the necessary infrastructure and configuration setup. It is a declarative authoring experience that lets you focus on what you know best: your application. The power of Porter is coming from using mixins giving CNAB authors smart components that understand how to adapt existing systems, such as Helm, Terraform, or Azure, into CNAB actions.


And of course, Docker App is a CNAB builder and installer that leverages the Docker Compose format to define your applications. To facilitate developer-to-operator handover, you can add metadata and run-time parameters. These applications can easily be shared using existing container registries. Docker App is available as part of the latest Docker release.

Bundle manifest file
As I said above, the specification defines the way of packaging distributed application of various formats. CNAB includes package definition (named bundle.json) used for describing an app, as well as a special image (also called invocation image) for its installation. A bundle.json is similar to a docker-compose.yml file in that it describes a complex configuration for image deployment. The difference is, the CNAB bundle is very clearly defined as to how it should be laid out, encoded, and where all associated files must reside. It contains:

  • The schema version.
  • Top-level package information.
  • Information on invocation images.
  • Map of images.
  • Specification for parameter override (with a reference to a validation schema).
  • List of credentials.
  • Optional description of custom actions.
  • A list of outputs produced by the application.
  • A set of schema definitions is used to validate input.
Here is a sample of bundle.json file below:
{ 
   "credentials":{ 
      "hostkey":{ 
         "env":"HOST_KEY",
         "path":"/etc/hostkey.txt"
      }
   },
   "custom":{ 
      "com.example.backup-preferences":{ 
         "frequency":"daily"
      },
      "com.example.duffle-bag":{ 
         "icon":"https://example.com/icon.png",
         "iconType":"PNG"
      }
   },
   "definitions":{ 
      "http_port":{ 
         "default":80,
         "maximum":10240,
         "minimum":10,
         "type":"integer"
      },
      "port":{ 
         "maximum":65535,
         "minimum":1024,
         "type":"integer"
      },
      "string":{ 
         "type":"string"
      },
      "x509Certificate":{ 
         "contentEncoding":"base64",
         "contentMediaType":"application/x-x509-user-cert",
         "type":"string",
         "writeOnly":true
      }
   },
   "description":"An example 'thin' helloworld Cloud-Native Application Bundle",
   "images":{ 
      "my-microservice":{ 
         "contentDigest":"sha256:aaaaaaaaaaaa...",
         "description":"my microservice",
         "image":"example/microservice:1.2.3"
      }
   },
   "invocationImages":[ 
      { 
         "contentDigest":"sha256:aaaaaaa...",
         "image":"example/helloworld:0.1.0",
         "imageType":"docker"
      }
   ],
   "maintainers":[ 
      { 
         "email":"[email protected]",
         "name":"Matt Butcher",
         "url":"https://example.com"
      }
   ],
   "name":"helloworld",
   "outputs":{ 
      "clientCert":{ 
         "definition":"x509Certificate",
         "path":"/cnab/app/outputs/clientCert"
      },
      "hostName":{ 
         "applyTo":[ 
            "install"
         ],
         "definition":"string",
         "description":"the hostname produced installing the bundle",
         "path":"/cnab/app/outputs/hostname"
      },
      "port":{ 
         "definition":"port",
         "path":"/cnab/app/outputs/port"
      }
   },
   "parameters":{ 
      "backend_port":{ 
         "definition":"http_port",
         "description":"The port that the back-end will listen on",
         "destination":{ 
            "env":"BACKEND_PORT"
         }
      }
   },
   "schemaVersion":"v1.0.0",
   "version":"0.1.2"
}
You may read more about bundle.json format at CNAB.io official page.

How about Azure?
For Azure, we also have got the solution, Azure CNAB Quickstarts library. It demonstrates how one can use bundles for deploying applications and solutions and how to create their own bundles. The library is designed to be optimized for bundles that use Azure resources but is not limited to Azure only. There is CI/CD workflow in the repository using custom GitHub Actions to enable the automatic building of bundles and publishing of bundles to a public Azure Container Registry. The library supports bundles made using Porter tool I mentioned above, a tool capable of building, publishing, invoking, and updating bundles.

Final thoughts
Likely CNAB becomes a game-changer for 2020, as we get more and more into containerized deployments and orchestrating them in the clouds. The specification is quite new and not too many companies are using it at the moment, but there is an ever-growing interest in it. Since all the major vendors are now ready, I am quite sure it will boost the whole industry in the coming months!

References
Hope you find this post helpful and get your own hand on CNAB shortly!