Experience Sitecore ! | February 2020

Experience Sitecore !

More than 200 articles about the best DXP by Martin Miles

Why CNAB could be a game-changer for Docker containers and how Sitecore can benefit from that?

Introduction

Currently, in order to run Sitecore in docker locally, one has to pull the code from a GitHub repository, build it (if not done yet, or pull already built images from a Docker registry), set the license file, and say "up" to docker-compose. Not to say the prerequisites required. If dealing with a cloud, then the deployment into Kubernetes is required, which also demands adequate skills.

Imagine an ideal situation where you don't need to do all the things, but just pull a "managing" image from a remote registry, and this image itself will care about running "all the things" internally, such as prerequisites, pulling the dependent images, preparing the environments, network and dependencies, doing alternative steps to docker-compose and much more.


Or going far beyond that: shipping is as a traditional GUI installer for non-technical people or deploying that same image into a cloud, any cloud, ready to use, does not that look as such desirable? What if I tell you this technology is already available and you can use it? Please welcome the new universal spec for packaging distributed apps created by Microsoft and Docker:

Cloud-Native Application Bundles or simply CNAB

Firstly, what the heck is CNAB at all? Using Cloud Native Application Bundle, a developer has an option of deploying the app either locally at his dev. machine, or in a public cloud as it bolts together containers and services into a seamless whole, placing a strong focus on standardization. It is an open-source specification that aims to facilitate the bundling, installing, and managing of containerized apps. With this bundle, users can define resources that can then be deployed to a number of run-time environments, such as Docker, Azure, Kubernetes, Helm, automation services (such as those used by GitOps), and more.

At a first glance, that task is supposed to be solved by Docker itself. However, when dealing with largely scaled hybrid infrastructures, its standard features become insufficient. Thus, CNAB is an attempt of standardizing the process of packaging, deployment, and lifecycle management of distributed apps on a basis of Kubernetes, Helm, Swarm, and others by using a unified JSON-based package format. 

Recently the CNAB spec reached version 1.0, which means it is ready for production deployment. The spec itself is now broken down into several chapters:

  • CNAB explains the fundamentals of the CNAB core 1.0.
  • CNAB Registry will describe how CNAB bundles can be stored inside of OCI Registries (this section is not yet complete).
  • CNAB Security explains the mechanisms for signing, verifying, and attesting CNAB packages.
  • CNAB Claims describes the CNAB Claims system, shows how records of CNAB installations formatted for storage
  • CNAB Dependencies describes how bundles can define dependencies on other bundles.

Tooling

Each of the organizations has provided its own tools that demonstrate CNAP capabilities: Microsoft released Duffle, while Docker shipped Docker app. Docker Desktop application is fully compatible with CNAB from May 2019.

CNAB is not the only solution for managing the cloud applications lifecycle. For example, Kubernetes has Сrossplane manager as well as package manager Helm. However, CNAB is the first ever solution that supports several most popular tools and is platform-agnostic. By the way, CNAB can also work with Helm and I came across a sample of it at GitHub.

Duffle is a simple command line tool that interacts with Cloud-Native Application Bundles - helping you package and unpackage distributed apps for deployment on whatever cloud platforms and services you use. Its goal is to exercise all parts of the specification and this tool also comes with very handy VS Code extensions, one of which named Duffle Coat allows you to create native executable installer (*.exe) of your bundle:


This results in the proper installer that will install and configure you Sitecore 9.3 locally from bundle image stored at docker registry:



Once again, instead of local Sitecore installations (like SIA does), we are having this CNAB installer that installs the platform from a Docker registry and with no prerequisites required at all! And CNAB bundle cares about all the dependencies and parameters. What magic!

Another tool, Porter, is Microsoft’s CNAB builder that gives you building blocks to create a cloud installer for your application, handling all the necessary infrastructure and configuration setup. It is a declarative authoring experience that lets you focus on what you know best: your application. The power of Porter is coming from using mixins giving CNAB authors smart components that understand how to adapt existing systems, such as Helm, Terraform, or Azure, into CNAB actions.


And of course, Docker App is a CNAB builder and installer that leverages the Docker Compose format to define your applications. To facilitate developer-to-operator handover, you can add metadata and run-time parameters. These applications can easily be shared using existing container registries. Docker App is available as part of the latest Docker release.

Bundle manifest file
As I said above, the specification defines the way of packaging distributed application of various formats. CNAB includes package definition (named bundle.json) used for describing an app, as well as a special image (also called invocation image) for its installation. A bundle.json is similar to a docker-compose.yml file in that it describes a complex configuration for image deployment. The difference is, the CNAB bundle is very clearly defined as to how it should be laid out, encoded, and where all associated files must reside. It contains:

  • The schema version.
  • Top-level package information.
  • Information on invocation images.
  • Map of images.
  • Specification for parameter override (with a reference to a validation schema).
  • List of credentials.
  • Optional description of custom actions.
  • A list of outputs produced by the application.
  • A set of schema definitions is used to validate input.
Here is a sample of bundle.json file below:
{ 
   "credentials":{ 
      "hostkey":{ 
         "env":"HOST_KEY",
         "path":"/etc/hostkey.txt"
      }
   },
   "custom":{ 
      "com.example.backup-preferences":{ 
         "frequency":"daily"
      },
      "com.example.duffle-bag":{ 
         "icon":"https://example.com/icon.png",
         "iconType":"PNG"
      }
   },
   "definitions":{ 
      "http_port":{ 
         "default":80,
         "maximum":10240,
         "minimum":10,
         "type":"integer"
      },
      "port":{ 
         "maximum":65535,
         "minimum":1024,
         "type":"integer"
      },
      "string":{ 
         "type":"string"
      },
      "x509Certificate":{ 
         "contentEncoding":"base64",
         "contentMediaType":"application/x-x509-user-cert",
         "type":"string",
         "writeOnly":true
      }
   },
   "description":"An example 'thin' helloworld Cloud-Native Application Bundle",
   "images":{ 
      "my-microservice":{ 
         "contentDigest":"sha256:aaaaaaaaaaaa...",
         "description":"my microservice",
         "image":"example/microservice:1.2.3"
      }
   },
   "invocationImages":[ 
      { 
         "contentDigest":"sha256:aaaaaaa...",
         "image":"example/helloworld:0.1.0",
         "imageType":"docker"
      }
   ],
   "maintainers":[ 
      { 
         "email":"matt.butcher@microsoft.com",
         "name":"Matt Butcher",
         "url":"https://example.com"
      }
   ],
   "name":"helloworld",
   "outputs":{ 
      "clientCert":{ 
         "definition":"x509Certificate",
         "path":"/cnab/app/outputs/clientCert"
      },
      "hostName":{ 
         "applyTo":[ 
            "install"
         ],
         "definition":"string",
         "description":"the hostname produced installing the bundle",
         "path":"/cnab/app/outputs/hostname"
      },
      "port":{ 
         "definition":"port",
         "path":"/cnab/app/outputs/port"
      }
   },
   "parameters":{ 
      "backend_port":{ 
         "definition":"http_port",
         "description":"The port that the back-end will listen on",
         "destination":{ 
            "env":"BACKEND_PORT"
         }
      }
   },
   "schemaVersion":"v1.0.0",
   "version":"0.1.2"
}
You may read more about bundle.json format at CNAB.io official page.

How about Azure?
For Azure, we also have got the solution, Azure CNAB Quickstarts library. It demonstrates how one can use bundles for deploying applications and solutions and how to create their own bundles. The library is designed to be optimized for bundles that use Azure resources but is not limited to Azure only. There is CI/CD workflow in the repository using custom GitHub Actions to enable the automatic building of bundles and publishing of bundles to a public Azure Container Registry. The library supports bundles made using Porter tool I mentioned above, a tool capable of building, publishing, invoking, and updating bundles.

Final thoughts
Likely CNAB becomes a game-changer for 2020, as we get more and more into containerized deployments and orchestrating them in the clouds. The specification is quite new and not too many companies are using it at the moment, but there is an ever-growing interest in it. Since all the major vendors are now ready, I am quite sure it will boost the whole industry in the coming months!

References
Hope you find this post helpful and get your own hand on CNAB shortly!

Sitecore Docker breaks when running Build.ps1 with "hcsshim::PrepareLayer failed in Win32: Incorrect function. (0x1)" message


Recently I was trying to build docker images from the main Sitecore repository, and got quite a weird message.

hcsshim::PrepareLayer - failed failed in win32 : Incorrect function. (0x1)

This message is not self-explanatory and it fired up when the actual build for the first image took place, in my case that was SQL server 2017. Some other details: quite fress and new instance of Windows 10 x64 build 1909, using Hyper-V, no docker / images previously been utilized at thtis computer.

After few hours long googling, I came to understanding that was caused by c:\windows\system32\cbfsconnectnetrdr2017.dll referenced by autorun. There is one halpful tool that helps out troubleshoouting such issues - Sysinternals Autoruns. Here is the output generated by Autoruns for my system:


What is that DLL and how did it get to my machine? CBFSConnect is a known library for creating and managing virtual filesystems for cloud storage providers such as such as Box, Dropbox etc., and this DLL is a kernel mode system drive. Sometimes it may be a leftover from a previously installed software, that was not uninstalled fully and properly.

Depending no your system you may see this DLL as either:

  • C:\Windows\system32\cbfsconnectnetrdr2017.dll  - for newer systems, or
  • C:\Windows\System32\drivers\cbfs6.sys
You may check if it is running or no tby running  a command-line utility for common minifilter driver management operations command (a command-line utility for common minifilter driver management operations):


Resolution
The easiest I found was to just simply rename this driver. Once restarted, docker build worked well.

Hope this helps someone

Tip of the day: running Sitecore Docker from within Hyper-V virtual machine is in fact possible

Why not to run Docker containers from within a Hyper-V virtual machine? 

Well, when you are trying to install Docker Desktop for Windows, it installs correctly. But running it will prompt you about Hyper-V is required for Docker to work.


Programs and Features on its turn shows you that Hyper-V is greyed out, so you cannot install it from there



Not everyone knows, but Hyper-V in fact allows having nested virtual machines. That means running docker is also possible from within a VM. All you need is just running one PowerShell command from outside of your container:

Set-VMProcessor -VMName _YOUR_VM_NAME_TO_ENABLE_ -ExposeVirtualizationExtensions $true

What it does - just adds virtualization features into a virtual processor of outer VM, similar to those you got at you physical CPU. Thus you need to run this command of the VM switched off.

Once done, Sitecore perfectly builds and work if Docker running within a virtual machine. For those who are total beginner and scary to mess the things around their host machines - this could be an option. However there is of course a performance penalty from double virtualization.



Finally, the main trick! Just because you've added virtualization extensions into a virtual processor of your VM, you may run docker in process isolation mode, relatively to the build number of VM's operation system. Will explain it on the following example: imagine, your host machine runs Windows 10 build 1809, but got no problems of running Windows 10 build 1909 in a Hyper-V machine there. Now, you after you enable virtualization extensions to that virtual machine (that runs 1909), you will be able to install docker within that virtual machine, and run 1909-built images in process isolation, natively to that VM: 1909-built images run on 1909- machine! But at the same time you will be able to run those same images only in hyper-v mode on the (outer) host machine, because it has 1809 and cannot run non-matching images in process isolation mode.

Hope this helps!

I have been awarded the Sitecore Most Valuable Professional Technology Award in 2020

Great news for me!

Just got an email saying I was awarded Sitecore Most Valuable Professional Technology Award for the oncoming year.

I feel very excited and carry on plenty of brave plans for the 2020. My main efforts will be applied in two main areas: containers and JSS, at least as I see that now. Apart from will take all of my best to get selected as a speaker for a big audience conference and keep going developing Sifon, as it is quite promissing tool (unfortunately it is so tought with finding and allocation long timeslots for it).

In any case, I am pleased to stay as MVP and hope to meet you at the events later this year.