Experience Sitecore ! | All posts tagged 'Docker'

Experience Sitecore !

More than 200 articles about the best DXP by Martin Miles

LTSC2022 images for Sitecore containers released: what does it mean to me?

Exciting news! Sitecore kept the original promise and released the new ltsc2022 container images for all the topologies of both the 10.3 and 10.2 versions of their platform.

The biggest benefits of new images are improved image sizes – almost 50% smaller than ltsc2019, and support for running Process Isolation on Windows 11.

Check it yourself:

So, what does that mean for developers and DevOps?

First and most, running Sitecore 10.3 on Windows Server 2022 is now officially supported. You may consider upgrading your existing solutions to benefit from Server 2022 runtime.

Developers working on Windows 11 now also got so much wanted support, containers built from the new images can run in Process isolation mode without a hypervisor. That brings your cluster performance to nearly bare metal metrics.


Let's try it in action!

I decided to give it a try and test if that would work and how effectively. I recently purchased a new Microsoft Surface  8 Pro laptop which had Windows 11 pre-installed and therefore useless for my professional purposes, so it seems to be excellent test equipment.

After initial preparation and installing all the prerequisites, I was ready to go. Choosing the codebase I decided to go with the popular Sitecore Containers Template for JSS Next.js apps and Sitecore 10.3 XM1 topology, as the most proven and well-preconfigured starter kit.

Since I initialized my codebase with -Topology XM1 parameter, all the required container configurations are located under /MyProject/run/sitecore-xm1 folder. We are looking for .env file which stores all the necessary parameters.

The main change to do here is setting these two environmental settings to benefit from ltsc2022 images:

SITECORE_VERSION=10.3-ltsc2022
EXTERNAL_IMAGE_TAG_SUFFIX=ltsc2022

The other important change in .env file would be changing to ISOLATION=process. Also, please note that TRAEFIK_ISOLATION=hyperv stays unchanged due to a lack of ltsc2022 support for Traefik, so sadly you still need to have Hyper-V installed on this machine. The difference is that it serves only Traefik, the rest of Sitecore resources will work in the Process mode.

I also did a few optional improvements upgrading important components to their recent versions:

MANAGEMENT_SERVICES_IMAGE=scr.sitecore.com/sxp/modules/sitecore-management-services-xm1-assets:5.1.25-1809
HEADLESS_SERVICES_IMAGE=scr.sitecore.com/sxp/modules/sitecore-headless-services-xm1-assets:21.0.583-1809

Also, changed node to reflect the recent LTS version:

NODEJS_VERSION=18.14.1

Please note, that sitecore-docker-tools-assets did not get any changes from the previous version of Sitecore (10.2), so I left it untouched.

Last thing – to make sure I indeed build and run in the Process isolation mode, I set ISOLATION=process changing this value from default. The rest of .env file was correctly generated for me by Init.ps1 script.

All changes complete, let’s hit .\up.ps1 in PowerShell terminal with administrative mode and wait until it downloads and builds images:


Advanced Part: building Traefik with ltsc2022

Now, let's get rid of the only left 1809-based container, which is Traefik. Luckily, its Dockerfile is available, so I can rewrite it to consume ltsc2022 images. In addition, I took the latest (by the time) version of it which is 2.9.8, while the officially supported is 2.2.0, so it would make sense to parametrize the version as well, taking its settings from .env settings.

I created a new docker\build\traefik folder and ended up with the following Dockerfile within there:

ARG IMAGE_OS
FROM mcr.microsoft.com/windows/servercore:${IMAGE_OS}

ARG VERSION
SHELL ["powershell", "-Command", "$ErrorActionPreference = 'Stop'; $ProgressPreference = 'SilentlyContinue';"]

RUN Invoke-WebRequest \
        -Uri "https://github.com/traefik/traefik/releases/download/$env:VERSION/traefik_${env:VERSION}_windows_amd64.zip" \
        -OutFile "/traefik.zip"; \
    Expand-Archive -Path "/traefik.zip" -DestinationPath "/" -Force; \
    Remove-Item "/traefik.zip" -Force

EXPOSE 80
ENTRYPOINT [ "/traefik" ]

# Metadata
LABEL org.opencontainers.image.vendor="Traefik Labs" \
    org.opencontainers.image.url="https://traefik.io" \
    org.opencontainers.image.source="https://github.com/traefik/traefik" \
    org.opencontainers.image.title="Traefik" \
    org.opencontainers.image.description="A modern reverse-proxy" \
    org.opencontainers.image.version=$env:VERSION \
    org.opencontainers.image.documentation="https://docs.traefik.io"

Because of that I also had to update the related docker-compose section of docker-compose.override.yml file:

  traefik:
    isolation: ${ISOLATION}
    image: ${REGISTRY}traefik:${TRAEFIK_VERSION}-servercore-${EXTERNAL_IMAGE_TAG_SUFFIX}
    build:
      context: ../../docker/build/traefik
      args:
        IMAGE_OS: ${EXTERNAL_IMAGE_TAG_SUFFIX}
        VERSION: ${TRAEFIK_VERSION}
    volumes:
      - ../../docker/traefik:C:/etc/traefik
    depends_on:
    - rendering

What I want to pay attention here - I am now using ${ISOLATION} as the rest of the containers are using instead of dedicated TRAEFIK_ISOLATION which can now be removed from .env.

Another thing is that I am passing fully parametrized image name:

image: ${REGISTRY}traefik:${TRAEFIK_VERSION}-servercore-${EXTERNAL_IMAGE_TAG_SUFFIX}

I intentionally do not prefix it with ${COMPOSE_PROJECT_NAME} so that this image becomes reusable between several solutions on the same machine, which saves some disk drive space.

Last step would be adding .env parameter TRAEFIK_VERSION=v2.9.8 and removing TRAEFIK_IMAGE parameter which is no longer needed. Good to go!


Outcomes and verdict

I tested all of the important features of the platform, including Experience Editor and it all works, and what is especially important – works impressively fast with the Process isolation mode. And since all the containers are built with ltsc2022 and run in Process isolation, one doesn't need Hyper-V at all!

As for me, I ended up having a nice and powerful laptop suitable for modern Sitecore headless operations.

Enjoy faster development!

All you need to know about Sitecore 10

As you might have already heard, Sitecore has just released a new major version of its product Sitecore 10, the one initially supposed to be numbered as Sitecore 9.4. Let's take a look at what does it have to offer to better understand why Sitecore decided to "release it louder" by giving a major version number.



During the course of this post I will be writing about:

  1. Overview
  2. Installation
  3. Containers
  4. ASP.NET Core
  5. CLI with Serialization
  6. Horizon
  7. Search
  8. XM / CMS-Only
  9. JSS and SXA
  10. Data Exchange and Connectors
  11. Privacy options
  12. Analytics
  13. EXM
  14. Marketing Automation
  15. Experience Editor
  16. Content Hub
  17. Other changes
  18. Further


1. Overview

Finally, Sitecore 10 has been released. With the release of Sitecore 10, a new era of Sitecore development has started, the mostly anticipated approaches becoming an official part of the platform. This version focuses on product updates and enhancements that provide more development and deployment options, increase usability and improve overall performance – all centered around enabling both Marketing and IT teams equally, thus making it easier and faster to launch and evolve digital customer experiences.


2. Installation

From now on, there are 3 option you may choose from in order to get Sitecore 10 up and running. 

Historically, there was an MSI installer to install Sitecore as well as a ZIP archive to handle things manually. With version 8 comes a third-party Sitecore Instance Manager to simplify the installation along with some housekeeping tasks. It is no longer with us.

1. From Sitecore version 9.0 there was one and only official way to install Sitecore - SIF. A bit complicated for newbies and those who aren't familiar with PowerShell and JSON configurations. 

2. Later on with Sitecore 9.2, we got one more option for that - SIA with its GUI. With every next release SIA became more mature and offered more option. As for now, SIA allows to install Solr on your behalf together with few more things, such as extra validations:

  • validates Solr connectivity
  • validates SQL Server connectivity
  • validates the Sitecore license file
  • displays a progress spinner during the installation process
  • supports the installation of Sitecore Experience Commerce

3. Since version 10.0, Sitecore now officially has support for Docker, Kubernetes and image repositories. Sitecore community has been around containers for a while and we made much commitment to make things happen - you may see my older posts on that. This for sure will help delivery teams move to now famous continuous delivery model, making infrastructure-as-code deployments for Sitecore smoother.

Of course, SIF is still available as the default option for local installation, for example on VMs or when your team is not yet ready for containers.

Register-PSRepository -Name SitecoreGallery -SourceLocation https://sitecore.myget.org/F/sc-powershell/api/v2
Install-Module SitecoreInstallFramework -Force

The actual SIF version for Sitecore 10.0 is 2.3.0, to ensure you've got it right type in:

Get-Module SitecoreInstallFramework –ListAvailable

Check if you have got 2.3.0 along with 2.3.0 folder as well in WindowsPowershell folder - then you're ready to install Sitecore 10 using SIF.

You may be interested in reading:


3. Containers

Firstly announced as supported in Sitecore 9.3, Sitecore released version 10.0 out-of-the-box support for containers - now it is an official option to develop or instantiate Sitecore and other services with containers. To help Sitecore developers learn, use, and get started quickly with Docker, there is comprehensive documentation available on developing with Sitecore on containers.

Sitecore has docker container images for Sitecore XP and XM along with non-prod images for MSSQL, Solr and Redis for developers, publicly available from Sitecore image repository. In this initial version, SXA is supported. Now you don't need to build images yourself. There are also liveness and readiness health check endpoints which are especially useful with containers.

1. Sitecore Experience Platform 10.0.0 uses Docker Compose as the container orchestrator on developer workstations. Docker Compose is a simple container deployment tool that is bundled with Docker for Windows. Sitecore container images can be deployed with other tools but we recommend that you use Docker Compose to deploy the containers that form the Sitecore Experience Platform.

2. Sitecore Experience Platform 10.0.0 uses Kubernetes (K8s) as the default orchestrator for deploying production environments. The Sitecore XP Kubernetes specification files that are used to map the minimum required configuration parameters between the Sitecore software containers are provided as a reference.

Sitecore customers are expected to extend these specifications to support their own requirements. It is the responsibility of each Sitecore customer to ensure that their production deployments meet the standards for stability and security set by their organization.

Why are containers important? In general, if you’ve been doing your development with virtual machines or local installations, moving to a Docker-driven development and infrastructure flow will be highly beneficial for you. In general, you can address four major areas:

  • No more need for a long onboarding process for new team members
  • Eliminates pain of having several Sitecore instances especially of various versions in parallel
  • No more "works on my machine" errors
  • Improved operational capabilities with new monitoring and security support
  • Better computer resources management due to process isolation opposing running full OS VM
  • Much more reliable continuous delivery model
  • No need to "install" Sitecore - images are already pre-built for you
  • Infrastructure-as-code approach (IaC) has proven its efficiency and finally landed the world of Sitecore

Adopting Docker and Kubernetes into your workflow will let your team focus on actual development rather than wasting time on repetitive ops, providing standard tooling they need to work on those changes.


4. ASP.NET Core

I don't think anyone missed out all the noise around ASP.NET Core which is an open-source, cross-platform framework for building fast, cloud-based web apps on Windows, macOS, or Linux. Some of Sitecore features prior to version 10 are already based on ASP.NET Core - Publishing Service, Identity Service, Sitecore Commerce websites, not to mention Sitecore Host to support many features like that. However, the core rendering of pages and components still remains dependent  on the legacy of .NET Framework - which terminates at version 4.8 and won't go ahead. With .NET 5 release by the end of 2020, Sitecore has a long term strategy with Sitecore Host and the rest of development moving to .Net Core.

.NET Framework is still there in Sitecore 10 for legacy rendering, but since now we've got a Headless development option allows teams to build applications faster on the latest .NET Core technology. It is really cool especially for those pure back-end developers who just don't want to learn JavaScript and those customers who don't need fully javascripted SPA-like application.

Headless development with ASP.NET Core brings developers same JSS-like advantages, development and runtime isolation for a site, delivering a better development experience. There is a new SDK for headless ASP.NET Core rendering in Sitecore 10. 
Using the same headless services that power JSS, you deliver experiences that work in Experience Editor, track and provide marketing capabilities, but work faster. You can use placeholders, field renderings, model bindings in ASP.NET Core as normal:

  • faster and lighter
  • run headless via the ASP.NET Rendering SDK
  • works in Experience Editor
  • supports routing
  • compiling your rendering code won't cause an app pool recycle in the platform
  • only connect to Sitecore when need to: making changes to the presentation layer, pushing them, debugging in a seconds
  • remote rendering host - developers can build straight from VS into the rendering host and test changes without recycling all of Sitecore every time. 


And a cherry on a cake - there's a Getting Started Template that demonstrates all the features for the release all together. Not to say that works well with your development version running in Docker containers. The code is available on GitHub. To install latest template type:
dotnet new -i Sitecore.DevEx.Templates --nuget-source https://sitecore.myget.org/F/sc-packages/api/v3/index.json
Looking forward to see full support for ASP.NET Core 5 in the next releases of Sitecore.


5. CLI with Serialization

As we've been shown at Sitecore MVP Summit last year, Sitecore developed Command Line Interface (CLI) to interact with a Sitecore instance. The expectation is to be able doing pretty much everything from command line and in that sense it will intersect (or may even absorb) PowerShell Extensions. In order to attract developers to CLI - they decided to start with serialization. 

It was quite a surprise that Sitecore got to proper built-in way to serialize items from or deploy items into Sitecore - only using third-party tooling. Now this release of the CLI Sitecore is standardizing the tooling and approaches for serialization. Its initial release allows you serializing, pushing items with further publishing, by combining the best of TDS and Unicorn.

To support the security one need to login in via interactive or automated authentication flows, integrated with Identity Server.

With next releases, CLI will add more functionality, but what is even more thrilling, a plugin system for adding new commands via NuGet is also expected. And since it will cover all the aspects of creating and managing content, one can even create a wrapper over it implementing own version of Content Editor with any set of features (in fact, there's no need to since we've got Horizon, keep reading below). 

For those who prefer GUI over command line, there is also a wrapper available over existing set of serialization CLI-features of Sitecore for Visual Studio, a new member of the Sitecore Developer Collection that does Sitecore Content Serialization from a GUI within Visual Studio - similar to TDS did before.

More reading:

6. Horizon

The first version of the new content editing experience called Horizon has been released with Sitecore 9.3 and there has been a lot of buzz around new editing interface since then. That first version was quite limited with supported functionalities.

Now with the second release of Sitecore, it supports more functionalities:

  • Multi-site and multi-lingual are now supported and added to the user interface
  • Drag & drop support in the page tree to organize your site’s structure
  • Content Hub integration
  • Keyboard shortcuts 
  • SXA sites are also supported with all the above
Horizon Content is a content manager that you use to manage content without a presentation layer. In Content you can see meta data for each field and you can edit content that is not accessible in Pages

Speaking of SXA sites, the integration to SXA has been improved to support editing directly on the page for all supported field types and also do site-specific images and datasources. A tighter integration between SXA and Horizon editing will be coming over the course of the Sitecore 10 series.

A new dedicated field editor experience allows you to edit page metadata that is not directly editable on the page. This allows you to edit other types of fields without the WYSIWYG interface. This is also useful for editing ‘headless’ content that is not intended for a page. Using the new Content view allows you to see all the content items, not just the pages, and edit your data in the Horizon field editor.

Extending Horizon with plugins is not yet available, however is expected with oncoming releases.

With assets in Sitecore Content Hub DAM, you can now leverage that rich media in the Horizon editor instead of only using the native Sitecore Experience Platform media library. This allows your marketing team to have a better flow from Content Hub to your web channel.


Sooner or later this should have happened - Sitecore XP 10.0.0 deprecates Azure Search and it will be completely removed ​in a future release​. If you are starting a new Sitecore project, please use Solr as your search engine.

What I especially love with Sitecore over the years, they do admit and fix previously done mistakes. Azure Search was not fully compatible since day one, I personally struggles so much with it. Finally it will leave Sitecore.

Sitecore XP no longer from passes malformed content to Solr​ ​-it now filters and validates content prior sending it to Solr​.

Last but not the least, Sitecore 10 now requires Solr version 8.4.0 to work with.


8. XM / CMS-Only

Few version ago, Sitecore came up with a lightweight and very fast CMS-only version of its product, opposing full XP with full-powered analytics and much more things around it. However for some reason there was not rule engine personalization coming with XM topology which was not right: for those not needing the full xConnect, still want get use use of some basic personalization or implementing custom conditions, especially valuable for SXA websites. Finally that got fixed and one can now run simple personalization on lightweight XM installations. This functionality contains session-based and device rules that do not depend on tracker or marketing definitions available in the XM package.

Please refer to the list of personalization conditions available for XM to find out more.


9. JSS 14.0 and SXA version 10.0

There are not much of new JSS features with this release, the changes are mostly relate to the stability improvements and support for new platform features. Thus, new JSS can now successfully handle forwarded deployment requests when it is installed behind a reverse proxy as it is in Docker. Exactly the same applies to SXA.

SXA keeps having its versions aligned to the platform release, in order to avoid the confusion.

For more details on what's new, please read the following:


10. Data Exchange and Connectors

Changes in Data Exchange Framework (DEF):

  • Additional field types added to CMP Connector
  • You can now sync a single contact
  • Respond to live events on the site

Sitecore Connect for Salesforce Marketing Cloud allows teams to work in Salesforce Marketing Cloud and leverage the rich visitor analytics data stored in xDB. Now, with the latest release, xDB information can be more easily sent to SalesForce. You can make use of this connection in real-time, no need to wait for a nightly batch. A new Sitecore Marketing automation activity, specifically for Salesforce Marketing Cloud, allows marketers to configure a realtime event to integrate with SFMC.

By connecting directly to the Salesforce Marketing Cloud, marketing teams can respond to events in Sitecore and immediately send information about the current visitor over to Salesforce Marketing Cloud and place visitors into Journey Builder marketing automation plans. This means that you can respond across all your channels to events, in real-time, in the marketing automation tool that you use.

Please read this link for more details on SalesForce Connector.


11. Privacy options

Previous releases of the platform have progressively been improving the privacy tools available to teams looking to meet with regulatory needs and customer privacy demands. In Sitecore 10, new support has been added to help you on your journey. Sitecore 10.0 and later provides API calls and configuration options that makes it easier to enforce explicit consent for tracking a contact's activity on your websites.“Update consent settings” now allows you to remove contacts from marketing messages or all messages. In general, there are two new "rights":

1. Right to be forgotten. With this release, Sitecore Forms makes it easier for you to support erasing this personal identifiable data that is saved via the out-of-the-box Save data submit action. This includes redacting submission values, removing any associations to xDB Contacts, and deleting any files uploaded by the visitor. At the same time this approach still allows marketing teams to see reporting data, despite being anonymized.

2. Right to object. A second change in this release regards making it easier to enforce and manage consent choices. Visitor tracking now requires that giving explicit consent.and marketing teams need to respect and store these choices and make decisions on them. The new consent management tools available in the platform allow for implementation teams to configure sites to require explicit consent before the Sitecore tracking will begin. The tracker is now able to be disabled based on the consent settings stored for the user, and xDB supports the ability to store these consent settings and persist it across visits for the contact.

This allows marketing teams to add additional consent requirements for specific sites that need to meet stricter privacy guidelines.


12. Analytics

Getting insights from Sitecore’s analytics reports is great, but previously one could not filter on specific segments. Now, there is a new filtering functionality available in all reports! 

This means that you can use Segments created from List Manager based on all sorts of segmentation rules as filters on your analytics. In the past when the “Use as analytics filter” was enabled in a Segment, you needed to apply some extra configuration to have this filter available in your reports. Now, this filter is available in the analytics reports without any extra configuration:

  • Target segments of contacts via predefined rules and then report based on that segment
  • Report on the performance of key audience groups
  • Push an email to a segmented list and then see the impact across the site

This is great, because with the extensive set of available segmentation rules, it’s very easy to zoom into the analytics for these specific segments focusing on a group of contacts.


13. EXM

Before Sitecore 9.2, there were no standard templates available OOB at all. Now, with Sitecore 10 we've got even more new additional templates added to the product: 

  • Left Image Block
  • Right Image Block
  • Image Focus

This saves much time for the customers using EXM from developing such email templates from scratch, as these templates can become a great starting point at the cost of a single click. But what is really great - these templates are completely customizable to using Scriban, similar to editing rendering variants in SXA.


14. Marketing Automation

Firstly, Marketing Automation has been reworked to allow for more efficient loading of xConnect data, there are performance improvements in XConnect for customers who do not use Marketing Automation. It does not load contacts and interactions unless there is an inactive or active plan.

Secondly, there are new rules around birthdays added to engage with contacts around their birth date. You can now do things like:

  • sending offers X days in advance of their birthday
  • sending greetings on their birthday
  • sending reminders and follow-up with your customer some number of days after their birthday

Thirdly, usability improved by rephrasing and redefining rules and settings, as well as security changes.


15. Experience Editor

Ladies and gentlemen, let me introduce the feature mostly wanted and requested by marketers which we finally got! The feature which on its own may save hundreds of hours for you content or marketing teams. And here is it:

All content pasted from Word into Rich Text Field is cleaned. Therefore no more Ms Word styling coming into a field, just pure content.

This might look as a minor improvement, but a lot of authors have been waiting for easier workflow going from their Ms Word documents into Sitecore rich text fields. Previously, authors often broke the styling on a site as their copy-pasted content included extra unwanted markup from their documents that conflicted with the styles already on a site. With the new changes, clean source HTML is created that preserves the content and the styling especially for those customers who aren't using Content Hub.


16. Content Hub

Once again I want to clarify very common confusion. Content Hub isn't just a replacement for Media Library. Quite opposite - Sitecore Content Hub is now the central hub for all media content in their DAM, but also content models defined in CMP. And Sitecore is just only one of the channels of content publishing with the web channel is only one channel consuming this content. Current release expands how much content can be managed in CMP and then integrated to the Sitecore Experience Platform. With support for additional field types, this allows marketing teams to centralize more of their content creation into Content Hub and still leverage this with their web teams. With the addition of a taxonomy import, marketing teams can also make sure that the taxonomy associations they make in Content Hub will persist into the content items in Sitecore.


17. Other changes

Just go through them in a form of a bullet list:

  • Sitecore PowerShell Extensions version 6.1.1 is accompanying this platform release.
  • With Sitecore 9.2 we've got Active Personalization report. Now with version 10 numerous performance optimizations have been put in the active personalization list, including infinite scrolling, caching, lazy loading, and back-end optimizations
  • You can now abort currently running jobs - another widely demanded feature.
  • Session State has been optimized for performance, expecting a 10-15% faster response time for CD servers.
  • The Shared Session State database is now disabled by default, but can be enabled if required.
  • Application map support has been one of the highest requested features for operations teams running their infrastructure on Azure, now it's possible to view and diagnose the xConnect stack directly in the application map, with monitoring and diagnosing allowing to start tracing requests.
  • Massive performance improvements to xDB change tracking achieved by changing the way changes are tracked in xDB. This allows for a faster processing when there is a high volume of changes being made, while simultaneously being less heavy on server resources.
  • A new page that automatically collects basic information about your Identity Server instance has been added.
  • You can configure service pages (Error, Not Found, and so on) for each website on content tree structure.
  • There is also already available Sitecore Commerce 10.0 that works on top of Sitecore XP 10.0.


18. Further reading

This is not the full list of changes and features coming with this so much promising Sitecore 10 release. You may use the links below for your own investigation:

Why CNAB could be a game-changer for Docker containers and how Sitecore can benefit from that?

Introduction

Currently, in order to run Sitecore in docker locally, one has to pull the code from a GitHub repository, build it (if not done yet, or pull already built images from a Docker registry), set the license file, and say "up" to docker-compose. Not to say the prerequisites required. If dealing with a cloud, then the deployment into Kubernetes is required, which also demands adequate skills.

Imagine an ideal situation where you don't need to do all the things, but just pull a "managing" image from a remote registry, and this image itself will care about running "all the things" internally, such as prerequisites, pulling the dependent images, preparing the environments, network and dependencies, doing alternative steps to docker-compose and much more.


Or going far beyond that: shipping is as a traditional GUI installer for non-technical people or deploying that same image into a cloud, any cloud, ready to use, does not that look as such desirable? What if I tell you this technology is already available and you can use it? Please welcome the new universal spec for packaging distributed apps created by Microsoft and Docker:

Cloud-Native Application Bundles or simply CNAB

Firstly, what the heck is CNAB at all? Using Cloud Native Application Bundle, a developer has an option of deploying the app either locally at his dev. machine, or in a public cloud as it bolts together containers and services into a seamless whole, placing a strong focus on standardization. It is an open-source specification that aims to facilitate the bundling, installing, and managing of containerized apps. With this bundle, users can define resources that can then be deployed to a number of run-time environments, such as Docker, Azure, Kubernetes, Helm, automation services (such as those used by GitOps), and more.

At a first glance, that task is supposed to be solved by Docker itself. However, when dealing with largely scaled hybrid infrastructures, its standard features become insufficient. Thus, CNAB is an attempt of standardizing the process of packaging, deployment, and lifecycle management of distributed apps on a basis of Kubernetes, Helm, Swarm, and others by using a unified JSON-based package format. 

Recently the CNAB spec reached version 1.0, which means it is ready for production deployment. The spec itself is now broken down into several chapters:

  • CNAB explains the fundamentals of the CNAB core 1.0.
  • CNAB Registry will describe how CNAB bundles can be stored inside of OCI Registries (this section is not yet complete).
  • CNAB Security explains the mechanisms for signing, verifying, and attesting CNAB packages.
  • CNAB Claims describes the CNAB Claims system, shows how records of CNAB installations formatted for storage
  • CNAB Dependencies describes how bundles can define dependencies on other bundles.

Tooling

Each of the organizations has provided its own tools that demonstrate CNAP capabilities: Microsoft released Duffle, while Docker shipped Docker app. Docker Desktop application is fully compatible with CNAB from May 2019.

CNAB is not the only solution for managing the cloud applications lifecycle. For example, Kubernetes has Сrossplane manager as well as package manager Helm. However, CNAB is the first ever solution that supports several most popular tools and is platform-agnostic. By the way, CNAB can also work with Helm and I came across a sample of it at GitHub.

Duffle is a simple command line tool that interacts with Cloud-Native Application Bundles - helping you package and unpackage distributed apps for deployment on whatever cloud platforms and services you use. Its goal is to exercise all parts of the specification and this tool also comes with very handy VS Code extensions, one of which named Duffle Coat allows you to create native executable installer (*.exe) of your bundle:


This results in the proper installer that will install and configure you Sitecore 9.3 locally from bundle image stored at docker registry:



Once again, instead of local Sitecore installations (like SIA does), we are having this CNAB installer that installs the platform from a Docker registry and with no prerequisites required at all! And CNAB bundle cares about all the dependencies and parameters. What magic!

Another tool, Porter, is Microsoft’s CNAB builder that gives you building blocks to create a cloud installer for your application, handling all the necessary infrastructure and configuration setup. It is a declarative authoring experience that lets you focus on what you know best: your application. The power of Porter is coming from using mixins giving CNAB authors smart components that understand how to adapt existing systems, such as Helm, Terraform, or Azure, into CNAB actions.


And of course, Docker App is a CNAB builder and installer that leverages the Docker Compose format to define your applications. To facilitate developer-to-operator handover, you can add metadata and run-time parameters. These applications can easily be shared using existing container registries. Docker App is available as part of the latest Docker release.

Bundle manifest file
As I said above, the specification defines the way of packaging distributed application of various formats. CNAB includes package definition (named bundle.json) used for describing an app, as well as a special image (also called invocation image) for its installation. A bundle.json is similar to a docker-compose.yml file in that it describes a complex configuration for image deployment. The difference is, the CNAB bundle is very clearly defined as to how it should be laid out, encoded, and where all associated files must reside. It contains:

  • The schema version.
  • Top-level package information.
  • Information on invocation images.
  • Map of images.
  • Specification for parameter override (with a reference to a validation schema).
  • List of credentials.
  • Optional description of custom actions.
  • A list of outputs produced by the application.
  • A set of schema definitions is used to validate input.
Here is a sample of bundle.json file below:
{ 
   "credentials":{ 
      "hostkey":{ 
         "env":"HOST_KEY",
         "path":"/etc/hostkey.txt"
      }
   },
   "custom":{ 
      "com.example.backup-preferences":{ 
         "frequency":"daily"
      },
      "com.example.duffle-bag":{ 
         "icon":"https://example.com/icon.png",
         "iconType":"PNG"
      }
   },
   "definitions":{ 
      "http_port":{ 
         "default":80,
         "maximum":10240,
         "minimum":10,
         "type":"integer"
      },
      "port":{ 
         "maximum":65535,
         "minimum":1024,
         "type":"integer"
      },
      "string":{ 
         "type":"string"
      },
      "x509Certificate":{ 
         "contentEncoding":"base64",
         "contentMediaType":"application/x-x509-user-cert",
         "type":"string",
         "writeOnly":true
      }
   },
   "description":"An example 'thin' helloworld Cloud-Native Application Bundle",
   "images":{ 
      "my-microservice":{ 
         "contentDigest":"sha256:aaaaaaaaaaaa...",
         "description":"my microservice",
         "image":"example/microservice:1.2.3"
      }
   },
   "invocationImages":[ 
      { 
         "contentDigest":"sha256:aaaaaaa...",
         "image":"example/helloworld:0.1.0",
         "imageType":"docker"
      }
   ],
   "maintainers":[ 
      { 
         "email":"matt.butcher@microsoft.com",
         "name":"Matt Butcher",
         "url":"https://example.com"
      }
   ],
   "name":"helloworld",
   "outputs":{ 
      "clientCert":{ 
         "definition":"x509Certificate",
         "path":"/cnab/app/outputs/clientCert"
      },
      "hostName":{ 
         "applyTo":[ 
            "install"
         ],
         "definition":"string",
         "description":"the hostname produced installing the bundle",
         "path":"/cnab/app/outputs/hostname"
      },
      "port":{ 
         "definition":"port",
         "path":"/cnab/app/outputs/port"
      }
   },
   "parameters":{ 
      "backend_port":{ 
         "definition":"http_port",
         "description":"The port that the back-end will listen on",
         "destination":{ 
            "env":"BACKEND_PORT"
         }
      }
   },
   "schemaVersion":"v1.0.0",
   "version":"0.1.2"
}
You may read more about bundle.json format at CNAB.io official page.

How about Azure?
For Azure, we also have got the solution, Azure CNAB Quickstarts library. It demonstrates how one can use bundles for deploying applications and solutions and how to create their own bundles. The library is designed to be optimized for bundles that use Azure resources but is not limited to Azure only. There is CI/CD workflow in the repository using custom GitHub Actions to enable the automatic building of bundles and publishing of bundles to a public Azure Container Registry. The library supports bundles made using Porter tool I mentioned above, a tool capable of building, publishing, invoking, and updating bundles.

Final thoughts
Likely CNAB becomes a game-changer for 2020, as we get more and more into containerized deployments and orchestrating them in the clouds. The specification is quite new and not too many companies are using it at the moment, but there is an ever-growing interest in it. Since all the major vendors are now ready, I am quite sure it will boost the whole industry in the coming months!

References
Hope you find this post helpful and get your own hand on CNAB shortly!

Tip of the day: running Sitecore Docker from within Hyper-V virtual machine is in fact possible

Why not to run Docker containers from within a Hyper-V virtual machine? 

Well, when you are trying to install Docker Desktop for Windows, it installs correctly. But running it will prompt you about Hyper-V is required for Docker to work.


Programs and Features on its turn shows you that Hyper-V is greyed out, so you cannot install it from there



Not everyone knows, but Hyper-V in fact allows having nested virtual machines. That means running docker is also possible from within a VM. All you need is just running one PowerShell command from outside of your container:

Set-VMProcessor -VMName _YOUR_VM_NAME_TO_ENABLE_ -ExposeVirtualizationExtensions $true

What it does - just adds virtualization features into a virtual processor of outer VM, similar to those you got at you physical CPU. Thus you need to run this command of the VM switched off.

Once done, Sitecore perfectly builds and work if Docker running within a virtual machine. For those who are total beginner and scary to mess the things around their host machines - this could be an option. However there is of course a performance penalty from double virtualization.



Finally, the main trick! Just because you've added virtualization extensions into a virtual processor of your VM, you may run docker in process isolation mode, relatively to the build number of VM's operation system. Will explain it on the following example: imagine, your host machine runs Windows 10 build 1809, but got no problems of running Windows 10 build 1909 in a Hyper-V machine there. Now, you after you enable virtualization extensions to that virtual machine (that runs 1909), you will be able to install docker within that virtual machine, and run 1909-built images in process isolation, natively to that VM: 1909-built images run on 1909- machine! But at the same time you will be able to run those same images only in hyper-v mode on the (outer) host machine, because it has 1809 and cannot run non-matching images in process isolation mode.

Hope this helps!

Starting with Docker and Sitecore

There was much of recent buzz around containers as technology and Docker in particular, especially now where more and more community efforts focused at Docker in conjunction with Sitecore. Plenty of articles do explain how it works on the very top level, what the benefits are but very rare do precise guidance. As for an ultimate beginner - I know how it is important to get a quick start, the minimal positive experience as a further point of development. This blog exactly about how to achieve the bare minimal Sitecore running in Docker.

Content


1. Terminology

  • Docker images - a blueprint for creating containers, is what you pull from remote registry
  • Docker container - an implementation of a specific image, you run and work with containers, not images
  • Docker repository - a logical unit containing one or many built images (specified by tags)
  • Docker registry - works like a remote git repo, it's a hosted solution to you push your built images into

There are plenty more of terminology, but these are the essentials for a demo below

2. Installing Docker

If you have it up and running, you may skip to the next part.

In order to operate the repository for given walk-through, you need to have Windows 10 x64 with at least build 1809.

The simplest way is to install it from Chocolatey gallery:

cinst docker-desktop

Missing something from your host OS installation? Docker will manage that itself!

Once done, you'll need to switch a mode. Docker for Windows can work in either Windows or Linux mode at the same time - which means you cannot mix types of containers.

One of the biggest issues at the moment is the size of Windows base images - the minimal Nano Server with almost everything cut off is already 0.5Gb and Server Core (this one either does not have UI - just a console) goes to 4Gb. That's too much comparing to minimal Linux images starting from as little as 5 megs. That's why it may seem very attractive to run Solr from Linux image (as both Solr and Java it requires are cross-platform and there are ready-to-use images there), same for Ms SQL Server which also has been ported on Linux and its images are also available.

Until the very-very recent the short answer was no - one could manage only single type of container at a time (while already running Linux container will keep up running unmanageable, there are also few workarounds how to make them work in parallel, but that's out of scope for now), but as from April 2019 it is doable (from Linux mode on Windows), I managed to combine NGINX on Linux with IIS on Windows.

Switching mode from UI is done by right-clicking Docker icon context menu from a system tray:


3. Docker registry

What you should do next is to provide a docker registry. Docker Hub is probably the first option for any docker beginner.

Docker Hub however allows only one private repository for free. You need to ensure sure all your repositories are private. Images you're building will contain your license file, having them in a public access will also be treated as sharing Sitecore binaries on your own, which you're not allowed - only Sitecore can distribute that publicly.

Alternatively, you may consider Canister project, they give up to 20 private repositories for free.

Pluralsight has a course on how to implement your own self-hosted docker registry.

But what is even more surprising, Docker itself provides a docker image with Docker Registry for storing and distributing Docker images.


4. Preparing images

Now let's clone Sitecore images repository from GitHub - https://github.com/sitecoreops/sitecore-images

If you don't have git installed - use Chocolatey tool already familiar from previous steps: cinst git (once complete - you'll need to reopen console window so that PATH variable gets updated).

To keep things minimal, I go to sitecore-images\images folder and delete everything unwanted - as for this demo, I keep only 9.1.1 images, removing the rest. So, I left only 9.1.1 images and sitecore-openjdk required for Solr:

Images folder contains instructions on how to build you new Sitecore images. As input data they require Sitecore installers and license file, so put them into this folder:

And the last but not the least, create build.ps1 PowerShell script.

Important: do not use an email as the username, it's not your email (in my case username is martinmiles), and from what I've heard many people find this confusing and were wondering why getting some errors.

This is my build script, replace usernames and password and you're good to go:

"YOUR_DOCKER_REGISTRY_PASSWORD" | docker login --username martinmiles --password-stdin

# Load module
Import-Module (Join-Path $PSScriptRoot "\modules\SitecoreImageBuilder") -Force

# Build and push
SitecoreImageBuilder\Invoke-Build `
    -Path (Join-Path $PSScriptRoot "\images") `
    -InstallSourcePath "c:\Docker\Install\9.1.1" `
    -Registry "martinmiles" `
    -Tags "*" `
    -PushMode "WhenChanged"


5. Building images

Run the build script. If receiving security errors, you may also be required to change execution policy prior to running build:

set-executionpolicy unrestricted
Finally you get your base images downloading and build process working:

As I said, the script pulls all the base images and builds, as scripted. Please be patient as it may take a while. Once built, your images will be pushed to the registry. Here's what I finally got - 15 images built and pushed to Docker Hub. Again, please pay attention to Private badge next to each repository:

Docker Hub has a corresponding setting for defining privacy defaults:


6. Running containers

Images are built and pushed to registry, so we are OK to run them now. Navigate to tests\9.1.1 rev. 002459\ltsc2019 folder where you see two docker-compose files - one for XM topology and another for XP. If simplified, docker compose is a configuration for running multiple containers together defining common virtual infrastructure, it is written in YAML format.

Since we are going the simplest route - we keep with XM topology, but that same principle works well for anything else.

Rename docker-compose.xm.yml to docker-compose.yml and open it in the editor. What you see is a declarative YAML syntax of how containers will start and interact with each other.

version: '2.4'

services:

  sql:
    image: sitecore-xm1-sqldev:9.1.1-windowsservercore-ltsc2019
    volumes:
      - .\data\sql:C:\Data
    mem_limit: 2GB
    isolation: hyperv
    ports:
      - "44010:1433"

  solr:
    image: sitecore-xm1-solr:9.1.1-nanoserver-1809
    volumes:
      - .\data\solr:C:\Data
    mem_limit: 1GB
    isolation: hyperv
    ports:
      - "44011:8983"

  cd:
    image: sitecore-xm1-cd:9.1.1-windowsservercore-ltsc2019
    volumes:
      - .\data\cd:C:\inetpub\sc\App_Data\logs
    isolation: hyperv
    ports:
      - "44002:80"
    links:
      - sql
      - solr

  cm:
    image: sitecore-xm1-cm:9.1.1-windowsservercore-ltsc2019
    volumes:
      - .\data\cm:C:\inetpub\sc\App_Data\logs
    isolation: hyperv
    ports:
      - "44001:80"
    links:
      - sql
      - solr

If your docker-compose has isolation set to process, please change it to hyperv (this is mandatory hosts on Windows 10, while on Windows Server docker can also run its process natively). In that case, processes will run in a hypervisor and is not a naked process next to native windows processes, that prevents you from memory allocations errors such as PAGE_FAULT_IN_NONPAGED_AREA and TERMINAL_SERVER_DRIVER_MADE_INCORRECT_MEMORY_REFERENCE

Notice data folder? This is how volumes work in docker. All these folders within data are created on your host OS file system - upon creation, a folder from container is mapped to a folder on a host system, and once container terminates, data still remains persistent on a host drive.

For example, one running SQL Server in docker can place and reference SQL database files (*.mdf and *.ldf files) on a volume externally in such a manner so that databases actually exist on a host OS and will not be re-created on each container run.

My data folder already has data folders mapped to data folders from all the various roles' containers run on previous executions (yours will be blank at that moment before the first run):

Just for a curiosity, below is an example of what you can find within cm folder, looks familiar, right?


Anyway, we are ready to run docker-compose:

docker-compose up

You'll then see 4 containers being created, then Solr container starts making its job, providing plenty of output:

In a minute you'll be able to use these containers. In order to log into Sitecore, one needs to know an IP address of a container running a particular role - so need to refer to a cm container. Every container has its hash value which serves as an identifier, so with docker ps command you can list all docker containers currently running, get hash of cm and execute ipconfig command within a context of that cm container (so that ipconfig runs inside of it internally):

Now I can call 172.22.32.254/sitecore in order to login to CMS:

What else can you do?

With docker you may also execute commands in interactive mode (sample) with -it switch, so you may do all the things such as deploying your code there (it is always good to deploy on top of clean Sitecore instance). That's how to enter an interactive session with remote command prompt:

docker exec -it CONTAINER_HASH cmd

You may go with more folder mappings using volume. Running XP topology offers even more interesting but safe playground for experiments.

Building other versions of Sitecore allows regression testing your code against legacy systems - always quick manner and always on clean! Going ahead you may use it for the development having only Visual Studio running on a host machine, with no IIS and no SQL server installed, publishing from VS directly in docker. Plenty other scenarios possible - it's only for you to choose from.


7. Stopping and clean-up

Stopping containers occurs in a similar way:

docker-compose down
After finished, you won't see any of the containers running by executing
docker ps
You'll be still able to see existing images on a system and their size occupied:
docker image ls

So if finally, after playing it over and want to clean-up you drive a noticed there's way less free disk space now. I want to beware you from doing one more common mistake - don't delete containers data manually! If you navigate to c:\ProgramData\Docker\windowsfilter folder, one can see plenty of them:

These are not container folders, they are symlinks (references) to windows system resources folders - deleting data from these symlinks actually deleted you resources from your host OS bringing you to a sorry state. Instead, use the command:

docker prune -a

This gets rid of all the images and containers from your host system correctly and safely.


8. Afterthoughts

Docker is a very strong and flexible tool, it is great for devops purposes. I personally find questionable using it for production purposes. That may be fine with Linux containers, but as for Windows... I'd rather opt out, for now, however I am aware of people doing that.

Proper use docker will definitely improve your processes, especially combining it with other means of virtualisation. Containers may take you a while to get properly into, but after getting your hands on you'll have your cookbook of docker recipes for plenty of day-to-day tasks.

As for Sitecore world, I do understand it is all only starting yet, but docker with Sitecore becomes more inevitable, as Sitecore drills deeper into microservices. Replacing Solr and SQL Server with Linux-powered images is only a matter of time, and what I am anticipating so much is XP and XC finally running together in Docker, facaded by IDS (ideally also on Linux) just in fractions after calling docker-compose. Fingers crossed for that.

Hope this material serves you as a great starting point for containers and Docker!