Experience Sitecore ! | More than 200 articles about the best DXP by Martin Miles

Experience Sitecore !

More than 200 articles about the best DXP by Martin Miles

Sharing Sitecore Identity Server between two independant instances of Sitecore

Imagine a case where you need having two Sitecore instances in parallel next to each other. That may be cause by several legit reasons.

WHY?

For example, in my case I am moving (by reworking, not just migrating) some functional areas from one legacy instance to another that will features SXA. The legacy instance has been passed from one hand to another with numerous configuration artifacts, with a limited maintenance options, so that it becomes next to impossible to combine it with a brand new SXA stuff under the same roof. I know, that is doable in principle (and have done it myself before), but the amount maintenance and lack of knowledge / documentation on existing codebase makes its maintenance inappropriately risky and non-acceptable. Therefore, it becomes reasonable keeping them both in isolation, only uniting at the URL level (by rewriting a (sub)domain of a new instance into a primary domain's folder level).

Things you have to consider in that case would turn to almost doubling your infrastructure and related expense as well as checking if your Sitecore licence permits you that. Currently doing quite an unusual setup where both the above concerns give me a green light for going ahead and I am OK to run both instances in parallel (as on-prem solution).

Once agreed, the next thoughts come to Identity Server, where keeping two instances for that same activity does not make much sense. Keeping them both is exhaustive, but the good news is that one can re-use and existing ID Server for any number of instances (namely CM boxes). That comes to making two extra steps and below I will show you how-to:

HOW?

Let's assume we have two instances, called old and new. Old one has all the bits configured and running, so we only want re-using ID Server of old instance with a new instance.

1. Get rid ow ID Server for a new instance (you can stop its web app and app pool for now). That makes sure it is not used.

2. Find Sitecore.Owin.Authentication.IdentityServer.config file on a new instance (App_Config\Sitecore\Owin.Authentication.IdentityServer.config) and substitute identityServerAuthority variable to point to an existing ID Server

    <sc.variable name="identityServerAuthority" value="https://old.identityserver" />


3. Now CM for a new instance knows which ID Server to talk through, but will ID Server accept those calls? The answer is no, unless you explicitly permit it doing so. Navigate to Config\production folder of old instance ID Server and add addition allowed CORS origin group into Sitecore.IdentityServer.Host.xml file. You will end up having smth. as below:

<AllowedCorsOrigins>
<AllowedCorsOriginsGroup1>https://old</AllowedCorsOriginsGroup1>
<AllowedCorsOriginsGroup2>https://new</AllowedCorsOriginsGroup2>
</AllowedCorsOrigins>


4. There is also Identity Server secret stored below at the same xml file, with a matching counterpart at App_Config\ConnectionStrings.config, so you also need updating config for new instance with the value from shared Identity Server:

<add name="sitecoreidentity.secret" connectionString="SECRET_from_ID_Server" />


5. Finally, recycle Identity Server application pool, then you're OK to test it. To make things more visual, I've recorded all the steps and testing it and sharing resulted video below:


Things to consider: as you're re-using existing old instance Identity Server, it will itself re-use all the assets. When it comes to Active Directory then it brings a desired result, but speaking about internal users (those normally you have got at Sitecore domain) - they will all get reused all as well, including admin. This comes because ID Server has a reference to core database (or security database extracted from the core) and that one belongs by default to an old instance too.

Hope you find this helpful!

What is a Reverse Proxy and what do you need one for?

There are a variety of Reverse Proxy solutions on the market. You may have already heard about some:

Major cloud providers also have their proprietary solutions:


But what is a Reverse Proxy? Why "Reverse"?
As Wikipedia says, a common type of proxy server that is accessible from the public network. Large websites and content delivery networks use reverse proxies - together with other techniques - to balance the load between internal servers. Reverse proxies can keep a cache of static content, which further reduces the load on these internal servers and the internal network. It is also common for reverse proxies to add features such as compression or TLS encryption to the communication channel between the client and the reverse proxy.

Reverse proxies are typically owned or managed by the web service, and they are accessed by clients from the public internet. In contrast, a forward proxy is typically managed by a client (or their company) who is normally restricted to a private, internal network. The client can, however, access the forward proxy, which then retrieves resources from the public internet on behalf of the client. Here's a reverse proxy in action from a very high:


What are typical scenarios for using a Reverse Proxy?

1. SSL Offload. Let's assume we've got a website which works at HTTP only, and for some reason (legacy, gone developers or being unable bringing changes into a running solution that may huge or any other) it is not possible to change the website itself - "If it works, don't touch it" paradigm in action. For compliance, we must add HTTPS support for that website.
With using a Reverse Proxy it comes to a really quick and easiy solution - we don't need developers at all. All we need is asking our Ops professional asking him to instantiate a proxy server with SSL Termination. (obviously, we'll also need SSL certificates for domain hostname(s) of a given website). Job done!


2. Load Balancer
. Next, we'd want horizontally scaling that website and even deployed two equal client facing copies of it. How do we "split" traffic to distributing it equally to both sites? In this case we introduce a Proxy Server functioning as a Load Balancer.
But what if one of websited dies or crashes half way down the road? Load Balancer needs somehow to know each of the "boxes" functions well and react the outages by re-distributing traffic to the rest of mchines functioning well. This is traditionally implemented by "pinging" a such called "HealthCheck" URL on each particular box. As soon as one of the healthchecks keeps failing, an alert is raised and the traffic is no longer routed to a faulty box (be careful with sticky sessions!).



3. Cybersecurity enforcement
. Sending specially formed packets hackers can undertake a Deny-of-Service attack, when sending a request comes times cheaper than serving it back. At some moment your servers won't cope with this parasitic workload and will fail.
In order to prevent that, dangerous traffic should not reach your servers, being filtered at a proxy. Namely, a Firewall with an adequate rule set that filters out all patters with anomalies, raises alerts and bypasses the legitimate requests.


4. Caching and compressing
. Even with purely legitimate traffic beyond the proxy, one may still get a large request payload. But how comes? Well, there may be different reasons, such as usage patterns where all users navigate to the same heavy-loaded area of a website; alternatively the website itself could be written by junior developers who did not enough care about the way if functions in the most optimal way once deployed. Regardless of the reason, we could still soften things up by identifying some of the popular endpoint that consume much of server's resources and cache it up right at the proxy level. Of course assuming that a given set of parameters always returns that same result, there is no longer a need spending expensive server resources on producing the results we're already got in past and have effectively cached at the proxy level and never reach servers at all. If we however must ensure this traffic reaching the end servers, we could at least compress / encode the "last mile" beyond the proxy.


5. Smooth automated deployments
? Why not, have you ever heard of Blue-Green Deployments? With that in action end users won't even realize that you're upgrading the solution while they're browsing your site.


6. A/B Testing
. As a result of previous point, it may be a case you've updated some but not the all end servers. You do not want updating them all, instead you'd like to perform an A/B Testing on both sets and based on a result decide to complete an update or rollback to the most recent version. This would be a pretty valid scenario that a reverse proxy can do for you.


7. URL and Links Rewriting
. What if you have a legacy website that functions perfectly well, but similar to a scenario (1) the is no way (and need) of maintaining it. The development team has gone and in any case there is no single reason of investing a lot into a smth to be dismissed at some stage. At the same time you got another website(s) that either could be successor(s) for a legacy one, or some additional areas, written in isolation by more modern tools and thus either incompatible or expensive to merge with an existing solution. However the business wants everything to function with the same main domain name, just in different "folders" under it, so that end users (and search robots!) see no difference between consisting parts and naturally experience them both as being a single solid website.
Achieving that is also possible with a Reverse Proxy by rewriting URLs. Please note that not just a external request coming to site.com/company1 will be rewritten to www.company1.com but also all the internal URLs within all the requested pages need to be rewritten as well. Please note that it becomes possible only in conjunction of SSL offload, otherwise the traffic gets encrypted and proxy becomes "a man in the middle".

Not just that - once 6 years ago I wrote a walkthrough how one can achieve that same result purely and entirely by the means of IIS on Windows


Conclusion.
This article give a high-level explanation on Reverse Proxies and their primary features. It intentionally does not focus on a specific implementation avoiding going deeper in technical details.

In real-world solutions of course you will meet Reverse proxy solutions implementing several of the features combined. This may be a typical workflow for processing inbound traffic with a Reverse Proxy:



In any case, I hope you found this helpful!

Merry Christmas and happy New Year!

Every year I create a special Christmas postcard to congratulate my readers with a new oncoming year, full of changes and opportunities. Wish you all the best in 2021!

My artwork for the past years (click to expand)
2020


2019


2018


2017


2016


The easiest way of installing Solr cores for SXA search

Few years ago, I made a walkthrough on setting up search for SXA-based website. However, it does not cover one important thing - out-of-the-box SXA uses two own indexes. As a good practice "one index - one core" it assumes you create two additional cores for each of those indexes.

It is also assuming that you may follow the same principle used for existing Sitecore Solr cores naming them <INSTANCEPREFIX>_sxa_master_index and <INSTANCEPREFIX>_sxa_web_index. But how do you create those cores?

I would like to introduce the easiest way achieving that using Sifon and its existing plugin for creating SXA Solr cores. All you need to do is updating plugins from public repository by executing "Plugins" - "Get Sifon plugins" command.

Once updated, you will see corresponding plugin at the menu:


And that's it! After a bit of waiting Sifon will complete installation of both new cores into a Solr instance references with the selected profile. You'll see the confirmation:


As it prompts, the cores have been created at the named location, managed schema has been also published but you still need to rebuild these new indexes. Sifon also has a plugin for that (fastest, but SPE Remoting must be enabled for that instance):


Alternatively, you can use Control Panel as you normally do:


Confirmation:


Hope this helps enjoying search with SXA which is really taken to the new level!

Cannot use a Hyper-V disk VHDX image you've been shared with? That's how to manage it properly

Cannot use a Hyper-V disk VHDX image you've been shared with? It can be your colleague's shared VM or a disk exported from Azure VM. Below I am going to describe the steps required to managed it properly.

The first thing one is likely to perform is either importing this disk or creating a new VM without disk and attaching it later. Despite it seems absolutely valid, that will end up with:

If you see the above error it means you've most likely created a VM of 2-nd Generation of Hyper-V, and attached you drive to it. That won't work, but why?

To answer this question let's define the difference between both generations. The main fact about Generation 1 is that it emulates hardware - all required hardware components must be emulated to make the virtual machine work! Special software that can imitate the behaviour of real hardware is included in Hyper-V, as a result the VM can operate with virtual devices. Emulated hardware (that behaves identical to real hardware) includes drivers to most operating systems in order to provide high compatibility.

Generation 1 the emulation works way less productive compared to native equipment and has numerous limitations. Among those:

  • a legacy network adapter
  • IDE controllers with only 2 devices could be attached to each
  • MBR with max disk size of 2TB and not more than 4 partitions.

The 2-nd Generation uses:

  • UEFI BIOS
  • GPT support (without size and partitions legacy limit) and Secure Boot
  • because of the above - VMs boot (and operate) much faster
  • fewer legacy devices and new faster synthetic hardware used instead
  • better CPU and RAM consumption

UEFI is not just a replacement of BIOS, UEFI extends support of devices and features, including GPT (GUID Partition Table).

Secure Boot is a feature that allows protection against modifying boot loaders and main system files, done by comparing the digital signatures that must be trusted by the OEM.


The solution is quite simple.

What you need to do is create a 1-st Generation VM (without creating a new disk drive) and reference existing VHDX file. That would work and OS will load (slowly but will do). The laziest person would probably stop at this stage, but we progress ahead with converting it to the 2-nd Generation VM. But, how do we?

All you need to do is convert legacy BIOS to UEFI running the following command executed within a 1-st Generation VM:

mbr2gpt.exe /convert /allowFullOS

Please pay attention to the warning: since now you need to boot in UEFI mode.

Now you can turn off the Windows VM, delete existing 1-st Generation VS from Hyper-V Manager (but not the disk drive!), and then create a new 2-nd Generation VM referencing the converted VHDX disk image. Also make sure the attached hard Drive stands first for at the boot sequence order (Firmware tab) and also you may want unchecking Enable Secure Boot switch from the Security tab.

Now you may successfully start your received Hard Drive with a fast and reliable 2-nd Generation VM that manages resources of your host machine in a much reliable and savvy way!

All you need to know about Sitecore 10

As you might have already heard, Sitecore has just released a new major version of its product Sitecore 10, the one initially supposed to be numbered as Sitecore 9.4. Let's take a look at what does it have to offer to better understand why Sitecore decided to "release it louder" by giving a major version number.



During the course of this post I will be writing about:

  1. Overview
  2. Installation
  3. Containers
  4. ASP.NET Core
  5. CLI with Serialization
  6. Horizon
  7. Search
  8. XM / CMS-Only
  9. JSS and SXA
  10. Data Exchange and Connectors
  11. Privacy options
  12. Analytics
  13. EXM
  14. Marketing Automation
  15. Experience Editor
  16. Content Hub
  17. Other changes
  18. Further


1. Overview

Finally, Sitecore 10 has been released. With the release of Sitecore 10, a new era of Sitecore development has started, the mostly anticipated approaches becoming an official part of the platform. This version focuses on product updates and enhancements that provide more development and deployment options, increase usability and improve overall performance – all centered around enabling both Marketing and IT teams equally, thus making it easier and faster to launch and evolve digital customer experiences.


2. Installation

From now on, there are 3 option you may choose from in order to get Sitecore 10 up and running. 

Historically, there was an MSI installer to install Sitecore as well as a ZIP archive to handle things manually. With version 8 comes a third-party Sitecore Instance Manager to simplify the installation along with some housekeeping tasks. It is no longer with us.

1. From Sitecore version 9.0 there was one and only official way to install Sitecore - SIF. A bit complicated for newbies and those who aren't familiar with PowerShell and JSON configurations. 

2. Later on with Sitecore 9.2, we got one more option for that - SIA with its GUI. With every next release SIA became more mature and offered more option. As for now, SIA allows to install Solr on your behalf together with few more things, such as extra validations:

  • validates Solr connectivity
  • validates SQL Server connectivity
  • validates the Sitecore license file
  • displays a progress spinner during the installation process
  • supports the installation of Sitecore Experience Commerce

3. Since version 10.0, Sitecore now officially has support for Docker, Kubernetes and image repositories. Sitecore community has been around containers for a while and we made much commitment to make things happen - you may see my older posts on that. This for sure will help delivery teams move to now famous continuous delivery model, making infrastructure-as-code deployments for Sitecore smoother.

Of course, SIF is still available as the default option for local installation, for example on VMs or when your team is not yet ready for containers.

Register-PSRepository -Name SitecoreGallery -SourceLocation https://sitecore.myget.org/F/sc-powershell/api/v2
Install-Module SitecoreInstallFramework -Force

The actual SIF version for Sitecore 10.0 is 2.3.0, to ensure you've got it right type in:

Get-Module SitecoreInstallFramework –ListAvailable

Check if you have got 2.3.0 along with 2.3.0 folder as well in WindowsPowershell folder - then you're ready to install Sitecore 10 using SIF.

You may be interested in reading:


3. Containers

Firstly announced as supported in Sitecore 9.3, Sitecore released version 10.0 out-of-the-box support for containers - now it is an official option to develop or instantiate Sitecore and other services with containers. To help Sitecore developers learn, use, and get started quickly with Docker, there is comprehensive documentation available on developing with Sitecore on containers.

Sitecore has docker container images for Sitecore XP and XM along with non-prod images for MSSQL, Solr and Redis for developers, publicly available from Sitecore image repository. In this initial version, SXA is supported. Now you don't need to build images yourself. There are also liveness and readiness health check endpoints which are especially useful with containers.

1. Sitecore Experience Platform 10.0.0 uses Docker Compose as the container orchestrator on developer workstations. Docker Compose is a simple container deployment tool that is bundled with Docker for Windows. Sitecore container images can be deployed with other tools but we recommend that you use Docker Compose to deploy the containers that form the Sitecore Experience Platform.

2. Sitecore Experience Platform 10.0.0 uses Kubernetes (K8s) as the default orchestrator for deploying production environments. The Sitecore XP Kubernetes specification files that are used to map the minimum required configuration parameters between the Sitecore software containers are provided as a reference.

Sitecore customers are expected to extend these specifications to support their own requirements. It is the responsibility of each Sitecore customer to ensure that their production deployments meet the standards for stability and security set by their organization.

Why are containers important? In general, if you’ve been doing your development with virtual machines or local installations, moving to a Docker-driven development and infrastructure flow will be highly beneficial for you. In general, you can address four major areas:

  • No more need for a long onboarding process for new team members
  • Eliminates pain of having several Sitecore instances especially of various versions in parallel
  • No more "works on my machine" errors
  • Improved operational capabilities with new monitoring and security support
  • Better computer resources management due to process isolation opposing running full OS VM
  • Much more reliable continuous delivery model
  • No need to "install" Sitecore - images are already pre-built for you
  • Infrastructure-as-code approach (IaC) has proven its efficiency and finally landed the world of Sitecore

Adopting Docker and Kubernetes into your workflow will let your team focus on actual development rather than wasting time on repetitive ops, providing standard tooling they need to work on those changes.


4. ASP.NET Core

I don't think anyone missed out all the noise around ASP.NET Core which is an open-source, cross-platform framework for building fast, cloud-based web apps on Windows, macOS, or Linux. Some of Sitecore features prior to version 10 are already based on ASP.NET Core - Publishing Service, Identity Service, Sitecore Commerce websites, not to mention Sitecore Host to support many features like that. However, the core rendering of pages and components still remains dependent  on the legacy of .NET Framework - which terminates at version 4.8 and won't go ahead. With .NET 5 release by the end of 2020, Sitecore has a long term strategy with Sitecore Host and the rest of development moving to .Net Core.

.NET Framework is still there in Sitecore 10 for legacy rendering, but since now we've got a Headless development option allows teams to build applications faster on the latest .NET Core technology. It is really cool especially for those pure back-end developers who just don't want to learn JavaScript and those customers who don't need fully javascripted SPA-like application.

Headless development with ASP.NET Core brings developers same JSS-like advantages, development and runtime isolation for a site, delivering a better development experience. There is a new SDK for headless ASP.NET Core rendering in Sitecore 10. 
Using the same headless services that power JSS, you deliver experiences that work in Experience Editor, track and provide marketing capabilities, but work faster. You can use placeholders, field renderings, model bindings in ASP.NET Core as normal:

  • faster and lighter
  • run headless via the ASP.NET Rendering SDK
  • works in Experience Editor
  • supports routing
  • compiling your rendering code won't cause an app pool recycle in the platform
  • only connect to Sitecore when need to: making changes to the presentation layer, pushing them, debugging in a seconds
  • remote rendering host - developers can build straight from VS into the rendering host and test changes without recycling all of Sitecore every time. 


And a cherry on a cake - there's a Getting Started Template that demonstrates all the features for the release all together. Not to say that works well with your development version running in Docker containers. The code is available on GitHub. To install latest template type:
dotnet new -i Sitecore.DevEx.Templates --nuget-source https://sitecore.myget.org/F/sc-packages/api/v3/index.json
Looking forward to see full support for ASP.NET Core 5 in the next releases of Sitecore.


5. CLI with Serialization

As we've been shown at Sitecore MVP Summit last year, Sitecore developed Command Line Interface (CLI) to interact with a Sitecore instance. The expectation is to be able doing pretty much everything from command line and in that sense it will intersect (or may even absorb) PowerShell Extensions. In order to attract developers to CLI - they decided to start with serialization. 

It was quite a surprise that Sitecore got to proper built-in way to serialize items from or deploy items into Sitecore - only using third-party tooling. Now this release of the CLI Sitecore is standardizing the tooling and approaches for serialization. Its initial release allows you serializing, pushing items with further publishing, by combining the best of TDS and Unicorn.

To support the security one need to login in via interactive or automated authentication flows, integrated with Identity Server.

With next releases, CLI will add more functionality, but what is even more thrilling, a plugin system for adding new commands via NuGet is also expected. And since it will cover all the aspects of creating and managing content, one can even create a wrapper over it implementing own version of Content Editor with any set of features (in fact, there's no need to since we've got Horizon, keep reading below). 

For those who prefer GUI over command line, there is also a wrapper available over existing set of serialization CLI-features of Sitecore for Visual Studio, a new member of the Sitecore Developer Collection that does Sitecore Content Serialization from a GUI within Visual Studio - similar to TDS did before.

More reading:

6. Horizon

The first version of the new content editing experience called Horizon has been released with Sitecore 9.3 and there has been a lot of buzz around new editing interface since then. That first version was quite limited with supported functionalities.

Now with the second release of Sitecore, it supports more functionalities:

  • Multi-site and multi-lingual are now supported and added to the user interface
  • Drag & drop support in the page tree to organize your site’s structure
  • Content Hub integration
  • Keyboard shortcuts 
  • SXA sites are also supported with all the above
Horizon Content is a content manager that you use to manage content without a presentation layer. In Content you can see meta data for each field and you can edit content that is not accessible in Pages

Speaking of SXA sites, the integration to SXA has been improved to support editing directly on the page for all supported field types and also do site-specific images and datasources. A tighter integration between SXA and Horizon editing will be coming over the course of the Sitecore 10 series.

A new dedicated field editor experience allows you to edit page metadata that is not directly editable on the page. This allows you to edit other types of fields without the WYSIWYG interface. This is also useful for editing ‘headless’ content that is not intended for a page. Using the new Content view allows you to see all the content items, not just the pages, and edit your data in the Horizon field editor.

Extending Horizon with plugins is not yet available, however is expected with oncoming releases.

With assets in Sitecore Content Hub DAM, you can now leverage that rich media in the Horizon editor instead of only using the native Sitecore Experience Platform media library. This allows your marketing team to have a better flow from Content Hub to your web channel.


Sooner or later this should have happened - Sitecore XP 10.0.0 deprecates Azure Search and it will be completely removed ​in a future release​. If you are starting a new Sitecore project, please use Solr as your search engine.

What I especially love with Sitecore over the years, they do admit and fix previously done mistakes. Azure Search was not fully compatible since day one, I personally struggles so much with it. Finally it will leave Sitecore.

Sitecore XP no longer from passes malformed content to Solr​ ​-it now filters and validates content prior sending it to Solr​.

Last but not the least, Sitecore 10 now requires Solr version 8.4.0 to work with.


8. XM / CMS-Only

Few version ago, Sitecore came up with a lightweight and very fast CMS-only version of its product, opposing full XP with full-powered analytics and much more things around it. However for some reason there was not rule engine personalization coming with XM topology which was not right: for those not needing the full xConnect, still want get use use of some basic personalization or implementing custom conditions, especially valuable for SXA websites. Finally that got fixed and one can now run simple personalization on lightweight XM installations. This functionality contains session-based and device rules that do not depend on tracker or marketing definitions available in the XM package.

Please refer to the list of personalization conditions available for XM to find out more.


9. JSS 14.0 and SXA version 10.0

There are not much of new JSS features with this release, the changes are mostly relate to the stability improvements and support for new platform features. Thus, new JSS can now successfully handle forwarded deployment requests when it is installed behind a reverse proxy as it is in Docker. Exactly the same applies to SXA.

SXA keeps having its versions aligned to the platform release, in order to avoid the confusion.

For more details on what's new, please read the following:


10. Data Exchange and Connectors

Changes in Data Exchange Framework (DEF):

  • Additional field types added to CMP Connector
  • You can now sync a single contact
  • Respond to live events on the site

Sitecore Connect for Salesforce Marketing Cloud allows teams to work in Salesforce Marketing Cloud and leverage the rich visitor analytics data stored in xDB. Now, with the latest release, xDB information can be more easily sent to SalesForce. You can make use of this connection in real-time, no need to wait for a nightly batch. A new Sitecore Marketing automation activity, specifically for Salesforce Marketing Cloud, allows marketers to configure a realtime event to integrate with SFMC.

By connecting directly to the Salesforce Marketing Cloud, marketing teams can respond to events in Sitecore and immediately send information about the current visitor over to Salesforce Marketing Cloud and place visitors into Journey Builder marketing automation plans. This means that you can respond across all your channels to events, in real-time, in the marketing automation tool that you use.

Please read this link for more details on SalesForce Connector.


11. Privacy options

Previous releases of the platform have progressively been improving the privacy tools available to teams looking to meet with regulatory needs and customer privacy demands. In Sitecore 10, new support has been added to help you on your journey. Sitecore 10.0 and later provides API calls and configuration options that makes it easier to enforce explicit consent for tracking a contact's activity on your websites.“Update consent settings” now allows you to remove contacts from marketing messages or all messages. In general, there are two new "rights":

1. Right to be forgotten. With this release, Sitecore Forms makes it easier for you to support erasing this personal identifiable data that is saved via the out-of-the-box Save data submit action. This includes redacting submission values, removing any associations to xDB Contacts, and deleting any files uploaded by the visitor. At the same time this approach still allows marketing teams to see reporting data, despite being anonymized.

2. Right to object. A second change in this release regards making it easier to enforce and manage consent choices. Visitor tracking now requires that giving explicit consent.and marketing teams need to respect and store these choices and make decisions on them. The new consent management tools available in the platform allow for implementation teams to configure sites to require explicit consent before the Sitecore tracking will begin. The tracker is now able to be disabled based on the consent settings stored for the user, and xDB supports the ability to store these consent settings and persist it across visits for the contact.

This allows marketing teams to add additional consent requirements for specific sites that need to meet stricter privacy guidelines.


12. Analytics

Getting insights from Sitecore’s analytics reports is great, but previously one could not filter on specific segments. Now, there is a new filtering functionality available in all reports! 

This means that you can use Segments created from List Manager based on all sorts of segmentation rules as filters on your analytics. In the past when the “Use as analytics filter” was enabled in a Segment, you needed to apply some extra configuration to have this filter available in your reports. Now, this filter is available in the analytics reports without any extra configuration:

  • Target segments of contacts via predefined rules and then report based on that segment
  • Report on the performance of key audience groups
  • Push an email to a segmented list and then see the impact across the site

This is great, because with the extensive set of available segmentation rules, it’s very easy to zoom into the analytics for these specific segments focusing on a group of contacts.


13. EXM

Before Sitecore 9.2, there were no standard templates available OOB at all. Now, with Sitecore 10 we've got even more new additional templates added to the product: 

  • Left Image Block
  • Right Image Block
  • Image Focus

This saves much time for the customers using EXM from developing such email templates from scratch, as these templates can become a great starting point at the cost of a single click. But what is really great - these templates are completely customizable to using Scriban, similar to editing rendering variants in SXA.


14. Marketing Automation

Firstly, Marketing Automation has been reworked to allow for more efficient loading of xConnect data, there are performance improvements in XConnect for customers who do not use Marketing Automation. It does not load contacts and interactions unless there is an inactive or active plan.

Secondly, there are new rules around birthdays added to engage with contacts around their birth date. You can now do things like:

  • sending offers X days in advance of their birthday
  • sending greetings on their birthday
  • sending reminders and follow-up with your customer some number of days after their birthday

Thirdly, usability improved by rephrasing and redefining rules and settings, as well as security changes.


15. Experience Editor

Ladies and gentlemen, let me introduce the feature mostly wanted and requested by marketers which we finally got! The feature which on its own may save hundreds of hours for you content or marketing teams. And here is it:

All content pasted from Word into Rich Text Field is cleaned. Therefore no more Ms Word styling coming into a field, just pure content.

This might look as a minor improvement, but a lot of authors have been waiting for easier workflow going from their Ms Word documents into Sitecore rich text fields. Previously, authors often broke the styling on a site as their copy-pasted content included extra unwanted markup from their documents that conflicted with the styles already on a site. With the new changes, clean source HTML is created that preserves the content and the styling especially for those customers who aren't using Content Hub.


16. Content Hub

Once again I want to clarify very common confusion. Content Hub isn't just a replacement for Media Library. Quite opposite - Sitecore Content Hub is now the central hub for all media content in their DAM, but also content models defined in CMP. And Sitecore is just only one of the channels of content publishing with the web channel is only one channel consuming this content. Current release expands how much content can be managed in CMP and then integrated to the Sitecore Experience Platform. With support for additional field types, this allows marketing teams to centralize more of their content creation into Content Hub and still leverage this with their web teams. With the addition of a taxonomy import, marketing teams can also make sure that the taxonomy associations they make in Content Hub will persist into the content items in Sitecore.


17. Other changes

Just go through them in a form of a bullet list:

  • Sitecore PowerShell Extensions version 6.1.1 is accompanying this platform release.
  • With Sitecore 9.2 we've got Active Personalization report. Now with version 10 numerous performance optimizations have been put in the active personalization list, including infinite scrolling, caching, lazy loading, and back-end optimizations
  • You can now abort currently running jobs - another widely demanded feature.
  • Session State has been optimized for performance, expecting a 10-15% faster response time for CD servers.
  • The Shared Session State database is now disabled by default, but can be enabled if required.
  • Application map support has been one of the highest requested features for operations teams running their infrastructure on Azure, now it's possible to view and diagnose the xConnect stack directly in the application map, with monitoring and diagnosing allowing to start tracing requests.
  • Massive performance improvements to xDB change tracking achieved by changing the way changes are tracked in xDB. This allows for a faster processing when there is a high volume of changes being made, while simultaneously being less heavy on server resources.
  • A new page that automatically collects basic information about your Identity Server instance has been added.
  • You can configure service pages (Error, Not Found, and so on) for each website on content tree structure.
  • There is also already available Sitecore Commerce 10.0 that works on top of Sitecore XP 10.0.


18. Further reading

This is not the full list of changes and features coming with this so much promising Sitecore 10 release. You may use the links below for your own investigation:

Sitecore RSS Feed revised by using Content Search API instead of unproductive Sitecore Query

I am working on a large multilingual website with several dozens of thousands pages are available in numerous languages. It also features news being released on a daily basis by tens of news editors under plenty of nested categories. Thus it was matter of time for me to be asked to implement an adequate RSS Feed solution to expose all the possible data.

Luckily, Sitecore has built-in RSS Feed feature, but unfortunately it has very limited implementation by data driven using Sitecore Queries, obviously this is quite a legacy feature was added into Sitecore way before version 7.0 with its revolutionary Content Search API. 

What is wrong with this old-fashioned Sitecore Query:

  • it is limited in its applicable functionality
  • has complicated synthax
  • isn't easy to debug and troubleshoot, especially on complex conditions
  • is slooooooow (even fast query is slow!)
  • talks to database and generates cache for touched items
That was absolutely obvious that I should use Content Search API. I googled around and came across some implementation done by Douglas Couto for Sitecore 7 and Lucene. I reworked it to be compatible for versions 9.*+ and added few new features.

It is available at my GitHub repository (and readMe file is quite explanatory there)
Once you get the items from either TDS serialization or Sitecore package, a new Content search section is added to RSS Feed template to drive the data out of Content Search API rather than Sitecore Query:


All the benefits of Content Search API are available to you from now on. Using this new section one may fine tune the data to be exposed by not just certain page templates, but also filter to be located under specific node, has certain tags and specify recency criteria for that filtered data.

Hope this helps!

Walkthrough: Using Publishing Targets in order to create preview environment with Sitecore 9.3

I made this 22-minutes long video in order to demonstrate how one can create a preview environment by creating a new publishing target on an example of vanilla Sitecore 9.3.

Adding new publishing target is very helpful when you:

  1. would allow content authors to demonstrate the content being not in a final state
  2. want to demonstrate that content to an audience without access to your Sitecore instance

This demo shows how to create a publishing target database, assign it to a designated hostname and then also to create an index for a given preview database. It also demonstrates the difference between publishing content in non-final-state into both live and preview databases.

Hope it helps!


Why CNAB could be a game-changer for Docker containers and how Sitecore can benefit from that?

Introduction

Currently, in order to run Sitecore in docker locally, one has to pull the code from a GitHub repository, build it (if not done yet, or pull already built images from a Docker registry), set the license file, and say "up" to docker-compose. Not to say the prerequisites required. If dealing with a cloud, then the deployment into Kubernetes is required, which also demands adequate skills.

Imagine an ideal situation where you don't need to do all the things, but just pull a "managing" image from a remote registry, and this image itself will care about running "all the things" internally, such as prerequisites, pulling the dependent images, preparing the environments, network and dependencies, doing alternative steps to docker-compose and much more.


Or going far beyond that: shipping is as a traditional GUI installer for non-technical people or deploying that same image into a cloud, any cloud, ready to use, does not that look as such desirable? What if I tell you this technology is already available and you can use it? Please welcome the new universal spec for packaging distributed apps created by Microsoft and Docker:

Cloud-Native Application Bundles or simply CNAB

Firstly, what the heck is CNAB at all? Using Cloud Native Application Bundle, a developer has an option of deploying the app either locally at his dev. machine, or in a public cloud as it bolts together containers and services into a seamless whole, placing a strong focus on standardization. It is an open-source specification that aims to facilitate the bundling, installing, and managing of containerized apps. With this bundle, users can define resources that can then be deployed to a number of run-time environments, such as Docker, Azure, Kubernetes, Helm, automation services (such as those used by GitOps), and more.

At a first glance, that task is supposed to be solved by Docker itself. However, when dealing with largely scaled hybrid infrastructures, its standard features become insufficient. Thus, CNAB is an attempt of standardizing the process of packaging, deployment, and lifecycle management of distributed apps on a basis of Kubernetes, Helm, Swarm, and others by using a unified JSON-based package format. 

Recently the CNAB spec reached version 1.0, which means it is ready for production deployment. The spec itself is now broken down into several chapters:

  • CNAB explains the fundamentals of the CNAB core 1.0.
  • CNAB Registry will describe how CNAB bundles can be stored inside of OCI Registries (this section is not yet complete).
  • CNAB Security explains the mechanisms for signing, verifying, and attesting CNAB packages.
  • CNAB Claims describes the CNAB Claims system, shows how records of CNAB installations formatted for storage
  • CNAB Dependencies describes how bundles can define dependencies on other bundles.

Tooling

Each of the organizations has provided its own tools that demonstrate CNAP capabilities: Microsoft released Duffle, while Docker shipped Docker app. Docker Desktop application is fully compatible with CNAB from May 2019.

CNAB is not the only solution for managing the cloud applications lifecycle. For example, Kubernetes has Сrossplane manager as well as package manager Helm. However, CNAB is the first ever solution that supports several most popular tools and is platform-agnostic. By the way, CNAB can also work with Helm and I came across a sample of it at GitHub.

Duffle is a simple command line tool that interacts with Cloud-Native Application Bundles - helping you package and unpackage distributed apps for deployment on whatever cloud platforms and services you use. Its goal is to exercise all parts of the specification and this tool also comes with very handy VS Code extensions, one of which named Duffle Coat allows you to create native executable installer (*.exe) of your bundle:


This results in the proper installer that will install and configure you Sitecore 9.3 locally from bundle image stored at docker registry:



Once again, instead of local Sitecore installations (like SIA does), we are having this CNAB installer that installs the platform from a Docker registry and with no prerequisites required at all! And CNAB bundle cares about all the dependencies and parameters. What magic!

Another tool, Porter, is Microsoft’s CNAB builder that gives you building blocks to create a cloud installer for your application, handling all the necessary infrastructure and configuration setup. It is a declarative authoring experience that lets you focus on what you know best: your application. The power of Porter is coming from using mixins giving CNAB authors smart components that understand how to adapt existing systems, such as Helm, Terraform, or Azure, into CNAB actions.


And of course, Docker App is a CNAB builder and installer that leverages the Docker Compose format to define your applications. To facilitate developer-to-operator handover, you can add metadata and run-time parameters. These applications can easily be shared using existing container registries. Docker App is available as part of the latest Docker release.

Bundle manifest file
As I said above, the specification defines the way of packaging distributed application of various formats. CNAB includes package definition (named bundle.json) used for describing an app, as well as a special image (also called invocation image) for its installation. A bundle.json is similar to a docker-compose.yml file in that it describes a complex configuration for image deployment. The difference is, the CNAB bundle is very clearly defined as to how it should be laid out, encoded, and where all associated files must reside. It contains:

  • The schema version.
  • Top-level package information.
  • Information on invocation images.
  • Map of images.
  • Specification for parameter override (with a reference to a validation schema).
  • List of credentials.
  • Optional description of custom actions.
  • A list of outputs produced by the application.
  • A set of schema definitions is used to validate input.
Here is a sample of bundle.json file below:
{ 
   "credentials":{ 
      "hostkey":{ 
         "env":"HOST_KEY",
         "path":"/etc/hostkey.txt"
      }
   },
   "custom":{ 
      "com.example.backup-preferences":{ 
         "frequency":"daily"
      },
      "com.example.duffle-bag":{ 
         "icon":"https://example.com/icon.png",
         "iconType":"PNG"
      }
   },
   "definitions":{ 
      "http_port":{ 
         "default":80,
         "maximum":10240,
         "minimum":10,
         "type":"integer"
      },
      "port":{ 
         "maximum":65535,
         "minimum":1024,
         "type":"integer"
      },
      "string":{ 
         "type":"string"
      },
      "x509Certificate":{ 
         "contentEncoding":"base64",
         "contentMediaType":"application/x-x509-user-cert",
         "type":"string",
         "writeOnly":true
      }
   },
   "description":"An example 'thin' helloworld Cloud-Native Application Bundle",
   "images":{ 
      "my-microservice":{ 
         "contentDigest":"sha256:aaaaaaaaaaaa...",
         "description":"my microservice",
         "image":"example/microservice:1.2.3"
      }
   },
   "invocationImages":[ 
      { 
         "contentDigest":"sha256:aaaaaaa...",
         "image":"example/helloworld:0.1.0",
         "imageType":"docker"
      }
   ],
   "maintainers":[ 
      { 
         "email":"matt.butcher@microsoft.com",
         "name":"Matt Butcher",
         "url":"https://example.com"
      }
   ],
   "name":"helloworld",
   "outputs":{ 
      "clientCert":{ 
         "definition":"x509Certificate",
         "path":"/cnab/app/outputs/clientCert"
      },
      "hostName":{ 
         "applyTo":[ 
            "install"
         ],
         "definition":"string",
         "description":"the hostname produced installing the bundle",
         "path":"/cnab/app/outputs/hostname"
      },
      "port":{ 
         "definition":"port",
         "path":"/cnab/app/outputs/port"
      }
   },
   "parameters":{ 
      "backend_port":{ 
         "definition":"http_port",
         "description":"The port that the back-end will listen on",
         "destination":{ 
            "env":"BACKEND_PORT"
         }
      }
   },
   "schemaVersion":"v1.0.0",
   "version":"0.1.2"
}
You may read more about bundle.json format at CNAB.io official page.

How about Azure?
For Azure, we also have got the solution, Azure CNAB Quickstarts library. It demonstrates how one can use bundles for deploying applications and solutions and how to create their own bundles. The library is designed to be optimized for bundles that use Azure resources but is not limited to Azure only. There is CI/CD workflow in the repository using custom GitHub Actions to enable the automatic building of bundles and publishing of bundles to a public Azure Container Registry. The library supports bundles made using Porter tool I mentioned above, a tool capable of building, publishing, invoking, and updating bundles.

Final thoughts
Likely CNAB becomes a game-changer for 2020, as we get more and more into containerized deployments and orchestrating them in the clouds. The specification is quite new and not too many companies are using it at the moment, but there is an ever-growing interest in it. Since all the major vendors are now ready, I am quite sure it will boost the whole industry in the coming months!

References
Hope you find this post helpful and get your own hand on CNAB shortly!

Sitecore Docker breaks when running Build.ps1 with "hcsshim::PrepareLayer failed in Win32: Incorrect function. (0x1)" message


Recently I was trying to build docker images from the main Sitecore repository, and got quite a weird message.

hcsshim::PrepareLayer - failed failed in win32 : Incorrect function. (0x1)

This message is not self-explanatory and it fired up when the actual build for the first image took place, in my case that was SQL server 2017. Some other details: quite fress and new instance of Windows 10 x64 build 1909, using Hyper-V, no docker / images previously been utilized at thtis computer.

After few hours long googling, I came to understanding that was caused by c:\windows\system32\cbfsconnectnetrdr2017.dll referenced by autorun. There is one halpful tool that helps out troubleshoouting such issues - Sysinternals Autoruns. Here is the output generated by Autoruns for my system:


What is that DLL and how did it get to my machine? CBFSConnect is a known library for creating and managing virtual filesystems for cloud storage providers such as such as Box, Dropbox etc., and this DLL is a kernel mode system drive. Sometimes it may be a leftover from a previously installed software, that was not uninstalled fully and properly.

Depending no your system you may see this DLL as either:

  • C:\Windows\system32\cbfsconnectnetrdr2017.dll  - for newer systems, or
  • C:\Windows\System32\drivers\cbfs6.sys
You may check if it is running or no tby running  a command-line utility for common minifilter driver management operations command (a command-line utility for common minifilter driver management operations):


Resolution
The easiest I found was to just simply rename this driver. Once restarted, docker build worked well.

Hope this helps someone