Experience Sitecore ! | All posts tagged 'Unicorn'

Experience Sitecore !

More than 200 articles about the best DXP by Martin Miles

Uploading scheduled auto-backups of editors content for your Helix / SXA website into Azure Blob storage

INTRODUCTION

I assume you are following good practices and develop Helix-powered websites (or SXA, that follows Helix by definition). In that case, you do separate actual user-generated content items from definition items, normally shipped along with each release and created by developers (if not, then please refer to an article I've previously written on that), so you end up having a Unicorn configuration that stores all of your author-edited content:

Will are going to split the process into 3 major steps

  1. Re-serialize Unicorn configuration for content items
  2. Archive serialized content
  3. Upload an archive into Azure Blob Storage

IMPLEMENTATION

1. Re-serialize Unicorn configuration for content items. Luckily, Unicorn provides us with MicroCHAP.dll library helping to automate the sync process as a part of your deployment process (note that the DLL and corresponding PowerShell module Unicorn.psm1  should be referenced from your code). The good news is that any verb can be passed to it, not just sync, so one can use 'Reserialize'. That call will look like:

$ErrorActionPreference = 'Stop'
$ScriptPath = Split-Path $MyInvocation.MyCommand.Path

Import-Module $ScriptPath\Unicorn.psm1
Sync-Unicorn -Verb 'Reserialize' -Configurations @('Platform.Website.Content') -ControlPanelUrl 'https://platform.dev.local/unicorn.aspx' -SharedSecret '$ecReT!'

2. Archiving serialized content would be the next step. If you click Show Config button for Platform.Website.Content configuration in Unicorn configuration page, you may find all the relevant information about it, including the physical folder where items are serialized. We need this folder to be archived. The step comes as:

$resource = "Content_$(get-date -f yyyy.MM.dd)"
$archiveFile = "d:\$resource.7z"
$contentFolder = "C:\inetpub\wwwroot\Platform.dev.local\App_Data\serialization\Project\serialization\Content"

7z a -t7z -m0=lzma -mx=9 -mfb=64 -md=32m -ms=on $archiveFile $contentFolder
I am using 7zip as an archiver, as my content is slightly more than 4GBs and traditional zip cannot handle that. As a hidden bonus, I am getting the best compression ratio coming with 7zip. Also, It would be worth checking if a file with such name exists at target and deleting it before archiving, especially if you run the script more often than daily.

3. Uploading to Azure Blob Storage concludes given routine. To make this happen you should have a subscription, and ideally a connection string to your Storage account. Then you may use the following code to achieve the result:
$containerName = "qa-serialization"
$ctx = New-AzureStorageContext -ConnectionString "DefaultEndpointsProtocol=https;AccountName=lab1234;AccountKey=y9CFAKE6PQYYf/vVDSFAKEzxOgl/RFv03PwAgcj8K80mSfQFDojdnKfakeaLMva0S9DbrQTzNjDMdGCp7rseRw==;EndpointSuffix=core.windows.net"

Set-AzureStorageBlobContent -File $archiveFile -Container $containerName -Blob $resource -Force -Context $ctx

Also, it is assumed you already have blob container created, if not you need to create it upfront:
New-AzureStorageContainer -Name $containerName -Context $ctx -Permission blob
Optionally, you may want to delete the temporal archive, once it's uploaded to blob storage.


RUNNING

Here's an entire code that works for me:
# Step 1: re-serialize user-generated content
$ErrorActionPreference = 'Stop'
$ScriptPath = Split-Path $MyInvocation.MyCommand.Path

Import-Module $ScriptPath\Unicorn.psm1
Sync-Unicorn -ControlPanelUrl 'https://platform.dev.local/unicorn.aspx' -Configurations @('Platform.Website.Content') -Verb 'Reserialize' -SharedSecret '$ecReT!'
# Step 2: archive serialized user-generated content with 7zip using best compression $resource = "Content_$(get-date -f yyyy.MM.dd)" $archiveFile = "d:\$resource.7z" $contentFolder = "C:\inetpub\wwwroot\Platform.dev.local\App_Data\serialization\Project\serialization\Content" 7z a -t7z -m0=lzma -mx=9 -mfb=64 -md=32m -ms=on $archiveFile $contentFolder # Step 3: upload generated content into Azure Blob Storage $containerName = "qa-serialization" $ctx = New-AzureStorageContext -ConnectionString "DefaultEndpointsProtocol=https;AccountName=lab1234;AccountKey=y9CFAKE6PQYYf/vVDSFAKEzxOgl/RFv03PwAgcj8K80mSfQFDojdnKfakeaLMva0S9DbrQTzNjDMdGCp7rseRw==;EndpointSuffix=core.windows.net"
Set-AzureStorageBlobContent -File $archiveFile -Container $containerName -Blob $resource -Force -Context $ctx #Step 4: clean-up after yourself Remove-Item $archiveFile -Force
I run it on a daily basis by Windows Task Scheduler in order to get a daily snapshot of editors' activity. The script produces the following output:


As a result of running script, I get an archive appearing in the Azure Blob Storage:



RESTORING

There's no sense in making backups unless you confirm restoring the data out of it works well. For content items, download an archive, restore and substitute content serialization folder with what you've extracted, then sync content configuration. As simple as that!

Note that your content should be aligned with the definition items, or it may not work well!

Hope this post helps!

Separating content items from definition using Unicorn's NewItemsEvaluator

The vast majority of my readers are already familiar with Unicorn (if not - please take your time to familiarize with it by this link) and understand the principles of how it works. In few words, it serializes your Sitecore items into text files so that you can store them under source control along with the rest of your solution code.Then you can sync these files back into Sitecore, either manually from its admin page, or automatically by PowerShell or deployment script on CI / CD pipeline.  If the item has was changed, Unicorn automatically modifies its serialization file (so that changed file goes to source control to all other developers) and the reverse - newer items from source control will update those in Sitecore. So far, so good.

Problem: your solution infrastructure has numerous environments from lower (Dev) to higher (Prod) where the actual site is running. But not just the site itself - Sitecore also is there. Your content editors will be using Sitecore on prod, bringing plenty changes. But with next deployment all that new content will be automatically overwritten by older serialized items. What is good - now we have Helix principles, that classifies two types of Sitecore items - definition items to be passed along with the rest of solution code and actual content item. For Content items source of truth is Production, while for definition items source of truth is Development. But how do we actually define and set up content items?

Solution: the best automated approach also comes with Unicorn, however almost not documented and very little people know about it. It is called NewItemsEvaluator. Let's take a look at how it works:

By default, Unicorn predicate runs with Master Evaluator, which always overwrites Sitecore item with whatever comes from serialization. Because of Master Evaluator, content items on production will be overwritten by serialization with each deployment's sync unless actions are taken. NewItemsEvaluator also creates an item in Sitecore, but only if that item does not (yet) exist in the target environment. But if the item with such ID already exists there - it won't be affected and overwritten.

Firstly, we need to define what content items are. That typically will be all the items underneath /sitecore/YourProjectName/Home including Home item itself. Also, that will be all items within /sitecore/media library/Project/YourProjectName and items within /sitecore/content/YourProjectName/Global including Global item itself and its folder-based children that have insert options assigned.

Secondly, we need to create a new unicorn predicate for those content items identified. That predicate should be driven by New Item Evaluator:

<evaluator type="Unicorn.Evaluators.NewItemOnlyEvaluator, Unicorn" singleInstance="true" />
Thirdly, we need to exclude content items from the default website predicate driven by Master Evaluator. Luckily, Unicorn comes with handy exclusion syntax (more examples can be found at this link):
<include name="Some children" database="master" path="/sitecore/content/YourWebsite">
  <exclude children="true">
    <except name="Settings" />
    <except name="Global" />
  </exclude>
</include>

Finally, your website project layer serialization configuration file will have two predicate configurations similar to one below:
<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/" xmlns:role="http://www.sitecore.net/xmlconfig/role/" xmlns:localenv="http://www.sitecore.net/xmlconfig/localenv/">
    <sitecore role:require="Standalone or ContentManagement">
        <unicorn>
            <configurations>
                <configuration 
                    name="Project.YourProjectName.Website" 
                    description="YourProjectName content" 
dependencies="Foundation.*,Feature.*,Project.Common" extends="Helix.Project"> <predicate> <include name="Project.YourProjectName.Layouts" database="master" path="/sitecore/layout/layouts/Project/YourProjectName" />
<include name="Project.YourProjectName.PlaceholderSettings" database="master" path="/sitecore/layout/placeholder settings/Project/YourProjectName" />
<include name="Project.YourProjectName.Content" database="master" path="/sitecore/content/YourProjectName">
<exclude path="/sitecore/content/YourProjectName/Home" />
<exclude childrenOfPath="/sitecore/content/YourProjectName/Global/Button Styles" />
<exclude childrenOfPath="/sitecore/content/YourProjectName/Global/Font Styles" />
<exclude childrenOfPath="/sitecore/content/YourProjectName/Global/Image Container Css" />
<exclude childrenOfPath="/sitecore/content/YourProjectName/Global/Landing Css Classes" />
<exclude childrenOfPath="/sitecore/content/YourProjectName/Global/Links/Footer Menu" />
<exclude childrenOfPath="/sitecore/content/YourProjectName/Global/Links/Header Menu" />
<exclude childrenOfPath="/sitecore/content/YourProjectName/Global/Links/Social Media" />
</include> <include name="Project.YourProjectName.Media" database="master" path="/sitecore/media library/Project/YourProjectName">
<exclude children="true" /> </include> <include name="Project.YourProjectName.ExperienceEditor" database="core" path="/sitecore/client/Applications/ExperienceEditor/Pipelines/InitializePageEdit" />
</predicate> <!-- roles and user data stores skipped here --> </configuration> <configuration name="Project.YourProjectName.Website.AuthoredContent"
description="Sample site content (New Item Provider)" dependencies="Foundation.*,Feature.*,Project.YourProjectNamev.Website"
patch:after="configuration[@name='Project.YourProjectName.Website']"
extends="Helix.Project"> <targetDataStore physicalRootPath="$(sourceFolder)\$(layer)\$(module)\serialization\Content" type="Rainbow.Storage.SerializationFileSystemDataStore, Rainbow" useDataCache="false" singleInstance="true" /> <evaluator type="Unicorn.Evaluators.NewItemOnlyEvaluator, Unicorn" singleInstance="true" /> <dataProviderConfiguration enableTransparentSync="false" type="Unicorn.Data.DataProvider.DefaultUnicornDataProviderConfiguration, Unicorn" /> <predicate type="Unicorn.Predicates.SerializationPresetPredicate, Unicorn" singleInstance="true"> <include name="Project.YourProjectName.Home" database="master" path="/sitecore/content/YourProjectName/Home" />
<include name="Project.YourProjectName.Global.ButtonStyles" database="master" path="/sitecore/content/YourProjectName/Global/Button Styles" />
<include name="Project.YourProjectName.Global.FontStyles" database="master" path="/sitecore/content/YourProjectName/Global/Font Styles" />
<include name="Project.YourProjectName.Global.HeroCss Classes" database="master" path="/sitecore/content/YourProjectName/Global/Hero Css Classes" />
<include name="Project.YourProjectName.Global.HeroMobile Background Image" database="master" path="/sitecore/content/YourProjectName/Global/Hero Mobile Background Image" />
<include name="Project.YourProjectName.Global.ImageContainerCss" database="master" path="/sitecore/content/YourProjectName/Global/Image Container Css" />
<include name="Project.YourProjectName.Global.LandingCssClasses" database="master" path="/sitecore/content/YourProjectName/Global/Landing Css Classes" />
<include name="Project.YourProjectName.Global.Recruiters" database="master" path="/sitecore/content/YourProjectName/Global/Recruiters" />
<include name="Project.YourProjectName.Global.HigllightCssClasses" database="master" path="/sitecore/content/YourProjectName/Global/Higllight Css Classes" />
<include name="Project.YourProjectName.Global.TabsCssClasses" database="master" path="/sitecore/content/YourProjectName/Global/Tabs Css Classes" />
<include name="Project.YourProjectName.Global.Leaders" database="master" path="/sitecore/content/YourProjectName/Global/Leaders" />
<include name="Project.YourProjectName.Global.Links.FooterMenu" database="master" path="/sitecore/content/YourProjectName/Global/Links/Footer Menu" />
<include name="Project.YourProjectName.Global.Links.HeaderMenu" database="master" path="/sitecore/content/YourProjectName/Global/Links/Header Menu" />
<include name="Project.YourProjectName.Global.Links.SocialMedia" database="master" path="/sitecore/content/YourProjectName/Global/Links/Social Media" />
<include name="Project.YourProjectName.Media.Content" database="master" path="/sitecore/media library/Project/YourProjectName" />
</predicate> </configuration> </configurations> </unicorn> </sitecore> </configuration>

Hope this helps!

Adding Unicorn icon to Sitecore Launchpad

Seriously, I cannot understand why no one hasn't done that earlier before myself!

Recently, yet another time bookmarking a unicorn.aspx for another specific environment, I caught myself on why not to have Unicorn icon as a standard launchpad tile icon. Benefits of having it there against bookmarking:

  • it becomes available not only for myself but for the rest of admin users in particular environment
  • yes, by saying admin, I want to say that you can customize security for that shortcut
  • if serialized, this icon shortcut becomes available on all the environments it is being deploys

So please welcome Unicorn Launchpad icon. You may download the installation package at the bottom of this blog post.


So, as you might know, all the Launchpad icons are taken from core database and you may find them by the following path:/sitecore/client/Applications/Launchpad/PageSettings/Buttons. So you may add a new icon by simply creating an item called Unicorn (of the template /sitecore/client/Applications/Launchpad/PageSettings/Templates/LaunchPad-Button) as a child of any LaunchPad-Group items (for example, Tools, along with Content Editor).


The much better option would be to duplicate already existing item that is available to admins only - in such case you'll also inherit permissions set) - AppCenter for instance. Once done, adjusts all the fields correspondingly. The most important field to set is Link, so make it pointing to /unicorn.aspx (but remember you may also include parameters, such as unicorn.aspx?verb=sync).

The next challenge is to set an icon. Unicorn is not a part of Sitecore, obviously, Sitecore won't have icon packs for it. Let's create our own icon pack for Unicorn.


Each of these drops down values above (Applications, Apps, Business, Controls etc.), in fact, is a zip archive underneath folder<SITE_ROOT>\sitecore\shell\Themes\Standard.


The structure of archive is the following - unicorn.zip\Unicorn\32x32\Unicorn.png - you may have icons for the other resolutions but I am referencing this one:


That's all the job and it took a couple of minutes!

Download ready to use package (19.7kb, keep in mind that Unicorn icon will be shown to admin users only).

Unicorn - the simplest way to share Sitecore items in the soure control along with your code

Annoyed of the necessity of creating packages with recent items in order to share that with your colleagues? Tired of items' versioning? Why not to version those items in source control then?


My favorite tool for achieving that is Unicorn. It does exactly what it suppose to - syncs certain Sitecore items (recursively with children) within a directory, as configured, so that you are able to check-in the folder with all serialized items, so that your colleagues can sync that changes into their database; that means new features / fixes deliver items deltas simultaneously with their code counterparts.

Here is the example of configuration, whatever sits under mentioned paths (recursively) will be serialized:

    





The greatest thing I love about Unicorn is simplicity - as simple as the following:

  • It installs as NuGet package.
  • Consists of 2 DLLs and a config patch file.
  • Just one control page at the web root to perform sync / revert
  • Simple configuration file.

From limitations I would only mention recursiveness - specifying an item in config processes it will all child items and that cannot be overridden. Of course, this is not a problem in more advanced solution - TDS, but are we here about simplicity?


Installation from NuGet Package Manager Console:

PM> Install-Package Unicorn

See also: GitHub source code

Update: now version 3 has been released with great, even revolutionary, changes and it became more friendly. So, please read: