Experience Sitecore ! | All posts tagged 'Serialization'

Experience Sitecore !

More than 200 articles about the best DXP by Martin Miles

XM Cloud: Beyond the Serialization

Distinguish Between Definition and Content Items

In XM Cloud, serialization is primarily a developer/ops concern, not a task for content authors. Developers and DevOps engineers use the Sitecore CLI to manage definition items (templates, layouts, SXA settings, etc.) in version control, whereas content authors continue to work in the Content Editor/Experience Editor and publish content via the usual publishing pipeline.  In contrast, content authors simply publish content; they shouldn’t be expected to run CLI commands: we reserve the CLI workflow for dev-defined items, and let content authors handle content via Sitecore’s authoring tools.

In XM Cloud, all templates and configurations are the definition items in the Sitecore tree - many of these are delivered as items-as-resources in protobuff resource files. By default, XM Cloud provides a base set of definition items in resource packages; custom definitions (such as your templates, renderings, layouts, SXA component definitions) should be added via serialization. The CLI can push/pull any item in the tree, so treat your custom definitions like any other serialized item. For example, you would include custom layouts (under /sitecore/Layout/Layouts/YourTenant), rendering definitions (/sitecore/Layout/Renderings/YourTenant), SXA rendering variants, themes, and placeholder settings in your modules. In fact, the Sitecore docs note that SCS modules allow you to "organize and separate serialized items according to their purpose".

You generally do not manually transfer definitions by hand; instead, you define them in your project and let the CLI handle serialization. As a special tool, Sitecore provides an Items as Resources CLI plugin (dotnet sitecore itemres) to package definition items into *.DAT resource files, but in most cases, you simply let sitecore ser push pack the YAML and deploy it to XM Cloud.

Note, that some environment-level settings are not items. For example, custom Sitecore security domains live in a config file on the server, not in the content tree, so they cannot be serialized or deployed via CLI. In those cases, you must handle them outside of SCS.

In summary: treat definition items as code: include them in your serialization modules and commit them, and let the Sitecore CLI bundle them into the XM Cloud resource IAR package. Content items, by contrast, should be managed by content workflows.

Content Serialization

Generally, don’t serialize day-to-day content. XM Cloud follows the old wisdom of treating content differently from code: the CLI is not intended as a general content migration tool. Instead, content authors create or edit content items in the CM interface or Pages Builder and then publish them to delivery targets. Serializing content will lead to conflicts and source-control bloat.

That said, there are a few exceptions where you might include minimal content stubs or settings in serialization. For example, it’s common to serialize the homepage item itself so that a brand-new environment has the correct site entry point immediately upon first serialization. You'd want to explicitly include the tenant and site root with CreateAndUpdate operations - this ensures that new environments get a placeholder homepage item (and updates propagate), without overwriting content. You might also serialize certain settings or dictionary items that are truly "configuration" (say, a language items folder or a set of shared data templates). But actual content (articles, product entries, news items) should be kept out of serialization.

In practice, restrict content serialization to very small, static subsets (often at most the single site-root node) and mark them as create-only or create-and-update. For example:

{
  "name" : "TenantA_SiteRoot",
  "path" : "/sitecore/content/TenantA/SiteA,"
  "scope": "singleItem",
  "allowedPushOperations" : "CreateUpdateAndDelete"
}

This would ensure the /SiteA item exists in all environments, but leave its children alone. The rest of /sitecore/content/TenantA/SiteA descendants (actual page content) would not be included.

Remember that after you push any content items, you still must publish them to make them live. The ser push command only writes into the CM database; it does not publish to Experience Edge. To serve content, you must run a publish command (dotnet sitecore publish --target Edge).

Summarizing: Limit content serialization to definition items with some exceptions like an empty homepage or settings, and let normal publishing handle real content.

Also, "do not include vanilla XM Cloud items – include only custom-created items". Content changes by authors should flow through Sitecore’s normal publishing workflows, not through SCS commits.

Serialization Scopes

Sitecore CLI modules use scopes to control how deeply an include or rule applies. The scope property can be one of:

  • SingleItem – only the specified item itself (no children).

  • ItemAndChildren – the item plus its immediate (one-level) children.

  • ItemAndDescendants – the item and all levels of descendants - the full subtree. That is a default value.

  • DescendantsOnly – all descendants of the item, but not the item itself.

  • Ignored – skips this branch entirely, used in rules to exclude subtrees.

For example, suppose you include a path /sitecore/content/TenantA/SiteA in a module. If not specified, that is ItemAndDescendants by default, so it would pull the entire site tree. If you want only the root item and none of its children, you would add a rule with "scope": "SingleItem". Conversely, to skip an unwanted subfolder, you could use "scope": "Ignored" on that path.

A concrete example of a rules section might look like:

"items": {
  "includes": [
    {
      "name": "SiteA",
      "path": "/sitecore/content/TenantA/SiteA",
      "allowedPushOperations": "CreateUpdateAndDelete",
      "rules": [
        { "path": "/sitecore/content/TenantA/SiteA/Data", "scope": "Ignored" },
        { "path": "/sitecore/content/TenantA/SiteA/Home", "scope": "Ignored" },
      ]
    }
  ]

This says "include the SiteA item (with all descendants by default) but ignore the entire Data and home Page subfolders." In practice, you’ll define scopes to precisely include what you need: broad (ItemAndDescendants) for full branches, and narrow (SingleItem or Ignored) to exclude or limit depth.

Serialization Order and Modules

Since you organize your serialized items into modules within each .module.json file in the Sitecore CLI, you must somehow define which modules depend on stuff from others. This each module can have an optional references list to other modules. The CLI uses these references to enforce a load order. For example, if you have a Foundation module with templates (Foundation.CoreTemplates) and a Feature module that depends on them (Feature.Shop), you would write:

{
  "namespace": "Feature.Shop",
  "references": [ "Foundation.CoreTemplates" ],
  "items": {
    "includes": [
      { "name": "ProductTemplates", "path": "/sitecore/templates/Feature/Shop", "allowedPushOperations": "CreateUpdateAndDelete" }
      // ...
    ]
  }
}

By specifying "references": [ "Foundation.CoreTemplates" ], you ensure the CLI pushes/pulls Foundation templates first, then the shop items. Wildcards are allowed ( "Foundation.*") to reference all Foundation modules. If you omit references, the CLI may process modules in alphabetical or file order, which can lead to missing-dependency errors, such as pushing a rendering before its template exists.

In practice, follow Helix conventions: have a base module for core templates/branch templates, then project-level modules that reference it and use the references array in each module to declare dependencies.

Alternatively, you can also tag modules in CLI commands (with -i) to run only subsets, but the fundamental sequencing comes from references. If dependencies are missing or misordered, you will see errors during push (like “item not found” or unresolved references). Always review dotnet sitecore ser info to verify your module graph.

SXA Serialization Considerations

When using SXA in XM Cloud, treat SXA definition items as code. As usual, you should serialize your SXA component definitions and site structure, but not your actual content. Key SXA assets to include are:

  • Rendering Variants and Templates: Anything under /sitecore/layout/Rendering Variants/... and the SXA Page/Partial designs under /sitecore/layout/Layouts/... or /sitecore/content/TenantName/SiteA/Presentation/....

  • Placeholder Settings and Layouts: SXA uses placeholders and page layouts, which live under /sitecore/Layout or under the site; include those as part of layouts and placeholders sections.

  • Themes and Styles: If your SXA site uses a custom theme (under /sitecore/media library/Themes or the SXA Theme item), serialize that.

  • SXA Site Settings: Under each site (e.g. /sitecore/content/Tenant/SiteA/Settings), include any site-level settings items that define colors, etc., if they are required for deployment.

  • Site Templates: SXA allows site templates in the dashboard; if you have custom site templates, serialize those under /sitecore/templates/Project/....

Do not serialize authored page content (articles, products, etc.) from SXA sites. Also consider shared vs. site-specific: if multiple sites under one tenant share variants or partial designs, put them in a shared module (under the tenant node). If a design is only for SiteA, put it under the SiteA module. In general, follow the same include rules as above, putting SXA assets under your project/feature paths. For example, include renderings and placeholder settings under /sitecore/Layout/Renderings/Project/... and /sitecore/Layout/Placeholder Settings/Project/... - these would cover your custom SXA renderings and placeholder definitions as well. By version-controlling SXA assets in the CLI, you ensure that your site designs, variants and styles are consistent across environments, while leaving content (pages and datasources) to the content pipeline.

Module Organization

For the multisite and especially multi-tenant data architecture, it becomes crucial to organize your modules and serialization in a totally isolated way. Thus, everything becomes self-contained so that removing one site/ tenant folder does not affect the rest of the serialized assets. To achieve that, I would recommend doing this:

  • Organize all the assets under a specific folder, for example: authoring/items/client
  • All the relevant modules fall into this folder. I usually split them into three groups: client.global.module.json; client.components.module.json and client.site.module.json
  • Each of these module files must include a path directive, which effectively defines a subfolders structure: "path": "components" or "path": "site". This is the most crucial bit.
  • Nothing else exists immediately under authoring\items folder other than clients and areas; typically there would be a global folder followed by client1, client2, etc.

I would also recommend reading this article, which I found to be helpful. Hope you find these tips helpful!

 

Separating content items from definition using Unicorn's NewItemsEvaluator

The vast majority of my readers are already familiar with Unicorn (if not - please take your time to familiarize with it by this link) and understand the principles of how it works. In few words, it serializes your Sitecore items into text files so that you can store them under source control along with the rest of your solution code.Then you can sync these files back into Sitecore, either manually from its admin page, or automatically by PowerShell or deployment script on CI / CD pipeline.  If the item has was changed, Unicorn automatically modifies its serialization file (so that changed file goes to source control to all other developers) and the reverse - newer items from source control will update those in Sitecore. So far, so good.

Problem: your solution infrastructure has numerous environments from lower (Dev) to higher (Prod) where the actual site is running. But not just the site itself - Sitecore also is there. Your content editors will be using Sitecore on prod, bringing plenty changes. But with next deployment all that new content will be automatically overwritten by older serialized items. What is good - now we have Helix principles, that classifies two types of Sitecore items - definition items to be passed along with the rest of solution code and actual content item. For Content items source of truth is Production, while for definition items source of truth is Development. But how do we actually define and set up content items?

Solution: the best automated approach also comes with Unicorn, however almost not documented and very little people know about it. It is called NewItemsEvaluator. Let's take a look at how it works:

By default, Unicorn predicate runs with Master Evaluator, which always overwrites Sitecore item with whatever comes from serialization. Because of Master Evaluator, content items on production will be overwritten by serialization with each deployment's sync unless actions are taken. NewItemsEvaluator also creates an item in Sitecore, but only if that item does not (yet) exist in the target environment. But if the item with such ID already exists there - it won't be affected and overwritten.

Firstly, we need to define what content items are. That typically will be all the items underneath /sitecore/YourProjectName/Home including Home item itself. Also, that will be all items within /sitecore/media library/Project/YourProjectName and items within /sitecore/content/YourProjectName/Global including Global item itself and its folder-based children that have insert options assigned.

Secondly, we need to create a new unicorn predicate for those content items identified. That predicate should be driven by New Item Evaluator:

<evaluator type="Unicorn.Evaluators.NewItemOnlyEvaluator, Unicorn" singleInstance="true" />
Thirdly, we need to exclude content items from the default website predicate driven by Master Evaluator. Luckily, Unicorn comes with handy exclusion syntax (more examples can be found at this link):
<include name="Some children" database="master" path="/sitecore/content/YourWebsite">
  <exclude children="true">
    <except name="Settings" />
    <except name="Global" />
  </exclude>
</include>

Finally, your website project layer serialization configuration file will have two predicate configurations similar to one below:
<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/" xmlns:role="http://www.sitecore.net/xmlconfig/role/" xmlns:localenv="http://www.sitecore.net/xmlconfig/localenv/">
    <sitecore role:require="Standalone or ContentManagement">
        <unicorn>
            <configurations>
                <configuration 
                    name="Project.YourProjectName.Website" 
                    description="YourProjectName content" 
dependencies="Foundation.*,Feature.*,Project.Common" extends="Helix.Project"> <predicate> <include name="Project.YourProjectName.Layouts" database="master" path="/sitecore/layout/layouts/Project/YourProjectName" />
<include name="Project.YourProjectName.PlaceholderSettings" database="master" path="/sitecore/layout/placeholder settings/Project/YourProjectName" />
<include name="Project.YourProjectName.Content" database="master" path="/sitecore/content/YourProjectName">
<exclude path="/sitecore/content/YourProjectName/Home" />
<exclude childrenOfPath="/sitecore/content/YourProjectName/Global/Button Styles" />
<exclude childrenOfPath="/sitecore/content/YourProjectName/Global/Font Styles" />
<exclude childrenOfPath="/sitecore/content/YourProjectName/Global/Image Container Css" />
<exclude childrenOfPath="/sitecore/content/YourProjectName/Global/Landing Css Classes" />
<exclude childrenOfPath="/sitecore/content/YourProjectName/Global/Links/Footer Menu" />
<exclude childrenOfPath="/sitecore/content/YourProjectName/Global/Links/Header Menu" />
<exclude childrenOfPath="/sitecore/content/YourProjectName/Global/Links/Social Media" />
</include> <include name="Project.YourProjectName.Media" database="master" path="/sitecore/media library/Project/YourProjectName">
<exclude children="true" /> </include> <include name="Project.YourProjectName.ExperienceEditor" database="core" path="/sitecore/client/Applications/ExperienceEditor/Pipelines/InitializePageEdit" />
</predicate> <!-- roles and user data stores skipped here --> </configuration> <configuration name="Project.YourProjectName.Website.AuthoredContent"
description="Sample site content (New Item Provider)" dependencies="Foundation.*,Feature.*,Project.YourProjectNamev.Website"
patch:after="configuration[@name='Project.YourProjectName.Website']"
extends="Helix.Project"> <targetDataStore physicalRootPath="$(sourceFolder)\$(layer)\$(module)\serialization\Content" type="Rainbow.Storage.SerializationFileSystemDataStore, Rainbow" useDataCache="false" singleInstance="true" /> <evaluator type="Unicorn.Evaluators.NewItemOnlyEvaluator, Unicorn" singleInstance="true" /> <dataProviderConfiguration enableTransparentSync="false" type="Unicorn.Data.DataProvider.DefaultUnicornDataProviderConfiguration, Unicorn" /> <predicate type="Unicorn.Predicates.SerializationPresetPredicate, Unicorn" singleInstance="true"> <include name="Project.YourProjectName.Home" database="master" path="/sitecore/content/YourProjectName/Home" />
<include name="Project.YourProjectName.Global.ButtonStyles" database="master" path="/sitecore/content/YourProjectName/Global/Button Styles" />
<include name="Project.YourProjectName.Global.FontStyles" database="master" path="/sitecore/content/YourProjectName/Global/Font Styles" />
<include name="Project.YourProjectName.Global.HeroCss Classes" database="master" path="/sitecore/content/YourProjectName/Global/Hero Css Classes" />
<include name="Project.YourProjectName.Global.HeroMobile Background Image" database="master" path="/sitecore/content/YourProjectName/Global/Hero Mobile Background Image" />
<include name="Project.YourProjectName.Global.ImageContainerCss" database="master" path="/sitecore/content/YourProjectName/Global/Image Container Css" />
<include name="Project.YourProjectName.Global.LandingCssClasses" database="master" path="/sitecore/content/YourProjectName/Global/Landing Css Classes" />
<include name="Project.YourProjectName.Global.Recruiters" database="master" path="/sitecore/content/YourProjectName/Global/Recruiters" />
<include name="Project.YourProjectName.Global.HigllightCssClasses" database="master" path="/sitecore/content/YourProjectName/Global/Higllight Css Classes" />
<include name="Project.YourProjectName.Global.TabsCssClasses" database="master" path="/sitecore/content/YourProjectName/Global/Tabs Css Classes" />
<include name="Project.YourProjectName.Global.Leaders" database="master" path="/sitecore/content/YourProjectName/Global/Leaders" />
<include name="Project.YourProjectName.Global.Links.FooterMenu" database="master" path="/sitecore/content/YourProjectName/Global/Links/Footer Menu" />
<include name="Project.YourProjectName.Global.Links.HeaderMenu" database="master" path="/sitecore/content/YourProjectName/Global/Links/Header Menu" />
<include name="Project.YourProjectName.Global.Links.SocialMedia" database="master" path="/sitecore/content/YourProjectName/Global/Links/Social Media" />
<include name="Project.YourProjectName.Media.Content" database="master" path="/sitecore/media library/Project/YourProjectName" />
</predicate> </configuration> </configurations> </unicorn> </sitecore> </configuration>

Hope this helps!

Creating a simple workflow in Helix

This may be a not as comprehensive guidance, as it should be, however, I am using this blog post mostly for leaving notes in a cheatsheet manner for later. So, there are several steps to make things happen.

1. Let's create a security domain for our website - that should typically be in a site config on a project layer (Website1.Website.config for my example):
<?xml version="1.0"?>
<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/" xmlns:role="http://www.sitecore.net/xmlconfig/role/">
    <sitecore>
        <domainManager defaultProvider="file">
            <patch:attribute name="defaultProvider">config</patch:attribute>
            <domains>
                <domain id="website1" type="Sitecore.Security.Domains.Domain, Sitecore.Kernel">
                    <param desc="name">$(id)</param>
                    <ensureAnonymousUser>false</ensureAnonymousUser>
                </domain>
            </domains>
        </domainManager>
    </sitecore>
</configuration>


2. Create 3 roles: website1\Editor and website1\Approver on the same domain, as well as website1\everyone that is a member of sitecore\Sitecore Client Authoring and is shared between first two. Make website1\Editor and website1\Approver members of website1\everyone and website1\everyone in turn member of sitecore\Sitecore Client Authoring.Also make website1\Approver member of sitecore\Sitecore Client Publishing


3. Create user accounts. I am creating them as part of website1 domain but they may be part of sitecore domain if need them instance-wide.


4. Assign users to their appropriate roles - editors or approvers. Once assigned - the user is able to log in and load content editor, however not able to insert new item for our website or change presentation details. User also is able to open a page in experience editor but again cannot edit this item because do not have write access to it.


Image above shows how LaunchPad looks when users log into Sitecore. However due to not having permissions they will see the following message in a Content Editor:

5. Create your workflow. I won't be original calling new workflow as Website1 Workflow. The easiest for a quick start would be to clone Sample Workflow and adjust states and other refs to point within corresponding items within that newly created workflow.


6. Then assign page relevant templates into Website1 Workflow. (Standard values -> Workflow section. Set default workflow into Website1 workflow). This ensures all the new items of this template will have Workflow field set into Website1 Workflow and State field will have state preselected as per workflow's Initial state field (workflow definition item)



7. Now give permissions to the role:
Open Access Viewer, click Account from the left top corner and select website1\Editor. Then having it selected give permissions to for everything under
/sitecore/content/Website1/Home,/sitecore/content/Website1/Global (but explicitly deny editing and deleting the top node itself), and do not forget media library for that project

Once complete - users will be able to Lock and Edit items and later submit for approval. So far so good.

8. Next step is to ensure that editors will not be able to approve items. It can be done by denying permission on Awaiting Approval state for editor.


That will result in the following permissions set for Editors in Access Viewer:


While Approvers' Acess Viewer shows Awaiting Approval state available:



9. One more thing to mention - if you got your Helix solution created from Habitat - you may come into a situation when certain fields are not editable. That happens due to write permission of fields for Feature-level templates are set to deny. I have written an explanation and the solution in a separate blog post.


10. Add language permissions to a shared role website1\everyone:



11. Last, but not the least - serialization. What you will serialize? Standard values for all templates that now became part of the workflow. Workflow itself (as a part of your Website1 project serialization configuration), Roles, possibly Users (however remember that there is no way to serialize a user with password - the only option is deserializing a user by Unicorn with setting a default password, also apart having the same default password these users will have Created field updated, which in turn will trigger source control changes). Also need to serialize Languages (foundation layer) with updated permissions - .\src\Foundation\Serialization\serialization\Foundation.Serialization.Languages\Languages.yml should be serialized.


12. Testing workflows in its basic falls into three steps routine:

- Firstly, you need to log in as an Editor and create a page and provide the rest of required content. Once done - submit that for approval. An important check is opening workbox to ensure that editors can only see Draft mode but not Awaiting approval


- Secondly, re-login as Approver and open Workbox. Now you should see Awaiting Approval section and will be able to Approve using it.


- Finally, login as an admin, switch to web database and make sure all the content has been published. That includes related and child items.

Unicorn - the simplest way to share Sitecore items in the soure control along with your code

Annoyed of the necessity of creating packages with recent items in order to share that with your colleagues? Tired of items' versioning? Why not to version those items in source control then?


My favorite tool for achieving that is Unicorn. It does exactly what it suppose to - syncs certain Sitecore items (recursively with children) within a directory, as configured, so that you are able to check-in the folder with all serialized items, so that your colleagues can sync that changes into their database; that means new features / fixes deliver items deltas simultaneously with their code counterparts.

Here is the example of configuration, whatever sits under mentioned paths (recursively) will be serialized:

    





The greatest thing I love about Unicorn is simplicity - as simple as the following:

  • It installs as NuGet package.
  • Consists of 2 DLLs and a config patch file.
  • Just one control page at the web root to perform sync / revert
  • Simple configuration file.

From limitations I would only mention recursiveness - specifying an item in config processes it will all child items and that cannot be overridden. Of course, this is not a problem in more advanced solution - TDS, but are we here about simplicity?


Installation from NuGet Package Manager Console:

PM> Install-Package Unicorn

See also: GitHub source code

Update: now version 3 has been released with great, even revolutionary, changes and it became more friendly. So, please read: