Experience Sitecore ! | More than 200 articles about the best DXP by Martin Miles

Experience Sitecore !

More than 200 articles about the best DXP by Martin Miles

Sifon 1.2.6 released with Sitecore 10.2 and Windows 11 support

Supports Sitecore 10.2. Supports Windows 11
SUPPORTS 10.2 ON Windows 11

It took me a month longer to conduct all the required testings for software and plugins supplied by me (in fact I was also stuck in Africa on the festive break), but finally, the new version of Sifon is available!

You can download it from Sifon.UK website, but the easier option would be using Chocolatey package manager:

cinst sifon

For the moment it is the only software that can install Sitecore on Windows 11. Let's take a look at Release Notes for this version:

  • supports Sitecore 10.2, supports Windows 11. Supports Sitecore 10.2 ON Windows 11!
  • comprehensive testing has been done - many bugs fixed or refactored
  • plugins were updated, consolidated with main functionality and each other
  • it is now possible to mark a plugin with ### Requires Profile: false to make it run even without an active profile
  • few features were temporarily suspended until got improved
  • lots of new plugins including installers for the latest Horizon, SXA, Publishing Service 6.0 etc
  • added support for Solr 8.8.2 and fixed a minor bug in Solr (un)installation script, making it genuinely universal
  • added syntax to present (double)clickable URLs right in the output to help users with supporting info
  • added Get-SitecoreVersion function that returns either object or string with current XP or XM release version
  • added Verify-NetCoreHosting function to ensure minimum required .NET Core version (passed as param) presents on a target system
  • since all the SQL Server activity goes through SqlServer PowerShell module (opposed to SQLPS) - it is now added to Sifon prerequisites
  • and much more minor issues got tested and fixed

I also recorded a video showing how easy one could install Sitecore without having anything installed at the target machine (except SQL server which could of course be accessible over the network):

Hope you find it useful!

Applying vulnerability fix to containerized environments

This advice was originally proposed by Peter Nazarov (Twitter, LinkedIn), who kindly asked me to give it a bigger spread.

The biggest question for the day is if the fix was already applied to all official Sitecore container images so that now we can just pull new Sitecore containers and rebuild to rebuild our own container images to apply the patch?

The KB article offers WDP and ZIP packages fixes but says noting about containers, just like containers are not supported by Sitecore:

Critical vulnerability applicable to all Sitecore versions related to XSS. 

This issue is related to a Cross Site Scripting (XSS) vulnerability which might allow authenticated Sitecore users to execute custom JS code within Sitecore Experience Platform (XP) and Sitecore Managed Cloud.

We encourage Sitecore customers and partners to familiarize themselves with the information below and apply the Solution to all affected Sitecore instances. We also recommend that customers maintain their environments in security-supported versions and apply all available security fixes without delay.


So below are some findings:


So to apply the fix for your Docker images you need to copy the patch files from the following Sitecore Docker assets images:

  • for XM1: scr.sitecore.com/sxp-pre/sitecore-xm1-assets:10.2.1.007064.169-10.0.17763.2366-1809
  • for XP0: scr.sitecore.com/sxp-pre/sitecore-xp0-assets:10.2.1.007064.169-10.0.17763.2366-1809
  • for XP1: scr.sitecore.com/sxp-pre/sitecore-xp1-assets:10.2.1.007064.169-10.0.17763.2366-1809

For example, if using XM1: scr.sitecore.com/sxp-pre/sitecore-xm1-assets:10.2.1.007064.169-10.0.17763.2366-1809 for XM 10.2.0.


Inside this Sitecore Docker assets image you find C:\platform\ directory which contains the directories for the corresponding Docker images that you need to patch:

  • \platform\cd
  • \platform\cm
  • \platform\id (it is empty and can be ignored)
You will need to copy the content of those directories to file system root C:\ of the corresponding container.
For your \docker\build\cm\Dockerfile  you would need a couple of new lines:
...
FROM scr.sitecore.com/sxp-pre/sitecore-xm1-assets:10.2.1.007064.169-10.0.17763.2366-1809 as kb1001489
...

FROM ${BASE_IMAGE}
...
WORKDIR C:\
COPY --from=kb1001489 /platform/cm/ ./


You would need to do similar changes to your \docker\build\cd\Dockerfile with only one difference that you copy the CD patch files instead of CM in the last line:
...
COPY --from=kb1001489 /platform/cd/ ./


Of course, you can introduce the .env variable for scr.sitecore.com/sxp-pre/sitecore-xm1-assets:10.2.1.007064.169-10.0.17763.2366-1809 and pass it to your docker files as an ARG.


Note: this patch changes the version of your Sitecore 10.2.0 instance to 10.2.1: Sitecore.NET 10.2.1 (rev. 007064 PRE) (see the screenshot below). Seeing this happening it feels that, sadly, Sitecore is unlikely to release 10.2.0 Docker that includes this patch - it would cause versioning issues:


The example above is good to learn how you apply a patch with Docker assets image-based patch to your containers.

However, this Cumulative fix for Sitecore XP 10.2 patch changes a lot of DLLs to the new version which are not exposed via NuGet feed, and changes your Sitecore version to a pre-release version (which does not exist). This would give you several challenges. Therefore, I would prefer in this specific case to apply just a standard .zip file-based fix as per Notes section on the page:

For Sitecore XP 10.1 and later, if it is not possible to apply the cumulative fix (pre-release update), the following patch can be applied alternatively: Sitecore.Support.500712.zip.

  • Changes multiple DLL versions (they are not available in NuGet feed)
  • Changes Sitecore version to a pre-release version (next version that is not released yet)


 

Sitecore.Support.500712.zip

  • Deploys new Sitecore.Support.500712.dll
  • Overwrites vulnerable \sitecore\shell\Applications\Content Manager\Execute.aspx page file so that it runs from Sitecore.Support.500712.dll which contains the fix.

Merry Christmas and happy New Year!

Every year I create a special Christmas postcard to congratulate my readers on a new oncoming year, full of changes and opportunities. Wish you all the best in 2022!

My artwork for the past years (click to expand)
2021


2020


2019


2018


2017


2016


Troubleshooting Marketing Automation service start for XP 10.2 installation on Windows 11

The problem

When installing Sitecore XP 10.2 for IIS 10.0.2*** like the one supplied for Windows 11, you will likely hit an error of Marketing Automation service unable to start. The SIF error starts with:

Install-SitecoreConfiguration : Failed to start service 'Sitecore Marketing Automation Engine'

That occurs due to a failure caused by TLS 1.3 so that the service fails to communicate with XConnect.

The solution

Just download and run this script prior to executing XP0-SingleDeveloper.ps1 main installation script. It will disable TLS 1.3 over TCP for the local IIS (beware as it may affect other non-Sitecore sites).

Alternatively, here's the code:

New-Item `
   'HKLM:\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\TLS 1.3\Server' `
   -Force | Out-Null
    
New-ItemProperty `
   -path 'HKLM:\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\TLS 1.3\Server' `
   -name 'Enabled' -value '0' -PropertyType 'DWord' -Force | Out-Null
    
New-ItemProperty `
   -path 'HKLM:\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\TLS 1.3\Server' `
   -name 'DisabledByDefault' -value 1 -PropertyType 'DWord' -Force | Out-Null

Hope that helps!

10+2 features of Sitecore 10.2 you must be aware of

Today Sitecore released its updated platform version 10.2 - you can download it by this link. Full Release Notes are also available.
Previously we've been told that Sitecore XP 10.1 supposed to become the last version supporting both classical and containerized installation. Due to a certain market inertion and slower products adoption, Sitecore has decided to extend that to version 10.2 as well. That means you can still install it will old good SIF 2.3.0, similar you did before. (they'll probably use a following major version of XP - 11.0, to become containers-only)

Below I would like to cover what comes new in Sitecore 10.2 going from the most important changes to the least. So there we go.

1. Sitecore MVC Component Static Generation


Headless Rendering 19.0 will include product enhancements and new documentation to facilitate the inclusion of Sitecore MVC components in the output of the Layout Service and Experience Edge, and conversion of existing Sitecore MVC sites to statically generated Jamstack architectures using Next.js. Next.js against JSS headless services (hosted on Vercel) is what heard a lot from Sitecore in recent times because of Sitecore establishing a partnership with them. But, as of 10.2, you can get there with MVC or SXA too, so, its really up to what you want.

That brings the ability to incrementally convert to headless Sitecore architectures without a full rebuild, providing a bridge to new Composable DXP. Sitecore is going to release an XM cloud in the future, so with this new feature that switch should be easier, whether you want not just .NET Core or JSS, but also bridging legacy MVC and SXA into the world of JavaScript and JamStack.

Why is this feature so important?

I personally have been involved in several Sitecore version upgrades, where the decision was taken to go for 10.* in containers, but without choosing a headless approach. In all cases, the reason was "having some legacy content or modules or other dependencies, that prevent going headless". All these existing customers must be taken aboard of headless progress, to benefit from modern approaches, such as Jamstack or Experience Edge.


2. Horizon updates go massive this time. In my opinion, finally, Horizon 10.2 became The Tool! As you know, unlike Experience Editor, Horizon is a decoupled modern version of the editing interface and this product is of even bigger importance when Sitecore ends up with a cloud-based editor.

So, firstly, welcome new Content explorer - an alternative way of navigating your content rather than (and in addition to) content tree. It features numerous queries and filters for narrow faceting your searches, also presents items without fixed (tree-based) hierarchy with a table view.

Secondly, since 10.2 headless rendering hosts built on our Next.js and ASP.NET Core SDKs can now utilize the Horizon editor and that is one of the biggest blockers preventing wide-spreading of Horizon.

Thirdly, not just headless, but also SXA is now properly supported in Horizon.

With all the above in mind, I can conclude Horizon now comes into our life with a decent editing experience, and that is what we've been waiting for!


3. DDS Container for EXM. DDS stands for emails sending Dedicated Dispatch Server for, now officially provided with container images for both Docker compose and Kubernetes. These images are available as sitecore-dds-${TOPOLOGY}-cm:${VERSION}.


4. Windows Server 2022 support is claimed for Sitecore 10.2. I assume that means we'll benefit from running Sitecore containers in Server 2022 with process isolation, resulting in the best performance and benefiting from the new Virtualization-based security of Server 2022 (you'll be able confirming that from the comprehensive Images and Tags List later).

Update: server 2022 support is delayed for the initial release, but is definitely coming later.


5. Modules Resources. As you might know, Sitecore 10.1 brought a highly demanded feature called Items as Resource that keeps default items outside of Sitecore databases in order to simplify version upgrades in the containerized environments. Now, this feature got extended to modules as well, meaning they're no more update packages for modules. That also includes module localizations.

Speaking about upgrades and containers, you may now find and follow a wonderful Upgrade Container Deployment Guide.


6. SXA brought really many changes and improvements with SXA version 10.2, two of them already highlighted above (Horizon support and that default SXA items are now provided as resources files rather than installable items packages). Apart from that, I'd mention support for Bootstrap 5, upgrade of Scriban library to version 3.6 which now supports responsive images and templates in JSON variants. Also, keep in mind that "JSS" sites are now renamed to "Headless" to account for the support of the .Net Core rendering host.


7. Headless supports Next.js 11 with Dynamic Component Imports and Vue 3.x. These changes and the rest of JSS covered with improved documentation for Sitecore Headless Rendering 19.0.


8. Sitecore Host got upgraded to .NET Core 3.1, and that affects everything that runs above it: Horizon, Publishing Service, Identity Server, etc. Sitecore Identity Server has been upgraded to version 6.0 based on IdentityServer 4.1.1 (which obviously runs on .NET Core 3.1). Sitecore Publishing Service no longer requires the resource files with default items to be copied to it before usage. The service will download the resource files automatically from your Sitecore instance. You can get the latest Publishing Service and the corresponding Publishing Module.


9. Reporting also got a few features. Firstly, you can now download reports data in a CSV format for further processing in Excel. Secondly, better defaults have been applied to report table columns to sort by more meaningful metrics out of the box. Thirdly,  can now see the impact of personalization on the conversions for individual goals.


10. Internals and "invisible" changes. 

Sitecore Kernel got 27 bugs fixed, that resulted from support tickets sent by us. Performance improved between Tracker and xConnect, to generate fewer requests. Speaking about xConnect, you can now purge data for interactions in a similar manner that was previously done for contacts. 


Bonus 1. CLI update is not a part of the current 10.2 release, as CLI upgrade to 4.0 was brought a month earlier. But as both are effectively a "part of the same thing", I will mention those below:

Firstly, as we've been asking for, finally we can serialize and deploy security roles.

Secondly, Items as Resource plugin should have come eventually and now it features improved release velocity and reliability by deploying items as files.

Thirdly, another highly requested feature: you can now exclude fields from serialization. 

There are other minor improvements as well as the Indexing plugin. All these CLI changes are also available for 10.1.


Bonus 2. Deprecates. Unwanted Azure Search and Update Center got deprecated - from 10.1 we've got Sitecore UpdateApp Tool and containers becoming the main deployment method. Thus Update Center functionality ceased to serve its purpose and is no longer available.


It seems to be very decent update. Hope this gives you enough encouragement to try the new Sitecore 10.2!

I intentionally left Experience Edge outside of scope of this post as given cloud feature alone worth of decent series of blog posts.

Sitecore 10 .NET Fundamental Developer Certification Exam is now available

Many of you have been asking about the version 10 developer exam availability, so it is finally there!

You may book it on the same page as you did all previous exams, same proctored exam. The price however is slightly higher - $350 as opposed to $300 for version 9 exams. For some European countries, there is also smth called "estimated tax" - not explained which exactly tax it is and how it got calculated. For example, buying an exam from the UK sets this tax to $70, totaling $420.

Metrics: you'll be given 100 minutes and the same amount of questions to answer as if was before - 50. Pass rate - 80%.

So, what to expect?

Firstly, I would admit the quality - the test has been reworked much to better! One thing I really disliked about the previous versions was large code snippets, with a challenge to select the correct one. Ironically these snippets were all related to item access API - something that very few people are doing nowadays due to ORMs, like Glass.

Secondly, you would expect questions on certain new stuff, like containers and CLI. In addition, the test takers must demonstrate knowledge of the existing familiar competencies, like security and roles/users management, layouts, placeholders, components, controls, renderings, and items management, and similar. Obviously, no SXA or JSS related questions are expected, as that is a Fundamental exam.

Thirdly, I want to notice incorrect answer options - those became more realistic, and therefore - confusing. With all my experience with the platform, I manage to miss a few, due to being not 100% attentive. That means - do read questions and answers very carefully and pay attention to details. Having 2 minutes per single question is a decent amount of time for doing that.

General feedback: the exam changed for the better but it is very unlikely to pass it without having actual hands-on all the expected competencies. With an 80% pass rate, one could incorrectly answer 10 questions. Let's imagine a person, who is proficient in version 9 but never has worked with 10. In that case, he/she is likely to fail more than 10 questions for containers and CLI, and given the rest of the answers are correct, the result is still - fail. On the other hand, the required level of experience is not that high, so everyone working with the latest features would pass without any doubt.

Wish you good luck with Sitecore 10 .NET Fundamental Developer Certification Exam!

Things beginners get incorrect about Kubernetes

On start playing with Kubernetes, one may face with one of the biggest delusions considering the K8S will work in the same way for both the development or testing environment. 

But It won't!

When it comes to containers in general and Kubernetes specifically, there is a big difference between occasional runs in a labs-alike conditions and in full production lifecycle. That is similar to a difference between just starting an app and long term running it full security and reliability enabled.

Not a Kubernetes exclusive problem, but is true for the entire variety of containers and microservices. Spin-up a container comes as relative simple task, while scaling containers as containerized microservices in the production turns to be more complicated.

Although Kubernetes has alternatives, it has quickly become a de-facto standard for orchestration. However there is a difference between launching K8S in a sandbox compared to a full production environment.



Delusion #1. Running containers with Kubernetes in the development or testing environment ensures that your operational needs will be satisfied.

The truth: the launch of Kubernetes in the development or testing environment allows cutting the corners, simplify things and not to bother with the operational load, which one faces when going live to Prod. Ops and safety considerations will become major areas of differences between K8S running in prod and in the development / testing environments. Failing a cluster in the labs conditions does not bring any losses.

For me it looks like a compromise between an agility and reliability: devs use containers to achieve flexibility while working with apps when developing and testing the code does its purpose. While the ops need to provide reliability, scaling, performance and safety provided by a sustainable, industry-proven platform. They are looking for a deployment automation for the clusters to ensure the repeatability and consistency. It also helps when restoring the system.

Versioning is also critical for operations. As far as possible, you need enabling versioning everywhere, including services deployment configuration, policies and infrastructure (applying the infrastructure-as-a-code approach). That results in environments becoming repeatable. As a good practice, avoid "latest" image versions, in order to avoid configuration drift effect.


Delusion #2. Both reliability and security got provided with Kubernetes

In reality: when using Kubernetes at non-production environments only, most unlikely reliability and security got provided, at least initially. Do not get discouraged, you will be there: it's a matter of designing an architecture before switching to the Prod.

Obviously, performance, scaling, availability and safety requirements are much higher in prod environments. This It is important to plan these requirements for the deployment of K8S into architecture, as well as build scaling and security plans into Helm-charts, etc.

But how could running a cluster in dev/testing environments lead to a false confidence?

This is common for non-production environments having all network connections open. It is acceptable that any service can refer to any other service: open connections are the defaults for Kubernetes. However such an approach is an evil practice for production environments and can lead to downtime. It also exposes larger areas for potential attack and increases threats to business.

When it comes to containers / microservices, one needs spending bigger effort for creating a highly available and reliable system. Orchestration itself helps a lot but isn't a "silver bullet", same applies to security. We will have to work hard to protect Kubernetes and reduce the surface of the attack. It is very important using RBAC with minimal privileges and enforce network policies, leaving only those channels services indeed use.

Also vulnerabilities of container images can rapidly turn ops into a critical state, while on development / testing environments this danger may absent at all. Pay attention to the base images used for building your containers: as far as possible, use trusted official images, or build your own. The last thing you want happening for your Kubernetes cluster is helping someone mining crypto coins.

It is recommended to refer to the security of containers as a ten-level system covering the container stack (host and registries), as well as questions related to the life cycle of containers (for example, API management). 


Delusion #3. Orchestration makes scaling a formality

Although Kubernetes considered being a completely necessary tool for scaling containers, it will be delusted to think that orchestration immediately sorts out scaling needs for the production environment. The volume of data at live environments is times more, please also keep in mind that monitoring may also need scaling. With increasing volumes, everything changes. 

It is impossible to ensure all K8S components implementing the interfaces correctly until you spin-up the prod: determining Kubernetes "working normally", and the API server and other controlled components get scaled according to your needs.

As I say, the development and testing environments go much easier. In local environments it is easy skipping basics like defining the right resources and restrictions for requests. Avoiding that can collapse you prod once later. 

Scaling the cluster both directions is a good example when the task goes easy locally, being clearly complicated at production: scaling prod clusters is more difficult than clusters for development/testing.

While Kubernetes makes it relatively simple scaling horizontally, DevOps still need keeping in mind some nuances, especially when it comes to maintaining services live when scaling an infrastructure. It is crucial to ensure that the main services, as well as a system monitoring and security alerts, were distributed across the cluster nodes and do work with stateful volumes so that data not being lost on scaling down.

Again, it all comes to proper planning and resources available. You need not just understand your needs for scaling when planning but most importantly - test them. Your production environment must be capable for handling much higher loads.


Delusion #4. Kubernetes works everywhere equally that same

In reality: differences in work in another environment may vary similar to those differences between running Kubernetes on the developer's laptop and prod server. The reality is that there may be serious differences depending on the vendor .Many believe that if the K8S works locally, it will work in any operational environment. 

Local environments commonly miss important components required by prod environments: monitoring, logging, certificate management and credentials. You need to keep that in mind, as that is another problem raised from a difference between prod  and development/testing environments.

However, that isn't Kubernetes exclusively, but applies to containers/microservices in general, especially in multicloud and hybrid cloud setups. Those Kubernetes implementations are more complicated than it seems initially, as many of the mandatory services are proprietary, like load balancing and firewalls. A container that works well locally may work unprotected (may not start at all) in the cloud with another setup of tools. Therefore, SERVICE MESH technologies like Istio attract so much attention. They guarantee the availability wherever your container works, so you do not need to think about infrastructure - which is the main reason for using containers.

I hope you can reach safer and more reliable production environments with Kubernetes keeping the above in mind!

Converting Sitecore back-end developer skills for a rapid kickstart with JSS & Next.js

Developers are crucial for Sitecore ecosystem!

With a count of several tens of thousands globally, only less than 15% of Sitecore developers feel confident with modern front-end tools. Resolving this bottleneck is very important as that slows down adoption of new generation of headless approaches: JSS and Next.js. This session is to show the quickest but still effective path for converting typical existing BE skills into the new development paradigm.

I have prepared a session for Symposium 2021 and below is my paper submission proposal. Symposium speech gets backed with a several blog posts and "how-to" videos that highlight the whole path for typical Sitecore developers with minimal knowledge of JS to the state of competency with Next.js

Update: sadly, this proposal was not chosen, leaving me frustrated, as the described topic is very sensitive to the most of us - developers and solution architects. Therefore, I am leaving my submission for historical purposes below.

The proposal

With much already said about the advantages of Next.JS with Jamstack fast delivery by pre-rendering, we won't focus on that as it's well-documented.

Instead, session will mostly cover converting a typical developers' experience from purely back-end skills to the state of confidence enough to start building own Next.js solutions. Outside of it there's almost nothing that describes that the actual learning curve which raise high level of frustration: the gap is too big to fill without knowing a shortcut to success.

The session gets based on my own experience: being such a typical back-end person, I carefully documenting all the way down to the wonderful world of headless Jamstack: necessary steps and traps that may not be obvious for the target audience get explained.

The mission is simplifying switching to a new gen. of development for as much of typical XP developers as possible by:
  • explaining the bare minimum of skills to obtain to be able using Next.js with Sitecore
  • explaining how to set up the necessary toolset, solution, dependencies and the fastest approach for getting the knowledge
  • briefly focusing on container environments being a part of overall experience
  • doing all the above with the minimum efforts possible
  • assuming the audience gets on a self-learning path after overcoming initial "studying gravity" with materials from this session

Speech Agenda

1. JavaScript
  • most important changes since years jQuery ruled the front-end world
  • starting with React: important basis to be used
  • all you need to know about TypeScript using it with Next.js Sitecore solutions
  • unobvious traps of front-end world to avoid

2. JSS
  • mapping the terminology of old dev experience to newer counterparts
  • explaining and troubleshooting GraphQL and layout service
  • JSS Styleguide, DOs and DON'Ts

3. Containers
  • brief introduction for those never had experience containers approach
  • Next.JS starter template
  • development considerations

4. Next.JS
  • understanding pre-rendering options: static generation vs server-side rendering vs incremental static generation
  • managing dynamic content with ISG
  • routing / dynamic routes
  • components rehydration
  • client-side personalization via callback to origin

5. Development Experience
  • understanding solution structure
  • organize CSS on component level
  • debugging and troubleshooting

6. Deployment and Going Live
  • brief architectural overview
  • is self-hosting the best option for your solution?
  • hosting at Vercel
  • Sitecore Experience Edge

7. Demo time covering some of Next.js features:
  • image optimization
  • error handling
  • unusual API routes

8. Conclusion
  • FAQs
  • take-away materials
  • further learning plan


Takeaway materials

By the time of the event, I am going to produce the following materials covering my presentation:

  • A series of blog posts covering a topic much wider
  • GitHub repo with a guidance and codebase from demo
  • A series of short YouTube videos for each use case

Hopefully, once my submission get selected for either SUGCON or next Symposium.

Evolutional approach to Next.js and its modes

As you've might hear, Sitecore has chosen Next.js to be used along with its JSS SDK. But what makes Next that great tool for most of us switching to a new paradigm of development for Sitecore? In this blog post, I'll go through Sitecore development evolution, starting with a review of Sitecore development progressing with time.

Old school development

A decade ago, we used classical ASP.NET WebForms to render a page on a server and pass it to the client. The whole idea of WebForms was faulty as it tried to mimic the event-based model of desktop development to make web development feel familiar to them. That was at cost of ignoring the stateless nature of HTML, creating weird ugly abstractions (ie. ViewState and EventValidation).

It was later made obsolete with an MVC approach which turned ASP.NET web development to what it should be in a better world: no state and events abstractions, server controls, and Master pages. Proper separation of code and markup (which itself went better and readable with the introduction of Razor views). It all benefited from an MVC architecture, proven with other web technologies, such as Ruby on Rails. Moreover, the implementation allowed extensibility at always every lifecycle of web request while ASP.NET MVC going open-source allowed writing your code aligned with the exact implementation of the framework.

MVC made a great step ahead and stayed a default way of making sites with Sitecore for as long as 5-8 years. Being so close to a raw request was a great strength but at a cost of having a lot of repetitive activities.

The introduction of SXA fixed most of these issues by strongly relying on Sitecore PowerShell for addressing most things that should and were in fact automated. The overall developers and editors' experience has improved with SXA due to the introduction of Page and Partial Designs, powerful components adjustable with rendering variants, most popular grid systems support, flexible search and SEO tools.

SXA was great in most aspects, except the one but most important - it was still based on top of MVC. That means web pages were generated at the server by rendering content into HTML views. Or in other words, it was not headless...

Headless

Meanwhile, the world of front-end development has experienced massive growth and after half-a-decade craziness of JS frameworks appearing one after another, a triad of winners stood out: React, Angular, and Vue. Those good old days of using jQuery came to an end giving way to industry-proven frameworks with the bigger feature sets and revised architecture that suits modern web development.

With time It became even harder and harder to split work between back-end and front-end teams (as for full-stack guys most of them tend to choose either side). Even bigger efforts have been spent on unwanted work of merging FE and BE teams in sync, which could not last long as both sides were struggling from that situation.

The headless approach was the right answer resolving all those issues with JSS being Sitecore response for that.

With the release of JSS, it became possible to separate BE and FE in a way that page each side becomes responsible for only its own duties. Front-end becomes free of previous limitations and could use React / Vue / Angular as much as they wanted. They did not need to use a heavily loaded web server with Sitecore for generating HTML pages - a new component called Rendering Host did that job exclusively for them. The only interaction with the back-end left was receiving just the necessary data asynchronously thanks to Layout Service and GraphQL.

NOTE: Actually headless means anything can consume the data from back-end services, not just FE frameworks. That is well done in ASP.NET Core renderings as an alternative option for headless implementation for Sitecore.

Client-Side Rendering

With a typical non-Sitecore single-page application the webserver firstly sends the browser an HTML page being in some initial state. Once that page gets loaded, the browser executes its JavaScript code which raises an asynchronous request(s) to an API endpoint in order to get actual data. As the user progresses with this app, more requests are sent by the browser, which will partially update content on a page without the whole page reloading from a web server. This approach is known as Client-Side Rendering, CSR and it brings lots of advantages such as apps responding faster and reducing traffic between client and server.

What's wrong with single-page applications?

Since single-page apps only load an initial HTML page once, this is the same as what search engine bots get. They struggle to obtain follow-up data from APIs and cannot index the page. Also without page reload the URL reaming the same and it can vary by only appending a #-anchor to a page URL. Often these URLs cannot be correctly processed when called directly.

Next.js

To address the above we have Next.js - a framework for statically generated and server-rendered React applications that opens up a lot of possibilities for developers: creating ready-to-use, zero-configuration applications, code separation, static HTML exporting, better UX, faster performance, and more. You can see many of its features below:

Next.js will ensure SEO without any extra actions from users beyond creating an application. Just to make clear, that results not from Next.js specifically, but from server-side rendering.

Once can do some SEO reports with Lighthouse even at earlier stages as you begin building your application.

But that still wasn't that....

SSG challenge

The idea behind Jamstack is truly attractive: instead of serving webpages in real-time (even when taking those from a cache), the webpages are already pre-rendered and deployed to CDN being globally accessible immediately upon publishing. In a simple scenario, one does not even have to keep a running server up as the traffic never reaches it going to CDN. Static content is fast, resilient to downtime, and gets indexed immediately by crawlers.

This approach however has some issues.

Let's think about a huge site with millions of pages. Deploying such a site may last hours rather than minutes due to static pages generation and the number of files to process. An increasing amount of content means increasing generation time. It seems to be reasonable to re-generating only those pages been updated, but it is only a small part of the solution (deployment becomes complicated and even one character change in a common part like a header will still make you process all the pages).

ISR

That is where Incremental Static Regeneration (ISR) comes into play. ISR is a new evolution step for Jamstack. Next.js allows you to create or update static pages beyond you’ve built a site. Incremental Static Regeneration enables developers and content editors to use static-generation on a per-page basis, without needing to rebuild the entire site. With ISR, you can benefit best from both worlds while scaling to millions of pages.

The principle difference is that now Static pages could be generated on-demand at runtime. The developers' job is now deciding which portion of pages you pre-generate, i.e. well known 80/20 Pareto's Law where 80% of traffic is served by only 20% of pages, while the other 80% of pages get the remaining 20% of traffic.

So it makes good sense to pre-generate that heavily used 20 % of pages. How to know which pages or sections to go through? You've got an arsenal of tools like analytics, A/B testing, alternative metrics - in any case, you got the flexibility to make your own tradeoff on build times, as the image below compares:

With being given a choice now, developers can define options A or B and choose between them: Selecting option A build time gets faster, while option B generates more pages.

This becomes crucial when working on large eCommerce implementations or headless CMSs such as Sitecore.

How that works

ISR relies on that same API being used for static sites generation getStaticProps. The difference is that by setting revalidate parameter to 60 we make Next.js using ISR for a page. Here's how the request goes with ISR:

  1. With Next.js one can define a revalidation time per page (ie. 60 seconds)
  2. The initial request to a product page will return the cached page with the original price
  3. At this stage, someone makes changes into a product data, affected in the database changes
  4. All requests to the page after the initial request but before 60 seconds are returned immediately as are cached.
  5. After a given 60-second window, the following request will still show the cached (old) page. But Next.js triggers background regeneration of that page. Once completed, it will update a cache for that single page or keep an old cached page upon a background regeneration failure.

Finding a compromise

Since all the sites vary by volume, audience, purpose, and internal architecture - there's no a silver bullet to cover them all with a universal solution. That is why Next.js is end-user-centric, offering developers shifting between solutions without leaving the bounds of the framework. It's for you to choose the right tool for a project.

Edge caching

In certain cases, ISR is not the best option, like some apps where live data display is crucial. Those would be better handled with server rendering, with some option of own Cache-Control headers with surrogate keys to invalidate content. Server rendered pages could get cached at some edge servers. With a hybrid framework, one can make own tradeoff and still stay within the framework.

SSR with edge server caching may look similar to ISR (especially with stale-while-revalidate headers for cache control).

The major difference comes from the way of handling the first request. With ISR it returns a statically rendered page that ensures the user will see a page even in case of API connectivity loss or database failure. SSR allows setting the pages depending on the specific features of requests.

One thing to care about in that case is using SSR whiteout caching may affect the performance as every millisecond of wait is important. In addition SSR with no cache badly impacts the TTFB metric (Time to First Byte) being used by Lighthouse.

In addition to that, ISR is not beneficial for small websites. That is reasonable if build time for the whole site is times lower than the revalidation parameter - just use classic SSR instead.

ISP fallback options

This is an important parameter with two potential options. When working with data that is fast to retrieve it makes sense using fallback: blocking. In that case, you do not need to display using a temporal "in progress" page while the data retrieval. That will guarantee users see the right page regardless of it is cached or not.

For uncertain or slow loading data the above approach will affect UX badly, therefore setting fallback: true makes an immediate display of the "please wait" page while data is processed.

SEO is the cause

SEO (search engine optimization) is a set of techniques (and even unobvious tricks) for changing your site in order to attract higher traffic from search engines. In order to increase the site's search rate, one needs to keep in mind many of them, such as:

Visitors won’t wait an eternity until your page loads. Performance is actually a crucial factor for SEO and therefore should be the main concern when building an app. In addition to FTFB (mentioned previously), there is another important parameter abbreviated as FCP (First Contentful Paint). Google uses FCP as a key metric for performance - FCP directly affects SEO rating. You can read more about improving FCP.

With Next.js you can analyze FCP and LCP (Largest Contentful Paint - time used for major content shown) by creating App component with a reportWebVitals function:

// pages/_app.js
export function reportWebVitals(metric)
{
  console.log(metric)
}

Once these parameters get calculated reportWebVitals function is called with all the metrics for you to log and analyze. Follow this link for more details about measuring performance with Next.js

I hope this post gives an overall highlight on the rendering evolution from the old days till ISR and nuances choosing them with Next.js.

Sitecore gets presented at Awesome List

After 3 months pull-requests-rejections-football I managed to squeeze Sitecore to be presented at Awesome List.

Awesome List logo

What is awesome list?

An awesome list is a list of "awesome things" curated by the community. There are awesome lists about everything from CLI applications to fantasy books. The main repository serves as a curated list of awesome lists, each of them represents the whole world presented in the most friendly way. If you never heard about it, I highly recommend start navigating if from the home page and can guarantee you'll find much great things there.

Until 2020 the list has missed Sitecore, so that I've fixed that. Now the repository contains comprehensive and well classified list of all known GitHub repositories. I personally find it useful for random lookups for certain code for a specific domain upon the demand - that saves much time! But apart from that, it's nice having the whole list of all the open source implementations just to review all the variety of things people did with Sitecore.


Existing categories

As for today, the whole Sitecore repositories are grouped into the below categories. I am leaving the direct links to each of them for the simplicity:

Everyone is welcome contributing to the repo as soon as you got any awesome stuff to add (by PR), but please be aware of the strict guidelines. 

Hope you find this list helpful!