Experience Sitecore ! | All posts tagged 'Architecture'

Experience Sitecore !

More than 200 articles about the best DXP by Martin Miles

Sharing Sitecore Identity Server between two independant instances of Sitecore

Imagine a case where you need having two Sitecore instances in parallel next to each other. That may be cause by several legit reasons.

WHY?

For example, in my case I am moving (by reworking, not just migrating) some functional areas from one legacy instance to another that will features SXA. The legacy instance has been passed from one hand to another with numerous configuration artifacts, with a limited maintenance options, so that it becomes next to impossible to combine it with a brand new SXA stuff under the same roof. I know, that is doable in principle (and have done it myself before), but the amount maintenance and lack of knowledge / documentation on existing codebase makes its maintenance inappropriately risky and non-acceptable. Therefore, it becomes reasonable keeping them both in isolation, only uniting at the URL level (by rewriting a (sub)domain of a new instance into a primary domain's folder level).

Things you have to consider in that case would turn to almost doubling your infrastructure and related expense as well as checking if your Sitecore licence permits you that. Currently doing quite an unusual setup where both the above concerns give me a green light for going ahead and I am OK to run both instances in parallel (as on-prem solution).

Once agreed, the next thoughts come to Identity Server, where keeping two instances for that same activity does not make much sense. Keeping them both is exhaustive, but the good news is that one can re-use and existing ID Server for any number of instances (namely CM boxes). That comes to making two extra steps and below I will show you how-to:

HOW?

Let's assume we have two instances, called old and new. Old one has all the bits configured and running, so we only want re-using ID Server of old instance with a new instance.

1. Get rid ow ID Server for a new instance (you can stop its web app and app pool for now). That makes sure it is not used.

2. Find Sitecore.Owin.Authentication.IdentityServer.config file on a new instance (App_Config\Sitecore\Owin.Authentication.IdentityServer.config) and substitute identityServerAuthority variable to point to an existing ID Server

    <sc.variable name="identityServerAuthority" value="https://old.identityserver" />


3. Now CM for a new instance knows which ID Server to talk through, but will ID Server accept those calls? The answer is no, unless you explicitly permit it doing so. Navigate to Config\production folder of old instance ID Server and add addition allowed CORS origin group into Sitecore.IdentityServer.Host.xml file. You will end up having smth. as below:

<AllowedCorsOrigins>
<AllowedCorsOriginsGroup1>https://old</AllowedCorsOriginsGroup1>
<AllowedCorsOriginsGroup2>https://new</AllowedCorsOriginsGroup2>
</AllowedCorsOrigins>


4. There is also Identity Server secret stored below at the same xml file, with a matching counterpart at App_Config\ConnectionStrings.config, so you also need updating config for new instance with the value from shared Identity Server:

<add name="sitecoreidentity.secret" connectionString="SECRET_from_ID_Server" />


5. Finally, recycle Identity Server application pool, then you're OK to test it. To make things more visual, I've recorded all the steps and testing it and sharing resulted video below:


Things to consider: as you're re-using existing old instance Identity Server, it will itself re-use all the assets. When it comes to Active Directory then it brings a desired result, but speaking about internal users (those normally you have got at Sitecore domain) - they will all get reused all as well, including admin. This comes because ID Server has a reference to core database (or security database extracted from the core) and that one belongs by default to an old instance too.

Hope you find this helpful!

What is a Reverse Proxy and what do you need one for?

There are a variety of Reverse Proxy solutions on the market. You may have already heard about some:

Major cloud providers also have their proprietary solutions:


But what is a Reverse Proxy? Why "Reverse"?
As Wikipedia says, a common type of proxy server that is accessible from the public network. Large websites and content delivery networks use reverse proxies - together with other techniques - to balance the load between internal servers. Reverse proxies can keep a cache of static content, which further reduces the load on these internal servers and the internal network. It is also common for reverse proxies to add features such as compression or TLS encryption to the communication channel between the client and the reverse proxy.

Reverse proxies are typically owned or managed by the web service, and they are accessed by clients from the public internet. In contrast, a forward proxy is typically managed by a client (or their company) who is normally restricted to a private, internal network. The client can, however, access the forward proxy, which then retrieves resources from the public internet on behalf of the client. Here's a reverse proxy in action from a very high:


What are typical scenarios for using a Reverse Proxy?

1. SSL Offload. Let's assume we've got a website which works at HTTP only, and for some reason (legacy, gone developers or being unable bringing changes into a running solution that may huge or any other) it is not possible to change the website itself - "If it works, don't touch it" paradigm in action. For compliance, we must add HTTPS support for that website.
With using a Reverse Proxy it comes to a really quick and easiy solution - we don't need developers at all. All we need is asking our Ops professional asking him to instantiate a proxy server with SSL Termination. (obviously, we'll also need SSL certificates for domain hostname(s) of a given website). Job done!


2. Load Balancer
. Next, we'd want horizontally scaling that website and even deployed two equal client facing copies of it. How do we "split" traffic to distributing it equally to both sites? In this case we introduce a Proxy Server functioning as a Load Balancer.
But what if one of websited dies or crashes half way down the road? Load Balancer needs somehow to know each of the "boxes" functions well and react the outages by re-distributing traffic to the rest of mchines functioning well. This is traditionally implemented by "pinging" a such called "HealthCheck" URL on each particular box. As soon as one of the healthchecks keeps failing, an alert is raised and the traffic is no longer routed to a faulty box (be careful with sticky sessions!).



3. Cybersecurity enforcement
. Sending specially formed packets hackers can undertake a Deny-of-Service attack, when sending a request comes times cheaper than serving it back. At some moment your servers won't cope with this parasitic workload and will fail.
In order to prevent that, dangerous traffic should not reach your servers, being filtered at a proxy. Namely, a Firewall with an adequate rule set that filters out all patters with anomalies, raises alerts and bypasses the legitimate requests.


4. Caching and compressing
. Even with purely legitimate traffic beyond the proxy, one may still get a large request payload. But how comes? Well, there may be different reasons, such as usage patterns where all users navigate to the same heavy-loaded area of a website; alternatively the website itself could be written by junior developers who did not enough care about the way if functions in the most optimal way once deployed. Regardless of the reason, we could still soften things up by identifying some of the popular endpoint that consume much of server's resources and cache it up right at the proxy level. Of course assuming that a given set of parameters always returns that same result, there is no longer a need spending expensive server resources on producing the results we're already got in past and have effectively cached at the proxy level and never reach servers at all. If we however must ensure this traffic reaching the end servers, we could at least compress / encode the "last mile" beyond the proxy.


5. Smooth automated deployments
? Why not, have you ever heard of Blue-Green Deployments? With that in action end users won't even realize that you're upgrading the solution while they're browsing your site.


6. A/B Testing
. As a result of previous point, it may be a case you've updated some but not the all end servers. You do not want updating them all, instead you'd like to perform an A/B Testing on both sets and based on a result decide to complete an update or rollback to the most recent version. This would be a pretty valid scenario that a reverse proxy can do for you.


7. URL and Links Rewriting
. What if you have a legacy website that functions perfectly well, but similar to a scenario (1) the is no way (and need) of maintaining it. The development team has gone and in any case there is no single reason of investing a lot into a smth to be dismissed at some stage. At the same time you got another website(s) that either could be successor(s) for a legacy one, or some additional areas, written in isolation by more modern tools and thus either incompatible or expensive to merge with an existing solution. However the business wants everything to function with the same main domain name, just in different "folders" under it, so that end users (and search robots!) see no difference between consisting parts and naturally experience them both as being a single solid website.
Achieving that is also possible with a Reverse Proxy by rewriting URLs. Please note that not just a external request coming to site.com/company1 will be rewritten to www.company1.com but also all the internal URLs within all the requested pages need to be rewritten as well. Please note that it becomes possible only in conjunction of SSL offload, otherwise the traffic gets encrypted and proxy becomes "a man in the middle".

Not just that - once 6 years ago I wrote a walkthrough how one can achieve that same result purely and entirely by the means of IIS on Windows


Conclusion.
This article give a high-level explanation on Reverse Proxies and their primary features. It intentionally does not focus on a specific implementation avoiding going deeper in technical details.

In real-world solutions of course you will meet Reverse proxy solutions implementing several of the features combined. This may be a typical workflow for processing inbound traffic with a Reverse Proxy:



In any case, I hope you found this helpful!

Sitecore Habitat

Few times I have been already demonstrated a new modular (I'd say revolutionary) architecture approach to Sitecore - Habitat. So my curiosity won, and I started investigating it. Meanwhile, I decided to share in one place all the useful links about Habitat I've managed to find.

What is Sitecore Habitat

As authors say, Habitat is a Sitecore solution example built on a modular architecture. The architecture and methodology focuses on simplicity, flexibility and extensibility. It will empower you with a consistent and discoverable architecture with ability of adding changes quickly and without worry, extensible with new features without steep learning curve.

Habitat is developed and managed by a team in Sitecore Product Marketing with members all over the world (Australia, UK and Ukraine).There are no plans to incorporate Habitat into the Sitecore product, but it will rather serve as a reference implementation showcasing aspect of Sitecore developing. Sitecore is and will continue to be committed to being an open development platform catering for many architectural and DevOps.

Sitecore encourages technical specialists to take part in Habitat. I was shown it for the first time on Sitecore 8 training in London and was amazed by architecture, how things looked differently to what used to have and logical and well-thought at the same time! So, here are some resources I managed to find and share about this project


References to Sitecore Habitat

Anything missed? Please let me know so that I will update the post.

Sitecore Technical User Group UK - January 2016 - Download presentation

Yesterday I have attended Sitecore Technical User Group in London and would like to share presentations from it. There were 3 presenters, talking about:

1. "CDN with Sitecore" presented by Kamruz Jaman, a Sitecore MVP 2013-2015 and in general one of the most experiences Sitecore architect, who seems to have an answer to any question (StackOverflow). He reviewed know approaches of implementing CDN into a solution in order to reduce bandwidth and improve you servers performance, and demonstrated the most complex - integrating Azure into Sitecore with media items being directly uploaded to the cloud storage,
Download presentation (521.5KB)

2. "Continuous Integration & Delivery for Sitecore" by Jason Bert.
Jason seems to know everything about CI with Sitecore. He stated, that is absolutely doable to achieve a working CI setup with Sitecore, GIT, TeamCity and Octopus Deploy and showed us how-to. Less theory in favor of a practical demo in real-time!
Download presentation (4.4MB)

3. "Sitecore 8.1: new Features and Improvements" presented by Steve McGill.
Steve guided us through serious list of new features and improvements in Sitecore 8.1 - there are so many sweeties I am anticipationg to start working with! At the end of presentation he also discussed Sitecore Habitat - a project bringing modular approach into Sitecore.
Download presentation (2.3MB)

Steve McGill and Kamruz Jaman:


Light of knowledge is coming out of Jason Bert


Thanks to presenters and everyone who has attended!

StackOverflow 2 questions on Multi-Site configuration: Multisite Best Practice for setting up Visual Studio Project and Manage web.config in multisite solution

Got to answer two question about multi-site configuration working with Sitecore:

1. Sitecore Multisite Best Practice for setting up Visual Studio Project (link to original question)

I'm seeking an advise on best practice that has worked for creating a asp.net MVC visual studio solution that supports multisite (multi tenant). One thing we would like to do is minimize the regression defects so that developer don't modify the wrong website code base etc.
That the solution needs to support more than 8 web sites.

2. Manage web.config in sitecore multisite solution (link to original question)

We have a multisite solution with individual visual studio solutions for each websites. Then we have master solution to build/deploy all websites.
Firstly, not sure whether it's a best practice to include web.config in Visual Studio solution. But I think all the nuget packages needs web.config to add their settings.
As a result, we have web.config for each solution. However when we deploy from master web.config gets overwritten by each sites. Could someone please suggest how this issue can be fixed?

Answers:

I have got one more alternative to address that. Currently working on a huge project, with dozens of developers and quite bureaucratic process of deployments, we have introduced the following approach:

  1. Project consists of multiple logical subparts, which in fact are individual websites.
  2. Each of those websites we set up as an MVC Area with own controllers, views etc
  3. Most important - we set up MVC Areas as individually pluggable DLLs. So each of that websites is a separate project under solution, with its own resources and statics; on build everything is copied under their required paths and DLL goes to the \bin folder.
  4. One more class library for shared (by all websites) resources, it is being referenced only by those sites, that need that functionality
  5. In Sitecore, we have the same strict principles - the content, layouts, renderings are isolated as higher as possible, no individual content may be re-used (unless from shared resources folder).

This approach serves us almost a year already and has proven for its agility, much easier deployment and fairly less code merge conflicts. If you are working under just one site - you don't need to re-compile and re-deploy everything - just replace one dll at bin folder.

Thus, combining that with your question, Approach 1 would be an answer.

References (more to read):

Regarding managing configuration, I would advise for each website (under its individual project) to create it's own App_Config/Include folder and create _project_name_.config within that folder in order to keep all site-specific settings there (for further merge into resulting config).

On build you set up (for each individual project) that file to be copied into main SITECORE_INSTANCE_WEB_ROOT\App_config\Include folder along with the rest of include patch config files.

Hope someone finds that helpful!

Sitecore Boilerplate - the repository of best practices all at the same place

I decided to create an ultimate "boilerplate" solution for Sitecore, implementing all the best Sitecore practices in one place, well documented and cross-linked with the support on this blog.

As a multi-language website with Experience Editor (ex. Page Editor) support utilizing with Glass Mapper, Lucene indexes and test-driven codebase and much more working well all together - it will be a perfect place for newbies to familiarize themselves with Sitecore platform. It aims also to simplify work of more senior Sitecore developers in terms of quickly searching for desired features and grabbing them into their working solutions.

The project originated out of my R&D activities as I decided it would be beneficial to share my workouts with Sitecore community. Any suggestions, comments and criticism are highly welcome!

List of the features I desire to supply into Sitecore boilerplate:

  • Support for Page Editor
  • Usage of Glass Mapper for ORM purposes
  • Unit testable code
  • Synchronization of user-editable content from CD environment to CM and further re-publish to the rest of CDs
  • Support for multi-language environment
  • Custom Lucene indexes
  • Custom personalisation of components and data
  • Workflows based on user permissions
  • Make all mentioned above working together as a solid and stable website
  • Implement new Sitecore 8 marketing features on top of that

.. for the moment I have planned and implemented several of mentioned features as a starting point, so it is coming soon on GitHub and further blog posts here.

Editing content on a CD server. Part 2. Event Queue to sync data with master

... this blog is a next step after Editing content on a CD server. Part 1. MVC ajax request to controller.

Now we got a question on how to implement server logic to store the value. It may seem pretty straightforward for a moment - get item by its ID, update the field and return result. But in fact it's not, due to Sitecore's architectural principles.


Our page is running on a CD (content delivery server) and the item we're trying to update comes from CD database (traditionally called "web"), so if we edit and update item on "web" database - we'll get into situation when "web" contains updated version, while "master" doesn't. Publishing is one-way process of copying items from CM to CD databases (or simply from "master" to "web") so with next publishing we may overwrite updated item in web with outdated previous version from master. Writing directly to CM database is a violation of architectural principles, while it is technically possible on your developer "default" Sitecore installation, in real world Sitecore CD servers do not keep "master" connection string, making this process impossible. So, what should we do in that case?


Luckily, there are several ways of solving this scenario, each has its own pros and cons, so it is worth of thinking well ahead which (and if) is applicable to your solution.

Solution 1: Allocate a separate database in parallel with web, to store all user editable content. Normal items and non-user content will remain in web database, as normally. Here is the great article describing that approach:

Solution 2: Employ Sitecore Event Queue no notify CM about CD changes, so that as soon CM receives update event, it updates itself with the latest change coming from CD and then re-publishes the change across the rest of CD databases in order to keep them in sync. We describe this approach below.


Sitecore has a mechanism that is called remote events and allows communication between instances. This is implemented via "core" database that has EventQueue table that is monitored by a minor periods of time (like 2 seconds). We always have connection string reference to core database on our CD, at least for authorization / security purpose but also to support Event Queue that servers transport for publishing operations.

So, the process of saving comment on back end now looks fairly complicated - as post request comes - we still save the changes into CD ("web") database, on CM we also add an additional handler to item:saved event that executes custom code with event handler, that updates the same item on master database. And finally, you may programmatically re-publish updated item to other CD instances (if many), that do not have an updated version yet.

Now let's look at the code that implements all described below. AjaxController will include additional code right before returning JSON back to browser:

UpdateCommentEvent evt = new UpdateCommentEvent();
evt.Id = item.ID.ToString();
evt.FieldName = fieldName;
evt.Value = database.GetItem(id).Fields[fieldName].Value;
                 
Sitecore.Eventing.EventManager.QueueEvent<UpdateCommentEvent>(evt); 

What we're doing here is just queueing an event. UpdateCommentEvent is a custom event written by us, it is normal C# class, but please pay attention to DataContract and DataMember attributes. This is required for serialization purposes. Here is how UpdateCommentEvent is defined within the code:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Runtime.Serialization;

namespace Website.Code.Events
{
    [DataContract]
    public class UpdateCommentEvent
    {
        [DataMember]
        public string Value { get; set; }

        [DataMember]
        public string Id { get; set; }

        [DataMember]
        public string FieldName { get; set; }

        public UpdateCommentEvent(string id, string fieldName, string value)
        {
            Value = value;
            Id = Id;
            FieldName = fieldName;
        }

        public UpdateCommentEvent()
        {
        }
    }
}

And, of course, we define UpdateCommentEventArgs that comes along with our new event:

namespace Website.Code.Events
{
    public class UpdateCommentEventArgs : EventArgs, IPassNativeEventArgs
    {
        private UpdateCommentEvent _evt;

        public UpdateCommentEventArgs(UpdateCommentEvent evt)
        {
            _evt = evt;
        }

        public string Id
        {
            get { return _evt.Id; }
        }

        public string FieldName
        {
            get { return _evt.FieldName; }
        }

        public string Value
        {
            get { return _evt.Value; }
        }
    }
}
Implementation on CM side: first of all we need need to specify a new hook.

    
 ...   

Here's the code referenced by hook specified above. We subscribe to our event and once it arrives - we call Run method that arranges local (for CM environment) event:

using System;
using Sitecore.Events.Hooks;
using Sitecore.Eventing;

namespace Website.Code.Events
{
    public class UpdateCommentHook : IHook
    {
        public void Initialize()
        {
            // and now raise event locally
            EventManager.Subscribe<UpdateCommentEvent>(new Action<UpdateCommentEvent>(UpdateCommentEventHandler.Run));
        }
    }
}

We need to specify new local event called updatecomment:remote in the configuration as associate it with a handler method:

    
    
    

And here is OnUpdateCommentRemote event handler that fires on CM. It gets event arguments, casts them to UpdateCommentEventArgs and extracts data out of arguments. In order to update an item on CM we require 3 parameters: Item ID, field name to be updated and the value to be updated with. All three are stored in event arguments so now we are able to update item on master database.

namespace Website.Code.Events
{
    public class UpdateCommentEventHandler
    {
        /// 
        /// The method is the method that you need to implement as you do normally
        /// 
        public virtual void OnUpdateCommentRemote(object sender, EventArgs e)
        {
            if (e is UpdateCommentEventArgs)
            {
                var args = e as UpdateCommentEventArgs;
                var master = Sitecore.Configuration.Factory.GetDatabase("master");

                using (new SecurityDisabler())
                {
                    var itemOnMaster = master.GetItem(args.Id);

                    using (new EditContext(itemOnMaster))
                    {
                        itemOnMaster[args.FieldName] = args.Value;
                    }
                }

// also do publishing to other CD here, if required
} } // This methos is used to raise the local event public static void Run(UpdateCommentEvent evt) { UpdateCommentEventArgs args = new UpdateCommentEventArgs(evt); Event.RaiseEvent("updatecomment:remote", new object[] { args }); } } }
If there are multiple CD environments (at least one apart from the one where we edit the comment) you will also need to re-publish from CM to those CDs in order to keep them all in sync.

So, in these two blog posts we described how to make an ajax MVC call to controller on back-end server and update user editable content on CD environment keeping it in-sync with other environments. Hope this post helps you to understand Sitecore architecture better.

See also (additional references):

Sitecore.Support.RemoteEventLogging
Tweak to log all item:saved:remote events of EventQueue that were just processed.
https://bitbucket.org/sitecoresupport/sitecore.support.remoteeventlogging/wiki/Home