Experience Sitecore ! | March 2024

Experience Sitecore !

More than 200 articles about the best DXP by Martin Miles

Cypress: a new generation of end-to-end testing

What is Cypress

Cypress is a modern JavaScript-based end-to-end (e2e) testing framework designed to automate web testing by running tests directly in the browser. Cypress has become a popular tool for web applications due to a number of distinctive advantages such as user-friendly interface, fast test execution, ease of debugging, ease of writing tests, etc.

Those who have already had any experience with this testing framework probably know about its advantages, which make it possible to ensure that projects are covered with high-quality and reliable autotests. Cypress has well-developed documentation, one of the best across the industry, with helpful recommendations for beginners, which is constantly being improved, as well as an extensive user community. However, despite the convenience, simplicity, and quick start, when we talk about Cypress tests, we still mean the code. In this regard, to work effectively with the persona behind Cypress requires not only an understanding of software testing as such but also the basics of programming, being more or less confident with JavaScript/TypeScript.

Why Cypress

Typically, to test your applications, you’ll need to take the following steps:

  • Launch the application
  • Wait until the server starts
  • Conduct manual testing of the application (clicking the buttons, entering random text in input fields, or submit a form)
  • Validate the result of your test being correct (such as changes in title, part of the text, etc.)
  • Repeat these steps again after simple code changes.

Repeating these steps over and over again becomes tedious and takes up too much of your time and energy. What if we could automate this testing process? Thanks to this, you can focus on more important things and not waste time testing the UI over and over again.

This is where Cypress comes into play. When using Cypress the only thing you need to do is:

  • Write the code for your test (clicking a button, entering text in input fields, etc.)
  • Start the server
  • Run or rerun the test

That’s it! The Cypress library cares about all the testing for you. It not only tells you if all your tests passed or not, but it also points to which test failed and why exactly.

How about Selenium

Wait, but we already have Selenium, is it still actual?

Selenium remained The King of automated testing for more than a decade. I remember myself back in 2015 creating a powerful UI wrapper for Selenium WebDriver to automate and simplify its operations for non-technical users. The application is named Onero and is still available along with its source code. But when it comes to Cypress – it already offers powerful UI straight out of the box, and much more useful tools and integrations – just keep reading to find this below.

Cypress is the next generation web testing platform. It was developed based on Mocha and is a JavaScript-based end-to-end testing framework. That’s how it is different to Selenium which is a testing framework used for web browser automation. Selenium WebDriver controls the browser locally or remotely and is used to test UI automation.

The principal difference is that Cypress runs directly in browser, while Selenium is external to a browser and controls it via WebDriver. That alone makes Cypress much perfectly handling async operations and waits, which were all the time issues for Selenium and required clumsy scaffolding around related error handling.

With that in mind, let’s compare Cypress vs. Selenium line by line:

With that in mind, let’s compare Cypress vs. Selenium line by line:

Cypress Selenium
Types of testing Front end with APIs, end-to-end End-to-end, doesn’t support API testing
Supported languages JavaScript/Typescript Multiple languages are supported, such as Java, JavaScript, Perl, PHP, Python, Ruby, C#, etc.
Audience Developers as well as testers Automation engineers, testers
Ease of Use For those familiar with JavaScript, it will be an easy walk. Otherwise, it will be a bit tricky. It is still developer-friendly being designed keeping developers in mind. It also has a super helpful feature called “travel back in time.” As it supports multiple languages, people can quickly start writing tests, but it’s more time-consuming than Cypress as you have to learn specific syntax.
Speed It has a different architecture that doesn’t utilize a web driver and therefore is faster, also Cypress is written with JavaScript which is native to browsers where it executes. Because of its architecture, it’s hard to create simple, quick tests. However, the platform itself is fast, and you can run many tests at scale, in parallel, and cross-browser.
Ease of Setup Just run the following command: npm install Cypress –save-dev. It requires no other component installation (unlike web driver) as Selenium does, you don’t even have to have a browser as it can use Electron. Also, everything is well-bundled. As it has two component bindings and a web driver. Installation is more complicated and time-consuming.
Integrations & Plugins It has less integrations which is compensated by a rich set of plugins. Perfectly runs in Docker containers and supports GitHub Actions. It integrates with CI, CD, visual testing, cloud vendors, and reporting tools.
Supported Browsers Supports all Chromium-based browsers (Chrome, Edge, Brave) and Firefox. All browsers: Chrome, Opera, Firefox, Edge, Internet Explorer, etc along with the “scriptable headless” browser – PhantomJs.
Documentation Helpful code samples and excellent documentation in general Average documentation.
Online Community & support A growing community, but smaller then Selenium gained over a decade. It has a mature online community.

Selenium is aimed more at QA automation specialists, while Cypress is aimed merely at developers to improve TDD efficiency. Selenium was introduced in 2004, so it has more ecosystem support than Cypress, which was developed in 2015 and continues to expand.

Installation and first run

You need to have Node.js and as Cypress is shipped with npm module.

npm -i init
npm install cypress -- save -dev

Along with Cypress itself, you will likely want to install XPath plugin, otherwise, you’re limited to only CSS locators.

npm install -D cypress-xpath

Once ready, you may run it:

npx cypress open

From there you'll see two screens: E2E Testing and Components Testing

Main Screen

Most of the time you will likely be dealing with E2E testing. That’s where you choose your desired browser and execute your tests:

E2e

By default you’ll find a live documentation in a form of a bunch of helpful pre-written tests exposing best of Cypress API in action. Feel free to modify, copy and paste as per your needs.

Tests

Here’s how Cypress executes tests from the UI on an example of sample test run:

Sample Run

But of course, in a basic most scenario you can run it from console, You can even pass a specific test spec file to execute:

npx cypress run --spec .\cypress\e2e\JumpStart\JumpStart.cy.js

Regardless of the execution mode, results will stay persistent:

From Console

Component Testing

This feature was recently added and stayed long time in preview. Now once it is out of beta, let’s take a look at what Component Testing is.

Instead of working with the entire application, with component testing you can simply connect a component in isolation. This will save you time by downloading only the parts you’re interested in, and will allow you to test much faster. Or you can test different properties of the same component and see how they display. This can be very useful in situations where small changes affect a large part of the application.

Component

In addition to initializing the settings, Cypress will create several support files, one of the most important is component.ts, located in the cypress/support folder.

import { mount } from 'cypress/react18'

declare global {
    namespace Cypress {
    interface Chainable {
        mount: typeof mount
    }
    }
}

Cypress.Commands.add('mount', mount)

// Example of usage:
// cy.mount(MyComponent)

This file contains the component mount function for the framework being used. Cypress supports React, Angular, Svelte, Vue, and even frameworks like Next.js and Nuxt.

Cypress features

  1. Time travel
  2. Debuggability
  3. Automatic waits (built-in waits)
  4. Consistence results
  5. Screenshots and videos
  6. Cross browser testing – locally or remotely

I want to focus on some of these features.

Time Travel. This is an impressive feature that allows you to see the current state of your application at any time while it is being tested.

Debuggability. Your Cypress test code runs in the same run loop as your application. This means you have access to the code running on the page, as well as the things the browser makes available to you, like document, window, and debugger. You can also leverage .debug() function to quickly inspect any part of your app right while running a test. Just attach it to any Cypress chain of commands to have a look at the system’s state at that moment:

it('allows debugging like a pro', ()=>{
    cy.visit('/location/page')
    cy.get('[data-id="selector"]').debug()
})

Automatic waits. Aa a key advantage over Selenium, Cypress is smart to know how fast an element is animating and will wait for it to stop animating before acting against it. It will also automatically wait until an element becomes visible, becomes enabled, or when another element is no longer covering it.

Consistence results. Due to its architecture and runtime nuances, Cypress fully controls the entire automation process from top to bottom, which puts it in the unique position of being able to understand everything happening in and outside of the browser. This means Cypress is capable of delivering more consistent results than any other external testing tool.

Screenshots and videos. Cypress can work on screenshots and videos. One can capture both the complete page and particular element screenshot with the screenshot command in Cypress. Cypress also has the in-built feature to capture the screenshots of failed tests. To capture a screenshot of a particular scenario, we use the command screenshot.

describe('Test with a screenshot', function(){
    it("Test case 1", function(){
        //navigate URL
        cy.visit("https://microsoft.com/windows")

        //complete page screenshot with filename - CompletePage
        cy.screenshot('CompletePage')

        //screenshot of the particular element
        cy.get(':nth-child(3) > section').screenshot()
    });
});

Produced screenshots appear inside the screenshots folder (in the plugins folder) of the project, but that’s configurable from the globals.

Cypress video capturing executes for tests. Enable it from cypress.config.ts:

import{ defineConfig } from 'cypress'

export default defineConfig({
    video: true,
})

Please refer to the official documentation that explains how to use screenshots and videos with Cypress.

GitHub Actions Integration

Cypress nicely allows to run tests in Cypress using GitHub Actions.

To do this on the GitHub Action server, you first need to install everything necessary. We also need to determine when we want to run tests (for example, run them on demand or every time new code is introduced). This is how we gradually define what GitHub Actions will look like. In GitHub Actions, these plans are called “workflows”. Workflow files are located under .github/workflows folder. Each file is a YAML with a set of rules configuring what and how will get executed:

name: e2e-tests

on:[push]

jobs:
    cypress-run:
    runs-on: ubuntu-latest
    steps:
        - name: Checkout
        uses: actions/checkout@v3
        - name: Cypress run
        uses: cypress-io/github-action@v5
        with:
            start: npm start
    

Let’s look at what’s going on in this file. In the first line, we give the action a name. It can be anything, but it is better to be descriptive.

In the first line, we give the action a name. In the second line, we define the event on which this script should be executed. There are many different events, such as push, pull_request, schedule, or workflow_dispatch (that allows you to trigger an action manually).

The third line specifies the task or tasks to be performed. Here we must determine what needs to be done. If we were starting from scratch, this is where we would run npm install to install all the dependencies, run the application, and run tests on it. But, as you can see, we are not starting from scratch, but using predefined actions – instead of doing that, we can re-use previously created macros. For example, cypress-io/github-action@v5 will run npm install, correctly cache Cypress (so installation will be faster next time), start the application with npm start, and run npx cypress run. And all this with just four lines in a YAML file.

Run Cypress in containers

In modern automated testing, setting up and maintaining a test environment can often be a time-consuming task, especially when working with multiple dependencies and their configurations, different operating systems, libraries, tools, and versions. Often one may encounter dependency conflicts, inconsistency of environments, limitations in scalability and error reproduction, etc., which ultimately leads to unpredictability and unreliability of testing results.

Using Docker greatly helps prevent most of these problems from occurring and the good news is that you can do that. In particular, using Cypress in Docker can be useful because:

  1. It ensures that Cypress autotests run in an isolated test environment. In this case, the tests are essentially independent of what is outside the container, which ensures the reliability and uninterrupted operation of the tests every time they are launched.
  2. For running it locally, this means the absence of Node.js, Cypress, any exotic browser on the host computer – that won’t become an obstacle. This not just allows to run Cypress locally on different host computers but also deploy them in CI/CD pipelines and to cloud services by ensuring uniformity and consistency in the test environment. When moving a Docker image from one server to another, containers with the application itself and tests will work the same regardless of the operating system used, the presence of Node.js, Cypress, browsers, etc. This ensures that Cypress autotests are reproducible and the results of running them predictable across different underlying systems.
  3. Docker allows you to quickly deploy the necessary environment for running Cypress autotests, and therefore you do not need to install operating system dependencies, the necessary browsers, and test frameworks each time.
  4. Speeds up the testing process by reducing the total time for test runs. This is achieved through scaling, i.e. increasing the number of containers, running Cypress autotests in different containers in parallel, parallel cross-browser testing capabilities using Docker Compose, etc.

The official images of Cypress

Today, the public Docker Hub image repository, as well as the corresponding cypress-docker-images repository on GitHub, hosts 4 official Cypress Docker images:

Limitations of Cypress

Nothing is ideal on Earth, so Cypress also has some limitations mostly caused by its unique architecture:

  1. One cannot use Cypress to drive two browsers at the same time
  2. It doesn’t provide support for multi-tabs
  3. Cypress only supports JavaScript for creating test cases
  4. Cypress doesn’t provide support for browsers like Safari and IE at the moment
  5. Reading or writing data into files is difficult
  6. Limited support for iFrames

Conclusion

Testing is a key step in the development process as it ensures that your application works correctly. Some programmers prefer to manually test their programs because writing tests requires a significant amount of time and energy. Fortunately, Cypress has solved this problem by allowing the developer to write tests in a short amount of time.

Storybook

You may have never heard of Storybook or maybe that was just a glimpse leaving you a feeling Storybook is such an unnecessary tool – in that case, this article is for you. Previously, I could share this opinion, but since I played with Storybook in action when building the XM Cloud starter kit with Next.Js, it changed.

Storybook

Why

With the advent of responsive design, the uniqueness of user interfaces has increased significantly – with the majority of them having bespoke nuances. New requirements have emerged for devices, browser interfaces, accessibility, and performance. We started using JavaScript frameworks, adding different types of rendering to our applications (CSR, SSR, SSG, and ISR) and breaking the monolith into micro-frontends. Ultimately, all this complicated the front end and created the need for new approaches to application development and testing.

The results of a 2020 study showed that 77% of developers consider current development to be more complex than 10 years ago. Despite advances in JavaScript tools, professionals continue to face more complex challenges. The component-based approach used in React, Vue, and Angular helps break complex user interfaces into simple components, but it’s not always enough. As the application grows, the number of components increases; in serious projects, there can be hundreds of them, which gives thousands of permutations. To even further complicate matters, interfaces are difficult to debug because they are entangled in business logic, interactive states, and application context.

This is where Storybook comes to the rescue.

What Storybook is

Storybook is a tool for the rapid development of UI components. It allows you to view a library of components and track the status of each of them. With StoryBook, one can develop components separately from the application, making it easier to reuse and test UI components.

Storybook promotes the Component-Driven Development (CDD) approach, where every part of the user interface is a component. These are the basic building blocks of an application. Each of them is developed, tested, and documented separately from the others, which simplifies the process of developing and maintaining the application as a whole.

A component is an independent fragment of the application interface. In Sitecore, in most cases, a component is equal to a rendering, for example, CTA, input, badge, and so on. If we understand the principles of CDD and know how to apply this approach in development, we can use components as the basis for creating applications. Ideally, they should be designed independently from each other and be reusable in other parts of the application. You can approach creating components in different ways: start with smaller ones and gradually combine them into larger ones, and vice versa. You can create them both within the application itself and in a separate project – in the form of a library of components.

With Storybook’s powerful functionality, you can view your interfaces the same way users do. It provides the ability to run automated tests, analyze various interface states, work with mock data, create documentation, and even conduct code reviews. All these tasks are performed within the framework of the so-called Story, which allows you to effectively use Storybook for development.

What is a Story

This is the basic unit of Storybook design and allows you to demonstrate different states of a component to test its appearance and behavior. Each component can have multiple stories, and each one can be treated as a separate test case to test the functionality of the component.

You write stories for specific states of UI components and then use them to demonstrate the appearance during development, testing, and documentation.

Using the Storybook control panel, you can edit each of the story function arguments in real time. This allows your team to dynamically change components in Storybook to test and validate different edge cases.

Storybook explained

Storybook Capabilities

Creating documentation

Storybook provides the ability to create documentation along with components, making the process more convenient. With its help, you can generate automatic documentation based on code comments, as well as create separate pages with examples of use and descriptions of component properties. This allows you to maintain up-to-date and detailed documentation that will be useful not only for developers but also for designers, testers, and users.

User Interface Testing

Another good use of Storybook – UI Tests identify visual changes to interfaces. For example, if you use Chromatic, the service takes a snapshot of each story in a cloud browser environment. Each time you push the code, Chromatic creates a new set of snapshots to compare existing snapshots with those from previous builds. The list of visual changes is displayed on the build page in the web application so that you can check if these changes are intentional. If they are not, that may be a bug or glitch to be corrected.

Accessibility Compliance

As The State of Frontend 2022 study found, respondents pay high attention to accessibility, with 63% predicting this trend will gain more popularity in the coming years. Accessibility in Storybook can be tested using the storybook-addon-a11y. Upon the installation, the “Accessibility” tab will appear, for you to see the results of the current audit.

Mocking the data

When developing components for Storybook, one should consider realistic data to demonstrate the capabilities of the components and simulate a real-life use case. For this purpose, mock data is often taken, that is, fictitious data that has a structure and data types similar to real ones but does not carry real information. In Storybook, you can use various libraries to create mock data, and you can also create your own mocks for each story. If a component itself needs to perform network calls pulling data, you can use the msw library.

Simulating context and API

Storybook addons can help you simulate different component usage scenarios, such as API requests or different context values. This will allow you to quickly test components in realistic scenarios. If your component uses a provider to pass data, you can use a decorator that wraps the history and provides a cloaked version of the provider. This is especially useful if you are using Redux or context.

Real-life advantages

Wasting resources on the user journey

Building a landing page may seem to be a simple exercise, especially in a development mode when you see changes appear in the browser immediately. However, the majority of cases are not that straightforward. Imagine a site with a backend entirely responsible for routing – and one may need to login first, answer the security questions, and then navigate through the complex menu structure. Say you only need to “change the color of a button” on the final screen of the application, then the developer needs to launch the application in its initial state, login, get to the desired screen, fill out all the forms along the way, and only after that check whether the new style has been applied to the button.

If the changes have not been applied, the entire sequence of actions must be repeated. Storybook solves this problem. With it, a developer can open any application screen and instantly see how it looks, taking into account the applied styles and the desired state. This allows you to significantly speed up the process of developing and testing components since they can be tested and verified independently of the backend and other parts of the application.

Development without having actual data

Often the UI development takes place before the API is ready from the backend developers. Storybook allows you to create components that stub data that will be retrieved from the real API in the future. This allows us to prototype and test the user interface, regardless of the presence or readiness of the backend, and use mock data to demonstrate components.

Frequently changing UI

On a project, we often encounter changes in layout design, and it is very important for us to quickly adapt our components to these changes. Storybook allows you to quickly create and compare different versions of components, helping you save time and make your development process more efficient.

Infrastructural issues

The team may encounter problems when a partner’s test environment or dependencies stop working, which leads to delays and lost productivity. However, with Storybook, it is possible to continue developing components in isolation and not wait for service to recover. Storybook also helps to quickly switch between your application versions and test components in different contexts. This significantly reduces downtime and increases productivity.

Knowledge transfer

In large projects, onboarding may take a lot of time and resources. Storybook allows new developers to quickly become familiar with components and how they work, understand the structure of the project, and start working on specific components without having to learn everything from scratch. This makes the development process easier and more intuitive, even for those not familiar with a particular framework.

Application build takes a long time

Webpack is a powerful tool for building JavaScript applications. However, when developing large applications, building a project can take a long time. Storybook automatically compiles and assembles components whenever changes occur. This way, developers quickly receive updated versions of components without a need to rebuild the entire project. In addition, Storybook supports additional plugins and extensions for Webpack, to improve performance and optimize project build time.

Installation

First, install Storybook using the following commands:

cd nextjs-app-folder
npx storybook@latest init

 

Once installed, execute it:

npm run storybook

This will run Storybook locally, by default on por 6066, but if the port is occupied – it will pick an alternative one.

Storybook

Storybook is released under the MIT license, you can access its source code in the GitHub repository.

Making it with Sitecore

When developing headless projects with Sitecore, everything stays in the same manner. As part of our mono repository, we set up Storybook with Next.js so that front-end developers don’t have to run an instance of Sitecore to do their part of the development work.

Upon installation, you’ll find a .storybook folder at the root of your Next.Js application (also used as a rendering host) which contains configuration and customization files for your Storybook setup. This folder is crucial for tailoring Storybook to your specific needs, such as setting up addons, and Webpack configurations, and defining the overall behavior of Storybook in your project.

  1. main.js(or main.ts): this is the core configuration file for Storybook. It includes settings for loading stories, adding add-ons, and custom Webpack configurations. You can specify the locations of your story files, add an array of addons you’re using, and customize the Webpack and Babel configs as needed.
  2. preview.js(or preview.tsx): used to customize the rendering of your stories. You can globally add decorators and parameters here, affecting all stories. This file is often used for setting up global contexts like themes, internationalization, and configuring the layout or backgrounds for your stories.

02

One of the best integrations (found from Jeff L’Heureux) allows you to use your own Sitecore context mock and also any placeholder to use any of your components (see the lines below decorators).

import React from 'react';
import {LayoutServicePageState, SitecoreContext} from '@sitecore-jss/sitecore-jss-nextjs';
import {componentBuilder} from 'temp/componentBuilder';
import type { Preview } from '@storybook/react';
import 'src/assets/main.scss';
export const mockLayoutData = {
  sitecore: {
    context: {
      pageEditing: false,
      pageState: LayoutServicePageState.Normal,
},
    setContext: () =>{
    // nothing
},
    route: null,
},
};
const preview: Preview = {
  parameters: {
    actions: { argTypesRegex: '^on[A-Z].*'},
    controls: {
      matchers: {
        color: /(background|color)$/i,
        date: /Date$/,
      },
    },
},
  decorators: [
    (Story) =>(
      <SitecoreContext
              componentFactory={componentBuilder.getComponentFactory({ isEditing: mockLayoutData.sitecore.context.pageEditing})}
              layoutData={mockLayoutData}
      >
        <Story />
      </SitecoreContext>
    ),
  ],
};

export default preview;

It is important to understand that getServerSide/getStaticProps are not executed when using Storybook. You are responsible for providing all the required data needed as well as context, so you need to wrap your story or component.

Component level fetching works nicely with Sitecore headless components using MSW – you can just mock fetch API to return the required data from inside of the story file.

Useful tips for running Storybook for Headless Sitecore

  • use next-router-mock to mock the Nextjs router in Storybook (or upgrade to version 7 with the @storybook/nextjs)
  • exclude stories from the componentFactory / componentBuilder file.
  • make sure to run npm run bootstrap before starting storybook or adding it to the package.json, something like: "prestorybook": "npm-run-all --serial bootstrap" – when the storybook script is invoked, this prestorybook will automatically run just before, using a default NPM feature.

Conclusion

The Storybook integration into a Sitecore headless project will require you to invest some time digging into it, but offers numerous benefits, including improved component visualization, and isolation for development and testing.

Using conditions with XM Cloud Form Builder

If you follow the release news from Sitecore, you’ve already noted the release of highly awaited XM Cloud Forms, which however did not have a feature of adding conditional logic. Until now.

The good news is that now we can do it, and here’s how.

See it in action

Imagine you have a registration form and want to ask if your clients want to receive the email. To do so you add two additional fields at the bottom:

  • a checkbox for users to define if they want to receive these emails
  • a dedicated email input field for leaving the email address

Obviously, you want to validate the email address input as normal and make it required. At the same time you want this field to only play on with a checkbook being checked, otherwise being ignored and ideally hidden.

Once we set both Required and Hidden properties, there is a hint appears saying that we cannot have them both together as it simply creates a deadlock – something I mentioned in my earlier review of XM Cloud Forms builder

01

So, how we achieve that?

From now on, there is an additional Logic tab that you can leverage to add some additional logic to your forms.

02

Let’s see what can do with it:

  • you can add new pieces of logic
  • apply multiple conditions within the piece of logic and define if all must comply or just any of them (logical “AND” and “OR”)
  • you can create groups of conditions to combine multiple OR logic clauses to work against the same single AND condition
  • for each condition, select a particular field of application from a dropdown
  • define what requirements you want to meet with it from another dropdown, which is content-specific:
    • strict match
    • begins or ends with
    • contains
    • checked / unchecked
    • etc.
  • not just having multiple conditional logic, you can also have multiple fields logic within a single condition with the same “and / “or”

Once all conditions are met, you execute the desired action against a specified field:

04

Coming back to a form with an optional email subscription defined by a checkbox, I created the following rule:

05

The conditional logic rules engine is very much intuitive and human-readable. I do not even need to explain the above screenshot as it is naturally self-explanatory. So, how does it perform? Let’s run Preview and see it in action.

When running the form as normal, we only see a checkbox disabled by default and can submit it straight away:

06

But checking Email me updates and promotions tick enables Email field, which is Required. The form won’t submit with an invalid email address until the checkbox is checked.

07

My expectation for the conditional logic was applying conditions for multipage forms at the page level. Say, I have a first page having some sort of branching condition, so that if the user meets it – it gets to pages 2 and 3, but if not – only page 4, with both branching ending up at page 5. Unfortunately, I did not find a way of doing that. Hopefully, the development teams will add this in the future, given how progressively and persistently they introduce new features. In any case, what we’ve been given today is already a very powerful tool that allows us to create really complicated steams of input user data.

GraphQL: not an ideal one!

You’ll find plenty of articles about how amazing GraphQL is (including mine), but after some time of using it, I’ve got some considerations with the technology and want to share some bitter thoughts about it.

GraphQL

History of GraphQL

How did it all start? The best way to answer this question is to go back to the original problem Facebook faced.

Back in 2012, we began an effort to rebuild Facebook’s native mobile applications. At the time, our iOS and Android apps were thin wrappers around views of our mobile website. While this brought us close to a platonic ideal of the “write once, run anywhere” mobile application, in practice, it pushed our mobile web view apps beyond their limits. As Facebook’s mobile apps became more complex, they suffered poor performance and frequently crashed. As we transitioned to natively implemented models and views, we found ourselves for the first time needing an API data version of News Feed — which up until that point had only been delivered as HTML.

We evaluated our options for delivering News Feed data to our mobile apps, including RESTful server resources and FQL tables (Facebook’s SQL-like API). We were frustrated with the differences between the data we wanted to use in our apps and the server queries they required. We don’t think of data in terms of resource URLs, secondary keys, or join tables; we think about it in terms of a graph of objects.

Facebook came across a specific problem and created its own solution: GraphQL. To represent data in the form of a graph, the company designed a hierarchical query language. In other words, GraphQL naturally follows the relationships between objects. You can now receive nested objects and return them all in a single HTTPS request. Back in the day, it was crucial for global users not to always have cheap/unlimited mobile tariff plans, so the GraphQL protocol was optimized, allowing only what users needed to be transmitted.

Therefore, GraphQL solves Facebook’s problems. Does it solve yours?

First, let’s recap the advantages

  • Single request, multiple resources: Compared to REST, which requires multiple network requests to each endpoint, GraphQL you can request all resources with a single call.
  • Receive accurate data: GraphQL minimizes the amount of data transferred over the wires, selectively selecting it based on the needs of the client application. Thus, a mobile client with a small screen may receive less information.
  • Strong typing: Every request, input, and response objects have a type. In web browsers, the lack of types in JavaScript has become a weakness that various tools (Google’s Dart, and Microsoft’s TypeScript) try to compensate for. GraphQL allows you to exchange types between the backend and frontend.
  • Better tooling and developer friendliness: The introspective server can be queried about the types it supports, allowing for API explorer, autocompletion, and editor warnings. No more relying on backend developers to document their APIs. Simply explore the endpoints and get the data you need.
  • Version independent: the type of data returned is determined solely by the client request, so servers become simpler. When new server-side features are added to the product, new fields can be added without affecting existing clients.

Thanks to the “single request, multiple resources” principle, front-end code has become much simpler with GraphQL. Imagine a situation where a user wants to get details about a specific writer, for example (name, id, books, etc.). In a traditional intuitive REST pattern, this would require a lot of cross-requests between the two endpoints /writers and /books, which the frontend would then have to merge. However, thanks to GraphQL, we can define all the necessary data in the request, as shown below:

writers(id: "1"){
    id
    name
    avatarUrl
    books(limit: 2){
        name
        urlSlug
    }
}

The main advantage of this pattern is to simplify the client code. However, some developers expected to use it to optimize network calls and speed up application startup. You don’t make the code faster; you simply transfer the complexity to the backend, which has more computing power. Also for many scenarios, metrics show that using the REST API appeared faster than GraphQL.

This is mostly relevant for mobile apps. If you’re working with a desktop app or a machine-to-machine API, there’s no added value in terms of performance.

Another point is that you may indeed save some kilobytes with GraphQL, but if you really want to optimize loading times, it’s better to focus on loading lower-quality images for mobile, as we’ll see, GraphQL doesn’t work very well with documents.

But let’s see what actually is wrong or could be better with GraphQL.

Strongly Typing

GraphQL defines all API types, commands, and queries in the graphql.schema file. However, I’ve found that typing with GraphQL can be confusing. First of all, there is a lot of duplication here. GraphQL defines the type in the schema, however, we need to override the types for our backend (TypeScript with node.js). You have to spend additional effort to make it all work with Zod or create some cumbersome code generation for types.

Debugging

It’s hard to find what you’re looking for in the Chrome inspector because all the endpoints look the same. In REST you can tell what data you’re getting just by looking at the URL:

Devtools 

compared to:

  Devtools

Do you see the difference?

No support for status codes

REST allows you to use HTTP error codes like “404 not found”, “500 server error” and so on, but GraphQL does not. GraphQL forces a 200 error code to be returned in the response payload. To understand which endpoint failed, you need to check each payload. Same applies for Monitoring: HTTP error monitoring is very easy compared to GraphQL because they all have their error code while troubleshooting GraphQL requires parsing JSON objects.

Additionally, some objects may be empty either because they cannot be found or because an error occurred. It can be difficult to distinguish the difference at a glance.

Versioning

Everything has its price. When modifying the GraphQL API, you can make some fields obsolete, but you are forced to maintain backward compatibility. They should still remain there for older clients who use them. You don’t need to support GraphQL versioning, at the price of maintaining each field.

To be fair, REST versioning is also a pain point, but it does provide an interesting feature for expiring functionality. In REST, everything is an endpoint, so you can easily block legacy endpoints for a new user and measure who is still using the old endpoint. Redirects could also simplify versioning from older to newer in some cases.

Pagination

GraphQL Best Practices suggests the following:

The GraphQL specification is deliberately silent on several important API-related issues, such as networking, authorization, and pagination.

How “convenient” (not!). In general, as it turns out, pagination in GraphQL is very painful.

Caching

The point of caching is to receive a server response faster by storing the results of previous calculations. In REST, the URLs are unique identifiers of the resources that users are trying to access. Therefore, you can perform caching at the resource level. Caching is a part of HTTP specification. Additionally, the browser and mobile device can also use this URL and cache the resources locally (same as they do with images and CSS).

In GraphQL this gets tricky because each query can be different even though it’s working on the same entity. It requires field-level caching, which is not easy to do with GraphQL because it uses a single endpoint. Libraries like Prisma and Dataloader have been developed to help with such scenarios, but they still fall short of REST capabilities.

Media types

GraphQL does not support uploading documents to the server, which is used multipart-form-data by default. Apollo developers have been working on a file-uploads solution, but it is difficult to set up. Additionally, GraphQL does not support a media types header when retrieving a document, which allows the browser to display the file correctly.

I previously made a post about the steps one must take in order to upload an image to Sitecore Media Library (either XM Cloud or XM 10.3 or newer) by using Authoring GraphQL API.

Security

When working with GraphQL, you can query exactly what you need, but you should be aware that this comes with complex security implications. If an attacker tries to send a costly request with attachments to overload the server, then may experience it as a DDoS attack.

They will also be able to access fields that are not intended for public access. When using REST, you can control permissions at the URL level. For GraphQL this should be the field level:

user {
    username <-- anyone can see that
    email <-- private field
    post {
      title <-- some of the posts are private
  }
}

Conclusion

REST has become the new SOAP; now GraphQL is the new REST. History repeats itself. It’s hard to say whether GraphQL will just be a popular new trend that will gradually be forgotten, or whether it will truly change the rules of the game. One thing is certain: it still requires some development to obtain more maturity.