OutScale Publishing in SDL Tridion

This was my topic in this year's SDL Tridion DX DEV India summit. We recently upgraded one of our client's system from SDL Tridion 2013 to SDL WEB 8.5, with more than 700 Websites and 200K items to publish, We need to build a system that supports the above scenarios with the high-performance rate.

So to maintain scalability we decided to use Deployer-Endpoint and Deployer-Worker rather than going with Deployer-Combined, we also used REDIS for Binary Storage and ActiveMQ for Massage queue/Notifications. Below is the Architecture Diagram.

Let understand the difference between Split Deployer and Deployer Combined

• Deployer Endpoint:- Receive the package sent by the Transport Service and index it in Binary Storage. 
• Deployer Worker:-  Process the package and saves the content in Broker DB or on the File System depending upon the configuration.  

Next, let's see how this architecture works step by step.

• Content is passed to the Deployer after a user Publishes an item in the CME.
• The Deployer Endpoint passes the Transport Package (.zip file) to the defined Binary Storage (File System or Redis Database). We are using Redis right now.
• The Deployer Endpoint also passes the item to the Queue in ActiveMQ (JMS).
• ActiveMQ triggers an event that informs the Worker Deployers that a new package has been received.
• The first available Worker Deployer picks up the job from the Queue and contacts the Binary Storage to get the respective Transport Package.
• After rendering the Transport Package the Worker Deployer passes the item to the Broker Database.
• The Deployer then gets the status of the job from the Broker Database, which is updated by the Worker Deployer responsible for that job.

Below is the screenshot of Redis Consumes the "Transport Package" and in the ActiveMQ we have "Queue/notifications" Enqueued and Dequeued.



 Happy Coding and Keep Sharing !!!

Configure Workers, ActiveMQ and Redis for Scalable Deployment in SDL WEB 8.5

This is in continuation of my previous blogs where we discussed how to install CM and publisher on a dedicated machine and then, we saw how we can implement scale out content deployment using workers, ActiveMQ and Redis. Today, we are going to see how to configure Workes, ActiveMQ and Redis.

To read more about last two blogs in this series:-

1. Scaling SDL WEB 8.5 Installing CM and Publisher on Dedicated Machines
2. Scaling Deployers in SDL WEB 8.5

In order to configure the scalable content deployment, we need Deployer(Endpoint) and Deployer-Worker.

Deployer Installation Media

Pre-requisites, we need to install ActiveMQ and Redis

1. How to install Redis please check my previous blogs.
2. Download the Apache ActiveMQ
1. Unzip the package in a suitable location and from command line execute command activemq start.
2. Open your browser and navigate to the Admin Console at http://localhost:8161

Next step is to configure Deployer-Endpoint and Deployer-Workers Microservices.

1. Both these services are required to update the deployer_conf.xml and Deployer-Workers required cd_storage_conf.xml.
2. In the Deployer-Endpoint installation media open deployer_conf.xml.
1. Here we need to update the <BinaryStorage> node and point this to Redis data store.

   

   Configure Redis in Deployer-Endpoint

2. we need to configure the <State> node. so that workers know where to update the status of a job and the endpoint knows from where to get the status of the job.

   

   Configure State

3. Next, is we need to configure ActiveMQ, we also need to make sure that all the other <Queue> entries related to FileSystem are commented out.

   

   Configure Apache ActiveMQ JMS

4. Save and close the deployer-conf.xml file and use this same file for Workers as well.
5. Next step is to configure the cd_storage_conf.xml for Deployer-Workers only.
1. Here we need to update the <Storage> node with the Broker Database details. The worker will deploy the content on this DB.
2. Update the License file path.
3. Save and close the file and we can use this same file on the second Deployer-Worker.
6. Now, we need to configure the ports for Deployer-Endpoint and Deployer-Workers.
7. Last run the installService.ps1 from the Deployer-Endpoint folder and from Deployer-Worker. You need to run the command more than once depending upon the number of Deployer-Workers you want.

   

   Deployer-Endpoint and Deployer-Workers are installed  

8. We have all the services configure and up-running.
9. The final step is to register your Deployer (Endpoint) with your Discovery Service and run the discovery-registration.jar.

Happy Coding and Keep Sharing

Scaling Deployers in SDL WEB 8.5

This is in continuation of my previous post where we discussed how to set up Publisher and CM on a dedicated machine to improve the publishing efficiency click here to Read More.

Today we are going to see how to scale out Deployers, using multiple deployers and workers.

High-Level Achrciture diagram 

Steps:

1. Content is passed to the Deployer after a user Publishes an item in the CME.
2. The Deployer Endpoint passes the Transport Package (.zip file) to the defined Binary Storage (File System or Redis Database). We are using Redis right now.
3. The Deployer Endpoint also passes the item to the Queue in ActiveMQ (JMS).
4. ActiveMQ triggers an event that informs the Worker Deployers that a new package has been received.
5. The first available Worker Deployer picks up the job from the Queue and contacts the Binary Storage to get the respective Transport Package.
6. After rendering the Transport Package the Worker Deployer passes the item to the Broker Database.
7. The Deployer then gets the status of the job from the Broker Database, which is updated by the Worker Deployer responsible for that job.

In the next blog, we'll see how to configure Workers, ActiveMQ and Redis.

Happy Coding and Keep Sharing 

Client Side Caching in SDL DXA using Redis

What is Redis?

Redis (Remote Dictionary Server)  is an in-memory data structure store, used as a database, cache and message broker. It supports data structures such as strings, hashes, lists, sets, sorted sets with range queries, bitmaps, hyperloglogs and geospatial indexes with radius queries.

Redis is good for cache, but it's much more than just a cache. It's high speed fully in-memory database.

Not just a cache:-

• In memory key-value storage.
• Support multiple datatypes (strings, hashes, lists, sets, sorted sets, bitmaps, and hyperloglogs).
• It provides an ability to store cache data into physical storage (if needed).
• Support the pub-sub model.
• Redis cache provides replication for high availability (master/slave).

SDL DXA comes up with the pre-defined configuration of Redis for client-side caching to improve the performance of Web-application.

Pre-requisites:-

• SDL Web 8.5
• SDL DXA 
• Redis
• Redis Desktop Manager
• Redis Server and Redis CLI
• And If you want to run this on docker run docker pull redis command. Check out the official Redis repository on the docker 

Installing the Redis

• Download the Redis and extract the zip file in a directory at any location.e;g in c:\redis

  

  Redis

    Next is open the command prompt and run following command

• redis-server.exe to start the Redis server

  

  Redis server

• redis-cli.exe to start the command line interface. redis-cli is the Redis command line interface, a simple program that allows to send commands to Redis and read the replies sent by the server, directly from the terminal.

  

  Redis CLI

Next, is configure DXA to use Redis for client-side caching go to DXA web.config and navigate to <sdl.web.delivery>  node and there we already have Redis configuration setup we just need to update the cacheName. You set the policy of expiry as per the requirement.

Redis Configuration in DXA Web.config

Restart the IIS and spin the DXA Website, the site will trigger the page caching and it will add it to the Redis database.

To test the data is saved in Redis or not install Redis Desktop manager and refresh the page to load the data. In the below image you see data is cached in the Redis.

Redis Desktop Manager

Happy Coding and Keep Sharing!!!