OutScale Publishing in SDL Tridion

This was my topic in this year's SDL Tridion DX DEV India summit. We recently upgraded one of our client's system from SDL Tridion 2013 to SDL WEB 8.5, with more than 700 Websites and 200K items to publish, We need to build a system that supports the above scenarios with the high-performance rate.

So to maintain scalability we decided to use Deployer-Endpoint and Deployer-Worker rather than going with Deployer-Combined, we also used REDIS for Binary Storage and ActiveMQ for Massage queue/Notifications. Below is the Architecture Diagram.

Let understand the difference between Split Deployer and Deployer Combined

• Deployer Endpoint:- Receive the package sent by the Transport Service and index it in Binary Storage. 
• Deployer Worker:-  Process the package and saves the content in Broker DB or on the File System depending upon the configuration.  

Next, let's see how this architecture works step by step.

• Content is passed to the Deployer after a user Publishes an item in the CME.
• The Deployer Endpoint passes the Transport Package (.zip file) to the defined Binary Storage (File System or Redis Database). We are using Redis right now.
• The Deployer Endpoint also passes the item to the Queue in ActiveMQ (JMS).
• ActiveMQ triggers an event that informs the Worker Deployers that a new package has been received.
• The first available Worker Deployer picks up the job from the Queue and contacts the Binary Storage to get the respective Transport Package.
• After rendering the Transport Package the Worker Deployer passes the item to the Broker Database.
• The Deployer then gets the status of the job from the Broker Database, which is updated by the Worker Deployer responsible for that job.

Below is the screenshot of Redis Consumes the "Transport Package" and in the ActiveMQ we have "Queue/notifications" Enqueued and Dequeued.



 Happy Coding and Keep Sharing !!!

Configure Workers, ActiveMQ and Redis for Scalable Deployment in SDL WEB 8.5

This is in continuation of my previous blogs where we discussed how to install CM and publisher on a dedicated machine and then, we saw how we can implement scale out content deployment using workers, ActiveMQ and Redis. Today, we are going to see how to configure Workes, ActiveMQ and Redis.

To read more about last two blogs in this series:-

1. Scaling SDL WEB 8.5 Installing CM and Publisher on Dedicated Machines
2. Scaling Deployers in SDL WEB 8.5

In order to configure the scalable content deployment, we need Deployer(Endpoint) and Deployer-Worker.

Deployer Installation Media

Pre-requisites, we need to install ActiveMQ and Redis

1. How to install Redis please check my previous blogs.
2. Download the Apache ActiveMQ
1. Unzip the package in a suitable location and from command line execute command activemq start.
2. Open your browser and navigate to the Admin Console at http://localhost:8161

Next step is to configure Deployer-Endpoint and Deployer-Workers Microservices.

1. Both these services are required to update the deployer_conf.xml and Deployer-Workers required cd_storage_conf.xml.
2. In the Deployer-Endpoint installation media open deployer_conf.xml.
1. Here we need to update the <BinaryStorage> node and point this to Redis data store.

   

   Configure Redis in Deployer-Endpoint

2. we need to configure the <State> node. so that workers know where to update the status of a job and the endpoint knows from where to get the status of the job.

   

   Configure State

3. Next, is we need to configure ActiveMQ, we also need to make sure that all the other <Queue> entries related to FileSystem are commented out.

   

   Configure Apache ActiveMQ JMS

4. Save and close the deployer-conf.xml file and use this same file for Workers as well.
5. Next step is to configure the cd_storage_conf.xml for Deployer-Workers only.
1. Here we need to update the <Storage> node with the Broker Database details. The worker will deploy the content on this DB.
2. Update the License file path.
3. Save and close the file and we can use this same file on the second Deployer-Worker.
6. Now, we need to configure the ports for Deployer-Endpoint and Deployer-Workers.
7. Last run the installService.ps1 from the Deployer-Endpoint folder and from Deployer-Worker. You need to run the command more than once depending upon the number of Deployer-Workers you want.

   

   Deployer-Endpoint and Deployer-Workers are installed  

8. We have all the services configure and up-running.
9. The final step is to register your Deployer (Endpoint) with your Discovery Service and run the discovery-registration.jar.

Happy Coding and Keep Sharing

Scaling SDL WEB 8.5 Installing CM and Publisher on Dedicated Machines

One of the biggest advantages of using SDL WEB (Tridion) is its scalability features. Last week, I was working on a task where I need to implement/Configure highly scalable SDL WEB 8.5 infrastructure.

Today, we are going to see one of them and that is "How to install CM and Publishers on Dedicated machines" to improve publishing efficiency.

Step I followed to configure CME and Publisher on a dedicated machine.

1. Install all the core components on your content manager server
2. Fill all the Database/ Website/ Topology related details.
3. Once your setup is completed and you are able to browse CME go to windows service and disable the following services from content manager server. 
1. Tridion Content Distributor Transport Service
2. Tridion Content Manager Publisher Service
3. This will stop the content manager server from doing the publishing

Set up Publisher servers

1. Run the installer again on Publisher servers.
2. Fill all the DB details and user accounts details.
3. Transfer the cd_transport_conf.xml file from Content Manager Server to the same location on the new Publisher Servers.
4. On Publisher servers we need to stop/disable the following services.
1. Tridion Content Manager Workflow Agent Service
2. Tridion Content Manager Service Host Service
3. Tridion Content Manager Search Indexer
4. Tridion Content Manager Batch Processor Service
5. Restart the following services 
1. Tridion Content Distributor Transport Service
2. Tridion Content Manager Publisher Service

Now, new publisher servers are ready to publish and transport the package to the deployers. We can scale this further depending upon the requirement. 

Happy Coding and Keep Sharing !!!