OutScale Publishing in SDL Tridion

This was my topic in this year's SDL Tridion DX DEV India summit. We recently upgraded one of our client's system from SDL Tridion 2013 to SDL WEB 8.5, with more than 700 Websites and 200K items to publish, We need to build a system that supports the above scenarios with the high-performance rate.

So to maintain scalability we decided to use Deployer-Endpoint and Deployer-Worker rather than going with Deployer-Combined, we also used REDIS for Binary Storage and ActiveMQ for Massage queue/Notifications. Below is the Architecture Diagram.

Let understand the difference between Split Deployer and Deployer Combined

• Deployer Endpoint:- Receive the package sent by the Transport Service and index it in Binary Storage. 
• Deployer Worker:-  Process the package and saves the content in Broker DB or on the File System depending upon the configuration.  

Next, let's see how this architecture works step by step.

• Content is passed to the Deployer after a user Publishes an item in the CME.
• The Deployer Endpoint passes the Transport Package (.zip file) to the defined Binary Storage (File System or Redis Database). We are using Redis right now.
• The Deployer Endpoint also passes the item to the Queue in ActiveMQ (JMS).
• ActiveMQ triggers an event that informs the Worker Deployers that a new package has been received.
• The first available Worker Deployer picks up the job from the Queue and contacts the Binary Storage to get the respective Transport Package.
• After rendering the Transport Package the Worker Deployer passes the item to the Broker Database.
• The Deployer then gets the status of the job from the Broker Database, which is updated by the Worker Deployer responsible for that job.

Below is the screenshot of Redis Consumes the "Transport Package" and in the ActiveMQ we have "Queue/notifications" Enqueued and Dequeued.



 Happy Coding and Keep Sharing !!!