Extending Content Delivery Storage in SDL WEB 8.5 - Part 2

In the previous blog, we set up the project and configured all the required config and based on that we published a DCP and saw custom logs are available which means everything is working fine.

Today we are going to read the content of DCP and will see how Storage Extension work when we publish, re-publish and un-publish an item.

The Content is available in the form of Bytes and we need to convert that, below is the code snippet that will allow you to access the DCP content in your custom code.

public void create(ComponentPresentation itemToCreate, ComponentPresentationTypeEnum componentPresentationType)

throws StorageException

{

          super.create(itemToCreate,componentPresentationType);

          log.debug("Custom Code Create Method COMID :- " + itemToCreate.getComponentId());

          log.debug("Custom Storage create Method:- "+ itemToCreate.getContent());

          log.debug(String.valueOf(itemToCreate.getComponentId()));

          byte[] dcpComponent= itemToCreate.getContent();

          try

          {

                  componentPresentation = new String(dcpComponent,"UTF-8");

                  log.debug("Custom Storage create DCP :- " + componentPresentation);

          }

          catch (UnsupportedEncodingException ex)

          {

                  log.error("Custom Storage create Unsupported " + ex);

          }

}

DCP content

Publish a new component and in the deployer log file, you will see the DCP content but what will happen when we re-publish the same component or un-publish this. How we are going to manage that in the custom code, will see in the next blog, until then.

Happy Coding and Keep Sharing !!!

Custom Deployer Extension in SDL WEB 8.5 Part - 1

Today, we are going to see how to extend the default SDL Tridion deployment functionality, using Custom Deployer Extension. Before we move forward and start building/set up the project let's first understand the use of Deployer Extension. When a user publishes content, the Content Deployer unpacks the incoming Transport Package and processes its transport instructions and we can extend the default behavior of the Content Deployer by creating a custom Module and adding it to a Step, or by extending an existing Module.

Deployer Extension:- 

• Deployer Extension is used to inject additional functionality in the default SDL Tridion deployment process. 
• If you want to implement your custom logic/data. "Event System might be useful here as well 🤔 We need to be absolutely sure about it".
• If you want to use some specific data/info which is only available at the time of deployment.   
• Based on JAVA.
• You can extend the default behavior of the Content Deployer by creating a custom Module and adding it to a Step, or by extending an existing Module.

In order to set up the JAVA project for the Deployer Extension, I have used Eclipse IDE, default JARs provided by SDL and JAVA 8. 

First, we need to configure the JAVA project using Eclipse.

1. Click on File --> New --> Other Project and then select Java Project and click Next.

Create New JAVA Project

2. Enter the Project Name

Enter the Project Name.

3. Create a new folder called "lib" which will contain all the JARs.

Create new Folder lib

4. Copy all the JARs in lib, you can find all the default JAR files in the deployer microservice lib folder. Once copied then we need to Add the Build PATH.

Add JARs Build Path

5 Next, is create a Java package and then create a CLASS file.

Create a new Java Package

Create a New Class

6. Start Importing all the Namespace that's required in order to extend the deployer.

Tridion Default Namespaces.

7. Next is we must implement the abstract method  Module.process(TransportPackage)

 package com.tridion.deployer.extension;  
 import org.slf4j.Logger;  
 import org.slf4j.LoggerFactory;  
 import com.tridion.configuration.Configuration;  
 import com.tridion.configuration.ConfigurationException;  
 import com.tridion.deployer.Module;  
 import com.tridion.deployer.ProcessingException;  
 import com.tridion.deployer.Processor;  
 import com.tridion.transport.transportpackage.TransportPackage;  
 public class PurgeCache extends Module {  
      protected static Logger log = LoggerFactory.getLogger(PurgeCache.class);  
        
      public PurgeCache(Configuration config, Processor processor) throws ConfigurationException  
      {  
           super(config,processor);  
      }  
      // This is called once for each Transport Package that is deployed.  
      @Override  
      public void process(TransportPackage data) throws ProcessingException   
      {  
            log.debug("This is custom logs");  
            int publicationId =data.getProcessorInstructions().getPublicationId().getItemId();  
            log.debug("PublicationId : " + String.valueOf(publicationId));  
      }  
 }  

That's it Export this into a JAR and next is the configuration in the deployer_config.xml of Deployer Microservice. "Don't forget to restart the service".

Click Export

Click Jar file and then Next

Custom Deployer Extension JAR file is ready

Finally, we need to deploy the custom JAR in the deployer microservice lib folder and open the deployer_config.xml file in your favorite editor add the below configuration, save the file Don't forget to take a backup before you start editing.

 <Step Factory="com.sdl.delivery.deployer.steps.TridionExecutableStepFactory" Id="PurgeCacheSteps">  
   <Module Type="PurgeCache" Class="com.tridion.deployer.extension.PurgeCache">  
   </Module>  
 </Step>  

 We also need to edit the logback.xml

 <logger name="com.tridion.deployer.extension">  
     <appender-ref ref="rollingDeployerLog"/>  
 </logger>  

Let's do some publishing and see if everything is working fine. Ideally, we should be able to see the logs entries. If you see the custom logs in the log file that means your custom code is interesting with default deployment process and you've built the extension successfully.

Log File

In the next blog, we will use the deployer extension with a very interesting use-case, until then.

Happy Coding and Keep Sharing !!!

Configure Workers, ActiveMQ and Redis for Scalable Deployment in SDL WEB 8.5

This is in continuation of my previous blogs where we discussed how to install CM and publisher on a dedicated machine and then, we saw how we can implement scale out content deployment using workers, ActiveMQ and Redis. Today, we are going to see how to configure Workes, ActiveMQ and Redis.

To read more about last two blogs in this series:-

1. Scaling SDL WEB 8.5 Installing CM and Publisher on Dedicated Machines
2. Scaling Deployers in SDL WEB 8.5

In order to configure the scalable content deployment, we need Deployer(Endpoint) and Deployer-Worker.

Deployer Installation Media

Pre-requisites, we need to install ActiveMQ and Redis

1. How to install Redis please check my previous blogs.
2. Download the Apache ActiveMQ
1. Unzip the package in a suitable location and from command line execute command activemq start.
2. Open your browser and navigate to the Admin Console at http://localhost:8161

Next step is to configure Deployer-Endpoint and Deployer-Workers Microservices.

1. Both these services are required to update the deployer_conf.xml and Deployer-Workers required cd_storage_conf.xml.
2. In the Deployer-Endpoint installation media open deployer_conf.xml.
1. Here we need to update the <BinaryStorage> node and point this to Redis data store.

   

   Configure Redis in Deployer-Endpoint

2. we need to configure the <State> node. so that workers know where to update the status of a job and the endpoint knows from where to get the status of the job.

   

   Configure State

3. Next, is we need to configure ActiveMQ, we also need to make sure that all the other <Queue> entries related to FileSystem are commented out.

   

   Configure Apache ActiveMQ JMS

4. Save and close the deployer-conf.xml file and use this same file for Workers as well.
5. Next step is to configure the cd_storage_conf.xml for Deployer-Workers only.
1. Here we need to update the <Storage> node with the Broker Database details. The worker will deploy the content on this DB.
2. Update the License file path.
3. Save and close the file and we can use this same file on the second Deployer-Worker.
6. Now, we need to configure the ports for Deployer-Endpoint and Deployer-Workers.
7. Last run the installService.ps1 from the Deployer-Endpoint folder and from Deployer-Worker. You need to run the command more than once depending upon the number of Deployer-Workers you want.

   

   Deployer-Endpoint and Deployer-Workers are installed  

8. We have all the services configure and up-running.
9. The final step is to register your Deployer (Endpoint) with your Discovery Service and run the discovery-registration.jar.

Happy Coding and Keep Sharing

Scaling Deployers in SDL WEB 8.5

This is in continuation of my previous post where we discussed how to set up Publisher and CM on a dedicated machine to improve the publishing efficiency click here to Read More.

Today we are going to see how to scale out Deployers, using multiple deployers and workers.

High-Level Achrciture diagram 

Steps:

1. Content is passed to the Deployer after a user Publishes an item in the CME.
2. The Deployer Endpoint passes the Transport Package (.zip file) to the defined Binary Storage (File System or Redis Database). We are using Redis right now.
3. The Deployer Endpoint also passes the item to the Queue in ActiveMQ (JMS).
4. ActiveMQ triggers an event that informs the Worker Deployers that a new package has been received.
5. The first available Worker Deployer picks up the job from the Queue and contacts the Binary Storage to get the respective Transport Package.
6. After rendering the Transport Package the Worker Deployer passes the item to the Broker Database.
7. The Deployer then gets the status of the job from the Broker Database, which is updated by the Worker Deployer responsible for that job.

In the next blog, we'll see how to configure Workers, ActiveMQ and Redis.

Happy Coding and Keep Sharing 

Multiple Destination Publishing Using - MIRROR strategy in SDL WEB 8.5

What is Multiple Destination Publishing?

Multiple destination publishing means that the transport service can publish to multiple deployer destinations (if you have more than one deployer).

DEFAULT strategy means that there will be only one deployer only one URL is registered in discovery-service).

MIRROR strategy means that you can have more than one deployer (multiple URLs are registered in discovery-service) and transport service will send the package to all of the deployer URLs register in discovery service.

How to configure Multiple Destination Publishing?

We can do this by defining the strategy for the DeployerCapability which is registered in the Discovery Microservice cd_storage.conf.xml:

If you have set up multiple Content Deployer destinations, do the following in order to setup MIRROR strategy:

1. Ensure that this Role element has a Strategy attribute, set to MIRROR.
2. Inside the Role element, insert a Urls subsection, itself containing two or more Url subelements. Each Url element must have a Value attribute, set to the URL of the destination, and can have a DestinationName attribute, set to the name of this destination

Strategy Attribute 

This Role has one Capability URL (value DEFAULT, to be specified in the Url attribute) 

or multiple ones value MIRROR, to be specified in the URLs subsection. Defaults to DEFAULT is not specified.

Happy Coding and Keep Sharing !!!