Multiple Destination Publishing Using - MIRROR strategy in SDL WEB 8.5

What is Multiple Destination Publishing?

Multiple destination publishing means that the transport service can publish to multiple deployer destinations (if you have more than one deployer).

DEFAULT strategy means that there will be only one deployer only one URL is registered in discovery-service).

MIRROR strategy means that you can have more than one deployer (multiple URLs are registered in discovery-service) and transport service will send the package to all of the deployer URLs register in discovery service.

How to configure Multiple Destination Publishing?

We can do this by defining the strategy for the DeployerCapability which is registered in the Discovery Microservice cd_storage.conf.xml:

If you have set up multiple Content Deployer destinations, do the following in order to setup MIRROR strategy:

1. Ensure that this Role element has a Strategy attribute, set to MIRROR.
2. Inside the Role element, insert a Urls subsection, itself containing two or more Url subelements. Each Url element must have a Value attribute, set to the URL of the destination, and can have a DestinationName attribute, set to the name of this destination

Strategy Attribute 

This Role has one Capability URL (value DEFAULT, to be specified in the Url attribute) 

or multiple ones value MIRROR, to be specified in the URLs subsection. Defaults to DEFAULT is not specified.

Happy Coding and Keep Sharing !!!

Published Summary Alchemy Plugin

In my previous blog, we saw how Published Summary plugin was conceptualized, its use cases and how it works and its different functionalities.

In this Plugin, I wrote the C# code that interacts with Core Service and forwards the JSON response to be consumed on FrontEnd.

Based on the use cases and feature of this plugin I wrote the Endpoints on the following scenarios and provided the JSON.

1. Get All Published Items from Publication, Structure Group, and Folder.
• Let's say if we select Publication then the response will be all Published  Pages, Components, and Categories in that publication.
• If we select this plugin from Structure group, then we will have all the published pages in the structure group and same goes for Folder selection all Published ComponentTemplates and Components.
2. Publish Items that are selected by the user from the Plugin UI.
3. Unpublish the Items that are select by the user from the Plugin UI.
4. Summary Panel, where we'll have a Snapshot of how many items are published in a particular publication, basically a GroupBy of itemsType with the count.
5. And Finally,  Get all Publication and Target Type.

Now, Let's look at the code at High Level

Get All Published Item

The GetAllPublishedItems method is the POST request method and deserialized JSON in C# modal.

JSON format is {'IDs':['tcm:14-65-4']}

 [HttpPost, Route("GetAllPublishedItems")]  
     public object GetAllPublishedItems(TcmIds tcmIDs)  
     {  
       GetPublishedInfo getFinalPublishedInfo = new GetPublishedInfo();  
       var multipleListItems = new List<ListItems>();  
       XmlDocument doc = new XmlDocument();  
       try  
       {  
         foreach (var tcmId in tcmIDs.IDs)  
         {  
           TCM.TcmUri iTcmUri = new TCM.TcmUri(tcmId.ToString());  
           XElement listXml = null;  
           switch (iTcmUri.ItemType.ToString())  
           {  
             case CONSTANTS.PUBLICATION:  
               listXml = Client.GetListXml(tcmId.ToString(), new RepositoryItemsFilterData  
               {  
                 ItemTypes = new[] { ItemType.Component, ItemType.ComponentTemplate, ItemType.Category, ItemType.Page },  
                 Recursive = true,  
                 BaseColumns = ListBaseColumns.Extended  
               });  
               break;  
             case CONSTANTS.FOLDER:  
               listXml = Client.GetListXml(tcmId.ToString(), new OrganizationalItemItemsFilterData  
               {  
                 ItemTypes = new[] { ItemType.Component, ItemType.ComponentTemplate },  
                 Recursive = true,  
                 BaseColumns = ListBaseColumns.Extended  
               });  
               break;  
             case CONSTANTS.STRUCTUREGROUP:  
               listXml = Client.GetListXml(tcmId.ToString(), new OrganizationalItemItemsFilterData()  
               {  
                 ItemTypes = new[] { ItemType.Page },  
                 Recursive = true,  
                 BaseColumns = ListBaseColumns.Extended  
               });  
               break;  
             case CONSTANTS.CATEGORY:  
               listXml = Client.GetListXml(tcmId.ToString(), new RepositoryItemsFilterData  
               {  
                 ItemTypes = new[] { ItemType.Category },  
                 Recursive = true,  
                 BaseColumns = ListBaseColumns.Extended  
               });  
               break;  
             default:  
               throw new ArgumentOutOfRangeException();  
           }  
           if (listXml == null) throw new ArgumentNullException(nameof(listXml));  
           doc.LoadXml(listXml.ToString());  
           multipleListItems.Add(TransformObjectAndXml.Deserialize<ListItems>(doc));  
         }  
         return getFinalPublishedInfo.FilterIsPublishedItem(multipleListItems).SelectMany(publishedItem => publishedItem, (publishedItem, item) => new { publishedItem, item }).Select(@t => new { @t, publishInfo = Client.GetListPublishInfo(@t.item.ID) }).SelectMany(@t => getFinalPublishedInfo.ReturnFinalList(@t.publishInfo, @t.@t.item)).ToList();  
       }  
       catch (Exception ex)  
       {  
         throw new HttpResponseException(Request.CreateErrorResponse(HttpStatusCode.InternalServerError, ex.Message));  
       }  
     }  

Publish and UnPublish Items

The Publish and UnPublish method is the POST request method and deserialized JSON  into C# Model.

{"TcmIds":[{Id:"tcm:14-65-4",Target:"staging"},{Id:"tcm:14-77-64",Target:"staging"}]}

  #region Publishe the items  
     /// <summary>  
     /// Publishes the items.  
     /// </summary>  
     /// <param name="IDs">The i ds.</param>  
     /// <returns>System.Int32.</returns>  
     /// <exception cref="ArgumentNullException">result</exception>  
     [HttpPost, Route("PublishItems")]  
     public string PublishItems(PublishUnPublishInfoData IDs)  
     {  
       try  
       {  
         var pubInstruction = new PublishInstructionData()  
         {  
           ResolveInstruction = new ResolveInstructionData() { IncludeChildPublications = false },  
           RenderInstruction = new RenderInstructionData()  
         };  
         PublishTransactionData[] result = null;  
         var tfilter = new TargetTypesFilterData();  
         var allPublicationTargets = Client.GetSystemWideList(tfilter);  
         if (allPublicationTargets == null) throw new ArgumentNullException(nameof(allPublicationTargets));  
         foreach (var pubdata in IDs.IDs)  
         {  
           var target = allPublicationTargets.Where(x => x.Title == pubdata.Target).Select(x => x.Id).ToList();  
           if (target.Any())  
           {  
             result = Client.Publish(new[] { pubdata.Id }, pubInstruction, new[] { target[0] }, PublishPriority.Normal, null);  
             if (result == null) throw new ArgumentNullException(nameof(result));  
           }  
         }  
         return "Item send to Publish";  
       }  
       catch (Exception ex)  
       {  
         throw new HttpResponseException(Request.CreateErrorResponse(HttpStatusCode.InternalServerError, ex.Message));  
       }  
     }  
     #endregion  
     #region Unpublish the items  
     /// <summary>  
     /// Uns the publish items.  
     /// </summary>  
     /// <param name="IDs">The i ds.</param>  
     /// <returns>System.Object.</returns>  
     /// <exception cref="ArgumentNullException">result</exception>  
     /// <exception cref="HttpResponseException"></exception>  
     [HttpPost, Route("UnPublishItems")]  
     public string UnPublishItems(PublishUnPublishInfoData IDs)  
     {  
       try  
       {  
         var unPubInstruction = new UnPublishInstructionData()  
         {  
           ResolveInstruction = new ResolveInstructionData()  
           {  
             IncludeChildPublications = false,  
             Purpose = ResolvePurpose.UnPublish,  
           },  
           RollbackOnFailure = true  
         };  
         PublishTransactionData[] result = null;  
         var tfilter = new TargetTypesFilterData();  
         var allPublicationTargets = Client.GetSystemWideList(tfilter);  
         if (allPublicationTargets == null) throw new ArgumentNullException(nameof(allPublicationTargets));  
         foreach (var tcmID in IDs.IDs)  
         {  
           var target = allPublicationTargets.Where(x => x.Title == tcmID.Target).Select(x => x.Id).ToList();  
           if (target.Any())  
           {  
             result = Client.UnPublish(new[] { tcmID.Id }, unPubInstruction, new[] { target[0] }, PublishPriority.Normal, null);  
             if (result == null) throw new ArgumentNullException(nameof(result));  
           }  
         }  
         return "Items send for Unpublish";  
       }  
       catch (Exception ex)  
       {  
         throw new HttpResponseException(Request.CreateErrorResponse(HttpStatusCode.InternalServerError, ex.Message));  
       }  
     }  
     #endregion  

Summary Panel

The GetSummaryPanelData method is the POST request method and deserialized JSON in C# modal.

JSON format is {'IDs':['tcm:14-65-4']}

 #region Get GetSummaryPanelData  
     /// <summary>  
     /// Gets the analytic data.  
     /// </summary>  
     /// <returns>System.Object.</returns>  
     [HttpPost, Route("GetSummaryPanelData")]  
     public object GetSummaryPanelData(TcmIds tcmIDs)  
     {  
       try  
       {  
         GetPublishedInfo getFinalPublishedInfo = new GetPublishedInfo();  
         var multipleListItems = new List<ListItems>();  
         XmlDocument doc = new XmlDocument();  
         foreach (var tcmId in tcmIDs.IDs)  
         {  
           var listXml = Client.GetListXml(tcmId.ToString(), new RepositoryItemsFilterData  
           {  
             ItemTypes = new[] { ItemType.Component, ItemType.ComponentTemplate, ItemType.Category, ItemType.Page },  
             Recursive = true,  
             BaseColumns = ListBaseColumns.Extended  
           });  
           if (listXml == null) throw new ArgumentNullException(nameof(listXml));  
           doc.LoadXml(listXml.ToString());  
           multipleListItems.Add(TransformObjectAndXml.Deserialize<ListItems>(doc));  
         }  
         List<Item> finalList = new List<Item>();  
         foreach (var publishedItem in getFinalPublishedInfo.FilterIsPublishedItem(multipleListItems))  
           foreach (var item in publishedItem)  
           {  
             var publishInfo = Client.GetListPublishInfo(item.ID);  
             foreach (var item1 in getFinalPublishedInfo.ReturnFinalList(publishInfo, item)) finalList.Add(item1);  
           }  
         IEnumerable<Analytics> analytics = finalList.GroupBy(x => new { x.PublicationTarget, x.Type }).Select(g => new Analytics { Count = g.Count(), PublicationTarget = g.Key.PublicationTarget, ItemType = g.Key.Type, });  
         var tfilter = new TargetTypesFilterData();  
         List<ItemSummary> itemssummary = getFinalPublishedInfo.SummaryPanelData(analytics, Client.GetSystemWideList(tfilter));  
         return itemssummary;  
       }  
       catch (Exception ex)  
       {  
         throw new HttpResponseException(Request.CreateErrorResponse(HttpStatusCode.InternalServerError, ex.Message));  
       }  
     }  
     #endregion  

Get All Publication and Target Type

 #region Get list of all publications  
     /// <summary>  
     /// Gets the publication list.  
     /// </summary>  
     /// <returns>List&lt;Publications&gt;.</returns>  
     [HttpGet, Route("GetPublicationList")]  
     public List<Publications> GetPublicationList()  
     {  
       GetPublishedInfo getPublishedInfo = new GetPublishedInfo();  
       XmlDocument publicationList = new XmlDocument();  
       PublicationsFilterData filter = new PublicationsFilterData();  
       XElement publications = Client.GetSystemWideListXml(filter);  
       if (publications == null) throw new ArgumentNullException(nameof(publications));  
       List<Publications> publicationsList = getPublishedInfo.Publications(publicationList, publications);  
       return publicationsList;  
     }  
     #endregion  
     #region Get List of all publication targets  
     /// <summary>  
     /// Gets the publication target.  
     /// </summary>  
     /// <returns>System.Object.</returns>  
     [HttpGet, Route("GetPublicationTarget")]  
     public object GetPublicationTarget()  
     {  
       var filter = new TargetTypesFilterData();  
       var allPublicationTargets = Client.GetSystemWideList(filter);  
       if (allPublicationTargets == null) throw new ArgumentNullException(nameof(allPublicationTargets));  
       return allPublicationTargets;  
     }  
     #endregion  

You can download the code from the Github.

Happy Coding and Keep Sharing !!!

SDL Web Hackathon 2018 - Published Summary Plugin

Published Summary Alchemy-Plug In project has been conceptualized and initiated for SDL Tridion Developer Summit, Amsterdam Hackathon for the year 2018.

The entry for Hackathon has been entered with the team name - "CB Ke Cheete" and for the project "Published Summary" Alchemy plugin.

Team Members 
The team - "CB ke Cheete" - having meanings as "Cheetahs of CB", comprises of three young and dynamic Content Bloom professionals. The team has following members:

• Pankaj Gaur - Director, Content Bloom | SDL Certified Dev and BA | SDL Tridion/Web MVP
• Hem Kant - Consultant, Content Bloom | SDL Certified Dev and BA | SDL Web MVP
• Priyank Gupta - Consultant, Content Bloom | SDL Certified Dev

This plugin is intended to do the following:

1. Get all items within a publication, folder or structure group published to one or more Publishing Target Types.
• Let's say if we select Publication then the response will be all Published  Pages, Components, and Categories in that publication.
• If we select this plugin from Structure group, then we will have all the published pages in the structure group and same goes for Folder selection all Published ComponentTemplates and Components.
2. Republish, Unpublish and open a specific item.
3. Republish or Unpublish multiple items on their respective Publishing Target Types.
4. Export in CSV.
5. Filter based on Publishing Target Types, Item Type (Component, Pages etc.), and Published Date Range.
6. Sorting based on Title, Published Date, Published By, Targets, and TCM URIs.
7. Searching.
8. Summary of published building blocks across all publishing target types.

The Published Summary Alchemy Plug-in can help in following scenarios:

1. You want to export a list of all published items from Tridion for a specific website (or for a specific structure group/folder within a website) in CSV format.
2. You want to see/export all published items from Tridion for a specific Publishing Target Type and compare among all publishing target types for a specific CMS instance.
3. You need to know, what all need to be published from Tridion in order to make a specific website up and running similar to an existing website.
4. You need to know a summary of "how many" specific items are published from Tridion to individual publishing target types.
5. You need to know about "delta" of published items across publishing target types of a CMS instance.
6. You need to sync your non-live websites with the live websites in terms of content managed from Tridion.

Published Summary

Happy Coding and Keep Sharing !!!

Client Side Caching in SDL DXA using Redis

What is Redis?

Redis (Remote Dictionary Server)  is an in-memory data structure store, used as a database, cache and message broker. It supports data structures such as strings, hashes, lists, sets, sorted sets with range queries, bitmaps, hyperloglogs and geospatial indexes with radius queries.

Redis is good for cache, but it's much more than just a cache. It's high speed fully in-memory database.

Not just a cache:-

• In memory key-value storage.
• Support multiple datatypes (strings, hashes, lists, sets, sorted sets, bitmaps, and hyperloglogs).
• It provides an ability to store cache data into physical storage (if needed).
• Support the pub-sub model.
• Redis cache provides replication for high availability (master/slave).

SDL DXA comes up with the pre-defined configuration of Redis for client-side caching to improve the performance of Web-application.

Pre-requisites:-

• SDL Web 8.5
• SDL DXA 
• Redis
• Redis Desktop Manager
• Redis Server and Redis CLI
• And If you want to run this on docker run docker pull redis command. Check out the official Redis repository on the docker 

Installing the Redis

• Download the Redis and extract the zip file in a directory at any location.e;g in c:\redis

  

  Redis

    Next is open the command prompt and run following command

• redis-server.exe to start the Redis server

  

  Redis server

• redis-cli.exe to start the command line interface. redis-cli is the Redis command line interface, a simple program that allows to send commands to Redis and read the replies sent by the server, directly from the terminal.

  

  Redis CLI

Next, is configure DXA to use Redis for client-side caching go to DXA web.config and navigate to <sdl.web.delivery>  node and there we already have Redis configuration setup we just need to update the cacheName. You set the policy of expiry as per the requirement.

Redis Configuration in DXA Web.config

Restart the IIS and spin the DXA Website, the site will trigger the page caching and it will add it to the Redis database.

To test the data is saved in Redis or not install Redis Desktop manager and refresh the page to load the data. In the below image you see data is cached in the Redis.

Redis Desktop Manager

Happy Coding and Keep Sharing!!!

Installing SDL Content Delivery Microservices in Docker

In the last couple of blogs, we learned how to use Docker with SDL DXA Html-Theming and SDL DXA. Today we are going to setup Microservices in Docker. The microservices, also called Content Interaction Services, are the server-side components of Content Delivery. The Microservices is based on Java Spring-boot which uses embedded Tomcat to run the Services.

To learn more about SDL DXA, DXA Html-theming and SDL WEB with docker, please follow below links.

1. SDL WEB, Docker and Cloud Containerization
2. Run DXA in Docker
3. DXA theming in Docker

After this, we can say that we managed to run SDL Content Delivery Environment in Docker.

I downloaded the latest CD HotFix from SDL FTP. In this blog, we are going to Install Content and Context Microservices in docker.

Steps:-

1. Download the latest CD HotFixes.
2. Create a new Folder and copy the Content standalone folder in it.
3. Configure cd_Storage_conf.xml file.
4. Copy License file in the config folder.
5. Update Logback.xml
6. Create Docker File in the root.

Docker File

1:  FROM java:8  
2:  COPY Contentstandalone /  
3:  RUN chmod +x /bin/start.sh  
4:  CMD bash -C '/bin/start.sh'  
5:  EXPOSE 8091  

Next, is we need to BUILD the Docker container.

 docker build -t content-service-staging .  

If everything goes well and it builds successfully next is RUN it.

 docker run -p 8091:8091 content-service-staging  

Below is the output of the above command.

Build Container 

Run the docker container 

Let's browse the Content service

Content Microservice running in Docker

SDL Content Microservice is up and running. Similarly, I followed the same steps in order to install the Context Microservice.

Context Microservice

Context Microservice is up and running in Docker

So, We've Dockerised SDL Content and Context Microservices, it was quick and simple most of the changes/configuration we did is the standard part of the installation of the SDL Microservices.

Happy Coding and Keep Sharing !!!!

Scaling SDL WEB 8.5 Installing CM and Publisher on Dedicated Machines

One of the biggest advantages of using SDL WEB (Tridion) is its scalability features. Last week, I was working on a task where I need to implement/Configure highly scalable SDL WEB 8.5 infrastructure.

Today, we are going to see one of them and that is "How to install CM and Publishers on Dedicated machines" to improve publishing efficiency.

Step I followed to configure CME and Publisher on a dedicated machine.

1. Install all the core components on your content manager server
2. Fill all the Database/ Website/ Topology related details.
3. Once your setup is completed and you are able to browse CME go to windows service and disable the following services from content manager server. 
1. Tridion Content Distributor Transport Service
2. Tridion Content Manager Publisher Service
3. This will stop the content manager server from doing the publishing

Set up Publisher servers

1. Run the installer again on Publisher servers.
2. Fill all the DB details and user accounts details.
3. Transfer the cd_transport_conf.xml file from Content Manager Server to the same location on the new Publisher Servers.
4. On Publisher servers we need to stop/disable the following services.
1. Tridion Content Manager Workflow Agent Service
2. Tridion Content Manager Service Host Service
3. Tridion Content Manager Search Indexer
4. Tridion Content Manager Batch Processor Service
5. Restart the following services 
1. Tridion Content Distributor Transport Service
2. Tridion Content Manager Publisher Service

Now, new publisher servers are ready to publish and transport the package to the deployers. We can scale this further depending upon the requirement. 

Happy Coding and Keep Sharing !!! 

SDL Web, Docker and Cloud Containerization

What are containers

•       Containers are a way to package software in a format that can run isolated on a shared operating system.

•       A container wraps an application in a complete package that contains everything to run the application – code, libraries, dependencies, runtimes.

•       A container image is a lightweight, stand-alone, executable package of a piece of software that includes everything needed to run it: code, runtime, system tools, system libraries, settings. Available for both Linux and Windows-based apps, the containerized software will always run the same, regardless of the environment

•       Containers isolate software from its surroundings, for example, differences between development and staging environments and help reduce conflicts between teams running different software on the same infrastructure.

What is Docker

•       Docker is a leading platform for Containerization of the application

•       Docker allows you to package all pieces of your application into a single container so that your application will run same everywhere without worrying about the environment dependency and complexities.

•       Docker is a tool that is designed to benefit both developers and system administrators, making it a part of many DevOps.

•       For developers, it means that they can focus on writing code without worrying about the system that it will ultimately be running on.

•       In a way, Docker is a bit like a virtual machine. But unlike a virtual machine, rather than creating a whole virtual operating system.

•       It is a tool designed to make it easier to create, deploy, and run applications by using containers. Containers allow a developer to package up an application with all of the parts it needs, such as libraries and other dependencies, and ship it all out as one package.

Containers Vs Virtual Machine

•       Containers are an abstraction at the app layer that packages code and dependencies together. 

•       Containers take up less space than VMs

•       Virtual machines (VMs) are an abstraction of physical hardware turning one server into many servers

•       Each VM includes a full copy of an operating system. VMs can also be slow to boot

Docker Advantages

•          Automation of repetitive and tedious deployment tasks.

•          Faster onboarding of new resource.

•          Faster shipment of new features and fixes without downtime.

•          Easy and real-time scaling of application.

•          Easy Distribution, Share, and Configuration of applications.

•          High security out of the box.

•          Improve efficiency of DevOps.

•          Modernize traditional apps faster and without investing huge without going through complete SDLC.

Docker Disadvantages

•          Better performance than Virtual Machines but slower than Bare-Metal.

•          Isolation and security are lesser as compared to the Virtual Machines.

•          Storing data such that it is available even after the container is shut down is quite complicated.

•       Not all applications benefit from containers – Cannot be blindly adopted for each and every application.

 Comparison between AWS, Google, and Azure Cloud Containerization

AWS Cloud – The AWS Cloud provides Amazon EC2 Container Services (Amazon ECS) which support container application and so let you run your Docker containers on a managed cluster of Amazon EC2 instances.

Google Cloud – Google offers Google Container Engine which lets you run Docker containers on Google Cloud.

MS Azure – Microsoft also offers Azure container service to allow deployment, management, and execution of Container applications.

Docker and SDL WEB 

With the release of SDL Web, the Dockerization becomes quite easy

SDL Web is quite Docker friendly because of the following:

·         Micro-service based architecture.

·         Parameterized config files.

             ·       License-free web application.

SDL WEB – Advantages of using Docker

•         Great time and effort saving for DevOps
•         Complexities of SDL CD Deployment is encapsulated and allows the non-Tridion experts to do the deployment.
•         Scaling of servers is quite easy.
•         The overall cost of resources will be optimized as compared to Virtual Machines.

Quick Links to learn more on SDL WEB with Docker.

http://hem-kant.blogspot.com/2017/08/how-to-customize-dxa-theme-step-2.html
http://hem-kant.blogspot.com/2017/11/storage-extension-using-elasticsearch-and-docker.html
http://hem-kant.blogspot.com/2018/03/run-sdl-dxa-inside-windows-docker.html

   Happy Coding and Keep Sharing !!!!