Storage Extension Using ElasticSearch and Docker

The latest release of the ElasticSearch4Web8 framework is much more clean, easy in use, less configuration and setup steps are required.To know more about the previous release refer this link.

In the previous version of this framework, apart from CMS and Deployer setup steps, we were required to host custom WCF based Index service, install and configure ElasticSearch. In this release, I've moved all these on Docker.

So, now instead of downloading code host the Index service and ElasticSearch all we need to do is Docker Pull and you will have all the pre-requisites up and running.

Steps to setup latest version of this framework:-

1. CMS setup
• Copy and paste the templating building block (TBB) to a location on your SDL WEB 8 CM Server
• Upload ESI4TIndexing.Templating.dll TBB to WEB 8 CMS using TcmUploadAssembly.exe
• Create a Component Template with following attributes
• Output Format – XML Fragment
• Add GetComponentAsXML TBB ,Publish Binaries in Package, Link Resolver and Cleanup Template.
2. To setup Elasticsearch run following docker commands.
• docker pull hemkant/elasticsearch .
• docker run -p 9200:9200 -m3gb hemkant/elasticsearch
3. To setup generic, WCF IndexService run following docker commands.
• docker pull hemkant/elasticsearchintegration4web8 .
• docker run -p 83:83 hemkant/elasticsearchintegration4web8
4. To verify all the docker containers are up and running run docker ps command
5. All these containers are available on hub.docker.com
6. The last step is setup Deployer service
• Open the cd_storage_config.xml Storage Configuration file from the /bin/config folder and add following node under the Storages section:
• <StorageBindings><Bundle src="CustomStorageDAOBundles.xml"/></StorageBindings>
• Copy and paste CustomStorageConfig.xml file to change the value of following nodes
• ServiceEndPoint - URL of the IndexService
• TemplateIdToIndex - Tcm Id Of component Template which we have created in step 1 CMS setup. 
• Copy the CustomStorageDAOBundles.xml XML file in the Content Delivery /bin/config folder

That's all we need. We have configured the ElasticSearch integration with SDL WEB  now publish the components and data will be stored in ElasticSearch.

Happy coding and keep sharing !!!

Bypass Model dependency while indexing data in ElasticSearch

Based on the feedback given in SDL Web Dev Summit in Delhi,India.

This is in continuation of my previous blog where we have used ElasticSearch as storage while publishing components data is getting saved in it.

Earlier whenever we need to index any component we first need to create schema model,that creates model based dependency to bypass that we now Serialize the DCP (Dymanic Component preseatation)which is in XML format into JSON and Elastic Search .NET based NEST API has this capability to index JSON directly.

Now, we just need to allow the schema in Dynamic Component Template which we have created to index the data in ElasticSearch.

In NEST API we have used

         var json = @"{

    ""BookName"": ""Book1"",

    ""ISBN"": ""978-3-16-148410-0"",

    ""chapter"" : [

        {

            ""chapter_name"": ""Chapter1"",

            ""chapter_desc"": ""Before getting into computer programming, let us first                     understand computer programs and what they...""

        },

        {

            ""chapter_name"": ""Chapter2"",

            ""chapter_desc"": ""Today computer programs are being used in almost every field,               household, agriculture, medical, entertainment, defense..""

        },

        {

            ""chapter_name"": ""Chapter3"",

            ""chapter_desc"": ""MS Word, MS Excel, Adobe Photoshop, Internet Explorer,                     Chrome, etc., are...""

        },

        {

            ""chapter_name"": ""Chapter4"",

            ""chapter_desc"": ""Computer programs are being used to develop graphics and                   special effects in movie...""

        }

    ]

}";

var indexResponse = client.LowLevel.Index<string>("fromelasticstoweb8", "esnews", json);

And Link Resolver TBB is used to generate the Link e;g i have multimedia component attached ,when you published it will resolve the link below is the DCP XML.

<blog Title='Testing of model' Id='tcm:6-13290' xmlns:xlink='http://www.w3.org/1999/xlink' xmlns:tcm='http://www.tridion.com/ContentManager/5.0'>

  <blogTitle>Testing of model</blogTitle>

  <blogDescription>Remove model dependency</blogDescription>

  <image Title='skyline' Id='tcm:6-291' Path='/DigitalAssets/skyline.jpg' xmlns:tridion='http://www.tridion.com/ContentManager/5.0'></image>

  <publication Id='tcm:0-6-1' Title='400 Example Site'></publication></blog>

Happy coding and keep sharing !!!

ElasticSearch Module in DXA

In my last two blogs we have discussed how to create components by consuming the data from Elasticsearch and then how to index data in Elasticsearch when publishing components.

Today,We are going to discuss and create Elasticsearch module in DXA which will have following features .DXA Modules allows us to extend its Functionality via creating new modules. Here we can create new modules which gives us flexibility & re-usability.

Features and Advantages :- 

• Free text search .
• Tag based filtering.
• Lighting fast response powered by Elasticsearch.
• Pagination and sorting based on fields.
• Article detail page.

In order to use the Elasticsearch module we need to setup the storage extension and Elasticsearch which we already discussed in my previous blog here.
Once that is done using Elasticsearch module in DXA we can search the content from Elasticsearch and render on DXA based application .

Let's discuss the CMS part first.

Elasticsearch Module structure 

1. Define module's folder structure
2. DXA gives us predefined structure where we can keep all the building blocks of the module.
3. For each new DXA module, we need to create a new folder in Modules Folder
4. The name of folder is usually module name.

Next is creating the Schema and Component Templates. I have create dynamic components and index them in Elasticsearch and render on the DXA based web application.

1. Publish all the dynamic components and index them in Elastic.
2. Create new area in you DXA web Application.

   

   DXA Module

3. We can call Elastic RESTFul API to get the data.
4. Data will be returned in the form of JSON.
5. Create model to deserialize the JSON.
6. Let's search for the text SDL.
   
7. Result can be further filtered based on tagging as well.
8. Compare the search result with the data index in Elastic

   

   Elastic Search RESTFul API

.Happy Coding and Keep Sharing !!!