ElasticSearch Integration 4 WEB 8

In my previous post we have discussed on how to create components by consuming data from Elastic Search.Today we are going to discuss how to index data in ElasticSearch when publishing components from WEB8.

To know more about the ElasticSearch advantages and how to setup please click here.

Steps to setup Storage extension and Generic Index service which will read data from and push in to the Elastic search.

1. CMS setup
• Copy and paste the templating building block (TBB) to a location on your SDL WEB 8 CM Server
• Upload ESI4TIndexing.Templating.dll TBB to WEB 8 CMS using TcmUploadAssembly.exe
• Create a Component Template with following attributes
• Output Format – XML Fragment
• Add GetComponentAsXML TBB ,Publish Binaries in Package, Link Resolver and Cleanup Template.
2. Setup ElasticSearch
• Download ES from here and unzip.
• Run bin/elasticsearch (or bin\elasticsearch.bat on Windows).
• Run http://localhost:9200.
• Default Elasticsearch instance is up and running now.
• With the help of Elasticsearch .NET high level client ,we can do all the CRUD operations in Elasticsearch.
3. Setup Deployer service
• Open the cd_storage_config.xml Storage Configuration file from the /bin/config folder and add following node under the Storages section:
• <StorageBindings><Bundle src="CustomStorageDAOBundles.xml"/></StorageBindings>
• Copy and paste CustomStorageConfig.xml file to change the value of following nodes
• ServiceEndPoint - URL of the IndexService
• TemplateIdToIndex - Tcm Id Of component Template which we have created in step 1 CMS setup. 
• Copy the CustomStorageDAOBundles.xml XML file in the Content Delivery /bin/config folder
4. ElasticSearch Index Service
• Copy and paste ElasticSearch IndexService on your server host it in IIS 
• I have used Strongly typed interface to Elasticsearch. Fluent and classic object initializer mappings of requests and responses. Uses and exposes Elasticsearch.Net
• Run Install-Package NEST
• Copy the configuration folder as well .
• You can update the log files path from Logging.config inside the configuration folder
• Update the path of configuration\logging.config file in web.config of index service
5. Here, I have created a default index  fromelasticstoweb8 (should be in all lower case other-wise you will get the error Invalid request) and Type esnews.
6. Once the data is indexed in the ES .
7. REST service will return data in JSON format.
• To get all the records http://localhost:9200/fromelasticstoweb8/_search?q=*:*

To test the index and search services

1. Index service


1. You can use fiddler for debugging 
2. I have created a sample schema article
1. title
2. description
3. imageurl
3. Run index service on fiddler
• http://localhost/Service1.svc/AddDocument
• Input JSON which will generated by custom storage 
• {"ServicePayload":{"DCP":"<esnews Title='About SDL WEB 8' Id='tcm:6-12273' xmlns:xlink='http://www.w3.org/1999/xlink' xmlns:tcm='http://www.tridion.com/ContentManager/5.0'><Title>About SDL WEB 8</Title><Description>SDL recently released the latest version of their web content management SDL WEB 8 an all-in-one global Web Experience Management solution</Description><shortTitle>About SDL WEB 8</shortTitle><authorName>SDL</authorName><Category>WEB 8</Category><relasedDate>2017-02-03T23:44:08.261+05:30</relasedDate><imageUrl>Images/SDL.png</imageUrl><publication Id='tcm:0-6-1' Title='400 Example Site'></publication></esnews>","LanguageInRequest":"en"}}Method Name: AddDocument
4. Execute this request in Fidler and verify the results returned
5. Check in ElasticSearchas well inside the collection\document which you have created and entered in the index service 
6. Result as 0 for success and 1 for failure

•  {"ResponseContext":{"EnvironmentContext":null,"FaultCollection":[]},"ServicePayload":{"ErrorMessage":"","Result":0}}

7. Test this by publishing the Component as well 
8. Log File Where Response Result :0 means successfully published and data is index in ElasticSearch.
2. Search Service
• To get all data http://localhost:9200/fromelasticstoweb8/_search?q=*:*
• To get the data based on the title http://localhost:9200/fromelasticstoweb8/esnews/AVo8K0bDJVbTUFCj_OQW
  
3. Here fromelasticstoweb8 is the node and esnews is type.

You can download the code here

Happy coding and keep Sharing !!!!