Connect SDL WEB 8 with Elasticsearch

What is ElasticSearch ?

Elasticsearch is an Apache Lucene-based search server. It was developed by Shay Banon and published in 2010.

Elasticsearch is a real-time distributed and open source full-text search and analytics engine. It is accessible from RESTful web service interface and uses schema less JSON (JavaScript Object Notation) documents to store data. It is built on Java programming language, which enables Elasticsearch to run on different platforms. It enables users to explore very large amount of data at very high speed.

Features of Elasticsearch are as follows:- 

1. Elasticsearch is scalable up to petabytes of structured and unstructured data.
2. Elasticsearch is open source and available under the Apache license version 2.0.
3. Elasticsearch uses denormalization to improve the search performance.
4. Elasticsearch can be used as a replacement of document stores like MongoDB.
5. Elasticsearch is one of the popular enterprise search engines, which is currently being used by many big organizations like Wikipedia, The Guardian, StackOverflow, GitHub etc.

key concepts of Elasticsearch:-

Node:- It refers to a single running instance of Elasticsearch. Single physical and virtual server accommodates multiple nodes depending upon the capabilities of their physical resources like RAM, storage and processing power.

Cluster:- It is a collection of one or more nodes. Cluster provides collective indexing and search capabilities across all the nodes for entire data.

Index:- It is a collection of different type of documents and document properties. Index also uses the concept of shards to improve the performance. For example, a set of document contains data of a social networking application.

Type/Mapping:- It is a collection of documents sharing a set of common fields present in the same index. For example, an Index contains data of a social networking application, and then there can be a specific type for user profile data, another type for messaging data and another for comments data.

Replicas:- Elasticsearch allows a user to create replicas of their indexes and shards. Replication not only helps in increasing the availability of data in case of failure, but also improves the performance of searching by carrying out a parallel search operation in these replicas.

Shard:-  Indexes are horizontally subdivided into shards. This means each shard contains all the properties of document, but contains less number of JSON objects than index. The horizontal separation makes shard an independent node, which can be store in any node. Primary shard is the original horizontal part of an index and then these primary shards are replicated into replica shards.

Advantages

Elasticsearch is developed on Java, which makes it compatible on almost every platform.

Elasticsearch is real time, in other words after one second the added document is searchable in this engine.

Elasticsearch is distributed, which makes it easy to scale and integrate in any big organization.

Creating full backups are easy by using the concept of gateway, which is present in Elasticsearch.

Now, Lets create connect  ElasticSearch  and SDL WEB 8 and  components in CMS.

1. Let's create an console based application using .NET 4.5.
2. Run the following command in the Package manager console .Install-Package NEST .Install Elasticsearch .NET high level client.
3. Download Elasticsearch click and unzip the Elasticsearch 
4. Run ../bin/elasticsearch
   
5. Run  http://localhost:9200/
   
6. Default Elasticsearch instance is up and running now .
7. With the help of Elasticsearch .NET high level client ,we can do all the CRUD operations in Elasticsearch.
8. Here, I have created a default index  fromelasticstoweb8 (should be in all lower case other-wise you will get the error Invalid resquest) and Type esnews.
9. So the data URL would be  http://localhost:9200/fromelasticstoweb8/esnews/AVoFKrUbqeqnhk6aE3Ca  the last parameter is the ID
   
10. To get all the records http://localhost:9200/fromelasticstoweb8/_search?q=*:*
    
11. Code snippet to write data in Elasticsearch node
    
12. Let read all the data from node fromelasticstoweb8
    
13. Using Coreservice we can create components in WEB 8 
1. Create coreservice client using ISessionAwareCoreService.
2. Create model Serialize that model into XML.
   
3. Using coreservice client.create method we can create the components in WEB 8.
   
4. I have created all the items available in ES node fromelasticstoweb8  into SDL WEB 8 as components .
   
14. To know more on how to use coreservice to interact with SDL WEB 8 and create CMS items such as components ,Keywords etc you can refer my previous blogs also .

Happy Coding and Keeping Sharing !!!!!