Our Work

AWS Cluster Creation and configuration

Updated 9 months and 25 days ago

For basic cluster creation you can choose smaller Ec2 instance types like t2.small like that. But if you really get Elasticity from elasticsearch you need to put one more node to share the load.So initially choose a 3 node cluster has 2 shards and 1 replica.

Step 1: Creating domain:

Click on create new domain.

Step 2:  Define name and version of ES you want

Step 3: Now choose instance types here is the main configuration

By default ES is giving m4.large Elasticsearch instance. So you can choose less configuration as well . start with t2.medium as of now


Note : as we are trying with small instance types for prototype we are not enabling zone awareness and dedicated master options.

These two are important for zone failover and cluster stability purpose it is recommended to enable these properties for production clusters

Step 4

If we enable dedicated master nodes the default configurations of EC2 types will be applicable to instance there is no option to choose master node hardware configurations. The default master node count is 3.(recommended) . this 3 is required to form a ES master quorum at least two master  nodes should be available to form a elastic cluster. In this case we can tolerate one node down. ( zen discovery min master nodes )


Note: If you want to try dedicated master nodes you can try with t2 medium.

SSD are more preferable to achieve speed. If you need advanced you can choose provisioned IOPS(SSD)

Leave snapshot default to aws as of now.if you need a automated snapshot please look at our blog.

https://ktree.com/aws-elasticsearch-service-snapshots-in-s3-buckets.html

Step 5

It is best practice to set indices field limit it is 30% of heap size recommended. Max clause query if you get any exceptions with your search cluster you can increase it based on your RAM size.

Step 6 Restrict ES to VPC or Your own private zone

By default aws ES domains available publicly . but on recent  update AWS released option to restrict to VPC. if you have VPC setup and ES bind to VPC it should be accessible for your machines only or through VPN connection. So our ES domain is safe.

Note: for old AWS ES domains , the above feature will not work you have to create new domain and restrict access and migrate your data.

If you choose public the domain should be accessible by anyone so your information is not secure so it is recommended to secure your domain if you have sensitive data.

Step 7 Confirm the properties.

Then your ES domain is ready and should be active in less than 10 mins

Important Points

You can create custom dashboards using cloud watch for your ES or use monitoring tab on aws ES console to see your logs
You can define alarms for you aws domain based on  your load and pressure on ES Domain.

Tags

    No tag results found for this post

Related Posts