Overview
Our Client is an vehicle parts provider of North America’s Aftermarket commercial, company is generating huge terabytes of transactions, data for their catalogue & orders which need to be searched in few milliseconds and also they needs a system which is 100% fault tolerant. KTree used Elasticsearch and AWS SQS to deliver this system and also to make it microservices enabled.
Requirements
- Order History Index is a place where all orders are stored and it is available for users for quick search.
- Each lambda functions is configured with environment variables predefined like indexname , mapping , keys , elasticsearch domain name and publishing versions based on it.
- Data Security & implementing best practices for protecting the endpoints traffic from only VPC/VPN or known applications/domains.
- Maintain different lamdba versions for environment like Dev, Beta, Prod etc we can write for order history we use micro services using Lambda To push data to elastic search which is highly available and server less architecture.
- Data shippers process data from sqs queue to ElasticSearch and this need 100% high availability too.
Order History Process
- App servers (Shopping carts ) push records to the respective SQS queue (order history).
- Lambda service will read the orders from the SQS queue (order history) and Lambda triggers to push it in batches to the ES index.
- ES index serves as a source for the reporting tool and search apps to display the reports and return the search results quickly.
- Lambda scheduled using cloud watch rules as cron which can run every one minute to read sqs records iteratively and push to elasticsearch and based on record successfully inserted at elasticsearch then we are removing that message from queue.
- We can write to searching and viewing orders can be visible through Angular.js application in cart.
- Records missing when internet got disconnected or not reachable by app server backfilling with automated jobs.
Results
Filters
-
With the usage of AWS SQS, we made the architecture microservices enabled and less coupling, and the development teams started adding features more quicker than before.
-
Using SQS and lambda helped the client in scaling transparently based on the demand.
-
Data shippers were implemented in AWS lambda (serverless architecture) which does not need any hardware setup or maintenance or downtime.
-
SQS offers a reliable, highly-scalable hosted queue for storing orders as they travel between applications or microservices.
-
Since migrating to AWS, Our client has seen measurable improvements in the bottom line, as 10 million+ of transactions are pushed to different elastic indices in multi-threaded distributed architecture with minimal or zero fault tolerance.
-
Moving to AWS has made our client’s IT department less reliant on IT support resources and client reported this has given them overall cost benefit of about 20-30%.
-
Usage of cloudwatch metrics & Alerts has helped us to respond in advance and do the maintenance activities for 100% HA.
-
By using SQS we have decoupled our standard Java application components successfully, and made our overall architecture as microservices enabled.
-
Multiple Lambda functions for the same queue in the peak time were created easily which helped reduced the ingestion time.
-
These lambda functions read SQS queue and ingest data directly to elasticsearch service indices.
Gallery
Read more about KTree
KTree, a Global IT Company
What and how the processes are structured in KTree?
KTree's processes follow CMMi to ensure consistent, high-quality outcomes. With a focus on quality coding , code reviewing, and performance and load testing in our QA processes, we deliver reliable, scalable, and high-performing applications to meet our clients' needs.
Where is KTree's Web and Mobile application Development done?
KTree's web and mobile application development primarily takes place at our Hyderabad development center. Hyderabad is not only abundant in IT talent but is also known for its rich heritage and exotic food.
How KTree makes outsourcing project a positive experience?
KTree delivers high-quality applications that meet our clients' needs, by prioritizing collaboration, communication, visibility, traceability, and requirements engineering. While we charge a little premium for our services, we believe that it is important to provide the necessary resources and expertise to ensure project success.
What is Success mantra of KTree, as one of the respected web development company?
The success mantra of KTree as a respected web development company is combining Agile methodology with strong documentation, powerful web and mobile frameworks, and a solid KTree team to deliver great web and mobile applications within a specified timeframe.
Any other Differentiation which KTree has, which it can offer?
KTree's tremendous knowledge and expertise, gained from working on hundreds of web and mobile application projects, gives us a significant advantage that allows us to provide a head-start on projects, saving our clients many man-months.
What Development methodology KTree follows and why?
KTree's development team follows the Scrum framework of the Agile methodology, which improves Agility through its simplicity and flexibility. This allows us to adapt to changing project requirements and deliver high-quality products within the specified timeframe.
Leverage the expertise of a Top Application development company for your enterprise web and mobile application needs.
Contact our solid KTree team to get started.
