Our Work

Overview

Our client is one of the largest auto parts distribution and marketing organizations in the world, marketing the Auto Parts sellers. With store and service center locations throughout North America and Europe, the client is a source for quality parts and service for over 4,600 parts stores and 2,500 certified service centers. Client engaged KTree for Elasticsearch Consulting to make their ominous data searchable, analyzable and presented in better way. 

Industry: Automobiles
USA, Canada & Mexico
Autospare parts industrry
  • Businesss Requirements and Challenges.

    • Design an ElasticSearch Cluster which can hold billions of records/documents with out fail and data search would be faster.
    • Design dashboards to view net sale status Daily, Weekly, Monthly, Yearly.
    • Extract information from Millions of XML files and CSV files and index to ElasticSearch.
    • Needed to Implement elasticsearch python Client to get snapshots automatically from cluster periodically.
    • Transform the data in to full form and Index data to elasticsearch.
    • Centralize logs from all application/web-servers to ElasticSearch Index.
  • Challenges

    • Extract Historical Sales Transactions data From Relational Database Management System
    • Integrate ElasticSearch to a third party Applications to filter data from Applications to Cluster directly.
    • Implement automatic cluster restart scripts for ElasticSearch.
    • Implement dashboards to visualize the data in a quick way.
    • Automatic Snapshot from Cluster with fail over mechanism at specified time & removing snapshots by keeping one week backups in repository.
    • Remove duplicates in elasticsearch while indexing data.
  • Solutions Provided - 1

    • Fast and Accurate Search:
      A powerful search capability is furnished that instead of searching the text directly, searches an index instead. As we have had millions of transactions to take into consideration, the search function is catered to browse large number of product descriptions for the best match for a specific phrase - suitably the optimal item matched to the query is depicted as result.
    • Dashboards For Data Visualization:
      scheme-free approach of elastic search, allowed us to calculate the aggregations within  seconds of time, which has been quite a considerable improvement over writing queries on other business intelligence tools. With Kibana, as Data Visualization engine , real-time summary and charting of streaming information allowed us to and slice and dice your data as we see fit.
    • Cluster Creation and Management:
      Elasticsearch is distributed by nature: it is perfectly adept with managing multiple nodes so as to provide high availability, scalability and agility.  We set up a multi-node production cluster big enough to search all the data that has been collected and to ensure that data is safe from hardware failure.Maintenance of clusters and shards is a routine activity we do as as to have supreme stability.
      The entire creation and management of clusters was carried out after thorough research and analysis and brimming on the facts that ES is equipped with ability to hold millions and trillions of data, which can be used to provide stellar customer experience.
  • Solutions Provided - 2

    • Processed millions of records from historical data to database & from there using ElasticSearch river concept we have indexed data to Elastic-search with a blazing speed. Later data is available for search & visualizations
    • Chosen Good hardware Configurations in order to made an effective cluster to hold such kind of huge data & make them available for search. This can help us to achieve fail over and high throughput from Elastic Cluster. Designed cluster configurations by Considering data growth in future.
    • Log Assessment and Management:
      Integration with Logstash facilitated in processing logs and other event data from a variety of stream systems. Logstash helped not just in parsing and normalizing this data and converge on a common format, but also assisted in inserting it into our analytics datastore. Efficient management and tracking of daily logs.
    • Third-party Application Framework:
      As elasticsearch offers wide range of API for other frameworks to integrate with, we ensured it manages the data repository in the most effective sense possible.
    • Eliminate Data Redundancy:
      Making the most of  primary key in elasticsearch, data duplication is avoided during indexing.
      Moreover to circumvent redundancy, only one shard is used from each shard group (primary and replicas)

    • Implemented elasticsearch python Client to get snapshots automatically from cluster periodically.
  • Solutions Provided - 3

    • Implemented log-stash shippers for each application server logs from that logs are pushing to elasticsearch so that we can analyze the logs in a quick way.
    • Like traditional databases we can define a primary key for elasticsearch also but we don't have any combination key for elasticsearch. By choosing this Primary key correctly we can remove duplicates from data and it will be helpful to update data at any time in efficient way.
    • Processed Historical data of XML files using Java Clients and push to Database from there we are indexing to elasticsearch so parts search is available for Client by hitting elasticsearch rather that going to each XML file. The search would be perform from millions of transactions even though response will get in milliseconds of time.
    • ElasticSearch have vast API's for all opensource frameworks which can integrate with third party Applications (like ERP, Java, PHP, Python, Perl etc..)
    • Implemented Kibana Dashboards by integrating with elasticsearch to view sales, parts data quickly and perform aggregations with in minutes of time. Designed monthly, weekly, daily dashboard to view sales data quickly.
    • Implemented automatic snapshots sweeping from repository if it is older than 7 days.

Free Software Consultation with our Experts!

No Obligation!

Read More

Our Previous Works

Come See Our Exemplary Work Done {{ service ? 'On '+service:''}}

KTree worked on hundreds of Web Application projects which can be further classified broadly into Web Application Development, SaaS application development, Single Page Applications, Portal Development, Ecommerce Development, ERP Implementation, Mobile APP Development, Point of Sale etc..

No Records Found.. You can use search for to find.

Read more about KTree

KTree, a Global IT Company

What and how the processes are structured in KTree?

KTree follows CMMi process to T. Defined and Mature processes for Coding, Code Reviewing Exercises, QA Process with focus on Performance testing & Load Testing

Where is KTree's Web and Mobile application Development done?

Most of our Development is done from our Hyderabad Development Center. Hyderabad is known for its rich heritage and exotic food apart from abundant IT Talent.

How KTree makes outsourcing project a positive experience?

True collaboration, transparent and constant communication, visibility and traceability are the key for success. The other major one being requirements engineering.

What is Success mantra of KTree, as one of the respected web development company?

Agile Methodology + Strong Documentation + Powerful Web & Mobile Frameworks + Solid KTree Team = Great Web & Mobile Applications within time.

Any other Differentiation which KTree has, which it can offer?

Having worked on hundreds of Web & Mobile Application projects, KTree teams tremendous knowledge & expertise gives very good head-start for projects which helps clients save many man-months

What Development methodology KTree follows and why?

KTree development team follows Agile methodology because of its suitability to web and mobile application development. Scrum helps improve Agility due to its simplicity and flexibility

Our Clients