Overview
-
Data Generation is happening at exponential rate and Industry (retailers, telco-providers, security agencies, scientists, etc.) needs more Insights from the Data for business Values & Customer Interests in Real time and High Performance Solutions.
-
Advance Analytics requires deep analytics across all data sources to capture 360 degree view for Business Insights.
-
For Deep and Real Time Analytics, We need In-Memory Solutions across our Databases -MySQL or Other RDBMS Databases and BigData Platform using Apache Spark ,Hadoop with Scalable architecture.
Apache Ignite
Apache Ignite, an in-memory computing platform which is strongly consistent, durable and highly available with access to powerful SQL, key-value, and processing APIs. It is an in-memory database that provides a variety of integration with existing technologies such as Cassandra, Hadoop, Spark, etc.
Problem Statement
-
The customer needs In-Memory solutions for two cases - To improve the performance of Healthcare application with .NET and SQL server Architecture. In second case, perform fast analytical queries on Apache Spark and Hadoop based community Data Warehouse.
-
The customer needs to perform both Online Transactional Processing (OLTP) and Online Analytical Processing (OLAP) on workloads distributed among various database stores such as RDBMS, NoSQL and Hadoop.
-
The customer needs, Real-Time Analytics and Query platform with high performance and consistency.
Solution Offered
We used Apache Ignite for In-Memory Database and with Apache Spark and Hadoop.
-
For First Use Case - We used Apache Ignite as in-memory database to improve the performance of SQL queries with Ignite In-Memory data fabric for .NET. For ACD transactions, SQL Queries and distributed SQL joins.
-
For Second Case - We used Apache Ignite as IGFS and shared memory layer Spark RDD using Ignite RDD and built Analytics Dashboard using Play framework which interacts with Apache Ignite using its API. The dashboard provisions user to upload semi-structured data in various formats such as CSV, JSON, etc and run analytical queries