CogCEP a Complex event processing (CEP) Solution using Microsoft® StreamInsight for high-throughput, low-latency processing of event streams at near real-time speed.

Microsoft StreamInsight provides a powerful platform for developing and deploying complex event processing (CEP) applications. CEP is a technology for high-throughput, low-latency processing of event streams. Typical event stream sources include data from manufacturing applications, financial trading applications, Web analytics, or operational analytics. The StreamInsight stream processing architecture and the familiar .NET-based development platform enable developers to quickly implement robust and highly efficient event processing applications.

The need for high-throughput, low-latency processing of event streams is common to the following business scenarios:
• Manufacturing process monitoring and control
• Clickstream analysis
• Financial services
• Power utilities
• Health care
• IT monitoring
• Logistics
• Telecom

Take an example of a bank. By using StreamInsight technology, it can monitor online transactions as they pass through the system. If any transaction fails to comply with the business rules defined in the CEP application, an event is raised that triggers a response to the threat, thereby ensuring security of its assets and those of the customers. On the other hand, banks can use this technology to mine data and generate profiles of their customers spending habits and design offers that will be most appealing to their customers.

StreamInsight never stores the data anywhere! Instead, the query is kept active at all times, and every new reading (we call them “events”) will immediately, upon arrival, trigger a new computation and generate a new result. The StreamInsight computation engine keeps just as much data in memory as necessary – for example, if the counter is kept for the duration of the 3-minute window. Every new incoming event causes an incremental update of StreamInsight’s internal data structures, and, if the query semantics require it, an output event.
A StreamInsight program brings incoming data to the queries using a data source, and the output of the queries is delivered through a data sink.

Microsoft introduced StreamInsight using the following approaches:
1. Continuous and incremental processing of unending sequences of events.
2. Lightweight streaming architecture that supports highly parallel execution of continuous queries over high-speed data.
3. The use of in-memory caches and incremental result computation provide excellent performance with high data throughout and low latency.
4. Low latency is achieved because the events are processed without costly data load or storage operations in the critical processing path.
5. All processing is automatically triggered by incoming events. In particular, applications do not have to incur any overhead for event polling.
6. Static reference or historical data can be accessed and included in the low-latency analysis.

A complex event process:

 
Taking it to the cloud:

Let us see some of the uses of StreamInsight in day to day life with Cognosys’ StreamInsight Solution.

Manufacturing process monitoring and control: Let us assume an automobile manufacturing unit testing its product. Here, we have an engine of car or a bus being tested. Every time the driver applies break, we get an event, this event is sent to the StreamInsight server as stream of data via a TCP Listener, The StreamInsight server then convert it to events to be queued into the StreamInsight query engine. The output is then streamed to a device through an output adapter also known as sink. This process can be time bound, that is, data is sent after every 3 minutes, or, purely based on events, as when they happen.

In the picture below the application is set to accept inputs from a vehicle as and when the driver applies the breaks. After every 10 seconds the application updates a column chart. The inset window is the input adapter with a start and stop button. The second pic displays the graph generated. The each dot in the upper part of the pic represents a breaking event received.

StreamInsight deployed on a mechanical process monitor control system.

StreamInsight at work in a banking portal:

The blue and red bars in the top part of the picture represent current and saving accounts opened in real-time. The middle section shows data in 5 seconds interval. The bottom part shows average amount in the accounts with respect to time.

StreamInsight with an ecommerce Portal:

The dots represent order price input. The bar chart represents orders received per second. The third part represents average price per second. The dial at the bottom left indicates grand total of all the orders received while the counter on the right represents total no of orders received.
 

StreamInsight with a finance portal:

The blue line represents the Last Traded Price (LTP) while the red candles represent the Last Traded Volume (LTV). The bottom part represents the Simple Moving Average (SMA).

Please email sales@cogno-sys.com for more details.