Introducing WSO2 Stream Processor

By Miyuru Dayarathna, WSO2 Sponsored Post.

1. Introduction

Stream Processors allow users to query continuous data streams for generating time critical business insights.

Figure 1: WSO2 Stream Processor Reference Architecture

WSO2 Stream Processor (WSO2 SP) let you write queries with Siddhi Streaming SQL, a SQL like query language that packs the power of SQL as well as temporal operators. It can ingest data from a wide variety of sources such as Kafka, JMS, MQTT, and HTTP. It can fuse data streams with the data from RDBMS, Hazelcast, MongoDB, and HBase. When queries trigger, WSO2 SP can visualize the output, generate alerts, carry out actions, or invoke APIs.

2. Why WSO2 SP?

First generation stream processors suffer from three problems.

• Custom Code: Commonly sought operators such as temporal patterns, and time windows need to be implemented as custom code. This consumes a lot of time and skills.
• Obese Deployments: Even a basic High Available (HA) deployment requires 5+servers. These deployments are complex and expensive to maintain.
• Poor Query Authoring Environments: Query authoring and application monitoring is complicated which stalls the application lifecycle (e.g. Facebook, Uber).

Let us explore how WSO2 SP handles them.

Avoid Complex Code with Streaming SQL

Siddhi Streaming SQL allows you to manipulate streaming data declaratively without having to write code using SQL and temporal operators.

WSO2 SP also has out of the box support for Machine Learning, supports Streaming Machine Learning (Streaming ML), PMML, and TensorFlow. WSO2 SP comes with a rich set of extensions for doing multiple additional processing. Incremental Aggregation Feature enables to implement batch processing scenarios with WSO2 SP.

Lean Deployments with 2-Node Highly Availability

When did you last handle an event rate greater than 50K? Often the answer is never. WSO2 SP can do more than 100K event per second. Hence creating a two node HA deployment with WSO2 SP you can handle most use cases and save the cost of a complex deployment.

If you are one of the rare few who has to handle more than 100K events per second, WSO2 SP can run distributed on top of Apache Kafka and scale to millions and support multi data center deployments.

Rapidly Adapt to Changes with Advanced Tooling Support and Multiple Deployment Options.

WSO2 SP comprises of a browser-based rich developer studio (See Figure 2), which provides time-saving development experience with query authoring, event simulation, playback and debugging support. It has a dashboard for monitoring complex deployments.

Figure 2: WSO2 Stream Processor Studio

WSO2 SP supports query configurations for business users though domain specific configuration UIs.

3. Summary

In this post, we introduced WSO2 SP and discussed its key strengths. Download and try it out using Quick Start Guide. To learn more, checkout the full features list and visit the official website.

Bio: Miyuru Dayarathna is a Senior Technical Lead at WSO2. He is a computer scientist with multiple research interests and contributions in stream computing, graph data management and mining, cloud computing, performance engineering, IoT, etc.