DataTurbine

"DataTurbine is a robust real-time streaming data engine that lets you quickly stream live data from experiments, labs, web cams and even Java enabled cell phones. It acts as a “black box” to which applications and devices send and receive data. Think of it as express delivery for your data, be it numbers, video, sound or text.

DataTurbine is a buffered middleware, not simply a publish/subscribe system. It can an receive data from various sources (experiments, web cams, etc) and send data to various sinks. It has “TiVO” like functionality that lets applications pause and rewind live streaming data.

DataTurbine consists of one or more servers accepting data from sources and serving them up to sinks. Each component can be located on the same machine or different machines, allowing for flexibility in the deployment.

Server

The DataTurbine server is contained in rbnb.jar it is the core of DataTurbine and is used as a center point that applications (sources and sinks) interface with.

It is not a replacement for a database and is designed for speed. Because of this although it is possible to store years of data in a DataTurbine server, for most applications data is also archived in permanent storage in a database.

The acronym RBNB stands for Ring Buffered Network Bus, and is the technology inside the DataTurbine server.  To data sources (applications that generate data), it acts as a middleware ring buffer which stores heterogeneous time-sequenced data. To data sinks (applications that read data), it acts as a consolidated repository of data.  Key to RBNB scalability is each source (ring buffer) and sink (network bus connection) act independently of each other.

Source

A DataTurbine Source (also refereed to as an ‘on-ramp’) is a program that takes data from a target (for example a sensor or file) and puts it into a DataTurbine server.

A source runs independently from the server as a separate application and uses the network to communicate. It can run on the same machine as the server or across the world.

Each source can contain multiple channels each with its own data type. It controls its own server-side memory and hard drive space allocation.

Sink

A DataTurbine Sink (also refereed to as an ‘off-ramp’) is simply a program that takes data from a DataTurbine Server and utilizes it, for example brings it up in Matlab or Real-time Data Viewer or puts it into a relational database or file for permanent storage.

Just like a source, a sink runs independently from the server as a separate application and uses the network to communicate. It can run on the same machine as the server or on a machine across the world.