Historian - Database Storage for Monitoring | Dewesoft

Historian is a service for database storage in long-term testing and monitoring applications. It provides easy cloud data logging with OPC UA support.

Historian Historian

Get a quote

Overview

Historian - Database Storage for Monitoring and Long-term Testing

Historian is a software service for database and cloud data logging. It provides storage into a time-series database for long-term monitoring applications.

Monitor your vibration, temperature, inclination, strain, pressure, and other data with self-hosted or fully cloud-managed service. Standard OPC UA protocol is supported for data access and integration into our Dewesoft X data acquisition software or SCADAs, ERPs, or any other OPC UA clients.

Historian database storage with Grafana client

Historian Highlights

TIME-SERIES DATABASE: Data is stored in a state-of-the-art open-source InfluxDB database. InfluxDB is an open-source time-series database developed by InfluxData. It is written in Go and optimized for fast, high-availability storage and retrieval of time series data in fields such as operations monitoring, application metrics, Internet of Things sensor data, and real-time analytics.
SELF-HOSTED OR FULLY MANAGED HOSTING: Historian service can either be installed locally on the measurement unit, your local intranet, or we can provide a fully cloud-managed service. For fully hosted managed service monthly fee applies for data storage and maintenance.
ACCESSIBLE FROM ANYWHERE AND ANYTIME: Data stored in the Historian database can be accessed from anywhere in the world and at any time using an instance of DewesoftX data acquisition software or any standard web browser on any device like computer, tablet, or smartphone.
VISUALISATION IN GRAFANA: Historian uses another great open-source project called Grafana for visualization of the stored data. Grafana is multi-platform open-source analytics and interactive visualization web application. It provides charts, graphs, and alerts for the web when connected to supported data sources. It is expandable through a plug-in system. End users can create complex monitoring dashboards using interactive query builders.
DATA SAFETY AND RE-TRANSMIT: In case of network failure or downtime, the measurement units will continue to store the data into the local buffer and will safely transfer the data to the Historian database when the network connection is re-established.
RAW AND REDUCED DATA: While raw data is always stored on the measurement unit for an in-depth analysis, Historian takes the role of long-term reduced data storage into the cloud database.
TRENDING AND ANALYTICS: Historical data can always be recalled and loaded from the Historian database and used for trend analysis as well as for in-depth analysis and root cause identification.

Talk to an Expert

Historian Introduction

The world is being flooded with sensors. We are measuring more parameters for longer periods than ever before, piling up huge amounts of data. Big data becomes difficult to store, access, and visualize. Especially when it comes to monitoring production facilities or conducting long term tests, the challenge is how and where to store the data in order to be easily accessible from anywhere in the world at any time.

Historian enables reliable data transfer from multiple Measurement Units into a time-series database that can be hosted on a server or in a cloud. On the other side, the database can be accessed through a web browser or an instance of Dewesoft X software.

The Measurement Unit runs an instance of Dewesoft X or Dewesoft RT DAQ software. It communicates via the OPC UA protocol with the database. On the server-side, a Historian Service is handling the communication with the Measurement Units, writes the data into the database or reads the data from the database, and transfers it to the clients.

Dewesoft Historian brings the data from the most capable DAQ hardware and software on the market into the cloud for historical and worldwide access, keeping core Dewesoft features intact: reliability, usability, and expandability.

InfluxDB Time-series Database Storage

At the core of Dewesoft Historian is a state-of-the-art time-series database based on the InfluxDB open source project. Time-series databases were developed specifically for long term monitoring projects with large data rates.

Measurement devices store data into a state-of-the-art open-source InfluxDB database. InfluxDB is an open-source time-series database developed by InfluxData. It is written in Go and optimized for fast, high-availability storage and retrieval of time series data in fields such as operations monitoring, application metrics, Internet of Things sensor data, and real-time analytics.

The historian can save the following channel types for an unlimited time-span:

Synchronous 1D channels
Asynchronous 1D channels
FFTs and Scope shots as asynchronous 2D channels

Data Visualisation in Web Client

Web client is based on another open-source product called Grafana. It provides visualization of long term data and is accessible on any platform - Windows, Linux, Mac OS, etc. on desktop or mobile devices such as tablets or smartphones. It is an ideal solution for a data dashboard.

Grafana provides comprehensive visualization capabilities for trend data and live values. Dewesoft developed 2D channel visualization extensions for Grafana to display FFTs and high-speed waveform scope shots.

Dewesoft also developed a Python-based Grafana Connector that optimizes data loading for long and short time intervals by dynamically adjusting the data density to be shown. This improvement can be clearly tested in this demonstration.

Basic math and threshold levels can be set on any display as well as email alerting based on the displayed data.

Visual displays in Grafana can be customized by the user or prepared as part of a turn-key solution by Dewesoft.

Historical Data Access in DewesoftX Software

In parallel to web clients, the database can be also accessed by an instance of Dewesoft X data acquisition software. Using the Historian Importer plugin in DewesoftX Analysis mode the data from a chosen time frame is loaded into a Dewesoft data file.

Dewesoft Historian Importer is typically running on a Client PC in an engineering office where the database needs to be accessed for further in-depth analysis and reporting.

Once imported into a Dewesoft *.dxd data file, the wide range of Dewesoft’s mathematical operations can be applied to the data.

Daily, weekly, monthly, etc. reports can be easily generated by applying a predefined display setup and exporting data as PDF, Excel file, or any other Dewesoft’s export format.

Historical Data Access in DewesoftX Software

Live data stream into Dewesoft X

The data can also be streamed directly from the measurement units to the Dewesoft X client using the Historian Client plugin, providing a detailed live view.

Live data stream is bypassing the database and can therefore display raw high-speed data that is not necessarily saved to the database. This enables on-demand data visualization and recording, for example after receiving an email notification about an alarm state.

Mathematical and display setup can be applied to the streamed data channels like any other Dewesoft channels, enabling further analysis on the client side.

Integration Into 3rd-party Enterprise Software

SCADA, CMMS, or ERP systems can be connected to Dewesoft Historian over OPC UA or directly to the InfluxDB database using its API.

The Historian Service acts as an OPC UA server, enabling data stream to a 3rd party OPC UA client.

InfluxDB database provides a well-documented open API that is easy to implement and enables historical data access, typically for integration into machine learning software.

Integration Into 3rd-party Enterprise Software

Data Safety and Re-transmit Functionality

In case of network failure or downtime, the measurement units will continue to store the data into the local buffer and will safely transfer the data to the Historian database when the network connection is re-established.

The local buffer size can be adjusted up to several hundred megabytes. Tolerated offline time depends on the buffer size, data rate, and available network bandwidth on reconnect.

OPC UA with retransmit function is so reliable that we even use it to send the data from the airplane over 4G providing a live telemetry view to several base stations.

Data Safety and Re-transmit Functionality

Self-hosted or Fully Managed Cloud Hosting

Historian service is platform-independent and can run on any operating system. The Historian can be installed on the measurement unit itself, your local intranet, your server, or in the cloud.

We also offer a fully managed Historian service. The fully managed Historian service is based on a monthly subscription fee for maintenance and storage. The subscription fee depends on the number of measured channels and the storage needed for the monitored data. We take care of everything else - updates, system scaling, and maintenance.

For estimating the monthly subscription fee for a fully managed Historian you can contact us and we can estimate the cost for you after discussion.

Self-hosted or Fully Managed Cloud Hosting

Data Retention Policy

It is possible to configure how much data is retained in the database and for how long. Data is reduced by taking the average of the higher data rate samples and downsampling. Durations and data rates can be configured using InfluxDB configuration scripts via SQL language.

Exemplary default configuration:

Data rate	Default duration	Comment
Raw	30 days	The original rate at which the channel is written to the database.
10 S/s	60 days	Raw data is reduced to 10 S/s after days and stored for 60 days.
1 S/s	90 days	Channels stored at more than 10 S/s are reduced to 10 S/s after 60 days and retained for 90 days.
1 S/min	360 days	Channels stored at more than 1 S/min are reduced to 1 S/min after 90 days and retained for a year.
1 S/h	infinite	Data of 1 sample per hour is never deleted.

Distributed Architecture

Historian is designed as a distributed software system. Each component can run on a different system:

Measurement Units are instances of DewesoftX (Windows) or DewesoftRT (Linux, cross-platform) software and require the Dewesoft OPC UA server plugin to communicate with Historian Service.
Historian Service is the core component of Dewesoft Historian. It handles data transfer between Measurement Units, Databases, and Clients. Historian Service and the database typically run on the same server, but this is not a requirement.
Grafana can also run on a different server if needed as it communicates with the database over https.
Historian Service supports connections of multiple Dewesoft clients at the same time from different systems over the internet or on a local network.
NTP synchronization on each component is sufficient for system operation.

Historian Applications and Case Studies

The historian can be used in many measurement and monitoring applications:

Machine condition monitoring
Structural health monitoring (bridge monitoring, high-rise building monitoring, construction site monitoring)
Geotechnical monitoring
Fleet monitoring
Electrical power monitoring