Introduction

VtrinLib is a client API that allows clients to access to any data source for which a driver is available. Data abstraction layer allows clients to use same interface, no matter what data source they are accessing. Key benefits are:

• Provide independent life cycles for the components by decoupling the clients from the data providers and hiding the details of the storage implementations.
• Connectivity to any data source with the same interface. Client can use any of the data sources and combine data from several of them with the same interface. Several data sources can be connected to one Data Abstraction Interface server.
• Provides an application architecture that is open and easy to integrate with other applications.
• Provide an extendable way to implement standard system interfaces such as OPC UA to handle application data without the need for changes in the middleware interfaces.
• Enforce controlled cyber security for the applications and systems with user authentication, secure transport, data access security, and audit logging.
• Take care of the common functions that provide additional value for most of the applications, such as online aggregation of the data in queries.
• Provide performance that is not limiting its use in the applications.
• Enable flexibility in the system topology and connectivity between networked systems.
• Provide the possibility to extend functionality with application-specific object models and business logic.

Drivers

Data Abstraction Layer (VtrinLib) provides object-based data access to any data source for which a driver is available. Currently there are drivers available for RTDB, SQL (example drivers for ODBC, OLEDB, and SQL Server), Oracle, PostgreSQL (TimescaleDB), OPC DA/HDA data sources, and for text files (for education purposes).

The currently available drivers are:

• ABB Ability™ History (RTDB)
• SQL (example drivers available for)
• ODBC driver
• OLEDB
• SQL Server
• PostgreSQL (TimescaleDB)
• OPC DA/HDA data sources
• Text file sample driver for education purposes
• Custom Oracle driver also in production use

Main features

Data access

🚧

The following API presentation is the VtrinLib API used on Mia Server, not in the JavaScript Client API (used in the browser)!

Time Handling

All timestamps in the VtrinLib are handled in UTC (Universal Coordinated Time) except when they are presented to the end user or when calculating local time based time intervals. By default the UI should display time stamps in server’s local time.

The .NET framework DateTime API does not contain a history of time conversion rules (which makes it impossible to convert other than the current time between different time zones). VtrinLib provides a separate API for this that makes it possible (UTC2ServerLocal, ServerLocal2UTC in the driver object).

Object Model Basics

• cDriverSkeleton is the base class for all data source drivers.
• A data source driver has a collection of the classes (driver.Classes as cDbClasses) in the data source.
• Each class definition (driver.Classes[“ClassName”] as cDbClass) contains a collection of properties (class.Properties as cDbPropertyInfo[]) and a class instance cache (class.Instances as cDbClassInstanceCache).
• The class instance cache provides an access to the class instances (class.Instances[instanceid] as cDbClassInstance) which provide the actual data in the data source.

For example, a typical mapping between a data source providing access to a relational database goes as follows:

Special Properties

Value and Raw Value

There are two concepts of property values in VtrinLib: Value and Raw value. Value is usually something that you want to show to the user. These are for example, timestamps in local time, resolved class references or language supported text representations of enumeration values.

Raw values are the actual values in the data storage (usually as native .NET types), for example timestamps in UTC, foreign keys (for class references), plain numbers (of enumeration values).

Fetching Data

Prerequisite for retrieving the data is a correct class definition (see Class Definitions).

📘

Note

The following examples are based on the equipment model defined in the Equipment model tutorial.

Fetching a single value

A single property value can be fetched using:

object value = Driver.Classes["ClassName"].Instances["Id/Name"]["PropertyIndex/PropertyName"];

For example, the following gets the level of the source tank:

var level = (double)Driver.Classes["Path_Tank"]
.Instances["Example site.Tank area.Source tank"]["Level"];

Null value checking should not be forgotten because class and property definitions are dynamic.

You can use cDbClassInstanceCache::Request to optimize queries between different parts of the code:

cDbClassInstanceCache.Request("Id/Name");

E.g. in a process display, the current value controls call Request in the initialization and execute fetch only in the painting phase so that multiple queries are merged into a single query.

Fetching multiple instances

Multiple instances can be fetched using the cDbClassInstanceCache::GetInstanceSet method. Parameters for GetInstanceSet can contain a list of instance IDs, names, a mask array or a combination of any of these.

For example, this returns all the tanks whose current level is over 500 mm:

var level = 500;
cDbClassInstance[] tanks = Driver.Classes["Path_Tank"].Instances.GetInstanceSet("Level > ?", level);

The same query can also be expressed by providing an array of cDbPropertyMask instances to GetInstanceSet:

var level = 500;
cDbClassInstance[] tanks = Driver.Classes["Path_Tank"].Instances.GetInstanceSet(
new cDbPropertyMask[] {
new cDbPropertyMask("Level", cDbPropertyMask.cMaskType.Greater, level)
}
);

Here is an example query with more conditions:

var minLevel = 300;
var maxLevel = 600;
cDbClassInstance[] tanks = Driver.Classes["Path_Tank"].Instances
.GetInstanceSet("Level > ? AND Level < ?", minLevel, maxLevel);

And the same query expressed using cDbPropertyMask:

var minLevel = 300;
var maxLevel = 500;
cDbClassInstance[] tanks = Driver.Classes["Path_Tank"].Instances.GetInstanceSet(
new cDbPropertyMask[] {
new cDbPropertyMask("Level", cDbPropertyMask.cMaskType.Greater, minLevel),
    new cDbPropertyMask("Level", cDbPropertyMask.cMaskType.Less, maxLevel),
}
);

The following query returns all the equipment instances under the path Example site:

cDbClassInstance[] equipmentInstances = Driver.Classes["Path"].Instances
.GetInstanceSet("Name LIKE 'Example site.*'");

And the same query using cDbPropertyMask:

cDbClassInstance[] equipmentInstances = Driver.Classes["Path"].Instances.GetInstanceSet(
new cDbPropertyMask[] {
new cDbPropertyMask("Name", cDbPropertyMask.cMaskType.WildCardStringMatch, "Example site.*")
}
);

Check the cDbPropertyMask.cMaskType for other filtering options.

Grouping

The following example sums the levels of all tanks:

object[][] data = Driver.Classes["Path_Tank"].Instances.GetInstanceData(
new cDbPropertyMask[] {
new cDbPropertyMask("Level", cDbPropertyMask.cGroupingStyle.Sum)
}
);

Updating Data

Creating and Updating Instances

Create a new instance:

cDbClassInstance instance = Driver.Classes["Path"].Instances.Add();

Update and instance:

cDbClassInstance newInstance = instance.BeginUpdate();

The return value of both of the above is a temporary cDbClassInstance object that should be used in the update. The original instance may be changed during the editing.

The property values can be set to the instance with two ways:

instance["PropertyName/PropertyIndex"] = value
instance.SetRawPropertyValue("PropertyName/PropertyIndex", value) // using the raw value

After setting the values, cDbClassInstance::CommitChanges must be called to commit the values:

instance.CommitChanges();

Multiple instances can be committed with:

cDbClassInstance[] instances;
Driver.Classes["Path"].Instances.CommitChanges(instances);

The commit calls will raise an exception if the committing fails.

After a successful commit the changes are stored in the database, and the temporary class instance can be abandoned. Updating can be cancelled by just abandoning the temporary instance.

Deleting Data

To remove one instance from the database:

instance.Remove()

To remove multiple instances:

cDbClassInstance[] instances;
Driver.Classes["Path_Tank"].Instances.Remove(instances);

or

object[] ids;
Driver.Classes["Path_Tank"].Instances.RemoveByIds(ids);

Listening to Events

Data change events are always triggered if the update operation is done through VtrinLib. Events triggered by external changes have to be generated by the data source driver.

The event handler can be set to following events:

The event handler will receive one or more of the following àrguments:

• The class affected by the event
• The event type (Insert, Modify, Delete, or InvalidateCache)
• The instance affected by the event
• A clone of the instance, OldInstance (Only if cDbClass.NonCacheable == false, otherwise null)

Command Classes

Command classes are used for executing parameterized commands.

Examples in ABB Ability™ History: ClearScenario, CommitScenario, RestoreFromBackupandUpdateHistory.

The command is executed by creating an instance (cDbClassInstanceCache.Add()), setting command parameters to the new instance as property values and calling CommitChanges() (as described above).

The driver has to implement a commit routine that handles the command.

All parameters are of IN/OUT type, so it is also possible to get return values from the command.

Fetching time series

A separate API (FetchGraphData, cGraphFetchParameters, cGraphData, ...) for fetching XY-series exist in the data source driver. This can be used for example in showing a series of history values in a trend graph or in a list. Any values from any class can be used in the fetch: The property for X-axis and the property for Y-axis can be selected from any property of the class. In cGraphFetchParameters masking can be defined to limit the amount of data returned in the fetch (for example to return values only from a certain period in time.)

var pumpname = "Example site.Water transfer system.Pump section.Pump";
var pump = RTDBDriver.Classes["Path"].Instances.GetInstanceByName(pumpname);

var graphFetchParams = ABB.Vtrin.Drivers.cGraphFetchParameters.CreateRawFetch(
RTDBDriver.Classes["EquipmentHistory"],
RTDBDriver.ServerTimeInUTC.AddHours(-1),
RTDBDriver.ServerTimeInUTC,
int.MaxValue,
"Path=? AND Property=?",
pump,
"Current power");

// Get iterator for the data
var iterator = RTDBDriver.GetGraphDataIterator(graphFetchParams);

// Print all the rows of data
while (iterator.MoveNext())
{
System.Console.WriteLine(iterator.XValue + ": " + iterator.YValue);
}

Fetching time series using Filter

Writing time series

The following examples will show you how to write values to history using a graph data iterator. Lists timestamps and values contain the data points we want to insert. Timestamps are in UTC. For example, timestamps[0] and values[0] make up the time and value for the first data point, timestamps[1] and values[1] for the second, and so on.

Using instance path and property

List<System.DateTime> timestamps;
    List<T> values;
        string path = "Example site.Tank area.Source tank";
        string property = "Level";

        cGraphFetchParameters parameters = cGraphFetchParameters.CreateScalarFetch(
        RTDBDriver.Classes["EquipmentHistory"],
        DateTime.UtcNow,
        "Path=? AND Property=?",
        path,
        property);

        var gdi = RTDBDriver.GetGraphDataIterator(parameters);

        for (var i = 0; i < timestamps.Count; i++)
        {
            gdi.Write(timestamps[i], values[i], cValueStatus.OK);
        }

        gdi.Close();

Using a variable

List<System.DateTime> timestamps;
    List<T> values;
        string variable = "My_Variable";

        cGraphFetchParameters parameters = cGraphFetchParameters.CreateScalarFetch(
        RTDBDriver.Classes["ProcessHistory"],
        DateTime.UtcNow,
        "Variable=?",
        variable);

        var gdi = RTDBDriver.GetGraphDataIterator(parameters);

        for (var i = 0; i < timestamps.Count; i++)
        {
            gdi.Write(timestamps[i], values[i], cValueStatus.OK);
        }

        gdi.Close();

Filters

There are several pre-defined filters that can be applied to the XY-series being fetched by defining the Filter parameter in cGraphFetchParameters. The Filter parameter is a string that consists of a single filter or a combination of filters.

If a suitable history of values already exists in the data source, an automatic optimization API allows the data source driver to use the history for faster processing.

Some of the filters currently available:

AAVG, ADEV, ADEVP, AMEDIAN, AMODE, AUTOCORR, AVARIANCE, AVARIANCEP, AVG, AVGRAW, CALC(expr), COUNTx, CUMSUM, DELTA, DEV, DURx, FFT, FIRST, HISTOGRAM, INTx, LAST, MAX, MIN, OPTIME, PERx, RANGE, STABILITY, STARTUP, SUM, SUMRAW, VARIANCE, WHEN(xxx), WORST, …

If the type of the X-axis property is a timestamp, a time period can be added to the filter to define the span for the values calculated by the filter.

For example:

If the X-axis property contains data other than timestamps, a number added at the end of the filter tells how many values the filter processes for each value.

For example:

Filters can be combined with the pipe character |, for example:

Security

VtrinLib offers a security model with Class, Property, and Instance-based security that can be defined with ACLs (Access Control Lists), Unix-like owner/group/other security or with flags. From these, the ACL is by far the most flexible solution but has also the highest overhead, which is the main reason for the co-existence of other security models. In case multiple models are used simultaneously for defining the security of an object, the matching "Owner" or "Group" will be used first, then ACL and finally the "Other" field. Rights granted by Flags are then combined with logical AND with these permissions to get final access rights, so users can never get permissions for operation by just a set of flags.

VtrinLib security has nothing to do with the underlying data source’s security. This is because there might be multiple different data sources providing information and all might have different security models and some might have no security model at all. VtrinLib implements all described security settings by itself and does not require any support from the driver implementation other than storage.

Transport security

The transport security in VtrinLib uses a 2048 bit RSA key exchange. Man-in-the-middle attacks are prevented by pre-shared and/or cached server keys. The data transfer is secured with 128bit AES encryption. The data is also compressed using Deflate compression to reduce the amount of traffic significantly.

The separate event transport (e.g. UDP multicast) is not encrypted and should be used only in secure environments. This can be disabled and events will be transported over the secure TCP connection.

Authentication

The authentication in VtrinLib is based on the authentication provided by the operating system (LogonUser API on Windows, PAM on Linux).

The Windows version supports Windows Vault and Kerberos. This allows the use of current credentials and secure storage to save the credentials.

The default authentication implementation in the driver can be overridden, although in general case, this is not recommended.

Access control

VtrinLib implements an access control mechanism that does not need support from the underlying data source by other means than storage. The same security definitions apply for both Vtrin and View/Mia connections.

❗️

Note

By default direct connections straight to the driver (not via Net Client connection) do not perform any security checks.

The data through VtrinLib is protected in three levels:

• Class level ("tables")
• Property level ("columns")
• Instance level ("rows")

Each level inherits access control information from the previous level by default. A special case is the Parent property. If Parent property exists and is not null, the access control is inherited from the instance defined in Parent.

The methods to define access control:

Users and groups

In Windows, you can prefix the user or group name with domain name, machine name or BUILTIN. (e.g. DOMAINNAME\user or BUILTIN\Administrators) \user will automatically expand to %COMPUTERNAME%\user.

Special exceptions to access control:

• Any user in Administrator group (AdministratorGroup in Vtrin-NetServer.exe.config) will bypass all the security checks. This is to prevent users from locking themselves out completely.
• Any user in the Robots group will bypass the update logging so that non-user based actions do not pollute the update log.

Permissions

Permissions are defined using bits of a byte:

Access control lists

Access Control Lists (ACLs) are similar to the Windows file security configuration where any number of groups or users can be granted a different set of permissions, which are inherited from parent objects unless overridden. For any given object (class/property/instance) you can allow/deny user/group-specific permissions. ACL entries per object are unlimited and no special properties are required in the class. For each class, ACLs can be turned on or off from the class definitions.

ACLs also allow defining time-based security. Each ACL entry can contain start and end time to limit the time that the definition is active. E.g. the user has rights to access February 2013 values for a certain variable in history, but no others.

Class Definitions

Special Classes

Class Definitions

Classes can be defined via code and dynamically, depending on your driver implementation.

UIClasses/UIClassInfo Example

The sample implementations use UIClasses and UIClassInfotables.

• UIClasses defines classes (one row / class).
• UIClassInfo defines properties (columns).

❗️

Note

UIClasses and UIClassInfo might contain also class and property specific access control information, so be careful not to overwrite access control information on upgrade.

Remember the special properties, especially Id.

Enumerations

With UIString class, it is possible to define translations from number values to texts in different languages.

Further reading

Equipment API .NET Client

EqM .NET Client API

EqM .NET Client API is a high-level C# implementation of the language-agnostic EqM API. It allows the user to use the EqM API without worrying about the low level details of the API. Another pro of using this implementation is that it uses VtrinLib's properietary binary format instead of JSON, in order to reduce network bandwidth usage.

Overview
The namespace of this API resides in ABB.Vtrin.Drivers.cMiaClient. This means that this API works only with a websocket connection (connection string starting with wss://).

The client API provides the concepts of EqM API in the following classes:

• cEquipment - Describes any equipment, its properties, functions, and events, and also any possible subequipment
• cEquipmentProperty - Describes a property of an equipment
• cEquipmentEvent - Describes an event of an equipment
• cEquipmentFunction - Describes a function of an equipment

Usage
The basic use is usually a two-step process: first the equipment(s) are introduced to the server, then the server responds with a subscription to a set (or all) of the properties. Then the client should start producing values to the equipment(s) that the server subscribed to.

The API to the server side has only two calls, SubscribeEquipmentSession and FetchEquipmentProperties.

SubscribeEquipmentSession

This is the only call needed to start using EqM API. This call takes a cEquipment as a parameter. The cEquipment object that is passed in should have event handlers set for events 'Ready' and 'Failed'. Those events are fired for the cEquipment object in case of a successful connection or a failure.
The required parameters for cEquipment are:

• equipmentid (can be null at first connect)
• equipmentname
• equipmenttype
• properties

See equipment subscription documentation for details of the parameters.

SubscribeEquipmentSession example
This is a minimal example of using the client API. It creates one property and writes one value after a successful connection to the server.

class Example
{
    static System.Threading.AutoResetEvent mPushReady = new System.Threading.AutoResetEvent(false);
    static ABB.Vtrin.Drivers.cMiaClient mDriver;
    static void Main(string[] args)
{
    ABB.Vtrin.cDataLoader loader=new ABB.Vtrin.cDataLoader();
    mDriver=(ABB.Vtrin.Drivers.cMiaClient)loader.Connect("wss://127.0.0.1/history", "", "", false);

    var myeqprops=new ABB.Vtrin.Drivers.cMiaClient.cEquipmentProperty[1] {
    new ABB.Vtrin.Drivers.cMiaClient.cEquipmentProperty(name: "My Property", type: typeof(System.Double))
};

    var eq=new ABB.Vtrin.Drivers.cMiaClient.cEquipment(equipmentid: null,
    equipmentname: "My Equipment",
    equipmenttype: "My Equipment Type",
    properties: myeqprops);

    // Remember to set handlers for these two events before calling SubscribeEquipmentSession
    eq.Ready += eq_ready;
    eq.Failed += eq_failed;

    mDriver.SubscribeEquipmentSession(eq);
    mPushReady.WaitOne(); // eq_ready and eq_failed are called asynchronously, that's why waiting for event here
}
    static void eq_ready(object sender, ABB.Vtrin.Drivers.cMiaClient.cEquipment eq)
{
    // Note that this event handler can be called multiple times by the EqM Client API
    // Subsequent calls happen for example when the equipment's properties are approved/disapproved

    // It is a good idea to wrap multiple pushes between BeginPushing/EndPushing calls to
    // allow driver to group the pushes as a single message, improving the performance.
    mDriver.BeginPushing();

    // Every property that the server subscribed to will remain in eq.Properties
    // the possible properties that the server does not subscribe to, are removed
    foreach(var prop in eq.Properties)
{
    prop.PushHandle.Push(1.0); // Produce a value to the database
}
    mDriver.CommitPushing();
    pushready.Set();
}
    static void eq_failed(object sender, System.Exception ex)
{
    System.Console.WriteLine(ex.ToString());
    pushready.Set();
}
}

Events

Equipment API supports events. You can use them with the client API by assigning an array of cEquipmentEvent objects to the constructor of cEquipment. Let's modify the first example a bit:

var myeqevents = new ABB.Vtrin.Drivers.cMiaClient.cEquipmentEvent\[] {
    new ABB.Vtrin.Drivers.cMiaClient.cEquipmentEvent("My first event"),
    new ABB.Vtrin.Drivers.cMiaClient.cEquipmentEvent("My second event")
};
var eq=new ABB.Vtrin.Drivers.cMiaClient.cEquipment(equipmentid: null,
equipmentname: "My Equipment",
equipmenttype: "My Equipment Type",
properties: myeqprops,
events: myeqevents);

Let's also produce values for those two events. Producing values is done the similar way that the value production for properties. Add following to the eq_ready function:

foreach(var evt in eq.Events)
{
    ((ABB.Vtrin.Drivers.cMiaClient.cEventPushHandle)evt.PushHandle).Push("Testing event production");
}

Note the casting of cPushHandle to cEventPushHandle. This is not strictly necessary, but it provides an additional Push function that is easy to use with events.

As documented in EqM API documentation, the event value production expects an array of message, optional attributes and optional event time. To demonstrate all these features, we can change the previous lines of code to following:

var attributes = new System.Collections.Generic.Dictionary\<string, object> {
    {"Attribute 1", "This is some attribute"},
{"Attribute 2", 10000}
};
foreach(var evt in eq.Events)
{
    ((ABB.Vtrin.Drivers.cMiaClient.cEventPushHandle)evt.PushHandle).Push("Testing event production", attributes, System.DateTime.UtcNow - System.TimeSpan.FromHours(1));
}

This produces two events with an event time of one hour behind the current time. You can see the one hour difference between EventTime and RowInsertionTime from the screenshot below.

The events are produced to a table called EquipmentEventLog. In Vtrin you can see the events from Maintenance -> System -> Equipment Model Configuration -> Equipment Events.

Functions

EqM API supports function calls from the server to the client, and returning the return value back to the server. This client API allows to specify normal C# functions to be called from the server through the EqM API.

Function call example
Similar to the previous examples, the functions need to be defined in the constructor of cEquipment. Due to technical limitations of the C# language, the functions then need to be wrapped into System.Funcobjects. The following example defines a function called SumOfTwoValues and assigns it to the cEquipment object.

static double SumOfTwoDoubles(double a, double b)
{
    System.Console.WriteLine("SumOfTwoDoubles ({0} + {1}) called", a, b);
    return a+b;
}

And later on before creating the cEquipment object:

var myfunctions=new System.Delegate\[] {
    new System.Func\<double, double, double>(SumOfTwoDoubles)
};
var eq=new ABB.Vtrin.Drivers.cMiaClient.cEquipment(equipmentid: new System.Guid("f91fb037-f1f4-44c5-a800-6e85ff0ec811"),
equipmentname: "My Equipment",
equipmenttype: "My Equipment Type",
properties: myeqprops,
events: myevents,
functions: myfunctions);

Also now the program needs to be running while the function is being called. One quick and easy thing to do is to remove the AutoResetEvent usage, and use System.Console.ReadKey() at the end of the main function instead.

Now, when the connection is open and the server calls a function, the SumOfTwoDoubles function will be run with the arguments and the result returned back to the server. If the connection is not open while the function is called, it will be queued so that it will be called right away when the client next connects to the server with SubscribeEquipmentSession.

Tutorials

We have gathered VtrinLib application examples under one how-to tutorial. You may find the examples here.

Recommended reading

• Vtrin Server article : Vtrin Server is the service that provides remote access to the Vtrin user interface client.
• Equipment Model article: The Equipment Model is one example of dynamic object models exposed by VtrinLib.
• Processing data section: This section describes how to refine the raw time series data to aggregated values that are needed in dashboarding and reporting.
• Software stack article: This article gives an overview of the layered architecture of ABB Ability™ History along with the Data Abstraction layer (VtrinLib).
•