Connector Transform Scripts

Introduction

You can send data from an external system to Clarity by configuring a Connector and sending HTTP POST requests to Clarity for that Connector. The HTTP POST body can be in any format, and the Transform Script you write for the Connector controls how the body gets interpreted.

The Transform Script is a piece of JavaScript defining a handler function that takes the request body as input, and should return data in the format that Clarity understands, which is described below.

If the input data is in the format described in Output Format, it is not necessary to write a transform script, and the "None" option can be selected.

Quick Example

Let's say your system will post text/plain data where the first line is an ISO Date/Time, and each following line is a key/value pair:

2021-04-27T18:37:26.991Z
battery/1/lifetimeWh=8.6
battery/1/soc=43.0
inverter/1/ac/power=0.0
meter/metered_demand=73.4
meter/pv/power=16.3
meter/site_demand=

Here's how the transform script could look:

function handler(request) {
  const [timestampLine, ...dataLines] = request.body.split(/\n/mg);
  const t = new Date(timestampLine).getTime();
  const data = {};
  for (const line of dataLines) {
    const [key, rawValue] = line.split('=');
    data[key.trim()] = parseFloat(rawValue);
  }
  return { t, data };
}

Script Format

The script must be either a single function declaration, or contain a function named handler.

Examples of Valid Scripts

function (request) {
  ...
}
(request) => {
    ...
}
function parseCsv(body) {
  ...
}

function handler(request) {
  const parsed = parseCsv(request.body)
  ...
}

Examples of Invalid Scripts

return request.body // no function declaration
const SEPARATOR = ':'

function (request) { // no function named handler
  ...
}

Input Format

Your function will be called with a request object. Currently it only has the request.body property.

request.body

This will be the body of your POST request. Its type depends on the Content-Type header of your request:

Content-Type

type of request.body

application/json

object or array

text/* (includes text/csv)

string

*/x-www-form-urlencoded

object parsed by qs

other

Buffer containing raw bytes

Output Format

Your function must return an object with the following optional properties:

t

The timestamp for data values and events in the output that don't specify their own timestamp. Defaults to Clarity's current time.

data

Realtime and/or historical data to ingest into Clarity. This must be an object where each key is a tag, and the associated value is data for that tag. Several different formats for the data are supported:

Format

Examples

single value (string, number, boolean, or null)

'high', 2, true, null

single value and timestamp

{ t: 1619550013153, v: 5 }

multiple values

{ t: [1619550013150, 1619550013160, v: [5, 6] }

Additionally, you can specify metadata for a given tag by including a metadata property alongside the optional t and v properties on the value object. The value of metadata must be an object with the following optional properties:

Property

Type

Description

name

string

The display name for the tag

dataType

'string', 'number', or 'boolean'

The data type for tag values

units

string

The units for tag values

min

number

The lower bound of the display range in dashboards

max

number

The upper bound of the display range in dashboards

displayPrecision

number between 0 and 6

The number of digits to show after the decimal place for number values

rounding

number >= 0

Rounding to apply to ingested values. For example with rounding: 0.05, values will be rounded to the nearest 0.05.

validTimeout

number

The length of time that data set on this tag will be considered valid. You should set this to slightly more time than the expected data interval. For example, if you expect to get data every 24 hours, 25 hours would be a reasonable.

events

Events to display notifications for in Clarity. This must be an array of objects where each object contains the following properties:

Property

Type

Required?

Description

tag

string

yes

The tag to associate the event with

t

number

no

The event timestamp, in milliseconds

severity

'warning' , 'alarm', or 'critical'

no (defaults to 'warning')

The severity to display in Clarity

message

string

no

The message to display in notifications

properties

JSON object

no

Arbitrary properties to store with the event

Output Examples

These are examples of the object you would return from your handler function.

In this example, an explicit timestamp is given for the foo datapoint, but the current time in Clarity will be used for bar:

{
  data: {
    foo: { t: 1619550013153, v: 5 },
    bar: { v: 6 }, // you could also just use `bar: 6`
  },
}

In this example, the top-level t timestamp will be used for bar instead of the current time in Clarity:

{
  t: 1619550013160,
  data: {
    foo: { t: 1619550013153, v: 5 },
    bar: { v: 6 },
  },
}

In this case, multiple values are given for bar:

{
  t: 1619550013160,
  data: {
    foo: { t: 1619550013153, v: 5 },
    bar: { t: [1619550013153, 1619550013160], v: [10, 12] },
  },
}

In this case, metadata is defined for some channels, some of which have values and some of which don't:

{
  data: {
    foo: {
      v: 5,
      metadata: {
        dataType: 'number',
        min: 0,
        max: 10,
      },
    },
    bar: {
      metadata: {
        dataType: 'number',
        rounding: 0.25,
      },
    }
  }
}

An example with events:

{
  events: [
    {
      t: 1619550013153,
      tag: 'pumps/1',
      severity: 'alarm',
      message: 'pressure is >= 200 kPa',
      properties: {
        comparison: '>=',
        threshold: 200,
        value: 213,
        units: 'kPa',
      }
    },
  ],
}

Limits

Quantity

Limit

Maximum number of tags per post (between data and events combined)

500

Maximum number of values per tag in a post

500

Maximum number of posts per second

10

Maximum transform script size

500000 characters

Maximum number of connectors per organization

20

Builtin Transforms

In a dedicated Clarity instance, builtin transforms can be included at the application level. If these are included, the builtin transforms will be shown in the drop-down selector.

Last updated