Writing Connector Transform Scripts

Introduction

You can send data from an external system to Clarity by configuring a Connector and sending HTTP POST requests to Clarity for that Connector. The HTTP POST body can be in any format, and the Transform Script you write for the Connector controls how the body gets interpreted.

The Transform Script is a piece of JavaScript defining a handler function that takes the request body as input, and should return data in the format that Clarity understands, which is described below.

Quick Example

Let's say your system will post text/plain data where the first line is an ISO Date/Time, and each following line is a key/value pair:

2021-04-27T18:37:26.991Z
battery/1/lifetimeWh=8.6
battery/1/soc=43.0
inverter/1/ac/power=0.0
meter/metered_demand=73.4
meter/pv/power=16.3
meter/site_demand=

Here's how the transform script could look:

function handler(request) {
  const [timestampLine, ...dataLines] = request.body.split(/\n/mg);
  const t = new Date(timestampLine).getTime();
  const data = {};
  for (const line of dataLines) {
    const [key, rawValue] = line.split('=');
    data[key.trim()] = parseFloat(rawValue);
  }
  return { t, data };
}

Script Format

The script must be either a single function declaration, or contain a function named handler.

Examples of Valid Scripts

function (request) {
  ...
}

(request) => {
    ...
}

function parseCsv(body) {
  ...
}

function handler(request) {
  const parsed = parseCsv(request.body)
  ...
}

Examples of Invalid Scripts

return request.body // no function declaration

const SEPARATOR = ':'

function (request) { // no function named handler
  ...
}

Input Format

Your function will be called with a request object. Currently it only has the request.body property.

request.body

This will be the body of your POST request. Its type depends on the Content-Type header of your request:

Content-Type	type of request.body
application/json	object or array
text/* (includes text/csv)	string
*/x-www-form-urlencoded	object parsed by qs
other	Buffer containing raw bytes

Output Format

Your function must return an object with the following optional properties:

t

The timestamp for data values and events in the output that don't specify their own timestamp. Defaults to Clarity's current time.

data

Realtime and/or historical data to ingest into Clarity. This must be an object where each key is a tag, and the associated value is data for that tag. Several different formats for the data are supported:

Format	Examples
single value (string, number, boolean, or null)	'high', 2, true, null
single value and timestamp	{ t: 1619550013153, v: 5 }
multiple values	{ t: [1619550013150, 1619550013160, v: [5, 6] }

Additionally, you can specify metadata for a given tag by including a metadata property alongside the optional t and v properties on the value object. The value of metadata must be an object with the following optional properties:

Property	Type	Description
name	string	The display name for the tag
dataType	'string', 'number', or 'boolean'	The data type for tag values
units	string	The units for tag values
min	number	The lower bound of the display range in dashboards
max	number	The upper bound of the display range in dashboards
displayPrecision	number between 0 and 6	The number of digits to show after the decimal place for number values
rounding	number >= 0	Rounding to apply to ingested values. For example with rounding: 0.05, values will be rounded to the nearest 0.05.
validTimeout	number	The length of time that data set on this tag will be considered valid. You should set this to slightly more time than the expected data interval. For example, if you expect to get data every 24 hours, 25 hours would be a reasonable.

events

Events to display notifications for in Clarity. This must be an array of objects where each object contains the following properties:

Property	Type	Required?	Description
tag	string	yes	The tag to associate the event with
t	number	no	The event timestamp, in milliseconds
severity	'warning' , 'alarm', or 'critical'	no (defaults to 'warning')	The severity to display in Clarity
message	string	no	The message to display in notifications
properties	JSON object	no	Arbitrary properties to store with the event

Output Examples

These are examples of the object you would return from your handler function.

In this example, an explicit timestamp is given for the foo datapoint, but the current time in Clarity will be used for bar:

{
  data: {
    foo: { t: 1619550013153, v: 5 },
    bar: { v: 6 }, // you could also just use `bar: 6`
  },
}

In this example, the top-level t timestamp will be used for bar instead of the current time in Clarity:

{
  t: 1619550013160,
  data: {
    foo: { t: 1619550013153, v: 5 },
    bar: { v: 6 },
  },
}

In this case, multiple values are given for bar:

{
  t: 1619550013160,
  data: {
    foo: { t: 1619550013153, v: 5 },
    bar: { t: [1619550013153, 1619550013160], v: [10, 12] },
  },
}

In this case, metadata is defined for some channels, some of which have values and some of which don't:

{
  data: {
    foo: {
      v: 5,
      metadata: {
        dataType: 'number',
        min: 0,
        max: 10,
      },
    },
    bar: {
      metadata: {
        dataType: 'number',
        rounding: 0.25,
      },
    }
  }
}

An example with events:

{
  events: [
    {
      t: 1619550013153,
      tag: 'pumps/1',
      severity: 'alarm',
      message: 'pressure is >= 200 kPa',
      properties: {
        comparison: '>=',
        threshold: 200,
        value: 213,
        units: 'kPa',
      }
    },
  ],
}

Limits

Quantity	Limit
Maximum number of tags per post (between data and events combined)	500
Maximum number of values per tag in a post	500
Maximum number of posts per second	10
Maximum transform script size	500000 characters
Maximum number of connectors per organization	20