Page tree

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Term or AcronymDefinition

Anchor
asset
asset

Asset

Anchor
batch_mode
batch_mode

A resource exposing one or more endpoints.

UI Text Box

Examples:

  • Water pump
  • IoT sensor
  • Airplane engine
  • Salesforce account
  • Billing system instance
  • Database instance
  • Mobile subscription identified by its SIM
  • Self-driving car
  • Base station


Bounded Stream

Anchor
collector
collector

A unidirectional stream that processes finite units of input, example complete files or fixed amount of rows in a database.

UI Text Box

Example:

Collecting .csv files via SFTP and only setting a "committed" checkpoint for a particular input file once it is guaranteed an entire file has been successfully processed.


Collector

Anchor
collection
collection

A function that collects data from a source.

Collection

Anchor
client_mode
client_mode

The process of Dazzler acquiring data from a source.

UI Text Box

Examples:

  • Collecting .csv files from an S3 bucket by acting as an HTTPS client
  • Collecting RESTful requests by acting as an HTTPS server


Client Mode

Anchor
decode
decode

A communication mode where collectors initiate outgoing requests.

UI Text Box
Examples:
  • Dazzler acting as an HTTPS client.
  • Dazzler acting as an SFTP client.
  • Dazzler acting as a JDBC client.


Custom References

Custom references refer to an optional feature that is used with SAP-related functions. When enabled it allows users to specify one or more typeCode and id values in the relevant usage records. 

Decode


Anchor
decode
decode



The process of parsing input from a source and converting it into records in a format that can be sent across a route.

Destination


Anchor
destination
destination


An endpoint to which data is sent from a stream.

Encode

Anchor
encode
encode

The process of formatting a record into a format that can be understood by a destination.

UI Text Box

Example:

Creating a JSON string from a record.


Endpoint

Anchor
endpoint
endpoint

An interface used by the service to connect to an asset. An endpoint may function in either server mode or client mode.

UI Text Box

Examples:

  • S3 bucket
  • SFTP server
  • HTTP server
  • HTTP client
  • Database instance


Anchor
flush
flush

Flush


The flush operation forces to pass the stored/buffered (aggregated) data to another function in the current stream or writes the data when the specified criteria is met.

Forwarder

Anchor
forwarding
forwarding

A function that sends data to a given destination.

Forwarding

Anchor
function
function

The process of sending data to a destination.

UI Text Box

Examples:

  • Sending JSON records to a billing system via HTTPS
  • Storing .csv files in an S3 bucket


Function

Anchor
record
record

A function accepts input, applies logic to it, and then returns a result. Functions may be stateful.

UI Text Box

Examples:

  • Filtering fields from records
  • Renaming fields in records
  • Aggregating multiple records.


HTTP Response Headers

This is an option found within the HTTP Client collector and processor function, intended to support the sending out of response headers in the network stream. The users can enter a custom value in the result header key and associated result key to configure the target web service. 

Notification Group

Anchor
Notification Group
Notification Group

 A notification group is an email list to which automated messages regarding erroneous events related to streams are being sent. The content of the error messages include stream execution errors, missing files, and related events. 

Record

Anchor
route
route

A unit of data that can be processed by a function or multiple functions and sent across routes.

UI Text Box

Examples:

  • A decoded line in a .csv file.
  • A decoded JSON payload in an HTTP request.
  • A decoded row in a relational database.
  • A decoded Kinesis message.


RegEx (Regular Expression)

The use of regular expressions implies the creation of custom schemes to which various operations can be executed. 

Route

Anchor
solution
solution

A route is a link between two functions, across which data flows.

UI Text Box

Example:

A function exposing two outgoing routes, e.g. records that are either successfully processed or erroneous records.


Session

Anchor
agg_session
agg_session

A representation of an aggregated record that is kept in storage before being flushed out. 

Solution

Anchor
source
source

A solution is a set of streams that together solve a particular customer problem.

UI Text Box

Examples:

  • Streams required to perform billing mediation for a cloud biller on incoming usage records
  • Streams required to acquire the product catalog data for a cloud biller


Source

Anchor
stream
stream

An endpoint from which data is collected into a stream using a given protocol and data format.

Stream

A graph representation of data flowing from a source, through a series of functions, to a destination

UI Text Box

Example:

A stream collecting CSV files from an Amazon S3 source and forwarding them to a destination database using JDBC.


Time To Leave (TTL) 

This is a mechanism that limits the lifespan of data in an environment. It is used in functions like aggregation to express the date and time where a record expires. 

Transaction

Anchor
transaction
transaction

A transaction is a unit of data that is being processed inside a stream. The definition of a completed transaction is when the stream processes it without any errors. The transactions by design are limited to their specified data sources.

A transaction can refer to a file that is being processed in a stream or a result set from a query. 

The data correction feature also creates transactions of reference and they are regarded as separate processes. 

For more information refer to Transaction Safety.

Transactions Per Second (TPS)

This is a metric showing the number of transactions per second (tps) that can be processed by a given web service. This value can be calculated using this formula:

Tps value for a function = (Total number of output events handled by the function) / (duration of the stream)

...