Modify Attunity Envelope to adhere to CloudEvents Specification

[GonzaloPerezPrim]

Hi Guys:

We would like to know what is the best approach to modify the Attunity Envelope sent by the Qlik Replicate to adhere to some of the principles of the CloudEvents 1.0 specification. https://cloudevents.io/

The Cloud Events Specification seems to be very flexible and some of the fields included at the Attunity Envelope comply to it. This is the Attunity Envelope in YAML for readability purposes (Please imagine this is a JSON file)

type: record
name: MessageEnvelope
fields:
    - name: magic
      type: fixed
      size: 5
    - name: type
      type: string
    - name: headers
      type: map
      values: string
    - name: messageSchemaId
      type: string
    - name: messageSchema
      type: string
    - name: message
      type: bytes

This is an example of the Cloud Events Specification 1.0 in YAML.

specversion: '1.0'
type: com.example.someevent
source: /mycontext
id: C234-1234-1234
time: '2018-04-05T17:31:00Z'
comexampleextension1: value
comexampleothervalue: 5
datacontenttype: application/json
data:
    appinfoA: abc
    appinfoB: 123
    appinfoC: true

On our understanding, this is similar to the envelope 

type: record
name: MessageEnvelope
fields:
    - name: specversion
      type: string
    - name: type
      type: string
    - name: source
      type: string
    - name: time
      type: timestamp
      value: '2018-04-05T17:31:00Z'
    - name: comexampleextension1 # This is extension
      type: string
      value: value
    - name: comexampleothervalue
      type: string
      value: 5
    - name: id
      type: string
    - name: datacontenttype
      type: string
      value: application/json
    - name: data
      type: json
      data:
        appinfoA: abc
        appinfoB: 123
        appinfoC: true

Just to reformulate the question, what is the best approach to extend the Message Envelope from Qlik Replicate to accommodate the fields from the Cloud Events specifications which are not found. This is very useful as it would allow to simplify the monitoring among other things.

Kind regards

Gonzalo Pérez-Prim

[Ola_Mayer]

Hello Gonzalo,

Your question requires research on our end. We are in touch with the team who can answer this question and they will respond shortly.

[EyalSilner · Accepted solution]

Hi Gonzalo,

We do not easily support direct production of CloudEvents specification and it may be supported as a built-in option in the future if the format becomes prevalent. But we still allow those message manipulations using mainly a combination of two of our features:

1. Global and Table levels transformation: for adding/removing/renaming target columns, dynamically values as well as fix attributes.
2. Custom Message formatting: where you can flatten our message headers and data fields, can change names like “data”, include/exclude headers and more.

Hope it helps,

Eyal Silner,

BigData Technical Product Manager of Qlik Replicate.