Call an API from Replicate

[MDAtl]

Hello:

We have a need to transform as we replicate using a 3rd party transformer that provides an API. Is there a way to invoke an API for each row as it is being replicated?

Thanks,

Manas.

[john_wang · Accepted solution]

Hello @MDAtl ,

Welcome to Qlik Community forum and thanks for reaching out here!

Qlik Replicate supports rich built-in transformations, and we also support User-defined transformations to extend calling tailor-made user modules, the UDTF should be written in C.

After the UDTF is compiled and setup done, the transformation can be seen in "User Defined" functions list, it take effect for each row in the replication task.

Hope this helps.

John.

[Heinvandenheuvel · Accepted solution]

@john_wang >> After the UDTF is compiled and setup done, the transformation can be seen in "User Defined" functions list, it take effect for each row in the replication task.

Nit-picking:  such UDT is called once for each column it is associated with for each row being processed.

The argument passed to the UDT is a pointer to the input column text. The return value is a pointer to the a string buffer with transformed text to return. There are pointers to other data types available, but last time I looked specifically the CLOB data type was NOT supported.

The Replicate installation comes with an an example. For Windows you can find that under C:\Program Files\Attunity\Replicate\addons . Brave souls should be able to figure out how to set this up (I did :-). Mere mortals may want to engage Qlik Professional Services to make sure to create a robust solution and to be aware of the latest  restrictions and features.

Hein.

[MDAtl]

Thank you @john_wang  and @Heinvandenheuvel . I now have enough info to get started.

[john_wang]

Thank you so much for your great support @MDAtl !

[noobDE]

Hi, mere mortal here 🙂

Could you maybe give some high-level pointers for this?

I have the same scenario where I use curl to call the API. I have been trying but have still had no luck.

For example, when creating the .dll file, do we need to link the curl library statically or dynamically?

[john_wang]

Hello @noobDE ,

You don't need to link the curl library in the C project since the UDTF operates within the Replicate framework. The 'curl' program and Replicate APIs function as independent modules.

Hope this helps.

John.

[noobDE]

Hi @john_wang , thank you so much for responding.

Just to clarify, the API I'd like to call is an external endpoint (let's say we have our company's internal API and we want to send POST request to it from Replicate), and here we've created the program using curl.h on our main function.

So this was the part that I would appreciate some pointers. We've managed to compile the sample code MyTransformation but still no luck when it comes to the calling API part.

If you can give list of factors/components as the checklist that we need to make sure that the final .dll can be loaded in Replicate.

Again, thank you for your help.

[Heinvandenheuvel]

First off, I think you need to create your own topic, with good, clear subject line.

Second, I still don't understand the main process flow. You wrote "I have the same scenario where I use curl to call the API."

The basic Replicate flow is 

1. for each source DML event
2. check transformation
3. if UDT(s) present call UDT(s) with param
4. update target column with transformed value
5. apply to target.

I suspect the in step 3 the you want the UDT code that is belng called by Replicate to activate a curl command (which is a program!) to poke another server and return a value to the udt. Yikes. That's a whole lot of overhead in a restricted context calling environment. Hopefully you mean that you calling LIBCURL that's one layer easier.

Free advice

• Carefully study the Replicate User Guide - K Replicate add-ons API
• Carefully study the Replicate User Guide - K Replicate add-ons API 5 more times
• build a test UDT which returns "Hello World"
• build a test UDT which returns "Hello {first-passed-parameter} World {second passed parameter}"
• Try call LIBCURL - login successful? 
• Try call LIBCURL - login, post, result. Works?
• Think about how to retain login context (memory pool?) - which might NOT be possible

If this fails - create fresh topic clearly stating where it all failed, maybe as soon as while linking, what error messages.

Personally I suspect this is one  level too complex for a UDT, but I'm aware of UDT calling out for encryption methods in libraries and other clever stuff. You may need Qlik Profession Services to help.

Hein

•  
•