topic Re: Genesis LIVE data extraction in Talend Studio

Genesis LIVE data extraction

Miya2 — Fri, 15 Nov 2024 22:41:51 GMT

Hi Team ,

Good Day!! I have a requirement to connect to genesis API and retrieve users data on an iterative manner and this needs to happen every 20 secs (500 users per every 20 secs) . I was able to do a rest call and make API calls and pull the reqd info but this seems to be challenging as the rate limit of the API is far less than the volume of data i am pulling and my job fails after few extracts saying Rate Limit exceeded .

Is there any alternative approach to pull more like LIVE data from the API by any other approach ?

I have also heard of the notification API which might help me achieve this , but to integrate this with talend , I would need a Websocket connection and i have absolutely no clue on how to establish this .

Can someone please help .

Thanks in advance!!

Re: Genesis LIVE data extraction

Anonymous — Mon, 01 Aug 2022 02:44:34 GMT

This is a limitation defined by the provider, go check with API provider whether it is possible to increase the rate limit number by changing the settings? Any parameter is available for increasing the number of users retrieved in a time?

Regards

Shong

Re: Genesis LIVE data extraction

Miya2 — Mon, 01 Aug 2022 02:58:04 GMT

Yes @Shicong Hong This is limitation from the API and it is non configurable. Hence looking for an alternative approach .

Re: Genesis LIVE data extraction

Anonymous — Mon, 01 Aug 2022 03:09:44 GMT

so we can do is to reduce the number of call in Talend. About notification API you mentioned, I have no information about it, any documentation or link is available?

Re: Genesis LIVE data extraction

Miya2 — Tue, 02 Aug 2022 09:32:38 GMT

Hi @Shicong Hong here's the link :

https://developer.genesys.cloud/notificationsalerts/notifications/