Do not input private or sensitive data. View Qlik Privacy & Cookie Policy.
Skip to main content

Announcements
Join us in Toronto Sept 9th for Qlik's AI Reality Tour! Register Now
cancel
Showing results for 
Search instead for 
Did you mean: 
Anonymous
Not applicable

How to access data in batches

First, I am new to component creation and I have very specific requirement. I am creating new component where I need to send data to web services. I am able to do so but I am sending data as single row in each request and this is not acceptable solution. My plan is to read multiple rows, send it to web-services for processing, retrieve the results, and then send it to output. I am not sure, how I can do this. Same time, web-services can be bottle neck and I can read data in multiple batches and spawn multiple threads to send request to web-services from each thread. So it it possible. Can I use any existing component for reference. Any help would be appreciated. Thanks
For ex, I have a .csv file having 10000 rows and I need to processing row no 2 using my web-server. Ideally, what I want is read data in 1000?s batch, create 5 threads to make web-services call and send/retrieve data, and put it in output in same order that it received.
Labels (3)
1 Reply
Anonymous
Not applicable
Author

Hi,
Sorry for the delay.
There is no such a component just for "thread process" but usually we set the option ?Enable parallel execution' on iterate to achieve that.
The job flow is tfileinputdelmited--main--tFlowToiterate--iterate--tFixedFlowInput--main--tWebservice--tLogRow
check out the option 'Enable parallel execution' on iterate link.
See the following screenshot
Best regards
Sabrina
0683p000009MDFy.png