Do not input private or sensitive data. View Qlik Privacy & Cookie Policy.
Skip to main content

Announcements
Want to see what's currently in public preview? We've pulled it all together in one place. Check it out here
cancel
Showing results for 
Search instead for 
Did you mean: 
Anonymous
Not applicable

How to access data in batches

First, I am new to component creation and I have very specific requirement. I am creating new component where I need to send data to web services. I am able to do so but I am sending data as single row in each request and this is not acceptable solution. My plan is to read multiple rows, send it to web-services for processing, retrieve the results, and then send it to output. I am not sure, how I can do this. Same time, web-services can be bottle neck and I can read data in multiple batches and spawn multiple threads to send request to web-services from each thread. So it it possible. Can I use any existing component for reference. Any help would be appreciated. Thanks
For ex, I have a .csv file having 10000 rows and I need to processing row no 2 using my web-server. Ideally, what I want is read data in 1000?s batch, create 5 threads to make web-services call and send/retrieve data, and put it in output in same order that it received.
Labels (3)
1 Reply
Anonymous
Not applicable
Author

Hi,
Sorry for the delay.
There is no such a component just for "thread process" but usually we set the option ?Enable parallel execution' on iterate to achieve that.
The job flow is tfileinputdelmited--main--tFlowToiterate--iterate--tFixedFlowInput--main--tWebservice--tLogRow
check out the option 'Enable parallel execution' on iterate link.
See the following screenshot
Best regards
Sabrina
0683p000009MDFy.png