<?xml version="1.0" encoding="UTF-8"?>
<rss xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns:taxo="http://purl.org/rss/1.0/modules/taxonomy/" version="2.0">
  <channel>
    <title>topic How to access data in batches in Talend Studio</title>
    <link>https://community.qlik.com/t5/Talend-Studio/How-to-access-data-in-batches/m-p/2250501#M34726</link>
    <description>First, I am new to component creation and I have very specific requirement. I am creating new component where I need to send data to web services. I am able to do so but I am sending data as single row in each request and this is not acceptable solution. My plan is to read multiple rows, send it to web-services for processing, retrieve the results, and then send it to output. I am not sure, how I can do this. Same time, web-services can be bottle neck and I can read data in multiple batches and spawn multiple threads to send request to web-services from each thread. So it it possible. Can I use any existing component for reference. Any help would be appreciated. Thanks&lt;BR /&gt;For ex, I have a .csv file having 10000 rows and I need to processing row no 2 using my web-server. Ideally, what I want is read data in 1000?s batch, create 5 threads to make web-services call and send/retrieve data,  and put it in output in same order that  it received.</description>
    <pubDate>Sat, 16 Nov 2024 12:08:32 GMT</pubDate>
    <dc:creator>Anonymous</dc:creator>
    <dc:date>2024-11-16T12:08:32Z</dc:date>
    <item>
      <title>How to access data in batches</title>
      <link>https://community.qlik.com/t5/Talend-Studio/How-to-access-data-in-batches/m-p/2250501#M34726</link>
      <description>First, I am new to component creation and I have very specific requirement. I am creating new component where I need to send data to web services. I am able to do so but I am sending data as single row in each request and this is not acceptable solution. My plan is to read multiple rows, send it to web-services for processing, retrieve the results, and then send it to output. I am not sure, how I can do this. Same time, web-services can be bottle neck and I can read data in multiple batches and spawn multiple threads to send request to web-services from each thread. So it it possible. Can I use any existing component for reference. Any help would be appreciated. Thanks&lt;BR /&gt;For ex, I have a .csv file having 10000 rows and I need to processing row no 2 using my web-server. Ideally, what I want is read data in 1000?s batch, create 5 threads to make web-services call and send/retrieve data,  and put it in output in same order that  it received.</description>
      <pubDate>Sat, 16 Nov 2024 12:08:32 GMT</pubDate>
      <guid>https://community.qlik.com/t5/Talend-Studio/How-to-access-data-in-batches/m-p/2250501#M34726</guid>
      <dc:creator>Anonymous</dc:creator>
      <dc:date>2024-11-16T12:08:32Z</dc:date>
    </item>
    <item>
      <title>Re: How to access data in batches</title>
      <link>https://community.qlik.com/t5/Talend-Studio/How-to-access-data-in-batches/m-p/2250502#M34727</link>
      <description>Hi, 
&lt;BR /&gt;Sorry for the delay. 
&lt;BR /&gt;There is no such a component just for "thread process" but usually we set the option ?Enable parallel execution' on iterate to achieve that. 
&lt;BR /&gt;The job flow is tfileinputdelmited--main--tFlowToiterate--iterate--tFixedFlowInput--main--tWebservice--tLogRow 
&lt;BR /&gt;check out the option 'Enable parallel execution' on iterate link. 
&lt;BR /&gt;See the following screenshot 
&lt;BR /&gt;Best regards 
&lt;BR /&gt;Sabrina 
&lt;BR /&gt; 
&lt;span class="lia-inline-image-display-wrapper" image-alt="0683p000009MDFy.png"&gt;&lt;img src="https://community.qlik.com/t5/image/serverpage/image-id/130998i09DA4885B84CF04B/image-size/large?v=v2&amp;amp;px=999" role="button" title="0683p000009MDFy.png" alt="0683p000009MDFy.png" /&gt;&lt;/span&gt;</description>
      <pubDate>Mon, 21 Jan 2013 06:14:35 GMT</pubDate>
      <guid>https://community.qlik.com/t5/Talend-Studio/How-to-access-data-in-batches/m-p/2250502#M34727</guid>
      <dc:creator>Anonymous</dc:creator>
      <dc:date>2013-01-21T06:14:35Z</dc:date>
    </item>
  </channel>
</rss>

