https://community.qlik.com/t5/Talend-Studio/Extract-Data-from-file-and-iterate-over-DB/m-p/2452811#M140646 <P>hi folks,&nbsp;</P> <P>&nbsp;</P> <P>i'm designing a job read +10K columns from file and take some columns and define them as context variable to run oracle DB function and then insert it to parquet file&nbsp;</P> <P>&nbsp;</P> <P>the design is below is working fine when testing insert the data to postgres table but now i need to create a parquet file , my issue with parquet file gives only action (create/overwrite) i cannot append on the created file before so by the end , only 1 record will be inserted in the file (because of iteration)</P> <P><span class="lia-inline-image-display-wrapper lia-image-align-inline" image-alt="mshafeeq_0-1715785833537.png" style="width: 400px;"><img src="https://community.qlik.com/t5/image/serverpage/image-id/166152i5A9051071B6FF319/image-size/medium?v=v2&amp;px=400" role="button" title="mshafeeq_0-1715785833537.png" alt="mshafeeq_0-1715785833537.png" /></span></P> <P>&nbsp;</P> Wed, 15 May 2024 15:13:11 GMT mshafeeq 2024-05-15T15:13:11Z https://community.qlik.com/t5/Talend-Studio/Extract-Data-from-file-and-iterate-over-DB/m-p/2452811#M140646 <P>hi folks,&nbsp;</P> <P>&nbsp;</P> <P>i'm designing a job read +10K columns from file and take some columns and define them as context variable to run oracle DB function and then insert it to parquet file&nbsp;</P> <P>&nbsp;</P> <P>the design is below is working fine when testing insert the data to postgres table but now i need to create a parquet file , my issue with parquet file gives only action (create/overwrite) i cannot append on the created file before so by the end , only 1 record will be inserted in the file (because of iteration)</P> <P><span class="lia-inline-image-display-wrapper lia-image-align-inline" image-alt="mshafeeq_0-1715785833537.png" style="width: 400px;"><img src="https://community.qlik.com/t5/image/serverpage/image-id/166152i5A9051071B6FF319/image-size/medium?v=v2&amp;px=400" role="button" title="mshafeeq_0-1715785833537.png" alt="mshafeeq_0-1715785833537.png" /></span></P> <P>&nbsp;</P> Wed, 15 May 2024 15:13:11 GMT https://community.qlik.com/t5/Talend-Studio/Extract-Data-from-file-and-iterate-over-DB/m-p/2452811#M140646 mshafeeq 2024-05-15T15:13:11Z https://community.qlik.com/t5/Talend-Studio/Extract-Data-from-file-and-iterate-over-DB/m-p/2452980#M140649 <P>Hi&nbsp;</P> <P>You can use a tUnite component before tFileOutputParquet to merge all data before they are written into parquet file.&nbsp;</P> <P>Regards</P> <P>Shicong</P> Thu, 16 May 2024 05:01:23 GMT https://community.qlik.com/t5/Talend-Studio/Extract-Data-from-file-and-iterate-over-DB/m-p/2452980#M140649 Shicong_Hong 2024-05-16T05:01:23Z