topic paralellized script to speed it up. in QlikView https://community.qlik.com/t5/QlikView/paralellized-script-to-speed-it-up/m-p/431636#M699419 <HTML><HEAD></HEAD><BODY><P>Hi to everyone that reads.</P><P></P><P>I have a scripting process that download data from files, and once it made all the calculations, split the results into several files splited by year-month. the files has the data in such way, but it can contains data from older periods.</P><P></P><P>So I start processing them, and the firsts ones make it through very fast, but once I filled several periods (last 3 years) the process slow down. The slowing, is not because file size or type of data, is because the spliting has to rearrange the data in several files across all periods. </P><P></P><P>The problem I face, is that I can fill one period of data in 1 minute without no problem. If the file has only one period, the total time of the script is something like 4 minutes. But the splitting, take 1 minute for each period. So If I have to fill 36 periods (even If I only need to insert one record), It tooks 36 minutes. Take in mind that this has to be done for 600 files. And this is a test, because we have to do the same process for other things.</P><P></P><P>So I have the idea to paralellize it somehow, but I don't see a way to call for another qvw inside my script, or even stop it from working while I call another one with parameters, to try to run 5 or more at the same time. We have several cpu´s on the server, so that is no problem.</P><P></P><P>Is there a way to accomplish this??? </P></BODY></HTML> Wed, 12 Dec 2012 17:42:46 GMT 2012-12-12T17:42:46Z paralellized script to speed it up. https://community.qlik.com/t5/QlikView/paralellized-script-to-speed-it-up/m-p/431636#M699419 <HTML><HEAD></HEAD><BODY><P>Hi to everyone that reads.</P><P></P><P>I have a scripting process that download data from files, and once it made all the calculations, split the results into several files splited by year-month. the files has the data in such way, but it can contains data from older periods.</P><P></P><P>So I start processing them, and the firsts ones make it through very fast, but once I filled several periods (last 3 years) the process slow down. The slowing, is not because file size or type of data, is because the spliting has to rearrange the data in several files across all periods. </P><P></P><P>The problem I face, is that I can fill one period of data in 1 minute without no problem. If the file has only one period, the total time of the script is something like 4 minutes. But the splitting, take 1 minute for each period. So If I have to fill 36 periods (even If I only need to insert one record), It tooks 36 minutes. Take in mind that this has to be done for 600 files. And this is a test, because we have to do the same process for other things.</P><P></P><P>So I have the idea to paralellize it somehow, but I don't see a way to call for another qvw inside my script, or even stop it from working while I call another one with parameters, to try to run 5 or more at the same time. We have several cpu´s on the server, so that is no problem.</P><P></P><P>Is there a way to accomplish this??? </P></BODY></HTML> Wed, 12 Dec 2012 17:42:46 GMT https://community.qlik.com/t5/QlikView/paralellized-script-to-speed-it-up/m-p/431636#M699419 2012-12-12T17:42:46Z