How can I extract whole year of raw files by using 1 script?

[#1]

Hi,

How can I automate the reloading without editing and reloading the Extractor.qvw and Transform.qvw per day.

This is how I manually Edit Script and Reload per day:

File-10-01-2016.csv  ----->  LET vToday=Num(MakeDate(2016,10,1));

File-10-02-2016.csv  ----->  LET vToday=Num(MakeDate(2016,10,2));

File-10-03-2016.csv  ----->  LET vToday=Num(MakeDate(2016,10,3));

File-10-04-2016.csv  ----->  LET vToday=Num(MakeDate(2016,10,4));

Problem 1:

My QlikView has error, I accidentaly rename 1 .qvw file while running the Management Console, after that Management Console hangs for almost 3 hours, that's the time I decided to "End Task" all that has QlikView on the Windows Task Manager. Upon opening the management console again , everything was messed up, "cannot reach connection" always appear etc.

Problem 2:

My Final .qvw uses Binary method but upon trouble shooting for almost half day, I just figured out that the "Binary" is not working and all of my QVD files are messed up. The data that I lost is almost half year. The approximate time for manually Extracting and Transforming is about 10~15minutes per day of data. It will took me 36~48 hours of continuous manually extracting and transforming data.

My Solution:

I will start Extracting and Transforming from the beginning to avoid error on the data.

My Scripts for Extractor.qvw

//LET vToday=Num(Today())-1;

LET vToday=Num(MakeDate(2016,10,1));

LET vTodayDate=Date('$(vToday)','YYYYMMDD');

LET vToday2=Upper(Date('$(vToday)','DD-MMM-YYYY'));

My Scripts for Transform.qvw

//LET vToday=Num(Today())-1;

LET vToday=Num(MakeDate(2016,10,1));

LET vTodayDate=Date('$(vToday)','YYYYMMDD');

Worse case scenario: This project was only turned over to me last month.

[Gysbert_Wassenaar]

I'd first extract all the data into one qvd file. Then I can later use that qvd file to do any transformations. Something like this should work:

LET vDayOfYear = DayNumberOfYear(Today());

HistoricalData:

LOAD 1 as Dummy AUTOGENERATE 0 ;

For vDay = 1 to $(vDayOfYear) 

     LET vToday = YearStart(Today()) + $(vDay) - 1;

     LET vDateString = Text(Date($(vToday),'DD-MM-YYYY'));

     CONCATENATE (HistoricalData)

     LOAD * FROM  [File-$(vDateString).csv] (utf8, txt, delimiter is ',' , embedded labels);    

Next

DROP FIELD Dummy;

STORE HistoricalData INTO HistoricalData.qvd (qvd);

If you'd rather work with each day's data separately then just replace the code inside the loop with whatever does what you need and get rid of the LOAD at the beginning and the DROP and STORE at the end.