I am using incremental load for to read data from files and store into QVD. The first time the data is read from files and read into QVD
The next time i run the reload, 0 files are read from files, since there is no new file, and the rest data is read from QVD files...I concatenate both and store it back into QVD.
When i see the log file, the same number of rows are being stored into QVD. However, The QVD file has changed from 995 kb to 907 kb. Can someone tell me why has the size changed even if there was no change in the number of rows. The script that I am using is given below. may be i am doing something wrong. Can someone please guide me
PATH: LOAD A as File_Type, B as File_Path FROM [Paths.xls] (biff, embedded labels, table is Sheet1$);
DATA_PATH: NOCONCATENATE LOAD File_Path RESIDENT PATH where File_Type='txt';
LET FILE_PATH_VAR=PEEK('File_Path',0,'DATA_PATH');
QVD_PATH: NOCONCATENATE LOAD File_Path RESIDENT PATH where File_Type='qvd';
LET QVD_PATH_VAR=PEEK('File_Path',0,'QVD_PATH');
IF FileSize('$(QVD_PATH_VAR)\MMC HIST.QVD') > 0 THEN [MMC MAX WEEK]: NOCONCATENATE LOAD MAXSTRING([MMC Plan Week]) AS [MAX PLAN WEEK] FROM [$(QVD_PATH_VAR)\MMC HIST.QVD](qvd); LET MMC_MAX_PLAN_WEEK=PEEK('MAX PLAN WEEK',0,'MMC MAX WEEK'); ELSE SET MMC_MAX_PLAN_WEEK='2001W01'; END IF
[MMC HIST]: NOCONCATENATE LOAD
trim(PLAN_WEEK) as [MMC Plan Week], PRODUCT_FAMILY AS [PF:MMC Product Family], trim(PRODUCT_FAMILY) AS [Product Family], trim(PLAN_PERIOD) as [MMC Plan Period], trim(PLANT) AS [MMC Plant], trim(WEEK) as [MMC Week], trim(PERIOD) as [MMC Period], AUTONUMBERHASH256(trim(PRODUCT_FAMILY),trim(PLANT),trim(PERIOD)) as [PF:PLNT:PRD], AUTONUMBERHASH256(trim(PRODUCT_FAMILY),trim(PLANT)) as [PF:PLNT], SUM(QTY) AS [MMC QTY] FROM [$(FILE_PATH_VAR)\MEASURES\HIST MMC DSA Hist Plans*.txt] (txt, codepage is 1252, embedded labels, delimiter is '\t', msq)
WHERE PLAN_WEEK>'$(MMC_MAX_PLAN_WEEK)' AND MEASURE_NAME='PACK MMC' AND PLAN_PERIOD< PERIOD
GROUP BY PLAN_PERIOD , PLAN_WEEK, PERIOD, WEEK , PRODUCT_FAMILY, PLANT;
IF FileSize('$(QVD_PATH_VAR)\MMC HIST.QVD') > 0 THEN CONCATENATE LOAD [MMC Plan Week], [Product Family], [MMC Plan Period], [MMC Plant], [MMC Week], [MMC Period], PF:PLNT:PRD, PF:PLNT, [MMC QTY] FROM [$(QVD_PATH_VAR)\MMC HIST.QVD](qvd); END IF
STORE [MMC HIST] INTO [$(QVD_PATH_VAR)\MMC HIST.QVD](qvd);
