App reloading from the last 20 hours ?? Need help

[vishalgoud]

Our final application with UI is relaoding from 19-dec-2017  13:59:32 to till.  almost 20 hours and still going on.

app details : earlier app size is 13 GB.

now few new columns are added to few tables so not sure about the size but definately more than 13 GB.

Prod server ram : 354 GB. 4 nodes.

For the first time facning this kind of issues Please help me.

Just explored more into log got to know that 7 left joins and 7 resident loads and 1 outer join and 1 concatination is taking more time than usual...

[vishalgoud]

Thanks again,

I have to add one more point here, in my fact extractor - it will exctract latest 2 months data and in the final tab i wrote one forloop which can be used to load the old qvds and add few new columns with default values to it and  store them in a different path with the same name..

that loop i wrote for 3 diff qvds after this i wrote exit script. PFB.

was suspecting the this loop is not ending there before exit script. that is the reason it is not ending.

YearData:
LOAD
  DISTINCT
  DATE(Date#(SubField(FileBaseName(),'_',3),'YYYYMMDD'),'YYYYMMDD')  AS YrMonth
From (qvd);

   let cnt=FieldValueCount('YrMonth'); 
  
FOR i =1  to cnt;
LET vYr_Split_StoreQVD = FieldValue('YrMonth',i);

  NoConcatenate
SBP_TEST_$(vYr_Split_StoreQVD):
LOAD
*,
0 as NoticePeriod,
0 as Workinghours,
0 as Rating

From (qvd);

STORE SBP_TEST_$(vYr_Split_StoreQVD) into E:\Development\ESI\MBP Test\Old Qvds\SBP_TEST_$(vYr_Split_StoreQVD).qvd(qvd);

DROP Table SBP_TEST_$(vYr_Split_StoreQVD);

Next i;

DROP TABLES YearData;

EXIT Script;

[sunny_talwar]

First thing to do is to add some TRACE statements to make sure you can see the progress in the log files.... Also, not sure if it makes any difference, I use $(i) inside FieldValue instead of just i..

YearData:
LOAD
  DISTINCT
  DATE(Date#(SubField(FileBaseName(),'_',3),'YYYYMMDD'),'YYYYMMDD')  ASYrMonth
From (qvd);

  LET cnt = FieldValueCount('YrMonth'); 
  TRACE $(cnt);

FOR i =1  to cnt;
LET vYr_Split_StoreQVD = FieldValue('YrMonth', $(i));

TRACE $(vYr_Split_StoreQVD);

Next i;

DROP TABLES YearData;

EXIT Script;

And once you see that nothing looks wrong above, try this

YearData:
LOAD
  DISTINCT
  DATE(Date#(SubField(FileBaseName(),'_',3),'YYYYMMDD'),'YYYYMMDD')  ASYrMonth
From (qvd);

  LET cnt = FieldValueCount('YrMonth'); 
  TRACE $(cnt);

FOR i =1  to cnt;
LET vYr_Split_StoreQVD = FieldValue('YrMonth', $(i));

TRACE $(vYr_Split_StoreQVD);

NoConcatenate
SBP_TEST_$(vYr_Split_StoreQVD):
LOAD
*,
0 as NoticePeriod,
0 as Workinghours,
0 as Rating

From (qvd);

STORE SBP_TEST_$(vYr_Split_StoreQVD) into E:\Development\ESI\MBP Test\Old Qvds\SBP_TEST_$(vYr_Split_StoreQVD).qvd(qvd);

DROP Table SBP_TEST_$(vYr_Split_StoreQVD);

Next i;

DROP TABLES YearData;

EXIT Script;

[vishalgoud]

Thanks sunny ,

Didnt get a chance to look into that Fact extractor again, But was reloaded the same final app(UI) in test env by poinitng that to prod qvds which we generated using the fact exctracotr...

Guess what now its reloaded in 50 Mins in test env with the same qvds.

but in prod it took more than 20 hours....Dont know how many surprises i will get in future , with this now things are bit more complicated..

Any pointers on this please..