Problem with Partial Load

[shane_spencer]

I'm set up a Partial reload to add new data in to Memory each day, however the new data is not linked to a seperate table like the existing data is:

for each File in FileList('*PerfMon_$(yestermonth).qvd')
//Only selects QVD Files to load data from if they're from yesterdays month
data: ADD Only LOAD Date,
     Server, 
     Hour, 
     Minute,
     Main_metric, 
     sub_Metric, 
     value
FROM '$(File)' (qvd)
WHERE Date='$(yesterdate)';
//Loads date only where the Date is yesterdays
next File

...this should be linked by Server to another table but the Partially Loaded data is not. Can any help?

[sebastiandperei]

Shane, I saw the app you sent.

The result of using Keep is showing that there aren't matching values between data and Server tables. Keep will only "keeps" the coincidences between both tables in the common field, and if you haven't any table (like your last app sended), is the reason that the field "Server" (common in both tables) hasn't any common value in both tables.

Check the "Server" column in GFA Utilisation Reporting Summary.xls and compare it with the values in your partial load. Is very extrange that these values in QV are empty.... Tell me, what kind of values do you have in this excel file? And what kind in *PerfMon*,qvd?

[shane_spencer]

The excel files has details about the server, Role, Application etc. I only want the data from that file if the Server exists in the "data" table.

HK000TKO4050_20120602.csv

is an example of the raw data I'm collecting

(n.b. I've got rid of the "Generate a report detailing the..." comment from future logs)

GSRS_Perfmon_processing.qvw

is the script I'm using for processing the hundreds of CSV files we have.

Part_processed_GSRS_201206.qvd

Is all these CSV's just loaded in to a single QVD for the month.

GSRS_Perfmon_201206.qvd

Is the processed "data" that gets loaded in to my Report/App (attached previously). As you can see there's some junk in there as the "date" somehow becomes a submetric which I think is where the blanks crop up in the Server column.

GFA Utilisation Reporting Summary.xls

Is the spreadsheet that contains further data about the servers that I link in my Report/App(attached previously).

...I can see the method I'm using isn't perfect but it's about the best I can do. 😕

[sebastiandperei]

Yes, the Submetric "date" is a problem. You can exclude these registers by Where sub_Metric<>'date'; at the load.

Also, like this way, you will get the field "value" with the right format (now, is date format, because of this entry in sub_Metric)

[shane_spencer]

Thanx! I had actually used something similar:

Where Len(Server)>0;

as I seem to have got some other junk data in some of my source files. But if I can get that cleaned up I can see that removing the 'data' from 'submetric' earlier on in the Processing would be a good idea.

[sebastiandperei]

I think your idea is better!!!