I am working on optimizing an application. It is 800 MB in size and the issue it that is takes 60 GB to run the script (300 mill rows).
Well I did get it optimized (20GB) but now the file size is almost double 1,350 MB. I have isolated the script that causes the increase I am just unsure as to why.
I have a date field stored as a numeric value (e.g. 123 = January 2007).
----EDIT--- The date field is not a timestamp, it is a 3 character numeric field----
I need to get the month name from this field so loaded all dates into a mapping table along with the month name into a mapping table:
MonthMap:
mapping load
TimeValue,
MonthName(TimeValueEndDate) as [Month Name]
from
..\TIME.qvd (qvd);
and then apply map in the load script:
applymap('MonthMap', TimeValue) as [Month Name]
this is what causes the increase in file size of 1,350 MB.
If I instead take this out and do a left join:
Left join (table1)
Load TimeValue,
MonthName(TimeValueEndDate) as [Month Name]
from
..\TIME.qvd (qvd);
drop field TimeValue;
The resulting file size is 805K (No other changes were made)
I can use the left join without a problem and have implemented this into the app but I am still concerned about the affect that the applymap had on file size since I use this alot in my development. Any idea why the mapping load would cause that large of an increase in file size?
Message was edited by: ryan hicks
