Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 
paulferguson80
Contributor III
Contributor III

Buffering a In Day Reload to improve reload time

Hi All,

This is just a question of peoples experience of using BUFFERS in development of applications.

In short we have files that reload every hour with In day and historical data.

We have set up the front end application to collect data from TODAY and HISTORICAL qvd files that cover the period we are reporting on.

The HISTORICAL has between 2 and 15 million rows dependant on when in the month is being reloaded, the TODAY has at the most half a million by the end of the day and we allow the application to auto concatenate both loads into one as they contain the same field names and we do not buffer the TODAY load as this data will transition throughout the day.

The HISTORICAL load isnt an optimised load due to a small amount of work needed to be done by the report front end.

I have done some testing over the last couple of days which has been successful it has done as its supposed to and done a fresh reload first thing and buffered for the rest of the day.

My question is has anyone done anything similar and encountered any problems?

If not maybe it would be useful for others?

LET VLASTRELOAD = date(reloadtime());

LET VTODAY = DATE(TODAY(),'DD/MM/YYYY');

LET VBUFFER = IF($(VLASTRELOAD) = $(VTODAY) , 'BUFFER');

FULL_TABLE:

$(VBUFFER) LOAD

A,

B,

C

FROM HISTORICAL.QVD

LOAD

A,

B,

C

FROM TODAY.QVD

Many Thanks
Paul Ferguson


0 Replies