I have a pretty big QVD file and am thinking about partition or breakup the file into smaller ones. Beside making the file more managable, I am just curious what other benefits might be. For example, will it reduce load time and enhance performance?
We are still in development stage, but just trying to think ahead. That siad, we eventually will have 36 months worth of data, and each month will have roughly 300 million rows, so I thought by breaking my original QVD file by YearMonth would make it easier to manage and maintain, and hopefully improve query performance.
I am also worry about user experience. Obviously, we will do what we can on the server side in terms of RAM and CPU, but I am just wondering if there is anything else we can do to improve performance or what QV best practices are when dealing with large volumn of data. For example,
does partitioning QVD into smaller ones help with user's queries?
If we have 36 months of data with total of, say, 500K account_id. What happens when a user select or drill down to one account_id and one month? Does it require QV to search through the entire record? Or QV is smart enough to know what to look for.
can we optimize QVD files?
p.s. I am still a QV newbie, so if my comments/questions do not make sense, I sincerely apologize.