Hi guys, In a directory I get daily (CSV) files that contain data that I need to add to my existing data. How can I do this without reading all the data in all csv files?
You can load all the csv files once into one table and then store that table into a qvd file. The next time you need only load the latest csv file and all the data from the qvd file and then store the complete table again in the qvd file. It's called incremental loading. The trick then is to know which csv file is the latest. Perhaps your csv files have a date in their file name and you can use that. Or you could check the creation times of the files. Here are some discussions on that topic:
