Does anybody know what happens "behind the scenes" during a drop table statement? Does QV drop the table all at once or does it iterate through the table and drop each field? I'm seeing unusual delays while dropping very wide tables, which doesn't seem to make sense if QV drops the table all at once...
I don't think I've dropped any tables with that many fields. I can't remember noticing a slowdown on the sizes of tables that I have dropped. I'm curious because I like to know things, not because I have a similar problem that needs to be solved. Sorry!
About all I can do is theorize, try to think it through based on what I've seen. Worth a shot.
It seems likely that the default QlikView joining by field name is only done at the end. As evidence, that's when you see the message about synthetic keys being formed. I also can create some temporary tables that would be really ugly if joined during the load, and then just remove them or resolve the problems before the end of the load, and never have a problem. So an initial conclusion might be that each table is a completely separate chunk of memory, and could be dropped whole and entire, since they aren't joined until the end of the load.
But I think that's taking the evidence too far. Because it also seems like QlikView is keeping track, even during the load, of each field and all the values for that field, regardless of on which table those values appear. As evidence, we can use expressions like fieldvalue('Date',iterno()) without referring to any table explicitly. I suspect that the only way to do that efficiently (and it's VERY efficient) is to have an explicit list in memory of every value for the field. There may well be other such memory structures that are built during the load.
So I would theorize that yes, there is a loop for each field. At a minimum, it looks like QlikView would have to go through every field on the table, and remove all values of the field from its internal list, if and only if that value doesn't also appear on some other table. I can see how that could take a little time. I'd expect it to be fast, in the sense that I'd expect QlikView to keep track of which values are on which tables and not have to scan each table all over again. But I can see how if you were dropping 100 fields, and particularly if those fields were on multiple tables or had a lot of values, it might take QlikView a little time to work it all out. And that might not be the only memory structure that it has to fix when you drop a table. I'd actually be surprised if it were the only one.
Well, I don't know what it's doing. But it can't be iteratively dropping the fields in the DROP FIELD sense, because that drops the field completely, not just from one table. But maybe there's some sort of loop internally. I'd certainly be curious.
I don't think I've dropped any tables with that many fields. I can't remember noticing a slowdown on the sizes of tables that I have dropped. I'm curious because I like to know things, not because I have a similar problem that needs to be solved. Sorry!
About all I can do is theorize, try to think it through based on what I've seen. Worth a shot.
It seems likely that the default QlikView joining by field name is only done at the end. As evidence, that's when you see the message about synthetic keys being formed. I also can create some temporary tables that would be really ugly if joined during the load, and then just remove them or resolve the problems before the end of the load, and never have a problem. So an initial conclusion might be that each table is a completely separate chunk of memory, and could be dropped whole and entire, since they aren't joined until the end of the load.
But I think that's taking the evidence too far. Because it also seems like QlikView is keeping track, even during the load, of each field and all the values for that field, regardless of on which table those values appear. As evidence, we can use expressions like fieldvalue('Date',iterno()) without referring to any table explicitly. I suspect that the only way to do that efficiently (and it's VERY efficient) is to have an explicit list in memory of every value for the field. There may well be other such memory structures that are built during the load.
So I would theorize that yes, there is a loop for each field. At a minimum, it looks like QlikView would have to go through every field on the table, and remove all values of the field from its internal list, if and only if that value doesn't also appear on some other table. I can see how that could take a little time. I'd expect it to be fast, in the sense that I'd expect QlikView to keep track of which values are on which tables and not have to scan each table all over again. But I can see how if you were dropping 100 fields, and particularly if those fields were on multiple tables or had a lot of values, it might take QlikView a little time to work it all out. And that might not be the only memory structure that it has to fix when you drop a table. I'd actually be surprised if it were the only one.
