I've run into a strange issue on some of our stored QVDs and was wondering if anyone else had encountered and new how to fix. Most of our QVDs store and contain the full set of metadata, however, some of our extractors (it seems potentially extractors that are quite old), store the QVD correctly but are missing metadata. For example, the below screenshot and code shows the metadata contained in one of these QVDs. You'll see that, for example, the NumberOfRows element is missing.
Any idea what might cause this and how I can fix it? Ideally don't want to copy all the code into a brand new app as I haven't yet identified all the QVDs that are impacted.
String: LOAD String%Table, %Key_QvdTableHeader_B94FCCAC68ED3E20 // Key to parent table: QvdTableHeader FROM [MyFile.qvd] (XmlSimple, Table is [QvdTableHeader/Fields/QvdFieldHeader/Tags/String]); QvdTableHeader: LOAD QvBuildNo, CreatorDoc, CreateUtcTime, SourceFileSize, TableName, [Fields/QvdFieldHeader/FieldName] as FieldName, [Fields/QvdFieldHeader/BitOffset] as BitOffset, [Fields/QvdFieldHeader/BitWidth] as BitWidth, [Fields/QvdFieldHeader/Bias] as Bias, [Fields/QvdFieldHeader/NoOfSymbols] as NoOfSymbols, [Fields/QvdFieldHeader/Offset] as Offset, [Fields/QvdFieldHeader/Length] as Length, [Fields/QvdFieldHeader/NumberFormat/Type] as Type, [Fields/QvdFieldHeader/NumberFormat/nDec] as nDec, [Fields/QvdFieldHeader/NumberFormat/UseThou] as UseThou, [Fields/QvdFieldHeader/NumberFormat/Fmt] as Fmt, [Fields/QvdFieldHeader/NumberFormat/Dec] as Dec, [Fields/QvdFieldHeader/NumberFormat/Thou] as Thou, %Key_QvdTableHeader_B94FCCAC68ED3E20 // Key for this table: QvdTableHeader FROM [MyFile.qvd] (XmlSimple, Table is [QvdTableHeader]);
Thanks for the suggestion.
Hmmm....so the XML header itself does seem to contain the header...Qlikview just doesn't seem to parse it correctly in the wizard as you've suggested. Any idea why that would be? I haven't previously come across anything like that. Loading in the header and assuming the columns are there works, so seems to just be the wizard that's not working correctly to build the script?
Maybe it's a bug in your release. But I could also imagine that's caused from some more restricted rules in regard what is a well-formed xml-structure or BNF or similar things. AFAIK these kind of rules are nowadays more strict as before (I remember of having seen such stuff within any of the release notes). And I wouldn't be surprised if there also some differences between the wizard/parser and the real process-engine (one more strict/forgiving as the other).
I think I would try to check this with some new, re-new and old qvd's - if those fields are there and could also be loaded or not. Further take a look on all clients/server which access these qvd's - if they have different releases it may cause such strange effects.
I concur with Marcus on his thoughts, but I wanted to toss out an idea for you to try, which would be to simply load that QVD into a new application and then immediately store it back out to a new QVD and then check that new QVD to see if it has the same problem or not. I would think it is most likely a defect in the release you are running, I do recall something along these lines from a while ago, if you are not on one of the following releases, you may want to try the closest one to see how it behaves.
12.20.21100/November 2017 SR10
12.30.20400/November 2018 SR4
12.40.20300/April 2019 SR3
Thanks for the response. I am running April 2019 SR1.
I recently ran into another issue and I'm wondering if this might be related (somewhat related to your suggestion to load in and then store again).
The site I'm working at is an old Qlik site (running since 2008). The applications I've noticed issues with are very old applications, some of them created close to that time. The other app I had an issue with was loading a table extremely slowly into memory (taking 6+ hours to load 130 million rows into memory which in our environment is very slow).
Resident load and loading from QVD were equally slow, but when I ran the same script in a new app the performance was significantly better. I couldn't understand why, and so I tried exporting the whole layout of the app in a macro and then importing that layout into a clean app. Suddenly reload performance was working as expected!
Is it possible that, for these older applications, that they're not being correctly "migrated" to newer versions of Qlik over time, or that something in the .QVW itself is broken in some unexplained way. The only change I made was to export and import the layout which I wouldn't have expected to make an impact. A similar process "fixed" the metadata storing I mentioned above.
Yep, I would concur completely with you given the new information on the 'old' apps, have seen similar behavior over the years, I started in '04 at the tail end of version 6, and the file formats have remained 'compatible' since version 6, but sometimes it is necessary to just recreate things in a new file. Actually, the easy thing to do may be to try to create prj files and the create a new version that way:
That is probably the best way to go to see what that does for you. The other options will likely be a bit more work, but again, if some of these apps are from version 8 or 9 etc., they could definitely be having some issues at this point. As I am sure you know, compatible does not always mean perfect, and I think you are definitely on the right track on this one, just shout if something more comes to light that does not make sense. What we may have to do is potentially have you/them create a support case if they can attach an app, and we might be able to get engineers to have a look at one of them to see if they can spot something, but if the rebuild works, not sure it will be worth the effort.