Autonumber in Script

[QFanatic]

hi all,

so as I understand I cannot use AutonumberHash if the data gets reloaded in 2 separate reloads. What I've seen from Help files is that I should use Hash128 etc, but that returns fields that are very long...

What do you do guys generally do to get a persistent value across loads?

Thanks

[marcus_sommer]

What's the aim for it? Why not just keeping the origin values - especially if they should be used within several applications?

- Marcus

[QFanatic]

Hi Marcus,

The key fields may potentially be made of of huge values - like '155.98.1237.263.178.32'. I though using autonumberhash might make things better if I could give a numeric value, say 9 for my example, instead of Qlik having to evaluate that terrible long value.

Thanks

[martinpohl]

so you have to load the datas in the same script, create an autonumberfield and store them into qvd. Then you can load the datas into your apps.

Regards

[QFanatic]

Hi Martin thank you for your reply.

My challenge comes that I create qvds in one load and comaume them in
another where I load a table from the dB as well based on the key values
from the qvds...

[marcus_sommer]

So the topic is not to use this (autonumber) key across multiple applications else you want to keep the qvd's smaller? How many records and distinct field-values are there? Does all field-values look like your example or are there more differences in length, numeric vs. alpha-numeric or strings, blanks, NULL's? Are any logics included?

- Marcus

[QFanatic]

Hi Marcus

I want to be able to reduce the size of the QVD's essentially by using Autonumber on those long key fields. Also make linking between tables that both contain these values, quicker.

The field values will always look like the example I gave ('155.98.1237.263.178.32') but will be different for every single key value of the Table - and we're talking billions of rows.

[marcus_sommer · Accepted solution]

Your dataset seems quite large. More important as the number of records is the number of distinct field-values. Means if there are just a few thousands distinct values by billions of rows your intended optimization won't be really significantly. But if there are quite many distinct values it would be different (just keep in mind that you could have max. 2 billions of distinct field-values within a single field).

I think I would in your case investigate if there is any logic within the values included so that you may capable to left the first or last numbers and/or replace/change other parts. If this isn't possible I would split the value into two pure numeric fields. In the target-application you could string-concat both parts within an autonumber() to get a valid key and to keep the RAM as small as possible.

- Marcus  

[QFanatic]

Thank you Marcus, your advice has been great.

[QFanatic]

just to be sure I understand you correctly, if the number of distinct values where more, the Autonumber would have a significant influence?

thanks