Data Structure

[naziralala]

Hello Everyone,

I have data that is in the form of a daily snapshot of the activity for that day.

I have created a key, by concatenating 3 fields and loaded 3 files (Snapshot Date-1/1/2016, 2/1/2016 and 3/1/2016)

Key                                        Snapshot date         Date1                   date2          Status

58 / J 234000 25 / REAL     1/1/2016                    1/1/2016          3/1/2016         REAL

58 / J 234000 25 / REAL     2/1/2016                     1/1/2016          3/1/2016         REAL

58 / J 234000 25 / INR        2/1/2016                    1/1/2016          3/1/2016          INR 

58 / J 234000 25 / REAL     3/1/2016                    1/1/2016          3/1/2016          REAL

58 / J 234000 25 / INR        3/1/2016                     1/1/2016          3/1/2016          INR 

58 / J 234000 25 / OUTR    3/1/2016                      1/1/2016          3/1/2016         OUTR 

Please advise if incremental load should be used here?

Should the Snapshot date be a part of the key?

Also, there is a one to many relationship existing on the Code J 234000 25...

Thanks in advance.

Nazira

[swuehl]

I think we can only answer your questions if we know a little more detailed what you are trying to do with your data.

For example, where is the Key linking to?

What kind of reports are you trying to create from that data?

W.r.t. using an incremental load or not, are you thinking about storing your table to a  QVD, then append only the latest

snapshot when reloading?

[naziralala]

Yes, I am trying to store only the latest snapshot date data into a qvd.

Also, what will be the implications if I do not store the other two snapshot dates.

To make any calculations, I need a unique key and not duplicates..but I have duplicates here..

Please advise.

Nazira

[swuehl]

I am a little confused. You are trying to keep only the latest snapshot date in your final model?

Why do you load all snapshots then?

I can't answer what the implications are if you do one or the other without knowing what you are trying to achieve.

Well, maybe try a different approach here in communication. Could you post some sample lines of input data records for each snapshot date, and then try to describe what you want to report from that data?

[naziralala]

I am loading all for testing.

But, when i am trying to load the last snapshot date data, by sorting first, I am getting only two entries on the last Snapshot date as below.

Key                                        Snapshot date         Date1                   date2          Status

58 / J 234000 25 / REAL     3/1/2016                    1/1/2016          3/1/2016          REAL

58 / J 234000 25 / INR        3/1/2016                     1/1/2016          3/1/2016          INR

and missing out the entry below:

58 / J 234000 25 / OUTR    3/1/2016                      1/1/2016          3/1/2016         OUTR

Is this due to the one to many relationship with another table with fields as below:

Date, Rev, Code, Desc, %Key, where %Key is also in the above table.

I am using XML file here and %Key is the XML file connector to the two tables.

Thanks,

Nazira

[swuehl]

How do you filter your snapshot data to the latest snapshot? Could you post your code snippet?