How to make Cartesian product easier because of large databbb

[uacg0009]

Hi all,

Because I have a large data, if I do the Cartesian product for all data, my computer will crash.

But I have finished the calculation what I want using small example. I just want to know if there is any better way to fulfill it.

I will give my example to you. I have data:

I want to calculate if a vip buy a selected ITEM, what ITEM did he buy in the past 12 months.

so I join the table and use interval to calculate the flag:

then it works:

But like what I said above, when I have a large data, the Cartesian product will occupy all my memory(64GB),

so do you have any ideas to make it easier?

Thank you so much.

p.s.:actually the column CAT1 is not necessary at first, but it is necessary for the before item, it comes from another table called ITEM.

p.s.1:now I just have one idea, do a delta load. just like load year(DATE)=2012 then do the Cartesian product, then 2013 year then concatenate and so on.

So if you have any better idea pls tell me and thank you so much!

[marcus_sommer]

There isn't a really good solution unless to split the data in pieces which runs on your hardware - independent if you create the cartesian product in script or in gui. Perhaps these links are helpful for you:

Re: Problems with Campaign ans a huge set of Data

Re: Pivot Challenge (Show all values)

Most often there aren't really business requirements to create cartesian products ... I have which created and the using from user is very near on nothing ... Firstly we need urgent, then great and very soon it's forgotten then often there are much better possibilities to improve the business ...

- Marcus

[uacg0009]

Hi Sommer,

thank you for your advise, I will look through the links, I think it will be helpful and if I have other questions I will tell you.