3 Replies Latest reply: Oct 17, 2012 5:01 AM by Lennart Frimannslund RSS

    Upper bound on num rows for bundle load?

      Hi,

       

      I am trying to do a bundle load, associating png images with some field values. When I test on a small toy application everything works fine, both when there is one image per possible field value, when there are more images in the bundle load statement than values in the original table, when there are more values than images --- and also for different widths of the table containing the field in question.

       

      So far, so good.

       

      In my real application, however, I have a table with 6 columns and about about 100 000 rows.  I want to associate two images with (two values of) a particular field, and these two values each occur about thirty times in the table.

       

      In this case the bundle load statement simply seems to be ignored. So is there an upper bound on the number of rows when it comes to bundle loads?

        • Re: Upper bound on num rows for bundle load?

          Update - I came up with a workaround using a data island, so my application now works as it should. Nevertheless, if anyone knows the answer to the original question it would be very interesting to know!

            • Re: Upper bound on num rows for bundle load?

              I'm new on this.

              I have the same problem.

              Can you explain me how to work with data island.

               

              Thanks a lot!!!

                • Re: Upper bound on num rows for bundle load?

                  Hey there!

                   

                  I'm writing this partially from memory, so hopefully I won't say anything which is wrong

                  What you would usually do is first to load your data, e.g.:

                   

                  Data:
                  LOAD Field1, Field2
                  From some_data_source;

                   

                  and then add images with a bundle load, in this case for Field2:

                   

                  BUNDLE LOAD * Inline [
                  Field2, second_name_can_be_arbitrary
                  1,path_to_image1.jpg
                  2,path_to_image2.jpg
                  3,path_to_image3.jpg
                  4,path_to_image4.jpg
                  ];

                   

                  And then you would refer to your images like so: qmem://Field2/1 --- and so on.

                   

                  However, this doesn't work for large data sets, as we know. So instead, I constructed a table which has no link to my original table, but which contains the (few) values which should have images.

                   

                  So, as before, we do

                   

                  Data:
                  LOAD Field1, Field2
                  From some_data_source

                   

                  Now we make a copy of some of the rows, namely those which are to have images:

                   

                  Island_with_image_rows:

                  NOCONCATENATE LOAD Field2 as copy_of_Field2
                  Resident Data
                  where some_clause_that_picks_the_image_rows;

                   

                  Then we do a bundle load on this new table, namely

                   

                  BUNDLE LOAD * Inline [
                  copy_of_Field2, second_name_can_be_arbitrary
                  1,path_to_image1.jpg
                  2,path_to_image2.jpg
                  3,path_to_image3.jpg
                  4,path_to_image4.jpg
                  ];

                   

                  So, now when you want to refer to your image, you need to write qmem://copy_of_Field2/1 instead of qmem://Field2/1, but as long as this string is generated using only() and whatnot, it should be doable --- at least it was in my case.

                   

                  Hope you figure it out!