Inner Join of two tables (19MB and 120KB) via script editor exceeding limit of 1.3GB by loading 2.6GB of data?!

[jzwi]

I am attempting to perform an inner join of two .csv files via the script editor.   The file sizes at 19.5MB and 125KB, respectively.

I am able to successfully load both of these tables  separately (but within the same load script), I receive confirmation of the load, and they both appear in the data model viewer.  

Started loading data

WebSales << eCommerce_ODS.WebSales

Lines fetched: 144,333

Product << Product Lines fetched: 606 Creating search index Search index creation completed successfully

App saved

Finished successfully

0 forced error(s)

0 synthetic key(s)

However, when attempting to run an Inner Join, I receive the following abort message upon load failure:

Started loading data

WebSales << eCommerce_ODS.WebSales

Lines fetched: 144,333 Product << Product

Lines fetched: 606

The following error occurred:

Your quota limit of 1342.2MB was exceeded. You loaded 2613.9MB. Reduce your data within this application or buy additional capacity.

Data has not been loaded. Please correct the error and try loading again.

Any suggestions as to what is going wrong and how I can fix the problem?

Thanks for any help.

[anat]

Can you share your script, how you are performing join? 

[jzwi]

Hi Anat, below is the script I am using.  Both the generic INNER JOIN and specified INNER JOIN(WebSales) caused the same problem.

Thanks

WebSales:
LOAD
DateKey,
ProductKey,
CustomerKey,
PromotionKey,
CurrencyKey,
SaleTypeKey,
OrderDate,
DueDate,
ShipDate,
SalesOrderNumber,
SalesOrderLineNumber,
RevisionNumber,
OrderQuantity,
UnitPrice,
UnitPriceDiscountPct,
ProductStandardCost,
TotalProductCost,
SalesAmount,
TaxAmt,
Freight,
ExtendedAmount

FROM [lib://DataFiles/eCommerce_ODS.WebSales.csv]
(txt, codepage is 28591, embedded labels, delimiter is ',', msq);

Product:
// Inner Join (WebSales)
// INNER JOIN
LOAD
SKU,
ProductSubcategoryKey,
ProductName,
StandardCost,
FinishedGoodsFlag,
"Color",
ReorderPoint,
ListPrice,
DaysToManufacture,
ProductLine,
DealerPrice,
Manufacturer,
Description,
Status,
SupplierKey

FROM [lib://DataFiles/Product.csv]
(txt, utf8, embedded labels, delimiter is ',', msq);

exit script;

[Vegar · Accepted solution]

It looks like you have no common field names in the two tables. In Qlik Sense the joins are based on common field names of the two tables.

Your join without common field names  will create the cartesian product of the two, leaving you with over 87 million rows.

[jzwi]

Thanks, Vegar.  Indeed I needed to edit the script to

SKU as ProductKey