topic Re: How can i reload data only select product in App Development

How can i reload data only select product

Anonymous — Thu, 15 Nov 2018 06:23:30 GMT

Hi,

Every month I received CSV file ( Large data or crore togther number of line items), when i was reload the data it will take time more than a day ( Very Slow). but i need to reload the data particular product.

If any option is there please suggest me,

Re: How can i reload data only select product

petter — Thu, 15 Nov 2018 07:07:20 GMT

Even if it is a few tens of millions of rows you must have many columns in the CSV-file if it takes more than a day to load.

If you can't do anything to optimize you Qlik load script you might have to use any of these:

PowerShell script to read and filter the CSV to get a smaller CSV with the relevant rows:
- https://stackoverflow.com/questions/16028787/powershell-filter-csv
Python script to do the same as above
- https://softwarerecs.stackexchange.com/questions/7463/fastest-python-library-to-read-a-csv-file
GAwk (awk): specialized text processing tool/language that is included in the Linux subsystem that can be installed on latest Windows versions.
csvkit: A ready made command line toolkit for CSV (written in Python):
- https://csvkit.readthedocs.io/en/1.0.3/index.html#
gocsv: - " - (written in Go) claims to be faster and use less memory than csvkit:
- https://github.com/DataFoxCo/gocsv
xsv: - "- (written in Rust):
- https://github.com/BurntSushi/xsv

Re: How can i reload data only select product

Anonymous — Tue, 20 Nov 2018 18:25:55 GMT

i have an issue like that

Re: How can i reload data only select product

Frank_Hartmann — Tue, 20 Nov 2018 20:29:48 GMT

Maybe use "Partial Reload".
This command executes only Load and Select (SQL) statements preceded by a Replace or Add prefix. Other data tables remain unaffected by the command.

hope this helps

Re: How can i reload data only select product

nikhil_deokar — Thu, 03 Jan 2019 06:43:36 GMT

Hi,

To reload data for specific products use where condition on table while loading data.

for example:

Load

.....,

....,

From table

Where Match(Product,'abc','xyz','klm');