topic Re: File as a source, directory wildcard in Qlik Replicate

File as a source, directory wildcard

allann — Tue, 01 Nov 2022 18:53:17 GMT

We created a task that has a bunch of files without file extensions by using /directory/* in the Location of full load data file(s). Even though the documentation states a full file path is needed, we're able to pull all the data. We're trying to create a similar task but these files are *.csv. When I use * it won't work.

When we used /*. *. it worked but it didn’t pick up any data. Any ideas for the wildcard in this situation?

Re: File as a source, directory wildcard

SwathiPulagam — Tue, 01 Nov 2022 19:42:54 GMT

Hi @allann ,

I tested with File as a source by giving just the directory\* keeping files with CSV extension and I see all the data loaded.

Please test one more time and share the task diagnostic package.

Thanks,

Swathi

Re: File as a source, directory wildcard

allann — Tue, 01 Nov 2022 20:11:47 GMT

When I do that, I get "Failed to init unloading table.
Failed to write record id: 2, Number of values: 5 is not equal to number of columns: 32."

Just to confirm the direct path to 1 file worked, I did put /filename.csv and that pulled in 111 rows.

Attached log file.

Re: File as a source, directory wildcard

SwathiPulagam — Tue, 01 Nov 2022 20:49:02 GMT

Hi @allann,

Try this option: Copy your CSV file to another folder and run the same mapping. let me know how it goes.

Thanks,

Swathi

Re: File as a source, directory wildcard

allann — Thu, 03 Nov 2022 11:22:17 GMT

We had to figure out a naming convention and use the * within the filename(s).

Re: File as a source, directory wildcard

Heinvandenheuvel — Thu, 03 Nov 2022 12:26:36 GMT

>> We had to figure out a naming convention and use the * within the filename(s).

for real? You use the "*" as a regular character in a filename? By choice? That's the silliest thing I've heard in years. Just Don't!

Hein.

Re: File as a source, directory wildcard

allann — Thu, 03 Nov 2022 13:29:58 GMT

No I mean instead of trying to do /*.csv, the asterix had to be within the file path.

Re: File as a source, directory wildcard

Heinvandenheuvel — Thu, 03 Nov 2022 14:04:54 GMT

>> of trying to do /*.csv, the asterix had to be within the file path.

Ok, I supposed that's what the topic tittle also suggested.

I don't think that is going to work but if it did it, then the syntax would be more like <fixed_part_of_path>/*/*.csv

no?

For 'complex' source file locations you may want to look at generating/using REFERENCE files instead of direct file location. Reference file hold points to files.

Hein

Re: File as a source, directory wildcard

Megha_More — Sat, 16 Dec 2023 06:14:08 GMT

Hi @SwathiPulagam ,

Can you please explain the steps for full load because there is an option to specify metadata (add field)? But when we give * that time, there might be changes in the metadata of files.

PFA for the same.

Regards,

Megha