topic Re: Task for CSV source fails - "cannot refresh metadata" and "could not get object" in Qlik Replicate

Task for CSV source fails - "cannot refresh metadata" and "could not get object"

jonna — Thu, 05 May 2022 14:19:31 GMT

hi folks

I am building a task which extracts data from a CSV source which fails at run time. In the logs i see a number of errors which I can't interpret - log is attached and an extract is pasted at the end of my post. The basic task details are as follows:

SOURCE

Source is a CSV file, it lives on a network server
the CSV file has two columns, it is comma separated and has less than 1,000 rows of data
"Test Connection" is successful when editing the endpoint
Only one tab exists in the CSV

TARGET

SQL server database, well established
Target table was created by the task, presume this was done before the task failed

TASK

No special processing included, other than chnage schema name
When editing the task after failure this error is thrown and shown on QEM >>> SYS-E-HTTPFAIL, Getting DB object (''QlikSense Server File Location'.'Web Chat'') info from repository failed.
The target table has been created on the Target but with no data
Log file attached

I have tried all kinds of alterations to the task and the source endpoint but can't seem to make any head way. All help would be greatly appreciated.

Thanks very much, jonna

Date and Time Level Message
05/05/2022 14:30:38 ERROR_GENERAL "Cannot refresh source table (1) metadata
Failed on getting table definition for table id: 1
Getting DB object (''QlikSense Server File Location'.'Web Chat'') info from repository failed."
05/05/2022 14:30:38 ERROR_GENERAL Could not get object: 'QlikSense Server File Location'.'Web Chat' from repository.
05/05/2022 14:30:38 ERROR_GENERAL "Stream component 'st_1_QlikSense Server File Location' terminated

Re: Task for CSV source fails - "cannot refresh metadata" and "could not get object"

KellyHobson — Thu, 05 May 2022 14:39:25 GMT

Hey @jonna ,

Is it possible to attach the task log for the Replicate task?

Thanks,

Kelly

Re: Task for CSV source fails - "cannot refresh metadata" and "could not get object"

jonna — Thu, 05 May 2022 16:06:14 GMT

Hi @KellyHobson

thanks for taking a look - here is the file as requested

cheerr, jonna

Re: Task for CSV source fails - "cannot refresh metadata" and "could not get object"

Steve_Nguyen — Thu, 05 May 2022 16:17:36 GMT

@jonna

from the log :

00006152: 2022-05-05T17:00:34 [SOURCE_UNLOAD ]E: Failed to write record id: 2195, Number of values: 1 is not equal to number of columns: 2. [1020417] (file_unload.c:344)
00006152: 2022-05-05T17:00:34 [SOURCE_UNLOAD ]E: Failed to init unloading table. [1020417] (file_unload.c:395)

this indicate that when you set the table it does not have the correct number of columns as compare to your csv file.

Re: Task for CSV source fails - "cannot refresh metadata" and "could not get object"

jonna — Thu, 05 May 2022 16:33:24 GMT

thanks @Steve_Nguyen

I have take a look at the task, the source and the target they all appear to have 2 columns.

cheers, jonna

Re: Task for CSV source fails - "cannot refresh metadata" and "could not get object"

Steve_Nguyen — Thu, 05 May 2022 17:08:52 GMT

do you have your table set in the endpoint?

see example screen-shot?

Re: Task for CSV source fails - "cannot refresh metadata" and "could not get object"

jonna — Thu, 05 May 2022 17:21:22 GMT

hi again @Steve_Nguyen

here is a screen shot of the table set up in Manage Endpoint Connections

Re: Task for CSV source fails - "cannot refresh metadata" and "could not get object"

KellyHobson — Thu, 05 May 2022 17:30:07 GMT

Hey @jonna

When you are in Task Designer-> Table Settings-> Transform

Do you see both columns in the input and output panes?

Thanks,

Kelly

Re: Task for CSV source fails - "cannot refresh metadata" and "could not get object"

jonna — Thu, 05 May 2022 17:31:57 GMT

yes @KellyHobson the pair of columns are in both input and output

Re: Task for CSV source fails - "cannot refresh metadata" and "could not get object"

Steve_Nguyen — Thu, 05 May 2022 19:21:56 GMT

@jonna

This could be that your csv file have more column or the delimiter is not correctly.

1. lets try to isolate the issue, could you create a small csv file with the two column and data (example 5 records) , using the delimiter that you define in your source endpoint.

2. then use the sample csv, does the task works ?

3. if the test sample csv works then it is your source CSV file that have more delimiter (causing more column) , therefore, error message.

Re: Task for CSV source fails - "cannot refresh metadata" and "could not get object"

jonna — Thu, 05 May 2022 20:13:14 GMT

I @Steve_Nguyen - I just built a brand new CSV, it has the same number of columns (two), it has a different name and is comma delimited. I also built a new task, stepped through it in case i missed something obvious but still getting the same result. Thanks for your help so far, I am finished for the day and will get back on this first thing. cheers, jonna

Re: Task for CSV source fails - "cannot refresh metadata" and "could not get object"

Steve_Nguyen — Thu, 05 May 2022 20:18:39 GMT

@jonna

1. could you provide the task Diagnostic Package?

2. could you provide a same of your CSV file ?

I want too see if i can duplicate the issue .

Re: Task for CSV source fails - "cannot refresh metadata" and "could not get object"

jonna — Fri, 06 May 2022 08:22:52 GMT

Here is the source file (nb - in .txt format, couldn't attach CSV

Re: Task for CSV source fails - "cannot refresh metadata" and "could not get object"

jonna — Fri, 06 May 2022 08:23:47 GMT

Here is the diagnostics package.

Re: Task for CSV source fails - "cannot refresh metadata" and "could not get object"

KellyHobson — Fri, 06 May 2022 11:55:27 GMT

Hey @jonna

Within your source file endpoint -> Tables tab -> open up 'Web Chat CSV'

Location of full load data file(s):

You need to reference the file name in your path, not just the folder.

current: \\VVL-QS-FS\QlikShare\Project Layer\Application Layer\Avaya\Reference Files

update to: \\VVL-QS-FS\QlikShare\Project Layer\Application Layer\Avaya\Reference Files\WebChat.csv

Ref: https://help.qlik.com/en-US/replicate/November2021/Content/Replicate/Main/File/Advanced_Settings.htm

Try pointing to the exact file path and let us know if able to run without error.

Best,

Kelly

Re: Task for CSV source fails - "cannot refresh metadata" and "could not get object"

jonna — Fri, 06 May 2022 12:05:10 GMT

thanks @KellyHobson - i am on it now 👍

Re: Task for CSV source fails - "cannot refresh metadata" and "could not get object"

jonna — Fri, 06 May 2022 12:21:03 GMT

@KellyHobson and @Steve_Nguyen - fantastic it worked a treat. Really impressed how you sorted out what should have been staring me in the face.

You folks are great.

Thanks, jonna

Re: Task for CSV source fails - "cannot refresh metadata" and "could not get object"

Heinvandenheuvel — Fri, 06 May 2022 13:28:40 GMT

Do NOT use Excel to visualize CSV files for support/understanding purposes. It tries to 'help' too much

Please just use NOTEPAD(++) or similar to show the exact contents with all details like quoted columns, seperators, dates which turn out to be day numbers and so on.

Hein.

Re: Task for CSV source fails - "cannot refresh metadata" and "could not get object"

KellyHobson — Tue, 10 May 2022 21:53:00 GMT

Hey @jonna ,

Glad to hear it is working now and appreciate the kind feedback! Keep asking away, we are happy to help.

All the best,

Kelly