topic Re: Error while loading data from csv to postgres using dynamic schema in Talend Studio

Error while loading data from csv to postgres using dynamic schema

Anonymous — Sat, 16 Nov 2024 07:59:55 GMT

Hi,

I have a requirement to load multiple csv files to different tables in a database. Since the CSVs are dynamic the schema also should be dynamic. I have a Talend licensed version and using dynamic schema. My work flow is like:

tFileList---->tFileInputFlowDelemited-->tMap-->tPostgresqlOutput

The schema i defined is

id : String

others: dynamic

The error I am getting at the time of postgres insert is

[ERROR]: local_project.s3_loader_0_1.S3_Loader - tPostgresqlOutput_1 - Batch entry 0 INSERT INTO "public"."S3-Loader-Test" ("id", ) VALUES (576,Test,41:02.4,TRUE,, ) was aborted. Call getNextException to see the cause.
caused by: ERROR: syntax error at or near ")"

It is not considering and splitting dynamic field "others" while inserting.

Also if I define schem with only one dynamic column then it is considering the whole CSV row as single column and record and trying to insert.

Could somebody correct me? or point me the right solution?

Re: Error while loading data from csv to postgres using dynamic schema

Anonymous — Tue, 10 Jul 2018 09:40:14 GMT

Hello,

For the purpose of testing, are you able to get all your output files by using tfileoutputdelimited component?

The work flow looks like:tfilelist-->tfileinputdelimited-->tfileoutputdelimited(Append option checked).

Best regards

Sabrina

Re: Error while loading data from csv to postgres using dynamic schema

Anonymous — Tue, 10 Jul 2018 19:51:26 GMT

Hi,

Thanks for the reply. I found the issue. The issue was related to the field separator. Once corrected it worked.

Re: Error while loading data from csv to postgres using dynamic schema

Anonymous — Fri, 13 Jul 2018 10:39:12 GMT

Hello,

Great it is fixed. Thanks for posting your solution here.

Best regards

Sabrina