tBigQueryOuput ASCII issues

[Anonymous]

Morning,

I was wondering if anyone can help or advise me on the issue which I am having.

I have checked the forum and tried the solutions provided but have been unsuccessful.

My issue is that when I try to load data from a csv into tBigQuery i get the Error: Bad Character (ASCII 0)

The encoding setting is UTF_8, however, I have tried it with "custom" - "windows 1252" as it did not work with UTF-8 and also with IOS

The csv file data type has been classed as string and then converted using tConvert to ensure the value pass through as string then converted, along with changing the advance setting of encoding and unfortunately still been unsuccessful.

Below is the errior message which i receive

Exception in component tBigQueryOutput_1_tBQBE (jo_GCP_ORWHCART_CSV2BQ)

java.lang.Exception: [{"location":"gs://jdis-dl-ods-csv-dev-temp/orwhcart_bulk.txt","message":"Error while reading data, error message: CSV table encountered too many errors, giving up. Rows: 234468; errors: 1. Please look into the errors[] collection for more details.","reason":"invalid"}, {"location":"gs://jdis-dl-ods-csv-dev-temp/orwhcart_bulk.txt","message":"Error while reading data, error message: Error detected while parsing row starting at position: 38484895. Error: Bad character (ASCII 0) encountered.","reason":"invalid"}, {"message":"You are loading data without specifying data format, data will be treated as CSV format by default. If this is not what you mean, please specify data format by --source_format.","reason":"invalid"}]

      at jdis_gcp.jo_gcp_orwhcart_csv2bq_0_1.jo_GCP_ORWHCART_CSV2BQ.tFileInputDelimited_1Process(jo_GCP_ORWHCART_CSV2BQ.java:6854)

      at jdis_gcp.jo_gcp_orwhcart_csv2bq_0_1.jo_GCP_ORWHCART_CSV2BQ.tJava_2Process(jo_GCP_ORWHCART_CSV2BQ.java:3395)

      at jdis_gcp.jo_gcp_orwhcart_csv2bq_0_1.jo_GCP_ORWHCART_CSV2BQ.tGSList_3Process(jo_GCP_ORWHCART_CSV2BQ.java:3108)

      at jdis_gcp.jo_gcp_orwhcart_csv2bq_0_1.jo_GCP_ORWHCART_CSV2BQ.tGSList_1Process(jo_GCP_ORWHCART_CSV2BQ.java:2616)

      at jdis_gcp.jo_gcp_orwhcart_csv2bq_0_1.jo_GCP_ORWHCART_CSV2BQ.runJobInTOS(jo_GCP_ORWHCART_CSV2BQ.java:14955)

      at jdis_gcp.jo_gcp_orwhcart_csv2bq_0_1.jo_GCP_ORWHCART_CSV2BQ.main(jo_GCP_ORWHCART_CSV2BQ.java:14631)

ORA-12899: value too large for column "TAL_AMC_V7"."TLOGCATCHER"."MESSAGE" (actual: 746, maximum: 255)

The ACSII character which is causing the issue is the "NULL" (ASCII code 00)

I ma not sure how to overcome this issue.

Please any help or advise would be appreciated.

Thanks

[Anonymous]

Try to change the Encoding to ISO-8859-15. I have succeed using this method.
Please let me know if the issue still exist.

[Anonymous]

@sinuraterickh 

Thank you for your respond.

I have tried IOS-8859-15 encoding and no look.

I have tried the available encoding and not been successful as yet.

[Anonymous]

@nthampi@@xdshi

 

Have you come across the below issue?