UTF-8 BOM Encoded File Processing

[Anonymous]

We are getting a daily file in UTF-8 BOM encoding because of which our Talend ETL Job always misses the first row of the file

Sample Data in File:

P, 1234, $10

Q,1235,$20

R, 1236, $15

Our actual flow is like

tFileList ==>> tFileInputDelimited ==>> fReplicate ==> tFilterRow ==>  tMSSqlSCD

Actually tFileInputDilimited is able to process all rows but when we use tFilterRow, but it always misses first row of every particular file

The condition for tFilterRow is column0  Equals  "P"

When we configured tLogRow we found few special characters prefixed with the first rows of all files. Example ???P

Also when we opened our CSV files in Notepad++ we discovered that File is encoded in UTF-8-BOM

We have option only for UTF-8 in Advanced settings of tfiledilimited

Let us know how can we process UTF-8-BOM file using Talend job

Thanks & Regards

[Anonymous]

Hello,

Are you using talend Data Preparation Free Desktop? Is there any error in app.log?

Best regards

Sabrina

 

[Anonymous]

Hi Justin,

When do you get this error? When trying to add a dataset or a preparation step? If so, did you follow the steps to disable App Nap as mentioned in the getting started guide? See https://help.talend.com/reader/mime3pPIS~faiBvVb~FT7w/b006EaGseLmCHC2Tx__deg for more details.

If you are facing the issue as soon as you access the product UI, can you share the log file (Library/Application Support/Talend/dataprep/logs/app.log)?

Regards,

Gwendal

[Anonymous]

Hello

Thanks for the advice.

I have now disabled App Nap, but still get the error when undertaking a preparation step.

Unfortunately the directory and log file that you refer to cannot be found on my computer.

Justin

[Anonymous]

Hi Justin,

A quick check before going any further: which version of Data Preparation Free Desktop are you using? That is either visible in the footer of the application (for older versions) or in the "About Data Preparation" menu on the top-right (for the latest versions).

Thank you,

Gwendal

[Anonymous]

It is version 2.5.1.

[Anonymous]

Hello @Moe,

The application log file is located here:

~/Library/Application Support/Talend/dataprep/logs/app.log

or if you use an absolute path: 

/Users/{your-username}/Library/Application Support/Talend/dataprep/logs/app.log

[Anonymous]

As mentioned before I’m afraid there is no Talend folder in my Application Support folder - see screenshot below. I have also searched for the app.log file and this is not present on my computer. Please could you advise me what to do.

Thank you

[Anonymous]

Hello

I am still having this problem on my new MacBook Pro. However I have also installing the same software onto my older iMac and it works fine without a problem. However, I do need to be able to use on my MacBook Pro.

Both machines are running the same version of Mac OSX Version 10.13.6 (17G65). The iMac is 21.5-inch, Mid 2011; 2.5 GHz Intel Core i5; 8 GB 1333 MHz DDR3. The MacBook Pro is 13-inch, 2017; 2.3 GHz Intel Core i5; 8 GB 2133 MHz LPDDR3

Can you offer any further advice to help it work on the MacBook Pro? Thanks

[Anonymous]

Having the same issue, tried the script:

defaults write org.talend.dataprep NSAppSleepDisabled -bool YES

But still having those errors, and in the logs you find:

2019-02-12 14:00:16.553 INFO 4086 --- [qtp1370856792-60] o.t.d.d.service.BaseDataSetService : New DataSet #2bbbd9fc-5c1c-421e-98e0-5b2895d9d378, name: Saved Cases, type: text/csv, from: local
2019-02-12 14:00:16.576 WARN 4086 --- [qtp1370856792-20] org.eclipse.jetty.server.HttpChannel : /api/datasets

To see the whole post, download it here
OriginalPost.pdf