Seems it is related a " Will Not Fix" Java Bug.
http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4508058
A quick and dirty workaround (in a tJava) :
//reading as UTF-16LE
FileInputStream fis = new FileInputStream("inpufile.txt");
BufferedReader r = new BufferedReader(new InputStreamReader(fis, "UTF-16LE"));
//writing as UTF-8
FileOutputStream fos = new FileOutputStream("ouputfile.txt");
Writer w = new BufferedWriter(new OutputStreamWriter(fos, "UTF-8"));
//copy data
for (String s = ""; (s = r.readLine()) != null;) {
w.write(s + System.getProperty("line.separator"));
w.flush();
}
//closing streams
w.close();
r.close();
The BOM is still wrong, but the encoding is right.
I did not find a convenient way to put binary files online.
So here is a small example of what I mean :
- Actual input data (readable) : ?
(LATIN CAPITAL LETTER A WITH DIAERESIS + DEGREE SIGN)
- Correct UTF-16LE (hexa) : FF FE 00 C4 00 B0
as written by talend in my case (supposed to be utf8)
- Actual output file (hexa mixed) : FF FE C3 84 C2 B0
after the above quick'n dirty conversion
- Expected output (hexa utf8) : EF BB BF C3 84 C2 B0
Edit : Oops seems that ultraedit converts automatically to utf16 when opening. Trying with a decent binary viwer/editor now.
