Spark Job using a tHMapFile component adds an extra newline character in the output file

TalendSolutionExpert — Fri, 09 Feb 2024 18:22:49 GMT

Problem Description

The output file generated by the tHMapFile component in Spark batch Job has an extra newline character after every record.

Root Cause

Creating a Flat representation adds an __osdtTerminator element at the end of the file. Also, in a Big Data environment, each output record is seen as its own file by the Flat representation. As a result, you get an extra newline character.

Solution

Remove the __osdtTerminatorelement from the file representation and clear the Output as delimited? check box in Flat representation.

article Spark Job using a tHMapFile component adds an extra newline character in the output file in Official Support Articles

Spark Job using a tHMapFile component adds an extra newline character in the output file

Problem Description

Root Cause

Solution