Unlock a world of possibilities! Login now and discover the exclusive benefits awaiting you.
Recent Documents
-
Spark Job using a tHMapFile component adds an extra newline character in the out...
Problem Description The output file generated by the tHMapFile component in Spark batch Job has an extra newline character after every record. ... Show MoreProblem Description
The output file generated by the tHMapFile component in Spark batch Job has an extra newline character after every record.
Root Cause
Creating a Flat representation adds an __osdtTerminator element at the end of the file. Also, in a Big Data environment, each output record is seen as its own file by the Flat representation. As a result, you get an extra newline character.
Solution
Remove the __osdtTerminatorelement from the file representation and clear the Output as delimited? check box in Flat representation.
-
Spark and cluster details to investigate for a TDM on Spark issue
Overview You are facing an issue on TDM on Spark. As a customer or a support engineer here are the environment details to check and collect. ... Show MoreOverview
You are facing an issue on TDM on Spark. As a customer or a support engineer here are the environment details to check and collect.
Resolution
-
Big data cluster in use: CDH, HDP, MapR, and so on
-
Component using Spark:
- Components configuration: Signature, Path and details
-
Spark configuration (Spark version, details and cluster config)
-
Spark runtime: Local or cluster (Standalone or Yarn Client)
-
-
How to isolate and replicate a TDM or a TDM on Spark issue
Overview Below are some steps that could help to isolate or replicate a TDM issue or a TDM on Spark issue. Description Check the last change don... Show MoreOverview
Below are some steps that could help to isolate or replicate a TDM issue or a TDM on Spark issue.
Description
- Check the last change done before break
- Find out what was changed in the Route/Job before issue surfaced
- Where was the change made – Structure/Mapping?
- What is after an upgrade/migration?
- Did the input document / format change?
- Scope of the issue: What works or does not work
- Is it a user/Studio/Machine specific issue?
- Is it Mapping/Structure related?
- Does it depends on a specific representation / data used?
- Size of input (in case Spark is not used)
- Connection to the cluster: check if working fine without TDM / TDM on Spark components
- Performance related issue – on Studio/Jobserver/Runtime/Cluster
- Working/non-working scenarios
- Find out if there is a specific input/use case that always fails
- Determine simplest case where problem occurs (process of elimination)
- Steps to reproduce
- Exact environment details (Talend version, Cluster details…)
- Create a mockup/sample Structure/Map/Route/Job in-house
- Customer data: get sample input/output data if it is data dependent issue
- Get the entire Job/Route (and in some cases, the project or workspace) for cases where issue cannot be easily replicated
- Check the last change done before break
-
Talend Data Mapper - Split Loop option
The purpose of this video is to show how to use the Split Loop option. The Split Loop option is useful when you want to map to a single output loop... Show More -
Talend Data Mapper - Custom Java Functions
The purpose of this video is to show you how to add custom Java functions to the Data Mapper repository, and then how to invoke the custom Java fun... Show More
