Talend Job with tHDFSxxx component fails with 'java.io.IOException: No FileSystem for scheme: maprfs' error

TalendSolutionExpert — Fri, 09 Feb 2024 18:22:49 GMT

Problem Description

When launched from Talend Administration Center (TAC), a Talend Job using the tHDFSxxx component fails with this error:

java.io.IOException: No FileSystem for scheme: maprfs 
at org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2615) 
at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2622) 
at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:91) 
at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2661) 
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2643) 
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:404) 
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:169) 
at etl4_t621.messagecontentstg_0_1.MessageContentSTG.tHDFSDelete_1Process(MessageContentSTG.java:9849) 
at etl4_t621.messagecontentstg_0_1.MessageContentSTG.tFixedFlowInput_1Process(MessageContentSTG.java:6744) at etl4_t621.messagecontentstg_0_1.MessageContentSTG$8.run(MessageContentSTG.java:19417)
[INFO ]: etl4_t621.messagecontentstg_0_1.MessageContentSTG - tSendMail_7 - Connection attempt to 'hostname [INFO ]: etl4_t621.messagecontentstg_0_1.MessageContentSTG - tSendMail_7 - Connection to 'hostname' has succeeded.
[ERROR]: etl4_t621.messagecontentstg_0_1.MessageContentSTG - tDie_4 - The die message: job failed 
[FATAL]: etl4_t621.messagecontentstg_0_1.MessageContentSTG - tDie_4 null

The problem is observed only in this specific environment. The Job runs correctly in another Talend environment. The Hadoop MapR version is 5.2.

Root Cause

The error message identifies the issue as related to the MapR-FS object definition. This issue is observed when the maprfs-5.2.0-mapr-6.2.0 JAR is not available for the Job.

The CommandLine library location, C:\Talend\6.4.1\cmdline\studio\configuration\.m2\repository\org\talend\libraries\maprfs-5.2.0-mapr\, does not include the maprfs-5.2.0-mapr-6.2.0 JAR.

Solution

To correct this problem, copy the maprfs-5.2.0-mapr-6.2.0 JAR from the C:\Talend\6.4.1\cmdline\studio\configuration\.m2\repository\org\talend\libraries\maprfs-5.2.0-mapr\ directory in a working environment to the same directory in your environment.

article Talend Job with tHDFSxxx component fails with 'java.io.IOException: No FileSystem for scheme: maprfs' error in Official Support Articles

Talend Job with tHDFSxxx component fails with 'java.io.IOException: No FileSystem for scheme: maprfs' error

Problem Description

Root Cause

Solution