At the end of a job, I need to decode a (long) base64-encoded string into a .zip file.
After several failed approaches, I'm writing it to a .txt file and trying to use tSystem to run an external program to decode it into a target (.zip) file but am getting:
Exception in component tSystem_1
java.io.IOException: Cannot run program "Decode" (in directory "C:\temp"): CreateProcess error=2, The system cannot find the file specified
I've reduced the job to just the tSystem component and moved the decoder program (b64dec.exe) and input .txt file into the c:\temp directory (direct execution there works fine), but the job does not find the program. Anybody see what I'm missing?
![]()
![]()
After several failed approaches, I'm writing it to a .txt file and trying to use tSystem to run an external program to decode it into a target (.zip) file but am getting:
Exception in component tSystem_1
java.io.IOException: Cannot run program "Decode" (in directory "C:\temp"): CreateProcess error=2, The system cannot find the file specified
I've reduced the job to just the tSystem component and moved the decoder program (b64dec.exe) and input .txt file into the c:\temp directory (direct execution there works fine), but the job does not find the program. Anybody see what I'm missing?
1 Solution
Accepted Solutions
Resolved!
YES, I needed to download the commons-codec-1.<7>.jar and install it in my Java Home directory.
Then the job ran but the resulting file was not an intelligible .zip file. Some of the bytes were bad, but most wer good and there was the correct number of them. I guessed the character encoding wasn't supposed to be the UFT-8 per the StringUtils.newStringUtf8() function so I tried a few others from the library and found that StringUtils.newStringIso8859_1() worked.
Voila!
![]()
YES, I needed to download the commons-codec-1.<7>.jar and install it in my Java Home directory.
Then the job ran but the resulting file was not an intelligible .zip file. Some of the bytes were bad, but most wer good and there was the correct number of them. I guessed the character encoding wasn't supposed to be the UFT-8 per the StringUtils.newStringUtf8() function so I tried a few others from the library and found that StringUtils.newStringIso8859_1() worked.
Voila!
Hi holberger,
Interesting case
If I had to do that, I wouldn't rely on external tools (such as b64dev.exe).
Indeed, tSystem here isn't the best option : it would make your job not portable anymore, and harder to debug/maintain/fix.
Instead, I'd do the decoding directly in the job. And there are plenty of ways to do that ! But let's look at my favorite one
1. Add a tLibraryLoad in your job. We'll rely on an external library that's called Apache Commons Codec, that provides a very neat way to deal with Base64. It's reliable, and open-source !
2. Add the library named 'commons-codec-1.6.jar'.
3. In the 'Advanced Settings' tab of the tLibrary, add the following two lines in the 'Import' :
(Note : These are the two Java classes needed to convert Base64)
4. In a tMap, use the following snippet to make the transformation :
(here base64.toDecode is the column you have to decode)
5. And then, you can write the decoded data into a file, using a tFileOutputDelimited. There's an option that enables to compress the content as a zip file.
Hope that helps,
Cyril.
Interesting case
If I had to do that, I wouldn't rely on external tools (such as b64dev.exe).
Indeed, tSystem here isn't the best option : it would make your job not portable anymore, and harder to debug/maintain/fix.
Instead, I'd do the decoding directly in the job. And there are plenty of ways to do that ! But let's look at my favorite one
1. Add a tLibraryLoad in your job. We'll rely on an external library that's called Apache Commons Codec, that provides a very neat way to deal with Base64. It's reliable, and open-source !
2. Add the library named 'commons-codec-1.6.jar'.
3. In the 'Advanced Settings' tab of the tLibrary, add the following two lines in the 'Import' :
import org.apache.commons.codec.binary.Base64;
import org.apache.commons.codec.binary.StringUtils;
(Note : These are the two Java classes needed to convert Base64)
4. In a tMap, use the following snippet to make the transformation :
StringUtils.newStringUtf8(Base64.decodeBase64(base64.toDecode));
(here base64.toDecode is the column you have to decode)
5. And then, you can write the decoded data into a file, using a tFileOutputDelimited. There's an option that enables to compress the content as a zip file.
Hope that helps,
Cyril.
Hello Cyril,
It almost worked.
I configured the tLibraryLoad component as described and used the StringUtils.newStringUtf8(Base64.decodeBase64(String #)) expression in the tMap in a test job that reads the XML-encapsulated string to be decoded. See the first images. For #, I used a substring expression that trims XML stuff from both ends of the encoded string. (It's been tested and is known to work.)
(Ignore the third image. It's from the tSystem attempt and refuses to be deleted.)
It seems the my installation could not find the Apache library. The Run process gets an error in the code building/compilation phase. It warns: Missing jars: commons-codec-1.6.jar. The code shows errors on the import statements. See the latter images.
Do I need to download commons-codec-1.6.jar and install it somewhere? (At the present, MDM, the Jboss server, and MSSqlServer are all running on my computer.
![]()
![]()
![]()
![]()
![]()
![]()
It almost worked.
I configured the tLibraryLoad component as described and used the StringUtils.newStringUtf8(Base64.decodeBase64(String #)) expression in the tMap in a test job that reads the XML-encapsulated string to be decoded. See the first images. For #, I used a substring expression that trims XML stuff from both ends of the encoded string. (It's been tested and is known to work.)
(Ignore the third image. It's from the tSystem attempt and refuses to be deleted.)
It seems the my installation could not find the Apache library. The Run process gets an error in the code building/compilation phase. It warns: Missing jars: commons-codec-1.6.jar. The code shows errors on the import statements. See the latter images.
Do I need to download commons-codec-1.6.jar and install it somewhere? (At the present, MDM, the Jboss server, and MSSqlServer are all running on my computer.
Resolved!
YES, I needed to download the commons-codec-1.<7>.jar and install it in my Java Home directory.
Then the job ran but the resulting file was not an intelligible .zip file. Some of the bytes were bad, but most wer good and there was the correct number of them. I guessed the character encoding wasn't supposed to be the UFT-8 per the StringUtils.newStringUtf8() function so I tried a few others from the library and found that StringUtils.newStringIso8859_1() worked.
Voila!
![]()
YES, I needed to download the commons-codec-1.<7>.jar and install it in my Java Home directory.
Then the job ran but the resulting file was not an intelligible .zip file. Some of the bytes were bad, but most wer good and there was the correct number of them. I guessed the character encoding wasn't supposed to be the UFT-8 per the StringUtils.newStringUtf8() function so I tried a few others from the library and found that StringUtils.newStringIso8859_1() worked.
Voila!
Great holberger ! Sorry for the late reply, but glad to see that this trick worked for your case !
Have fun with Talend
Have fun with Talend