[resolved] GlobalMap and context variables passed to a component result in error in Talend Cloud only, but work in Talend Studio or if set to the equivalent static string inputs

[dan2]

Edited the title to help future troubleshooters and marked the resolution comment.

When using the tS3Get and other S3 commands, I am unable to use GlobalMap or context variables specifically in Talend Cloud. The same work fine in Data Studio. I can update the Bucket and Key fields to the equivalent string and it will work without issue, however I have a need to dynamically define the Bucket and Key since the bucket is different in each environment.

 

 

 

 

 

Then I go back and hard-code the same exact values:

 

 

@rhall

[gjeremy1617088143]

Hi, maybe the issue come from / ,could you try \\ instead in your globalMap or context var ?

Send me love and kudos

[dan2]

I made the change to \\, but it just changes the error message from Status Code 400 to 404:

I also tried \/, which gets the same 404.

[gjeremy1617088143]

you could try the debug perspective and see when you go throw

com.amazonaws.services.s3.model.GetObjectRequest getObjectRequest_tS3Get_1 = new com.amazonaws.services.s3.model.GetObjectRequest();

if you have the right value inside when you use globalVar or contextVar

[gjeremy1617088143]

maybe i suspect issue from joblet context

you can try to write a routine like this (here i named it Tester) :

package routines;public class Tester { private static String bucket;

 private static String key;

/**

 * @return the bucket

 */

public static String getBucket() {

return bucket;

}

/**

 * @param bucket the bucket to set

 */

public static void setBucket(String bucket) {

tester.bucket = bucket;

}

/**

 * @return the key

 */

public static String getKey() {

return key;

}

/**

 * @param key the key to set

 */

public static void setKey(String key) {

tester.key = key;

}

}

I assume here you have two context variable : key and bucket

at the begining of the job(not the joblet) in a tJava :

Tester.setKey(context.key);

Tester.setBucket(context.bucket);

in the component fields :

for key:

Tester.getKey()

for bucket:

Tester.getBucket()

[dan2]

The problem though is that this works as intended when I run it from Talend Studio. It only breaks when run as a Task in Talend Cloud via Management Console running from a Cloud Engine. Is there a way to get that level of debug in Cloud?

I have tried changing one of the inputs to hard-coded and the other to get the global to see if it is either the Bucket or Key input that is breaking. Either is breaking unfortunately, so hardcoding Key while making the bucket a variable does not resolve and vice-versa.

[dan2 · Accepted solution]

@guenneguez jeremy​ Thanks for this. It ultimately helped solve the problem. The issue was that I should not have been inputting the context parameters encapsulated with double-quotes. The use of (String) casting at the beginning of the function implicitly takes care of that. To avoid future complications, I amended the routine to strip the quotes and made it, so that I can use it for cleaning up other variables. Thanks again for all your help with this one. I'm a newbie with this particular tool and Java. Hopefully this will help someone in the future.

I do feel that there is a logical bug here though that Talend should address. Namely, Talend Studio seems to strip excess quotes for you, hence why the job does not fail in Talend Studio whereas Talend Cloud does not do this. That inconsistency causes problems when the version 8 compilers should match across systems.

Here is the updated routine:

package routines;

public class contextHandler {

private static String bucket;

private static String key;

private static String filename;

/**

* @return the bucket

*/

public static String getBucket() {

return bucket.replace("\"", "");

}

/**

* @param bucket the bucket to set

*/

public static void setBucket(String bucket) {

contextHandler.bucket = bucket;

}

public static void setFileName(String filename) {

contextHandler.filename = filename;

}

/**

* @return the key

*/

public static String getKey() {

return key.replace("\"", "");

}

public static String getKeyFileName() {

return key.substring(key.lastIndexOf("/") + 1 ).replace("\"", "");

}

public static String getFileName() {

return filename.replace("\"", "");

}

/**

* @param key the key to set

*/

public static void setKey(String key) {

contextHandler.key = key;

}

}