How to extract values from Dynamic JSON fields?

[jerownimow]

Hi Folks,

Good day!

I am new to Talend and badly needed your help. I have this job that consist of the following:

1st subjob:
processing logic -> tHashOutput (contains invoice and other fields)

2nd subjob:
Hash (contains the source of AggGrp - see attachment Source and Expected Output.json) -> tMap (to create JSON RequestBody) -> tAggregateRow (to get only the last generated string) -> tReplicate -> RestClient1 and RestClient2

I can get the response from both of these successfully. The problem is that they have dynamic JSON fields (doesn't look like a JSON field but a value).

Please see attached Response_1.json and Response_2.json for RestClient1 and RestClient2, respectively.

How can I extract these fields and process them to these:

Source and Expected Output.json
(This includes both Source and expected output)

P.S. There are also descriptions on the files on how it will be mapped. Let me know if you need more info.

Please help me. Thank you so much.

[jerownimow]

Hi @rhall ,
Good day! sorry to bother you again. Seems that your solution in here Parse hashmap inside a json  will work on my problem but I don't know how to execute it. I'm quite confused with point 1 and the output myColumn. On the issue posted, he wants 2 outputs both the key and the value if I'm not mistaken.

Can you please explain it to me like you explained the tJavaFlex on my other post?

Thank you so much.

[Anonymous]

This is incredibly hard for someone who is new. If you do not have any Java knowledge, this will make it almost impossible to describe here. So I have built a job using TOS 7.2. This will extract the data from Response1.txt. I have had to download the org.json Jar and import it into the job. You will probably need to do this on your side (I don't believe my export will include that). There is a tLibraryLoad component which needs to have this Jar configured. You can find the Jar here: https://jar-download.com/artifacts/org.json

I hope this helps. Unfortunately I cannot spend a great deal more time on this problem. Hopefully you can extrapolate from this if you need to do more. If not, you will need to find a friendly Java developer to help you with this. By the way, Java is REALLY useful with Talend 🙂

DynamicJSON.zip

[jerownimow]

No worries @rhall . Thank you so much for your time. Any online trainings if there's free or readable materials you can recommend for me to learn java?

Also, I got an error when tried to export the job. It says that 'Job Designs GetDynamicJSON 0.1' The item 'Job Designs GetDynamicJSON 0.1' from project 'Local_Project' was invalid.

[jerownimow]

Hi @rhall  seems that it is not compatible with my studio TOS for Big Data 7.1.

[Anonymous]

You will need to use v7.2 to open it. I used TOS ESB v7.2. 

[Anonymous]

Sorry, I've had a lot of alerts to go through and I am responding in order. I've just noticed you sent me another message. As I said, try installing v7.2 and the job should import. With regard to learning Java, there are loads of online methods. Depending on your level of existing experience, there are different options. For an absolute beginner, I would recommend this approach https://www.bluej.org/

[jerownimow]

Hi @rhall 

No problem. Sorry for the spam, I didn't noticed that you already had a reply.
Unfortunately, I am not able to have 7.2 on my PC. Is it possible to just send me the screenshots like you did on the other thread we have?

Sorry for requesting this much and thank you for your tips! Will definitely grind on that.

[Anonymous · Accepted solution]

OK, here are some screenshots and the code I used.....

The tLibraryLoad component needs the Jar file I mentioned in my earlier post. After configuring that, you will need to set the other components up then save, close the job, then re-open the job. This is a *feature* that some people experience with Java libraries not being "seen" by the job. Hopefully you will not have any issues there.

The tFileInputJSON component is configured like below....

Essentially you are passing the whole JSON from the file to the next component in one go.

The tJavaFlex config can be seen below. I am just including the code. You can figure out the schema from the code. This must be copied and pasted exactly as it is shown.

Start Code

// start part of your Java code
String json = ((String)globalMap.get("json"));
org.json.JSONObject jsonObj = new org.json.JSONObject(json);
org.json.JSONObject appMapObj = jsonObj.getJSONObject("applicationMap");
org.json.JSONObject taxesObj = appMapObj.getJSONObject("taxes");
java.util.Iterator<String> taxKeysIt = taxesObj.keys();

while(taxKeysIt.hasNext() ) {
	String taxKey = (String)taxKeysIt.next();
	
	org.json.JSONObject taxKeyObj = taxesObj.getJSONObject(taxKey);

	java.util.Iterator<String> innerKeysIt = taxKeyObj.keys();
	
	while(innerKeysIt.hasNext() ) {
		String innerKey = innerKeysIt.next();
		
		org.json.JSONArray innerArr = taxKeyObj.getJSONArray(innerKey);
		
		java.util.Iterator<Object> innerArrIt = innerArr.iterator();
		
		while(innerArrIt.hasNext()){      

Main Code

			org.json.JSONObject innerObj = (org.json.JSONObject)innerArrIt.next();
			row2.taxes = taxKey;
			row2.innerTaxCode = innerKey;
			row2.amount = !innerObj.isNull("amount") ? innerObj.getDouble("amount") : 0;
			row2.initialAmount = !innerObj.isNull("initialAmount") ?  innerObj.getDouble("initialAmount") : 0;
			row2.description = !innerObj.isNull("description") ? innerObj.getString("description") : "";
			row2.taxItemId = !innerObj.isNull("taxItemId") ? innerObj.getInt("taxItemId") : 0;
			
			
			

End Code

// end of the component, outside/closing the loop
		}
	}
}

The tLogRow just prints out the results.

[jerownimow]

Thank you so much @rhall . It really worked, and I was able to proceed with the rest of the transformations.
Thank you so much for your time! God bless you, man!