parsing and extract json

[Anonymous]

Hello,

 In fact, I just started working with Talend open studio for data integration; I want to export an array long json and I want to transform the data coming from JSON by using Talend to organize and store them in a SQL database and sqlserver

my json:  https://security-tracker.debian.org/tracker/data/json

How I would like the output to look like:

Package	CVE	Scope	Debian bug	description	status	fixed_version	urgency

I attach all the configuration photos of my components and the code; can someone help me?

I have two problems :

1: first I couldn't extract the package name; I attach a picture of my json file; in fact CVEs are categorized in different packages, CVEs extract good but I want to add a column Package in my DB ; I don't know what to add in tjava for node extraction from each cves that is called package

2: I found a problem my codes do not extract all the CVEs well for example in my source I have CVE-2019_3462 but my job does not extract it, I could not find what is  the problem 

Does anyone have any advice or ideas on how I can achieve this?

I use "Talend open studio for data integration version 7.1.1.2018102681147

Thank you very much for your time and help

Best regard

[fdenis]

first try to work without using java.
look at document data type and txmlmap. it may be helpfull.

[Anonymous]

thank you, but without java code it extracts nothing

[Anonymous]

I have adjusted your code above to extract the Package Name and the JSON for that Package. You will need to further process this to get the CVEs from it each row of JSON returned. My job looks like below....

The tFileInputJSON_1 component is configured very similarly to yours. But the JSONPath for the content is set to "." instead of "*". This component is linked to a tSetGlobalVar_1. This stores the whole JSON into a String. 

The tJavaFlex is configured with the following code.....

Start Code

Object obj = new org.json.simple.parser.JSONParser().parse(((String)globalMap.get("content")));
org.json.simple.JSONObject jobj = (org.json.simple.JSONObject) obj;
Set<String> jkeys = jobj.keySet();
Iterator<String> jit = jkeys.iterator();
while(jit.hasNext()){

Main Code

String jit1 = jit.next().toString();
row2.package_name = jit1;
row2.package_json = ((org.json.simple.JSONObject)jobj.get(jit1)).toJSONString();

End Code

}

This will return a row for each Package section. You then need to do something very similar to retrieve the CVEs from the Packages. 

[Anonymous]

first of all, thank you for your time and answer,

could you put the tSetGolabalVar component configuration and the tJavaFlex and advanced setting scheme of tJavaFlex
regards

[Anonymous]

I could, but you have all the information you need in what I have already given you. The tSetGlobalVar receives the data from the tFileInputJSON_1 and stores it with a key of "content". This can be worked out from the Start Code of the tJavaFlex. The schema of the tJavaFlex can be figured out by looking at the Main Code.

I'm not trying to be difficult, but I have thrown away my example code and extrapolating from the information that is there will help you learn Talend. 

[Anonymous]

Actually, I asked you because despite that I don't change much I have the following error

regards

[Anonymous]

Add....

row1.content

....where you have "" in your tSetGlobalVar. No quotes. You are referencing the row.column that is returned by your tFileInputJSON_1 component

[Anonymous]

Thank you, I fixed that but I have other problems java heap space despite that I have already added XMS and XMX

regards  

[Anonymous]

How much RAM does your machine have? Also, do you get this error if you switch off everything after the tLogRow_1?