Extract strings from log file

[Parikhharshal]

Hi experts

I have got log file like below from which I want to extract Applicant ID, Agent Applicant ID and Error:

OES Admission Application Process Log at 2020-02-18 09:12:34.733

***** START OF LOGS *****

Application Name:ERWGERG Application ID: A-5562 Cal Applicant ID: 431234 CRM Applicant ID: C-4357623
[INFO] - New Applicant ID 431234 created for Agent Applicant ID 0035Q000002hzDmQAL.
Preference 1 for application 123 has been successfully loaded into Applicant Portal.
No document has been inserted as no information has been provided.
Applicant Application data has been inserted, even though error(s) did occur. The data that caused the error has been ignored or the error has been ignored.

[ERROR] - Field <SCNDRY><COMPLETION_YR> that has been defined as mandatory by the institution has not been supplied.
Secondary data supplied has been matched to data that already exists for the applicant therefore the data has been ignored.
Tertiary data supplied has been matched to data that already exists for the applicant therefore the data has been ignored.
Either Planned Date or Date Achieved must be entered.

What's the best way to extract and if regex needs to be written, what it should be?

Thanks

Harshal.

[Parikhharshal]

@shong 

The values I am getting are like this from tjavarow using println (using arraylist to output):

Applicant_id: [2084506, 2084507, 2084508, 2084509, 2084510]

Account_Id: [0035O000002lpB3QAI, 0035O000002lowPQAQ, 0035O000002lpBrQAI, 0035O000002lpCaQAI, 0035O000002lpD9QAI]

[Anonymous]

If you are storing these values in ArrayLists, you are almost there. I assume that you have used a tJavaFlex. I imagine you created the ArrayLists in the Start Code section and populated them in the Main Code section. To release these values to a row so that you can update your DB, you can either alter your code to release these values in Main Code section. Simply create the columns you want to output and assign the values to those columns instead of adding them to ArrayLists. Alternatively, if you need to assign the values to ArrayLists, you can read those ArrayLists using another tJavaFlex. Consider the Start Code as the area to open your loop. Maybe like this....

//Assuming your ArrayList is called myArrayList
for(int i = 0; i<myArrayList.size(); i++){

Then in the Main Code extract the values in the myArrayList using the value of "i" and assign them to output columns.

You have to remember that you need to close your for loop with the following code in the End Code section....

}

[Parikhharshal]

@rhall  Could you please share job design or something which can show me how would you accept it as column and store? That would be really helpful because that's where I am stuck and not able to proceed.

[Parikhharshal]

@rhall  I am suing tjavarow and below is my code:

String str = row18.line;

String strFind = "Callista Applicant ID:";

String strFind1= "for Agent Applicant ID";

String strFind2= "[ERROR] -";

String strFind3= "Application Name:";

int count = 0, fromIndex = 0;

int count1 =0, fromIndex1= 0;

int count2 =0, fromIndex2= 0;

List<String> results1 = new java.util.ArrayList<String>();

List<String> results2 = new java.util.ArrayList<String>();

while ((fromIndex = str.indexOf(strFind, fromIndex)) != -1 ){

// System.out.println("Found at index: " + fromIndex);

String str1 = str.substring(strFind.length() + fromIndex, strFind.length() + fromIndex + 9);

str1 = str1.trim();

//System.out.println("Callista applicant ID " + str1);

//globalMap.put("Applicant_Id",str1);

//System.out.println("Applicant_Id: "+((String)globalMap.get("Applicant_Id")));

results1.add(str1);

//results[count] = globalMap.get("Applicant_Id");

count++;

fromIndex++;

}

while ((fromIndex1 = str.indexOf(strFind1, fromIndex1)) != -1 ){

// System.out.println("Found at index: " + fromIndex1);

String str2 = str.substring(strFind1.length() + fromIndex1, strFind1.length() + fromIndex1 + 19);

str2 = str2.trim();

results2.add(str2);

//System.out.println("Account ID " + str2);

//globalMap.put("Account_Id",str2);

//System.out.println("Account_Id: "+((String)globalMap.get("Account_Id")));

//results[count1] = ","+globalMap.get("Account_Id");

count1++;

fromIndex1++;

}

//globalMap.put("Applicant_id",results1);

//globalMap.put("Account_id",results2);

String[] arr1 = new String[results1.size()];

arr1 = results1.toArray(arr1);

//globalMap.put("App_id",arr1);

//System.out.println("App_Id: "+((String)globalMap.get("App_id")));

System.out.println("Applicant_id: "+ java.util.Arrays.toString(arr1));

String[] arr2 = new String[results2.size()];

arr2 = results2.toArray(arr2);

// globalMap.put("Acc_id",arr2);

  //System.out.println("Acc_id: "+((String)globalMap.get("Acc_id")));

System.out.println("Account_Id: "+java.util.Arrays.toString(arr2));

[Anonymous]

Look at this post (https://community.talend.com/t5/Design-and-Development/Dynamic-column-export-with-order/m-p/205749/h...). The tJavaFlex_2 section covers what you want to do. 

[Parikhharshal]

@rhall  I am using tjavarow. Does it mean I will have to change my code to accomodate according to tJavaflex and end both the loops in end code?

[Anonymous]

When you are reading the data, you can do that in a tJavaFlex without the Loop starting in the start code and ending in the end code. The "loop" is controlled by the rows of data coming in (your log file). You will need to save the data (the ArrayLists) using the globalMap in the end code though. When reading the data in, it should work exactly the same way as the example I sent you. However, you will probably want to use a for loop instead of an iterator with a while loop. This is because you will want to retrieve data from two ArrayLists. I'm assuming that they will have the same number of elements.

[Parikhharshal]

Yes that's right both will have same number of elements.

But how do I write for loop for both together? What I don't get is how do you create two columns in first component and assign the values there using for loop? If you could help here, would really appreciate. I am literally stuck here since many days.

[Parikhharshal]

In my case I must use while or some loop to find multiple occurrences of same attribute. If you see my pasted code you will know why I’m using loop.

[Anonymous]

Yes, I see that in your input code (populating the ArrayLists), but in your output code (getting the data from the ArrayLists) it will probably be better to use a for loop