Extract strings from log file

[Parikhharshal]

Hi experts

I have got log file like below from which I want to extract Applicant ID, Agent Applicant ID and Error:

OES Admission Application Process Log at 2020-02-18 09:12:34.733

***** START OF LOGS *****

Application Name:ERWGERG Application ID: A-5562 Cal Applicant ID: 431234 CRM Applicant ID: C-4357623
[INFO] - New Applicant ID 431234 created for Agent Applicant ID 0035Q000002hzDmQAL.
Preference 1 for application 123 has been successfully loaded into Applicant Portal.
No document has been inserted as no information has been provided.
Applicant Application data has been inserted, even though error(s) did occur. The data that caused the error has been ignored or the error has been ignored.

[ERROR] - Field <SCNDRY><COMPLETION_YR> that has been defined as mandatory by the institution has not been supplied.
Secondary data supplied has been matched to data that already exists for the applicant therefore the data has been ignored.
Tertiary data supplied has been matched to data that already exists for the applicant therefore the data has been ignored.
Either Planned Date or Date Achieved must be entered.

What's the best way to extract and if regex needs to be written, what it should be?

Thanks

Harshal.

[Anonymous]

Hi
The format of log messages are always the same? and what Error string do you want to extract?

Regards
Shong

[Parikhharshal]

@shong  The format will remain same but I will have multiple messages in same log file like below:

Admission Application Process Log at 2020-02-26 16:34:06.124

***** START OF LOGS *****

Application Name: FRT450as TEST Application ID: A-445664 Callista Applicant ID: 1234567 CRM Applicant ID: C-453234
[INFO] - New Applicant ID1234567 created for Agent Applicant ID 0035P000002lpE3QAI.
Preference 1 for application 123 has been successfully loaded into Applicant Portal.
No document has been inserted as no information has been provided.

[ERROR] - Field <SCNDRY><COMPLETION_YR> that has been defined as mandatory by the institution has not been supplied.
Only one of Institution Code and Institution name is required.
Your application is incomplete. You must provide at least one qualification for secondary education, tertiary education, other qualification or work experience before submitting
Applicant Application data has not been inserted as an error has occurred.

Application Name: FREF23 TEST Application ID: A-345656 Callista Applicant ID: 2345678 CRM Applicant ID: C-A0534546
[INFO] - New Applicant2345678 created for Agent Applicant ID 0035O000002lowAWAQ.
Preference 1 for application 123 has been successfully loaded into Applicant Portal.
No document has been inserted as no information has been provided.

[ERROR] - Field <SCNDRY><COMPLETION_YR> that has been defined as mandatory by the institution has not been supplied.
Only one of Institution Code and Institution name is required.
Your application is incomplete. You must provide at least one qualification for secondary education, tertiary education, other qualification or work experience before submitting
Applicant Application data has not been inserted as an error has occurred.

So I want to capture anything after [ERROR] and capture it.

[Parikhharshal]

Hi @rhall  @shong  Could you please reply?

[Parikhharshal]

@rhall  Could you pls respond to this?

[Anonymous]

The easiest way is to standard Java String methods (https://docs.oracle.com/javase/7/docs/api/java/lang/String.html). You know the words that precede the data you want. So you need to find their positions in the String (using something like "indexOf()") then you will need to look for the first space character after your String has started. The easiest way to do this will be to use code similar to this (assuming the text is in row1.body)....

String startOfApplicantId = row1.body.subString(row1.body.indexOf("Application ID:"));
String ApplicantId = startOfApplicantId.trim().subString(0,startOfApplicantId.trim().indexOf(" "));

The above code was written off the top of my head, so may need tweaking. But that is essentially what you need to do.  

[Parikhharshal]

Hi @rhall 

I wrote below code and achieved it.

String str = row18.line;

String strFind = "Callista Applicant ID:";

String strFind1= "for Agent Applicant ID";

String strFind2= "[ERROR] -";

String strFind3= "Application Name:";

        int count = 0, fromIndex = 0;

        int count1 =0, fromIndex1= 0;

        int count2 =0, fromIndex2= 0;

        while ((fromIndex = str.indexOf(strFind, fromIndex)) != -1 ){

          //  System.out.println("Found at index: " + fromIndex);

            String str1 = str.substring(strFind.length() + fromIndex, strFind.length() + fromIndex + 9);

            str1 = str1.trim();

            System.out.println("Callista applicant ID " + str1);

             count++;

            fromIndex++; 

        }

       while ((fromIndex1 = str.indexOf(strFind1, fromIndex1)) != -1 ){

          //  System.out.println("Found at index: " + fromIndex1);

            String str2 = str.substring(strFind1.length() + fromIndex1, strFind1.length() + fromIndex1 + 19);

            str2 = str2.trim();

            System.out.println("Account ID " + str2);

             count1++;

            fromIndex1++; 

       }

However, I want to store value of str1 and str2 in db and do further processing. How cna I do that?

[Anonymous]

Do you write these code on tJavaRow or tJavaFlex component? We usually define context variables and store the value of str1 and str2, and then these context variable can be used later on other components.

Regards
Shong

[Parikhharshal]

@shong I know you can store in context variable.

I have got multiple values (multiple applicant ID, multiple Account Id) to be captured from log file. Then how do I capture in tJava like this below

Applicant Id, Account ID

123,345

234,677

456,466

and push it into DB?

[Anonymous]

As I mentioned, I usually use context variable to store the value. Then, i will use tFixedFlowInput to generate the data flow and insert them into DB.
What is the value of str1 and str2 you can get now? Show me the value.