topic Re: Reading and parsing Google Calendar ics file in Talend Studio

Reading and parsing Google Calendar ics file

AlessandroCic — Sun, 02 Feb 2014 16:32:48 GMT

Hi forum
In another post I asked about to query Google Calendar item
url=http://www.talendforge.org/forum/viewtopic.php?pid=126354#p126354
I hope that the developer find a way to do this.
Meantime I try another method to read Google Calendar item: in Google there is the possibility to export the calendar in a structured text file with ics extension that it's something like this:

BEGIN:VEVENT
DTSTART:20140107T090000Z
DTEND:20140107T120000Z
DTSTAMP:20140129T164854Z
UID:@google.com
CREATED:20140108T080811Z
DESCRIPTION:Descriprion 1° event
LAST-MODIFIED:20140108T080831Z
LOCATION:
SEQUENCE:1
STATUS:CONFIRMED
SUMMARY:Object 1° event
TRANSP:OPAQUE
END:VEVENT
BEGIN:VEVENT
DTSTART:20131224T073000Z
DTEND:20131224T080000Z
DTSTAMP:20140129T164854Z
UID:@google.com
CREATED:20131223T165249Z
DESCRIPTION:Descriprion 2° event
LAST-MODIFIED:20140107T075355Z
LOCATION:
SEQUENCE:0
STATUS:CONFIRMED
SUMMARY:Object 2° event
TRANSP:OPAQUE
END:VEVENT
.....

It's a sequential file where every event is distinct between the sections

BEGIN:VEVENT

END:VEVENT

Isn?t guaranteed that each section have the same number of rows and the same sequence.
Therefore it would be necessary generate a cycle that it identify the begin and the end of the section to create new DB row.
Then on have to identify the data row by row to assign to a corresponding field.
Is there a Talend component that it running this big job?
Or what components could I use to develop this?
Regards
Alessandro

Re: Reading and parsing Google Calendar ics file

Anonymous — Sun, 02 Feb 2014 23:01:24 GMT

To your question, no this is not guaranteed. If a event has same attributes unlike others than the entries can differ.
Currently there is no component dedicated to this format. I would do this job with a Java class parsing this file line by line and accumulate the Events in objects. It is very easy to do.
Do you think, this task will be done often? I suggest creating a JIRA feature request for a new component to handle ICS files.

Re: Reading and parsing Google Calendar ics file

AlessandroCic — Mon, 03 Feb 2014 13:56:45 GMT

Hi Jlolling thanks for your replay.
I had already begun to develop Java code for this function.
I'm using the component tFileInputFullRow to read my .ics file then with an "Iterate" collector I pass to a tJava component.
Is what the name of the variable to use to get the row value from tFileInputFullRow?
I suppose something like this

((String)globalMap.get("tFileInputFullRow_1_xxxx"));

Regard

Re: Reading and parsing Google Calendar ics file

AlessandroCic — Wed, 05 Feb 2014 20:35:55 GMT

Hi forum
can anyone tell me the property to use to extract the value of the line?
The work that I am developing is as shown in the image but I do not know the right parameter ("tFileInputFullRow_1_line" is wrong!)
Thanks to everyone.
Alessandro

Re: Reading and parsing Google Calendar ics file

AlessandroCic — Fri, 07 Feb 2014 18:35:17 GMT

Up..

Re: Reading and parsing Google Calendar ics file

AlessandroCic — Thu, 13 Feb 2014 07:13:44 GMT

Hi forum
no suggestions for my issue?
In another job where I'm using tFileList I received the suggestion to use the property tFileList_1_CURRENT_FILE, tFileList_1_CURRENT_FILEDIRECTORY, tFileList_1_CURRENT_FILEPATH which have been very useful for the development.
Using tFileInputFullRow I think there are some similar properties but I could not find documentation on this.
Help me please.
Thanks to everyone.
Alessandro

Re: Reading and parsing Google Calendar ics file

Anonymous — Thu, 13 Feb 2014 07:43:55 GMT

Hi Alessandro
The global variable (Integer)globalMap.get("tFileInputFullRow_1_NB_LINE") on tFileInputFullRow_1 is used to count the total number of lines are read from the file. I don't understand why you are using the iterate link. What data do you want to extract from the file? or you just need to count the number of events (that starts from BEGIN:VEVENT and ends with END:VEVENT)?
Shong

Re: Reading and parsing Google Calendar ics file

AlessandroCic — Thu, 13 Feb 2014 14:44:40 GMT

Hi Shong
Thanks for your attention.
With tFileInputFullRow I would like to read line by line the .ics file, fot this I think to use the iterate link.
In a tJava component I would like to have the contents of the line and to parse it to identify the kind of row.
Then if I my row start (es.) with SUMMARY: or DTSTART: I take the value after ":" and I save the value in a structure with a row for each event with the field required.
The global variable (Integer)globalMap.get("tFileInputFullRow_1_NB_LINE") can to be helpful but in first time I must have the content of the row.
What is the global variable to get the row string? How can I find documentation about this?
Regards
Alessandro

Re: Reading and parsing Google Calendar ics file

AlessandroCic — Wed, 19 Feb 2014 09:23:40 GMT

Hi people
Is it really so hard to have this information?
I have not found documentation about global variables, Is it necessary from someone to help me.
Thanks, regards.
Alessandro

Re: Reading and parsing Google Calendar ics file

AlessandroCic — Fri, 21 Feb 2014 08:43:37 GMT

Up.
Help me please.
The information required is:
the global variable related to component tFileInputFullRow to get the content of current the row.
Then I would like to manipulate it in a jJava component.
Thanks, regards.
Alessandro

Re: Reading and parsing Google Calendar ics file

Anonymous — Fri, 21 Feb 2014 10:25:33 GMT

Hi Alessandro

The global variable (Integer)globalMap.get("tFileInputFullRow_1_NB_LINE") can to be helpful but in first time I must have the content of the row.
What is the global variable to get the row string? How can I find documentation about this?

Not all of the components documents its global variables right now, we had reported this request to our Doc team, and we will explain the global variable related to the component one by one in future versions of document.

Then if I my row start (es.) with SUMMARY: or DTSTART: I take the value after ":" and I save the value in a structure with a row for each event with the field required.

tFileInputFullRow is the best component to read the entire row one by one from the source file, and then you can use a tFilterRow to tMap to filter rows that starts from SUMMARY: or DTSTART:, then extract the value after ":" on tJavaRow, for example:

output_row.SUMMARY=input_row.line.substring(input_row.line.indexOf(":")+1);

For more details, please see my screenshots. (In this case, I use a tFixedFlowInput to generate the source data provided by you)
Hope this can give you tip!
Shong

Re: Reading and parsing Google Calendar ics file

AlessandroCic — Sun, 23 Feb 2014 19:24:32 GMT

Hi Shong, hi forum
thank you, thank you very very much for this suggestions and the complete example about my issue.
Unfortunately, the example is not adequate to process the file .ics Google because it has a more complex structure than what I initially described.
For example between the lines:

BEGIN:VEVENT

END:VEVENT

it can to be more that one lines that begin with "DESCRIPTION:" and this it destroys the index of the data in the same section of the event.
But this is not a problem, nevertheless the suggestions and the example are very very useful for the development that I'm thinking of doing because I discovered the component tJavaRow which is better than the global variable that I thought to use.
During the next days I will work on tJavaRow and let you know.
Thanks, regards.
Alessandro

Re: Reading and parsing Google Calendar ics file

Anonymous — Mon, 24 Feb 2014 02:12:38 GMT

Alessandro
Yes, the example job is designed based on the example data you gave, however, I am sure it will give you some hints to achieve your request. If each event may contains more than one "DESCRIPTION:", you can also extract other property such as DTSTAMP that can be an identifier for each event, the identifier will be the key for merging the columns later if needed.
Shong