http://www.talendforge.org/forum/viewtopic.php?pid=46295#p46295 There is a possiblity that the issue in this post may be a bug. Here is the text....
---post 1---
Hey guys. I've got a problem specifically when using the tinfobrightoutput object. I'm pulling data from a modified apache log and when I do the load I get the following error from infobright
2010-10-19 15:52:37 The Infobright storage engine has encountered an unexpected error. The current transaction has been rolled back.
2010-10-19 15:52:37 Error: Wrong data or column definition. Row: 152271, field: 18.
2010-10-19 15:52:37 Error: An unknown system exception error caught.
I have checked this row of data and it appears to be the same as everything else. And I don't know how something unexpected could be passed to the loader as I'm not getting any errors from talend regarding the input that it's getting.
I was just wondering if anyone else has gotten this error while working with tinfobrightoutput.
---post 2---
I want to add that I tried this process again, though the second time I had it output to a delimited file so I could see exactly what it was passing back to the infobright loader at line 152271 and in the csv file the data looks exactly as it should.
In this case the error message is coming from infobright, not from talend. There is something that Talend is sending to the infobright loader that it doesn't like. I don't think this is a problem with the format of the table or else the load would fail on the first record dues to data not being the correct type. I don't think this is a prblem with the log itself as it's being parsed correctly when I send it to a csv file.
I don't think the tmap would send something that's outside of it's scop (IE: I don't think that something in the output side of the tmap that is set to INT would pass a string to infobright) so I'm at a bit of a loss. While there is a distinct possibility that I might be doing something wrong, I am going to repost this to the bug forum as it sounds like that might be the case.
- Peter
Sorry, one more quick addition. After loading the data into the MYISAM table I created another job to transfer the data from there into the infobright table and ran into the same problem. The only guess that I have at this point is that when Talend uses the infobright data loader you would figure that it would have to specify how it's delimiting it's columns when passing it to the infobright data loader. It's possible that whatever it's using as a delimiter may be contained in one of the text fields that I'm passing to infobright, though I find that to be a bit of a long shot as I have other processes that parse delimited files and pass them to the infobright DB, but I can't think of anything else that this could be.
Does anyone know what tinfobrightoutput uses as a delimiter before passing the data to the infobright data loader?
- Peter
Hi,
tInfobrightOutput uses (internally) ',' as a delimiter, ' " ' (double quote) as an enclosure, and ' \ ' backslash) as escape character.
It is more than likely an embedded quote character or something similar. If you wanted to share the offending row, I can confirm.
Geoffrey Falk, Infobright
Hi, tInfobrightOutput uses (internally) ',' as a delimiter, ' " ' (double quote) as an enclosure, and ' \ ' backslash) as escape character. It is more than likely an embedded quote character or something similar. If you wanted to share the offending row, I can confirm. Geoffrey Falk, Infobright
It's been speculated that there is a known issue with Infobright remote data loader, where if you try to pass too much to it it will give this error. I've broken the log files we're using into smaller chunks of 25k rows per file and that has improved the situation, but I still can't say 100% that this is what resolved the issue as I'm still working out other bugs with the process. Once I have something more conclusive I'll be posting all the details here.
- Peter
