Data Profiling Report Generation Failure

[hiitzsdg]

Hi,
I performed a Columnar analysis and then I am trying to generate the "Basic" report. Being new to Talend DQ, some of the reports are generated successfully. But for few, I am getting an error message and the report generation fails. The error message that I am getting is "Error during historization of data of report <Report_Name>: could not insert : [org.talend.dataprofiler.datamart.hibernate.TdqValues].

The error message that I am seeing in the Error Log is as under:
org.hibernate.exception.GenericJDBCException: could not insert: [org.talend.dataprofiler.datamart.hibernate.TdqValues]
at org.hibernate.exception.SQLStateConverter.handledNonSpecificException(SQLStateConverter.java:103)
at org.hibernate.exception.SQLStateConverter.convert(SQLStateConverter.java:91)
at org.hibernate.exception.JDBCExceptionHelper.convert(JDBCExceptionHelper.java:43)
at org.hibernate.persister.entity.AbstractEntityPersister.insert(AbstractEntityPersister.java:2202)
at org.hibernate.persister.entity.AbstractEntityPersister.insert(AbstractEntityPersister.java:2595)
at org.hibernate.action.EntityInsertAction.execute(EntityInsertAction.java:51)
at org.hibernate.engine.ActionQueue.execute(ActionQueue.java:248)
at org.hibernate.engine.ActionQueue.executeActions(ActionQueue.java:232)
at org.hibernate.engine.ActionQueue.executeActions(ActionQueue.java:139)
at org.hibernate.event.def.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:298)
at org.hibernate.event.def.DefaultFlushEventListener.onFlush(DefaultFlushEventListener.java:27)
at org.hibernate.impl.SessionImpl.flush(SessionImpl.java:1000)
at org.hibernate.impl.SessionImpl.managedFlush(SessionImpl.java:338)
at org.hibernate.transaction.JDBCTransaction.commit(JDBCTransaction.java:106)
at org.talend.dq.persistence.DatabasePersistence.persist(DatabasePersistence.java:156)
at org.talend.dataquality.reporting.engine.ReportDocGenerator.historizeData(ReportDocGenerator.java:505)
at org.talend.dataquality.reporting.engine.ReportDocGenerator.generate(ReportDocGenerator.java:275)
at org.talend.dataprofiler.core.tdq.action.LocalReportDocGenerator.generate(LocalReportDocGenerator.java:129)
at org.talend.dataprofiler.core.tdq.ui.action.GenerateReportFileAction$2.run(GenerateReportFileAction.java:342)
at org.eclipse.core.internal.jobs.Worker.run(Worker.java:54)
Caused by: java.sql.SQLException: ORA-12899: value too large for column "DQ_PORTAL"."TDQ_VALUES"."VAL_STRING" (actual: 500, maximum: 255)

at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:445)
at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:396)
at oracle.jdbc.driver.T4C8Oall.processError(T4C8Oall.java:879)
at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:450)
at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:192)
at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:531)
at oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:207)
at oracle.jdbc.driver.T4CPreparedStatement.executeForRows(T4CPreparedStatement.java:1044)
at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1329)
at oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3584)
at oracle.jdbc.driver.OraclePreparedStatement.executeUpdate(OraclePreparedStatement.java:3665)
at oracle.jdbc.driver.OraclePreparedStatementWrapper.executeUpdate(OraclePreparedStatementWrapper.java:1352)
at org.hibernate.jdbc.NonBatchingBatcher.addToBatch(NonBatchingBatcher.java:23)
at org.hibernate.persister.entity.AbstractEntityPersister.insert(AbstractEntityPersister.java:2182)
... 16 more

As per my understanding this is because of space. So should I contact with the Admin Team to have the space increased for TDQVALUES tables? Can the DBA's change this size without any impact?

Your help in this regards is highly appreciated.

[Anonymous]

Hi,
Could you please indicate what's the build version you are using? Is there any special characters in your column? Screenshots of data profiling setting will be preferred.
Best regards
Sabrina

[hiitzsdg]

Hi,
I am using Talend Big Data Platform (6.1.1.20151214_1327). There are Chinese characters in columns. We are performing Simple, Advanced, Pattern Frequency.

[Anonymous]

Hi,
This issue is probably caused by Charset(Some special characters get 2 bytes in UTF-8, while they get only one byte in other charset.)
Did you set file delimilted metadata for your column? If so, what does file setting look like? Are you using UFT-8 Encoding?
Best regards
Sabrina

[hiitzsdg]

We are using Oracle Database as our metadata connection. 

[Anonymous]

Hi,
Have you tried to    modify the length of this column in DB side to see if it works?
Best regards
Sabrina

[hiitzsdg]

Modify to what length? 500 as being suggested in the error message?

[hiitzsdg]

Hi,

I modified the field length of the column TDQ_VALUES.VAL_STRING to 500 and the report generation was successful for most of the analysis. But it failed for one of the reports with the below error message in the Error_Log:

java.lang.StringIndexOutOfBoundsException: String index out of range: -3
at java.lang.String.substring(Unknown Source)
at org.jfree.text.G2TextMeasurer.getStringWidth(G2TextMeasurer.java:80)
at org.jfree.text.TextUtilities.nextLineBreak(TextUtilities.java:294)
at org.jfree.text.TextUtilities.createTextBlock(TextUtilities.java:235)
at org.jfree.chart.axis.CategoryAxis.createLabel(CategoryAxis.java:1140)
at org.jfree.chart.axis.CategoryAxis.refreshTicks(CategoryAxis.java:1106)
at org.jfree.chart.axis.CategoryAxis.reserveSpace(CategoryAxis.java:823)
at org.jfree.chart.plot.CategoryPlot.calculateDomainAxisSpace(CategoryPlot.java:3144)
at org.jfree.chart.plot.CategoryPlot.calculateAxisSpace(CategoryPlot.java:3211)
at org.jfree.chart.plot.CategoryPlot.draw(CategoryPlot.java:3257)
at org.jfree.chart.JFreeChart.draw(JFreeChart.java:1230)
at org.jfree.chart.JFreeChart.draw(JFreeChart.java:1114)
at net.sf.jasperreports.charts.util.DrawChartRenderer.render(DrawChartRenderer.java:62)
at net.sf.jasperreports.engine.export.JRPdfExporter.exportImage(JRPdfExporter.java:1505)
at net.sf.jasperreports.engine.export.JRPdfExporter.exportElements(JRPdfExporter.java:854)
at net.sf.jasperreports.engine.export.JRPdfExporter.exportPage(JRPdfExporter.java:818)
at net.sf.jasperreports.engine.export.JRPdfExporter.exportReportToStream(JRPdfExporter.java:695)
at net.sf.jasperreports.engine.export.JRPdfExporter.exportReport(JRPdfExporter.java:417)
at org.talend.dataquality.reporting.JasperReportBuilder.run(JasperReportBuilder.java:300)
at org.talend.dataquality.reporting.engine.ReportDocGenerator.generateReport(ReportDocGenerator.java:679)
at org.talend.dataquality.reporting.engine.ReportDocGenerator.generateDoc(ReportDocGenerator.java:413)
at org.talend.dataprofiler.core.tdq.action.LocalReportDocGenerator.generateDocInternal(LocalReportDocGenerator.java:175)
at org.talend.dataprofiler.core.tdq.action.LocalReportDocGenerator.access$0(LocalReportDocGenerator.java:174)
at org.talend.dataprofiler.core.tdq.action.LocalReportDocGenerator$1.run(LocalReportDocGenerator.java:163)
at org.talend.repository.RepositoryWorkUnit.executeRun(RepositoryWorkUnit.java:99)
at org.talend.core.repository.model.AbstractRepositoryFactory.executeRepositoryWorkUnit(AbstractRepositoryFactory.java:258)
at org.talend.repository.localprovider.model.LocalRepositoryFactory.executeRepositoryWorkUnit(LocalRepositoryFactory.java:3217)
at org.talend.core.repository.model.ProxyRepositoryFactory.executeRepositoryWorkUnit(ProxyRepositoryFactory.java:2024)
at org.talend.dataprofiler.core.tdq.action.LocalReportDocGenerator.generateDoc(LocalReportDocGenerator.java:168)
at org.talend.dataquality.reporting.engine.ReportDocGenerator.generate(ReportDocGenerator.java:293)
at org.talend.dataprofiler.core.tdq.action.LocalReportDocGenerator.generate(LocalReportDocGenerator.java:129)
at org.talend.dataprofiler.core.tdq.ui.action.GenerateReportFileAction$2.run(GenerateReportFileAction.java:342)
at org.eclipse.core.internal.jobs.Worker.run(Worker.java:54)

[Anonymous]

Hi,
For your  Talend Big Data Platform  product, could you please report a ticket on talend support portal so that we can give you a remote assistance on your issue through support cycle with priority?
Best regards
Sabrina