Do not input private or sensitive data. View Qlik Privacy & Cookie Policy.
Skip to main content

Announcements
Qlik Open Lakehouse is Now Generally Available! Discover the key highlights and partner resources here.
cancel
Showing results for 
Search instead for 
Did you mean: 
TCommunity1615436108
Contributor
Contributor

Talend Cloud Databricks Error- modification_time

Hi,

I started using Talend Cloud very recently and the use case I am trying is to read data from Delta Lake using Databricks Spark Configuration. I created a Spark Batch job and used my Databricks workspace and cluster in spark configuration. When I execute the job, it is failing immediately with the following error. I tried every runtime version, but it always immediately fails. I couldn't find much conversation on "modification_time" error in Talend. Did anyone else face the same issue?

 

 

org.talend.bigdata.launcher.utils.BigDataLauncherException: Error getting content of remote folder '/FileStore/talend/'

at org.talend.bigdata.launcher.fs.DatabricksFileSystem.list(DatabricksFileSystem.java:197)

at org.talend.bigdata.launcher.fs.DatabricksFileSystem.list(DatabricksFileSystem.java:142)

at org.talend.bigdata.launcher.fs.DatabricksFileSystem.updateDatabricksJarList(DatabricksFileSystem.java:565)

at org.talend.bigdata.launcher.databricks.DatabricksJob.uploadJars(DatabricksJob.java:153)

at org.talend.bigdata.launcher.databricks.DatabricksJob.executeJob(DatabricksJob.java:386)

at edap_rbdf.test_0_1.test.runClientJob(test.java:676)

at edap_rbdf.test_0_1.test.runJobInTOS(test.java:480)

at edap_rbdf.test_0_1.test.main(test.java:342)

Caused by: org.talend.bigdata.jackson.databind.exc.UnrecognizedPropertyException: Unrecognized field "modification_time" (class org.talend.bigdata.launcher.fs.DatabricksFileSystem$FileInfo), not marked as ignorable (3 known properties: "is_dir", "file_size", "path"])

 at [Source: (ByteArrayInputStream); line: 1, column: 131] (through reference chain: org.talend.bigdata.launcher.fs.DatabricksFileSystem$FileInfoList["files"]->java.util.ArrayList[0]->org.talend.bigdata.launcher.fs.DatabricksFileSystem$FileInfo["modification_time"])

at org.talend.bigdata.jackson.databind.exc.UnrecognizedPropertyException.from(UnrecognizedPropertyException.java:61)

at org.talend.bigdata.jackson.databind.DeserializationContext.handleUnknownProperty(DeserializationContext.java:840)

at org.talend.bigdata.jackson.databind.deser.std.StdDeserializer.handleUnknownProperty(StdDeserializer.java:1206)

at org.talend.bigdata.jackson.databind.deser.BeanDeserializerBase.handleUnknownProperty(BeanDeserializerBase.java:1592)

at org.talend.bigdata.jackson.databind.deser.BeanDeserializerBase.handleUnknownVanilla(BeanDeserializerBase.java:1570)

at org.talend.bigdata.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:294)

at org.talend.bigdata.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:151)

Labels (4)
1 Reply
Anonymous
Not applicable

@Talend Community​ , I have seen some related issues open on our bugtracker, have you raised a ticket on Talend Support Portal? If not, please do it and check with our support team if a patch is available.

 

Regards

Shong