Hi,
Sylvain Blanckeart from Talend has developping a route for us who takes parquets file generated from a previous job and put theses files in a minio bucket.
The previous job takes a table from a data base and put it in a csv file maybe more than one for a big table before to convert the files in parquet format in a directory who is monitoring by a route with a cFile.
I have two issuses, the first is if the route is running and I put a parquet file in the directory monitoring by a "cFile", the file who is put in minio is empty but not in the original directory. I dont have this probleme if I run the route after the files are in the directory.
I'm asking me, maybe the file is send to minio before the parquet file is filled ?
The second issus I have is when I deploy this route in a remote engine, I get this error:
I don't get this error in my talend studio.
org.apache.camel.CamelExecutionException: Exception occurred during execution on the exchange: Exchange[3DB65D25E6CD4DE-0000000000000240]
at org.apache.camel.CamelExecutionException.wrapCamelExecutionException(CamelExecutionException.java:45) ~[bundleFile:3.11.1]
at org.apache.camel.support.AbstractExchange.setException(AbstractExchange.java:589) ~[bundleFile:3.11.1]
at org.apache.camel.support.DefaultExchange.setException(DefaultExchange.java:27) ~[bundleFile:3.11.1]
at org.apache.camel.support.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:69) [bundleFile:3.11.1]
at org.apache.camel.processor.SendDynamicProcessor.lambda$process$0(SendDynamicProcessor.java:197) [bundleFile:3.11.1]
at org.apache.camel.support.cache.DefaultProducerCache.doInAsyncProducer(DefaultProducerCache.java:318) [bundleFile:3.11.1]
at org.apache.camel.processor.SendDynamicProcessor.process(SendDynamicProcessor.java:182) [bundleFile:3.11.1]
at org.apache.camel.processor.errorhandler.RedeliveryErrorHandler$SimpleTask.run(RedeliveryErrorHandler.java:463) [bundleFile:3.11.1]
at org.apache.camel.impl.engine.DefaultReactiveExecutor$Worker.schedule(DefaultReactiveExecutor.java:179) [bundleFile:3.11.1.tesb1]
at org.apache.camel.impl.engine.DefaultReactiveExecutor.scheduleMain(DefaultReactiveExecutor.java:64) [bundleFile:3.11.1.tesb1]
at org.apache.camel.processor.Pipeline.process(Pipeline.java:184) [bundleFile:3.11.1]
at org.apache.camel.impl.engine.CamelInternalProcessor.process(CamelInternalProcessor.java:398) [bundleFile:3.11.1.tesb1]
at org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:492) [bundleFile:3.11.1]
at org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:245) [bundleFile:3.11.1]
at org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:206) [bundleFile:3.11.1]
at org.apache.camel.support.ScheduledPollConsumer.doRun(ScheduledPollConsumer.java:190) [bundleFile:3.11.1]
at org.apache.camel.support.ScheduledPollConsumer.run(ScheduledPollConsumer.java:107) [bundleFile:3.11.1]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) [?:?]
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305) [?:?]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305) [?:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
at java.lang.Thread.run(Thread.java:829) [?:?]
Caused by: java.lang.NoClassDefFoundError: org/simpleframework/xml/strategy/Strategy
at io.minio.S3Base.createMultipartUpload(S3Base.java:1689) ~[?:?]
at io.minio.S3Base.putObject(S3Base.java:1165) ~[?:?]
at io.minio.MinioClient.putObject(MinioClient.java:1666) ~[?:?]
at org.apache.camel.component.minio.MinioProducer.putObject(MinioProducer.java:171) ~[?:?]
at org.apache.camel.component.minio.MinioProducer.process(MinioProducer.java:79) ~[?:?]
at org.apache.camel.support.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:66) ~[bundleFile:3.11.1]
... 19 more
Caused by: java.lang.ClassNotFoundException: org.simpleframework.xml.strategy.Strategy cannot be found by wrap_file__E__Talend_8.0.1-R2022-10-RT_runtime_system_io_minio_minio_8.2.2_minio-8.2.2.jar_0.0.0
at org.eclipse.osgi.internal.loader.BundleLoader.findClassInternal(BundleLoader.java:511) ~[org.eclipse.osgi-3.13.300.jar:?]
at org.eclipse.osgi.internal.loader.BundleLoader.findClass(BundleLoader.java:422) ~[org.eclipse.osgi-3.13.300.jar:?]
at org.eclipse.osgi.internal.loader.BundleLoader.findClass(BundleLoader.java:414) ~[org.eclipse.osgi-3.13.300.jar:?]
at org.eclipse.osgi.internal.loader.ModuleClassLoader.loadClass(ModuleClassLoader.java:153) ~[org.eclipse.osgi-3.13.300.jar:?]
at java.lang.ClassLoader.loadClass(ClassLoader.java:522) ~[?:?]
at io.minio.S3Base.createMultipartUpload(S3Base.java:1689) ~[?:?]
at io.minio.S3Base.putObject(S3Base.java:1165) ~[?:?]
at io.minio.MinioClient.putObject(MinioClient.java:1666) ~[?:?]
at org.apache.camel.component.minio.MinioProducer.putObject(MinioProducer.java:171) ~[?:?]
at org.apache.camel.component.minio.MinioProducer.process(MinioProducer.java:79) ~[?:?]
at org.apache.camel.support.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:66) ~[bundleFile:3.11.1]
... 19 more
The route is configured like this:
It begin with a cFile and send to minio with a cJavadslProcessor and there is a cConfig for adding this library: mvn:org.talend.libraries/camel-minio-alldep-3.11.1/6.0.0/jar
the cJavaDSLProcessor code is: .toD("minio://" + context.Ctx_Minio_BucketName + "?accessKey=" + context.Ctx_Minio_AccessKey + "&secretKey=" + context.Ctx_Minio_SecretKey + "&endpoint=" + context.Ctx_Minio_Endpoint + "®ion=" + context.Ctx_Minio_Region + "&keyName=${headers.CamelFileName}")
N.B.: My prefered language is french, so, you could response to me in french if you could.
Thank you
