topic Re: JDBC to Clickhouse? in Talend Studio

JDBC to Clickhouse?

Anonymous — Sat, 16 Nov 2024 05:42:38 GMT

Hi friends,

Has anyone of you guys tried to connect Talend Open Studio 7.1.1 (for Data Integration) to ClickHouse before?

I tried connecting from Repository > Metadata > Db Connection > Right Click > Add New Connection > Give it a name > Next > and then key in details provided here

https://github.com/yandex/clickhouse-jdbc

but no luck. When "Test Connection" button is clicked, i could see incoming connection in clickhouse log but Talend Open Studio "Finish" button is still disabled

Also, tried using tDBInput component and basically configure as above, give a little nice query "SELECT 1", and then run the job. I got this exception :

Error Line: 474
Detail Message: ru.yandex.clickhouse.ClickHouseDriver cannot be resolved to a variable
There may be some other errors caused by JVM compatibility. Make sure your JVM setup is similar to the studio.

	at org.talend.designer.runprocess.JobErrorsChecker.checkLastGenerationHasCompilationError(JobErrorsChecker.java:338)

	at org.talend.designer.runprocess.DefaultRunProcessService.checkLastGenerationHasCompilationError(DefaultRunProcessService.java:510)

	at org.talend.designer.runprocess.RunProcessService.checkLastGenerationHasCompilationError(RunProcessService.java:316)

	at org.talend.designer.runprocess.ProcessorUtilities.generateBuildInfo(ProcessorUtilities.java:820)

	at org.talend.designer.runprocess.ProcessorUtilities.generateCode(ProcessorUtilities.java:601)

	at org.talend.designer.runprocess.ProcessorUtilities.generateCode(ProcessorUtilities.java:1861)

	at org.talend.designer.runprocess.RunProcessContext$1.run(RunProcessContext.java:601)

	at org.eclipse.jface.operation.ModalContext.runInCurrentThread(ModalContext.java:466)

	at org.eclipse.jface.operation.ModalContext.run(ModalContext.java:374)

	at org.eclipse.jface.dialogs.ProgressMonitorDialog.run(ProgressMonitorDialog.java:527)

	at org.eclipse.ui.internal.progress.ProgressMonitorJobsDialog.run(ProgressMonitorJobsDialog.java:284)

	at org.eclipse.ui.internal.progress.ProgressManager.run(ProgressManager.java:1190)

	at org.talend.designer.runprocess.RunProcessContext.exec(RunProcessContext.java:548)

	at org.talend.designer.runprocess.ui.ProcessComposite.exec(ProcessComposite.java:1401)

	at org.talend.designer.runprocess.ui.views.ProcessView$RunAction.run(ProcessView.java:701)

	at org.talend.designer.runprocess.ui.actions.RunProcessAction.run(RunProcessAction.java:58)

	at org.talend.designer.core.debug.JobLaunchConfigurationDelegate$1.run(JobLaunchConfigurationDelegate.java:84)

	at org.eclipse.swt.widgets.RunnableLock.run(RunnableLock.java:35)

	at org.eclipse.swt.widgets.Synchronizer.runAsyncMessages(Synchronizer.java:136)

	at org.eclipse.swt.widgets.Display.runAsyncMessages(Display.java:4147)

	at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:3764)

	at org.eclipse.e4.ui.internal.workbench.swt.PartRenderingEngine$9.run(PartRenderingEngine.java:1151)

	at org.eclipse.core.databinding.observable.Realm.runWithDefault(Realm.java:332)

	at org.eclipse.e4.ui.internal.workbench.swt.PartRenderingEngine.run(PartRenderingEngine.java:1032)

	at org.eclipse.e4.ui.internal.workbench.E4Workbench.createAndRunUI(E4Workbench.java:148)

	at org.eclipse.ui.internal.Workbench$5.run(Workbench.java:636)

	at org.eclipse.core.databinding.observable.Realm.runWithDefault(Realm.java:332)

	at org.eclipse.ui.internal.Workbench.createAndRunWorkbench(Workbench.java:579)

	at org.eclipse.ui.PlatformUI.createAndRunWorkbench(PlatformUI.java:150)

	at org.talend.rcp.intro.Application.start(Application.java:265)

	at org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:196)

	at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:134)

	at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:104)

	at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:380)

	at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:235)

	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

	at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)

	at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)

	at java.lang.reflect.Method.invoke(Unknown Source)

	at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:648)

	at org.eclipse.equinox.launcher.Main.basicRun(Main.java:603)

	at org.eclipse.equinox.launcher.Main.run(Main.java:1465)

When trying to connect directly (from cmd or terminal), it succeeded.

How can I connect ClickHouse to Talend Open Studio?

Re: JDBC to Clickhouse?

Anonymous — Thu, 30 May 2019 08:07:46 GMT

Hello,

Could you show job and tDBInput components configuration screenshots? Please mask your sensitive data.

Best regards

Sabrina

Re: JDBC to Clickhouse?

Anonymous — Thu, 30 May 2019 11:30:14 GMT

As simple as it could be, this is the job when i tried to connect from tDBInput.

I downloaded the whole clickhouse jars and its dependencies from

https://jar-download.com/?search_box=clickhouse-jdbc

and then follow their instruction on how to input driver class from

https://github.com/yandex/clickhouse-jdbc

Blank User Id and Password? Yes. I did not set user and password for clickhouse database.

Clickhouse client and server are installed on "remote machine"

On local machine browser, it connects successfully to clickhouse-server on remote machine.

On remote machine itself, clickhouse-client is connected successfully to the database without username and password.

This picture below might explain better :

But, this is what i got when the above job is run :

Am I missing something obvious?

Re: JDBC to Clickhouse?

Anonymous — Fri, 31 May 2019 03:36:01 GMT

(It's weird, this is my second time replying to this, hope my reply could show up because the first time it just did not although i clicked "Post" already)

As simple as it is, here is the job config :

So, clickhouse is installed in "remote machine"

When i tried accessing it from other machine (via browser) or from the remote machine itself, it works as shown below :

Unfortunately, found no luck accessing it from the above job configuration. When I run the job, it shows me this :

Am I missing something obvious?

Re: JDBC to Clickhouse?

Anonymous — Fri, 31 May 2019 06:01:33 GMT

Finally, I am able to connect.

For future reference to others visiting this topic,

although it is automatically filled in by TOS, the "Driver Class" should be wrapped in double-quote.

Thanks to this topic :

https://community.talend.com/t5/Design-and-Development/Error-compiling-quot-cannot-be-resolved-to-a-variable-quot/td-p/45212

I am now able to connect to clickhouse via jdbc.

Remember to put all dependencies found here :

https://jar-download.com/artifact-search/clickhouse-jdbc

Cheers

Re: JDBC to Clickhouse?

Anonymous — Fri, 31 May 2019 06:07:29 GMT

Finally, I am able to connect.

For future reference to others visiting this topic,

although it is automatically filled in by TOS, the "Driver Class" should be wrapped in double-quote.

Thanks to this topic :

https://community.talend.com/t5/Design-and-Development/Error-compiling-quot-cannot-be-resolved-to-a-variable-quot/td-p/45212

I am now able to connect to clickhouse via jdbc.

Remember to put all dependencies found here :

https://jar-download.com/artifact-search/clickhouse-jdbc

Cheers

Re: JDBC to Clickhouse?

Anonymous — Fri, 31 May 2019 08:17:44 GMT

Hello,

Is it OK with you when you locate your clickhouse-jdbc-0.1.54.jar in local machine?

Best regards

Sabrina

Re: JDBC to Clickhouse?

Anonymous — Tue, 04 Jun 2019 07:05:55 GMT

hi xdshi,

Yes, it was ok when i locate clickhouse-jdbc-0.1.54.jar
What I did was :
1. in TOS, go to Windows > Preference > Java > Build Path > User Libraries > import the jar file with all its dependencies
2. in tDBInput component > Component Tab > Drivers > locate all necessary jar files

That worked for me!

Thank you for your support