topic Re: ScyllaDB insertion in Talend Studio

ScyllaDB insertion

Anonymous — Fri, 26 Apr 2019 00:52:34 GMT

Has anyone successfully performed insertion with Open Studio for Big Data (7.1.1) into ScyllaDB?

I've inserted into single node [regular] Cassandra and multi- [3] node Cassandra without any problems... but can't get ScyllaDB Docker instance to insert at all. Talend connects to the instance and creates keyspace and column family just fine if it does not exist... but does not insert data. I can of course connect just fine to Scylla using cqlsh and any DB IDE. ScyllaDB docker is running as localhost (127.0.0.1); has anyone had any luck with this?

Re: ScyllaDB insertion

Anonymous — Fri, 26 Apr 2019 01:11:34 GMT

...and of course, almost immediately after posting this (and clued in by a potential bit from the scylla.yaml documentation below) you need to uncheck the "Use unlogged batch" radio button from the tCassandraOutput component in order to get the insertions to transpire.

I'm still not sure why my "periodic" synch did not show the insertions as they should have shown up in short order; nevertheless, this does appear to be the solution for the interim period until I can figure out how to tweak this paramater more thoroughly. The snippet from my scylla.yaml follows:

Thanks,
Jonathan

# When in batch mode, Scylla won't ack writes until the commit log
# has been fsynced to disk. It will wait
# commitlog_sync_batch_window_in_ms milliseconds between fsyncs.
# This window should be kept short because the writer threads will
# be unable to do extra work while waiting. (You may need to increase
# concurrent_writes for the same reason.)
#
# commitlog_sync: batch
# commitlog_sync_batch_window_in_ms: 2
#
# the other option is "periodic" where writes may be acked immediately
# and the CommitLog is simply synced every commitlog_sync_period_in_ms
# milliseconds.
commitlog_sync: periodic
commitlog_sync_period_in_ms: 10000

Re: ScyllaDB insertion

Anonymous — Fri, 26 Apr 2019 06:48:30 GMT

Hello,

So far, ScyllaDB is not list on the supported DB table.

For more information, please have a look at this online installation guide about:TalendHelpCenterupported systems, databases and business applications by Talend components for talend open studio bigdata V 7.1.1

Feel free to create a workitem/new feature jira issue on talend bug tracker.

Best regards

Sabr