Re: How can I read from a cassandra table using a ... - Qlik Community

Anonymous · ‎2014-12-01

Hi,
I am trying to read data from Cassandra column family (see attached schema) which has a composite row key but when I read it through tCassandraInput, it gives an exception mentioned below.

Exception in component tCassandraInput_2
java.nio.BufferUnderflowException
	at java.nio.Buffer.nextGetIndex(Unknown Source)
	at java.nio.HeapByteBuffer.getLong(Unknown Source)
	at bigdatademo.testjob_0_1.TestJob.tCassandraInput_2Process(TestJob.java:916)
	at bigdatademo.testjob_0_1.TestJob.tCassandraConnection_1Process(TestJob.java:397)
	at bigdatademo.testjob_0_1.TestJob.runJobInTOS(TestJob.java:1277)
	at bigdatademo.testjob_0_1.TestJob.main(TestJob.java:1134)

How can I read from a Cassandra table using a composite key? I have attached job screenshot as well.

Anonymous · ‎2014-12-03

Can you please answer how I can use tCassandraInput with composite keys?
I have been waiting for reply on this and your help is appreciated

Anonymous · ‎2014-12-03

Hi,
Whether this works with https://help.talend.com/search/all?query=tCassandraRow
Have you tried that?
Vaibhav

Anonymous · ‎2014-12-05

sanvaibhav,
Thanks for your response. I am just doing simple read from a table having composite primary keys, I have tried two options. Can you suggest what's the possible error

using tCassandraInput to read from a table with composite keys but receiving this error

Exception in component tCassandraInput_2
java.nio.BufferUnderflowException
	at java.nio.Buffer.nextGetIndex(Unknown Source)
	at java.nio.HeapByteBuffer.getLong(Unknown Source)
	at bigdatademo.testjob_0_1.TestJob.tCassandraInput_2Process(TestJob.java:916)
	at bigdatademo.testjob_0_1.TestJob.tCassandraConnection_1Process(TestJob.java:397)
	at bigdatademo.testjob_0_1.TestJob.runJobInTOS(TestJob.java:1277)
	at bigdatademo.testjob_0_1.TestJob.main(TestJob.java:1134)

using tCassandraRow as you suggested but it is also giving this error

Starting job TestJob at 11:59 05/12/2014.
 connecting to socket on port 3433
 connected
: me.prettyprint.cassandra.connection.CassandraHostRetryService - Downed Host Retry service started with queue size -1 and retry delay 10s
: me.prettyprint.cassandra.service.JmxMonitor - Registering JMX me.prettyprint.cassandra.service_cluster_tCassandraConnection_1_37pScd:ServiceType=hector,MonitorType=hector
: me.prettyprint.cassandra.model.ConfigurableConsistencyLevel - Default write ConsistencyLevel set to: ONE.
: me.prettyprint.cassandra.connection.HConnectionManager - MARK HOST AS DOWN TRIGGERED for host localhost(127.0.0.1):9160
: me.prettyprint.cassandra.connection.HConnectionManager - Pool state on shutdown: <ConcurrentCassandraClientPoolByHost>:{localhost(127.0.0.1):9160}; IsActive?: true; Active: 1; Blocked: 0; Idle: 15; NumBeforeExhausted: 49
: me.prettyprint.cassandra.connection.ConcurrentHClientPool - Shutdown triggered on <ConcurrentCassandraClientPoolByHost>:{localhost(127.0.0.1):9160}
: me.prettyprint.cassandra.connection.ConcurrentHClientPool - Shutdown complete on <ConcurrentCassandraClientPoolByHost>:{localhost(127.0.0.1):9160}
: me.prettyprint.cassandra.connection.CassandraHostRetryService - Host detected as down was added to retry queue: localhost(127.0.0.1):9160
: me.prettyprint.cassandra.connection.HConnectionManager - Could not fullfill request on this host CassandraClient<localhost:9160-1>
: me.prettyprint.cassandra.connection.HConnectionManager - Exception: 
me.prettyprint.hector.api.exceptions.HectorTransportException:  org.apache.thrift.transport.TTransportException
	at me.prettyprint.cassandra.service.ExceptionsTranslatorImpl.translate(ExceptionsTranslatorImpl.java:43)
	at me.prettyprint.cassandra.service.ExceptionsTranslatorImpl.translate(ExceptionsTranslatorImpl.java:26)
	at me.prettyprint.cassandra.model.CqlQuery$1.execute(CqlQuery.java:195)
	at me.prettyprint.cassandra.model.CqlQuery$1.execute(CqlQuery.java:136)
	at me.prettyprint.cassandra.service.Operation.executeAndSetResult(Operation.java:104)
	at me.prettyprint.cassandra.connection.HConnectionManager.operateWithFailover(HConnectionManager.java:241)
	at me.prettyprint.cassandra.model.ExecutingKeyspace.doExecuteOperation(ExecutingKeyspace.java:102)
	at me.prettyprint.cassandra.model.CqlQuery.execute(CqlQuery.java:135)
	at bigdatademo.testjob_0_1.TestJob.tCassandraRow_1Process(TestJob.java:544)
	at bigdatademo.testjob_0_1.TestJob.tCassandraConnection_1Process(TestJob.java:442)
	at bigdatademo.testjob_0_1.TestJob.runJobInTOS(TestJob.java:1010)
	at bigdatademo.testjob_0_1.TestJob.main(TestJob.java:867)
Caused by: org.apache.thrift.transport.TTransportException
	at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
	at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
	at org.apache.thrift.transport.TFramedTransport.readFrame(TFramedTransport.java:129)
	at org.apache.thrift.transport.TFramedTransport.read(TFramedTransport.java:101)
	at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
	at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:362)
	at org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:284)
	at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:191)
	at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69)
	at org.apache.cassandra.thrift.Cassandra$Client.recv_execute_cql_query(Cassandra.java:1656)
	at org.apache.cassandra.thrift.Cassandra$Client.execute_cql_query(Cassandra.java:1642)
	at me.prettyprint.cassandra.model.CqlQuery$1.execute(CqlQuery.java:150)
	... 9 more
: me.prettyprint.cassandra.connection.HConnectionManager - Client CassandraClient<localhost:9160-1> released to inactive or dead pool. Closing.
Exception in component tCassandraRow_1
me.prettyprint.hector.api.exceptions.HectorException:  All host pools marked down. Retry burden pushed out to client.
	at me.prettyprint.cassandra.connection.HConnectionManager.getClientFromLBPolicy(HConnectionManager.java:401)
	at me.prettyprint.cassandra.connection.HConnectionManager.operateWithFailover(HConnectionManager.java:232)
	at me.prettyprint.cassandra.model.ExecutingKeyspace.doExecuteOperation(ExecutingKeyspace.java:102)
	at me.prettyprint.cassandra.model.CqlQuery.execute(CqlQuery.java:135)
	at bigdatademo.testjob_0_1.TestJob.tCassandraRow_1Process(TestJob.java:544)
	at bigdatademo.testjob_0_1.TestJob.tCassandraConnection_1Process(TestJob.java:442)
	at bigdatademo.testjob_0_1.TestJob.runJobInTOS(TestJob.java:1010)
	at bigdatademo.testjob_0_1.TestJob.main(TestJob.java:867)
: me.prettyprint.cassandra.connection.HConnectionManager - Shutdown called on HConnectionManager
: me.prettyprint.cassandra.connection.CassandraHostRetryService - Downed Host retry shutdown hook called
: me.prettyprint.cassandra.connection.CassandraHostRetryService - Downed Host retry shutdown complete
: me.prettyprint.cassandra.connection.NodeAutoDiscoverService - Auto Discovery retry shutdown hook called
: me.prettyprint.cassandra.connection.NodeAutoDiscoverService - AutoDiscovery retry shutdown complete
: me.prettyprint.cassandra.service.JmxMonitor - Unregistering JMX me.prettyprint.cassandra.service_cluster_tCassandraConnection_1_37pScd:ServiceType=hector,MonitorType=hector
 disconnected
Job TestJob ended at 11:59 05/12/2014.

Anonymous · ‎2014-12-08

It looks like you have connection issue...
can you connect using cql to the server from command prompt and fire the same query you used?
- Can you connect successfully to database using connection component?
Vaibhav

Anonymous · ‎2014-12-08

The connection works fine as you can see attached screen shot in my initial post.
Yes I can connect using cql Shell and can execute the same query with no issues.
The connection component is not giving any error.
But I don't know there might be some configuration related issue. I am using single Cassandra node with default configurations, no authentication and default replication strategy and factor. Does this can cause such an issue?

Anonymous · ‎2014-12-08

As per error, it is connection related issue... you can re-verify that after making some changes to the configuration.
Vaibhav

Anonymous · ‎2015-01-27

hi, maybe could help if you create a NOsql Conecction in the Metadata.
Later you have to import schema.
regards.
Salvador

Anonymous · ‎2015-12-18

Hi ,
I am facing same issue, while executing the job connecting to Cassandra.
Error:
Exception|tCassandraOutput_1|me.prettyprint.hector.api.exceptions.HectorException: All host pools marked down. Retry burden pushed out to client.|1
2015-12-18 14:32:51|20151218143238_jVY2j|20151218143238_jVY2j|20151218143238_jVY2j|636184|TEST|Data_Load_CASSANDRA|_LI2r4J9qEeWcDeROBqLAgA|0.1|UnixPaths||end|failure|12232
disconnected
Exception in component tCassandraOutput_1
me.prettyprint.hector.api.exceptions.HectorException: All host pools marked down. Retry burden pushed out to client.
at me.prettyprint.cassandra.connection.HConnectionManager.getClientFromLBPolicy(HConnectionManager.java:401)
at me.prettyprint.cassandra.connection.HConnectionManager.operateWithFailover(HConnectionManager.java:232)
at me.prettyprint.cassandra.model.ExecutingKeyspace.doExecuteOperation(ExecutingKeyspace.java:102)
at me.prettyprint.cassandra.model.MutatorImpl.execute(MutatorImpl.java:265)
at rdf.data_load_phm_0_1.Data_Load_CASSANDRA.tTeradataInput_1Process(Data_Load_CASSANDRA.java:3139)
at rdf.data_load_phm_0_1.Data_Load_CASSANDRA.tJava_1Process(Data_Load_CASSANDRA.java:1737)
at rdf.data_load_phm_0_1.Data_Load_CASSANDRA.tFileInputDelimited_1Process(Data_Load_CASSANDRA.java:1538)
at rdf.data_load_phm_0_1.Data_Load_CASSANDRA.runJobInTOS(Data_Load_CASSANDRA.java:6356)
at rdf.data_load_phm_0_1.Data_Load_CASSANDRA.main(Data_Load_CASSANDRA.java:6089)
through NoSQL iam able to connect and retrview schema.

Please help me out.
Bhargavi

How can I read from a cassandra table using a composite key?

Big Data

Java

v5.x