topic Re: How to get data from AWS Kinesis in Talend data streams in Archived Discussions

How to get data from AWS Kinesis in Talend data streams

Anonymous — Thu, 08 May 2025 17:40:29 GMT

Hi,

I am currently trying out Talend Data Streams on AWS and Kinesis.

I have lambda operation that puts data into kinesis stream and I also created a connector to Kinesis srteam.

My Kinesis stream monitoring shows successful getRecords from Talend Data Stream yet I keep having this error

Cannot fetch sample from dataset 29409e8b-f3d6-4161-9ec6-62ee23917236

NB:

I have added an IAM role my EC2 instance that has full Kinesis rights.

my dataset type is CSV even though I am passing a string to Kinesis as data

I wish to know what I am missing out. Please help

Thanks

Re: How to get data from AWS Kinesis in Talend data streams

Anonymous — Wed, 19 Jun 2019 11:41:01 GMT

Sorry about the delay in getting back to you. Have you tried this use case in Pipeline Designer (the new evolution of the Data Streams idea)? You may find that this is better handled in Pipeline Designer.

Re: How to get data from AWS Kinesis in Talend data streams

Pressley1647 — Mon, 28 Mar 2022 09:16:36 GMT

Amazon Kinesis Data Streams is a massively scalable, highly durable data ingestion and processing service optimized for streaming data. You can configure hundreds of thousands of data producers to continuously put data into a Kinesis data stream. Data will be available within milliseconds to your Amazon Kinesis applications, and those applications will receive data records in the order they were generated.

MCDVOICE

Re: How to get data from AWS Kinesis in Talend data streams

IronSloth — Wed, 01 Jun 2022 10:45:01 GMT

Awesome and interesting article. Great things you’ve always shared with us. Thanks. Just continue composing this kind of post.

MyMileStoneCard Login

Re: How to get data from AWS Kinesis in Talend data streams

milestone — Wed, 11 Dec 2024 17:54:43 GMT

To resolve the error you're encountering in Talend Data Streams with AWS Kinesis, ensure that your dataset schema matches the data format being sent to Kinesis. Since you're passing a string but have set the dataset type to CSV, this mismatch could be causing the issue. Additionally, verify that the IAM role attached to your EC2 instance has the necessary permissions for both reading from and writing to the Kinesis stream. Lastly, check the connector configuration for any discrepancies. For more information and discussion please contact us at Mymilestonecard.