Unlock a world of possibilities! Login now and discover the exclusive benefits awaiting you.
currently we are using tKinesisInput component to consume data from kinesis stream. it works when data is in readable format ex- json, but with aws cloud watch log data , subscription filters gzip's data and publishes into the stream. so when
tKinesisInput component reads this data, it cannot unzip it ( field type for payload is string, and it cannot be modified to byte[]) , we see unreadable data and unzip java code says , payload is not a zipped file/format.
we raise a support ticket to give flexibility to change payload data type from string to byte[] , so to unzip using java code, but its been a while and no ETA yet form talend r&d.
im checking in here to see, if there are any alternative solutions to unzip data ( I tried to place data in file and use tunarchive comoponent but it gives same error - not a zipped file/format.
Hello,
Could you please give us your support case Id? In this way, we will check your case status.
Best regards
Sabrina
Hello, below are talend and its jira tickets
talend case -
For Case 00202441, How to read and decompress data from AWS Kinesis in Big Data Streaming job, currently the JIRA ( TCOMP-1822 (https://tlnd-prod--zsfjira.na119.visual.force.com/apex/zsfjira__ZIssueDetails?issueKey=TCOMP-1822) ) Unable to decompress data from AWS Kinesis in Big Data Streaming job has been assigned under review for approval with no ETA Date on Fix or Release.
JIRA:
https://jira.talendforge.org/browse/TBD-11753
Unable to decompress data from AWS Kinesis in Big Data Streaming job
Hello,
Our colleagues are working on your feature request and we will keep you up to date about it.
Best regards
Sabrina