Failed to produce kinesis message with record id <3578> to partition <0> in stream 'dev-wood-fiber-ops'.

[cbwillia]

Whenever I tried to load a particular table, it fails.  One of the error message is "ProvisionedThroughputExceededException: Rate exceeded for shard shardId-000000000000 in stream dev-wood-fiber-ops under account 536159048647.."

Is there some setting I can change to allow this table to be loaded?

[SwathiPulagam · Accepted solution]

Hi @cbwillia ,

The number of messages and size of messages outputted to the Kinesis Target can be controlled with the following Internal Parameters for the Kinesis Target:

maxMessagesInBatch
maxBatchSize

Please review your CloudWatch for metrics and limit output accordingly.

https://aws.amazon.com/premiumsupport/knowledge-center/kinesis-readprovisionedthroughputexceeded/

Thanks,

Swathi