Failed to produce Eventhubs message with record id

[MoeyE]

Hi Guys.

The log message is

]I: Failed to produce Eventhubs message with record id <32266985> to partition <-1> in hub 'example'. System.Exception: Operation SendBatchAsync failed.

Has anyone ever encountered this issue for Event Hubs? It says that the message was not produced.

I did some researching and it says that memory exception errors are caused by not enough contiguous memory. But this doesn't mean total  system memory from what I've read. Correct?

Looking through some logs I only see this issue just once. Anyone know general potential causes or contributing factors?

I may open a support case later if needed but just seeing if I can get some quick info. Thanks.

Kind Regards,

Mohammed

[john_wang]

Hello @MoeyE ,

Thanks for reaching out!

Many different reasons may lead SendBatchAsync() call failed eg message batch size, network connectivity, and buffer memory (as you mentioned) etc. Please make sure the maximum batch size does not exceed the 256k limit in target endpoint setting.

I also noticed the partition ID "-1" is not reasonable value (should be 0 ~ maximum partition number). Please confirm the generated partition number (for example partition ID is generated by an expression in table or global transformation) is in correct range because Kafka Client will route the event to that specific event hubs partition.

BTW, from the piece of log line looks to me it's an INFORMATION level message, there should be retry implicitly or explicitly. please check the task log file to see if any clue.

Hope this helps.

Regards,

John.

[MoeyE]

Hi John,

Thanks for the reply. I don't see the limitation of 256,000 byte limit for Event Hubs endpoint. Where can I find more info on this? 

Also I don't understand the line about the partition. What does -1 mean and what does 0 mean? How can it be changed?

Thanks.

Kind Regards,

Mohammed

[john_wang]

Hello @MoeyE ,

I hope the page Azure Event Hubs quotas and limits helps. Regarding the partition, do you have any transform on it ? or what's the partition setting in table/task level?

Regards,

John.

[MoeyE]

Hi John,

Yep, thanks for the the Event Hubs page it was helpful and insightful. I am currently unable to answer the rest of the questions. Thanks for the help!

Regards,

Mohammed