topic Re: Performance testing in Talend Studio

Performance testing

Anonymous — Sat, 16 Nov 2024 08:53:47 GMT

Hello community,

I'm a performance testing expert and completely new with Talend.

My company has implemented many jobs using Talend and they asked me to valide its performance under a load test.

So far, I worked only with load testing tools such as Jmeter or Loadrunner but it seems it's not suitable for this case.

I heard about a bulk load mode but don't know much.

When I test a web based app, I validate metrics such as response time, CPU, hits per sec, throughput, transaction ok/ko etc...

But what about a Talend job?

What relevant metrics or information should I highlight in my analysis?

And what would be the best way to load test a Talend job? Any good tools for it?

Any help would be greatly appreciated.

I may have more information later about the technical architecture.

Antoine.

lojdr — Fri, 05 Jan 2018 15:50:51 GMT

Hello Antoine,

Bulk load is way how to import data to table directly from a file (e.g. CSV). Read about it for MySQL here:

Similar operations exist also for other databases.

Metrics which can be important:

Number of records per second processed
How long the job runs (seconds, milliseconds...)
Resources consumed (can be merged via OS monitoring and PID provided by tStatCatcher)

One of the ways is to run the job in Talend Studio and see performance counters.

Another way is to collect statistics using tStatCatcher. It will return you the information you can merge with e.g. OS statistics (via PID number).

These statistics can be exported to a CSV file, database... The schema on the screenshot is generated the tStatCatcher.

Hope it helps.

Regards

lojdr

Anonymous — Tue, 09 Jan 2018 11:21:53 GMT

Thank you Lojdr, I really appreciate your help!

I gonna try it soon but I'm almost sure it will solve my problem.

I'll keep you updated.

Thank you again.

Antoine.