topic Re: sampling with TOS? in Talend Studio

sampling with TOS?

Anonymous — Sat, 16 Nov 2024 13:34:51 GMT

Hi!
Are there any other methods to create a sample apart from tSampleRow?
I want a random sample or better a stratified sample.
if there is only tSampleRow please give me the reply that it isn´t possible to do it on an easy way.
thanks and best regards
matte

Re: sampling with TOS?

Anonymous — Mon, 01 Feb 2010 14:33:20 GMT

Hello

I want a random sample or better a stratified sample.

Would you like to get some randow rows? Can you give us an example?
Best regards
shong

Re: sampling with TOS?

Anonymous — Mon, 01 Feb 2010 14:51:19 GMT

for example i have 1.000.000 Customers with different attributes and to optimize the processing time i want to get only 30 % of the whole Data. i dont want the first 300.000 customers. i want any of the customers (random).
and with stratified sample mean for example:
i have 1.000.000 customers too but i want a random sample that contains 40 % men and 60 % women.
like this:
Database:
id Name Gender ...
1 a m
2 b m
3 c f
4 d m
5 e f
6 f f
7 g f
8 h m
9 i m
10 j f
11 k m
12 l m
13 m f
14 n m
15 o f
16 p f
17 q f
18 r m
19 s m
20 t f

Sample 50 % with 40 % men and 60 %
id Name Gender ...
2 b m
3 c f
7 g f
8 h m
10 j f
12 l m
15 o f
17 q f
18 r m
20 t f
i hope this is more understanable

best regards
matte

Re: sampling with TOS?

Anonymous — Mon, 01 Feb 2010 16:34:45 GMT

i think there dont't have a componet can do this, i think you can use first get the ids of m and f
like
M's ID in DB
id
1
2
4
8
9
11
12
14
18
19

F's ID in DB
id
3
5
6
7
10
13
15
16
17
20
and difine a java method, use it to get random ids

Re: sampling with TOS?

Anonymous — Tue, 02 Feb 2010 10:01:38 GMT

Hello matte
As ylp_1 said, it need to define a Java method in a routine to get the random ids as your required, and then call this routine and get some random ids before quering data from table. Are you a Java programmer? If not, I can create a demo for you later.
Best regards
shong

Re: sampling with TOS?

Anonymous — Wed, 03 Feb 2010 15:32:33 GMT

Hi!
Are there any other methods to create a sample apart from tSampleRow?
I want a random sample or better a stratified sample.
if there is only tSampleRow please give me the reply that it isn´t possible to do it on an easy way.
thanks and best regards
matte

If you are using a sql db as source, you can specify your select as
select * from mytable where rand() < 0.3
to fetch an approximate sample 30% of rows

Re: sampling with TOS?

Anonymous — Thu, 04 Feb 2010 09:48:39 GMT

thanks for your answers.
yes i´m a java programer. not the best but i can implement a random sampling.
i asked because i want to evaluate the function from tos that are given by the components.