topic Re: how to access Mapper Counter value in a Reducer? in Talend Studio

how to access Mapper Counter value in a Reducer?

_AnonymousUser — Sat, 16 Nov 2024 12:10:59 GMT

I want to acces the myCounter.my value in reducer :

public static class Map extends Mapper<LongWritable, Text, ImmutableBytesWritable, ImmutableBytesWritable>
{
public static enum myCounter{my};

@Override
public void map(LongWritable key, Text value, Context context)
{
context.getCounter(myCounter.my).increment(1);
context.write( new ImmutableBytesWritable ( ),new ImmutableBytesWritable() );
}
}

public static class Reduce extends Reducer<ImmutableBytesWritable, ImmutableBytesWritable, Text, Text>
{
@Override
public void reduce(ImmutableBytesWritable key,Iterable<ImmutableBytesWritable> result,Context context)
{

}
}

how to make it work for new API(hadoop 0.20.0 API) ?
**Or**
I want to know the total number of mapper output ? Is there any better way ? (i am not able to access counter in Reducer:
`Group Name->org.apache.hadoop.mapred.Task$Counter Counter Name->MAP_OUTPUT_RECORDS`)

Thanks

Re: how to access Mapper Counter value in a Reducer?

Anonymous — Mon, 08 Oct 2012 12:07:39 GMT

Hi,

How come you could execute map reduce using talend? Is it by using tLibraryLoad and tJava? Is there any other way?

Re: how to access Mapper Counter value in a Reducer?

_AnonymousUser — Tue, 22 Jan 2013 09:55:22 GMT

Hi ,,
I want to know is there any way to perform map-reduce for weblog data in talend vig data v.5.2.1
Thanks

Re: how to access Mapper Counter value in a Reducer?

Anonymous — Wed, 06 Feb 2013 00:32:41 GMT

Hi
What's the use-case? Talend tries to avoid the need to write map-reduce code through abstraction the transformation into a set of graphical components.
Have you reviewed the Talend Youtube big data videos?
http://www.youtube.com/user/TalendChannel?feature=g-high-u

Ciaran