topic Re: How to redirect standard output to a file in Talend Studio

How to redirect standard output to a file

_AnonymousUser — Sat, 16 Nov 2024 12:55:39 GMT

Hi,
I need to know if there is a way to redirect the standard output from a Talend job to a file. I know it can be done by modifying the scripts using the >> operator, but I need to know if there is a way to do it via Talend.
We would need this to redirect the output of a tLogRow, for example, to a file. At the moment we are doing this by using a customised version of tLogRow, which adds a file as a parameter, and appends the tLogRow output to this file, but surely there is a better way of doing this ?
If it cannot be done, is there a way to dynamically adjust the ".bat" and ".sh" files generated by talend, to add the ">>" operator automatically each time the job is exported ?
Thank you,
Louis

Re: How to redirect standard output to a file

Anonymous — Thu, 12 May 2011 10:30:22 GMT

Hi Louis,
You can simply use a tFileOutputDelimited, that you put after your tLogRow, it will put in a file the same data that goes in the tLogRow.
Just create a row from your tLogRow to the tFileOutputDelimited.

Re: How to redirect standard output to a file

_AnonymousUser — Thu, 12 May 2011 10:37:24 GMT

Hi Louis,
You can simply use a tFileOutputDelimited, that you put after your tLogRow, it will put in a file the same data that goes in the tLogRow.
Just create a row from your tLogRow to the tFileOutputDelimited.

Thank you vgalopin, however this solution doesn't work for me for several reasons :
I'd like to be able to reproduce the behaviour of the tLogRow component (the pretty table), without having to recode it each time I want to log data.
I'd also like to log the full stack trace of a java exception ie the full

Exception in component tFileInputProperties_1
java.io.FileNotFoundException: P:\export 2Talend\assistance_equipe_0.1\assistance_equipe_0.1\assistance_equipe\louisdevdivers\assistance_equipe_0_1\contexts\Default.properties (Le chemin d'accès spécifié est introuvable)
	at java.io.FileInputStream.open(Native Method)
	at java.io.FileInputStream.<init>(Unknown Source)
	at louisdevdivers.assistance_equipe_0_1.assistance_equipe.tFileInputProperties_1Process(assistance_equipe.java:443)
	at louisdevdivers.assistance_equipe_0_1.assistance_equipe.runJobInTOS(assistance_equipe.java:1977)
	at louisdevdivers.assistance_equipe_0_1.assistance_equipe.main(assistance_equipe.java:1848)

Instead of the "message only" output I get from using a tLogCatcher.
Louis

Re: How to redirect standard output to a file

Anonymous — Thu, 12 May 2011 10:56:11 GMT

Louis,
If you want to reproduce the behaviour of the tLogRow, you can try to take the java code that it produces, paste it in a tJavaRow, and modify it to have a single column out of your tJavaRow that will contain the data parsed the same way than the tLogRow did.
To see the code produced by the tLogRow, click on the component, then click on the Code Viewer tab on the bottom left. You will be able to see the start, main and end parts of the component.
Here is what I got in the start part (it creates the table name and columns):

 class Util_tLogRow_1 {
        String[] des_top = { ".", ".", "-", "+" };
        String[] des_head = { "|=", "=|", "-", "+" };
        String[] des_bottom = { "'", "'", "-", "+" };
        String name="";
        java.util.List<String[]> list = new java.util.ArrayList<String[]>();
        int[] colLengths = new int;
        public void addRow(String[] row) {
            for (int i = 0; i < 0; i++) {
                if (row!=null) {
                  colLengths = Math.max(colLengths, row.length());
                }
            }
            list.add(row);
        }
        public void setTableName(String name) {
            this.name = name;
        }
            public StringBuilder format() {
            
                StringBuilder sb = new StringBuilder();
                
                return sb;
            }
        }
        Util_tLogRow_1 util_tLogRow_1 = new Util_tLogRow_1();
        util_tLogRow_1.setTableName("tLogRow_1");
        util_tLogRow_1.addRow(new String[]{});        
		int nb_line_tLogRow_1 = 0;

Re: How to redirect standard output to a file

Anonymous — Thu, 12 May 2011 10:59:11 GMT

Avoid if possible stuff like ">>"... these are shell specifics concept and absolutely not portable...
I see that under the hood tlogrow uses plain System.out object.
So you could, at the beginning of you job, inside a tjava, redirect globally the standard output... insert this code (add buffering if you need):
-------------------------
java.io.File file = new java.io.File("/temp/mylogfile_test.txt");
java.io.PrintStream ps = new java.io.PrintStream(new java.io.FileOutputStream(file));
System.setOut(ps);
-------------------------
this will redirect at the job level to the file specified.
hope it helps

Re: How to redirect standard output to a file

_AnonymousUser — Thu, 12 May 2011 11:10:30 GMT

Avoid if possible stuff like ">>"... these are shell specifics concept and absolutely not portable...
I see that under the hood tlogrow uses plain System.out object.
So you could, at the beginning of you job, inside a tjava, redirect globally the standard output... insert this code (add buffering if you need):
-------------------------
java.io.File file = new java.io.File("/temp/mylogfile_test.txt");
java.io.PrintStream ps = new java.io.PrintStream(new java.io.FileOutputStream(file));
System.setOut(ps);
-------------------------
this will redirect at the job level to the file specified.
hope it helps

This is extremely helpfull, and exactly what I was looking for, thank you !
vgalopin, I understand the concept of reproducing the code behind the tLogRow, (I created a custom component that does just that), I was looking for a more permanent solution, and redirecting the output via a tJava seems like it will do the trick.
Thank you.

Re: How to redirect standard output to a file

Anonymous — Thu, 12 May 2011 18:31:49 GMT

emaxt6,
That is exactly what I was looking for as well!!! Thanks a lot for the idea! so simple and elegant.
Peter.

Re: How to redirect standard output to a file

rbaldwin — Thu, 12 May 2011 19:21:44 GMT

Don't forget that if you want to capture exception stack traces, you'll also have to do the same with System.err with System.setErr(...);

Re: How to redirect standard output to a file

Anonymous — Wed, 28 Aug 2013 07:05:52 GMT

Interesting. Is it also possible to capture ALL output (Standard AND Error) into a string variable and email it at the end of the Talend Job?

Re: How to redirect standard output to a file

alevy — Thu, 29 Aug 2013 14:51:26 GMT

There's a tRedirectOutput component on Exchange (see http://community.talend.com:80/t5/Deployment/resolved-save-output-in-run-window-to-txt-file/m-p/8414#M219) that will capture the output to a text file. You could then use that as an attachment for tSendMail.