5 Replies Latest reply: Sep 3, 2013 5:40 PM by John Lifter RSS

    Expressor Dataflow Start & End Times

    Jamie Hicks

      I am currently working with partitioning to improve throughput times. I am loading a fact table that has approx 50 million rows. Our test server has 24 cores, so I am testing different ways of utilizing the cores with partitioning. I have copied the Results text and pasted it into a text file for review. I quickly found a start Date/Time, but I cannot find an end date/time. Is there an end time stamp in the results? If not, is there a way to get both a start & end time?

       

      Thanks

        • Re: Expressor Dataflow Start & End Times

          By Results text, you mean the log information displayed in the Results tab, correct?

           

          You will probably need to write out the start/stop times from code within your dataflow.  For example, there is a runtime parameter dataflow.start that will give you the time at which processing started.  So, in the initialize function of some operator, you could write this value to the log (the content displayed in the Results tab).  Use the log.information function:

          function initialize()

          log.information(string.concatenate("Start time: ", datetime.string(dataflow.start))

          end

           

          Note that each partition will print out this value, so you will see it repeated multiple times in the log.

           

          To get the end time, you need to print out the current time.

          function finalize()

          log.information(string.concatenate("End time: ",datetime.string(datetime.timestamp())

          end

           

          Again, each partition will print this out, so you may want to include the record.partition runtime parameter in the logging entry.  Perhaps

          log.information(string.concatenate("Partition: ",record.partition,"\tEnd time: ",datetime.string(datetime.timestamp())