5 Replies Latest reply: Apr 23, 2014 12:17 PM by John Lifter RSS

    Cannot Compare a Persistent Value

    Paul Madden

      I have a project with 2 DataFlows ("DataFlow1" and "DataFlow2"). In "DataFlow1", I store a persistent value using the following:

       

      varStartDate = "2014-01-01 00:00:00"

      utility.store_string("varStartDate", varStartDate)

       

      I would like to use this value in a Filter in "DataFlow2". My Filter rule looks like the following:

       

      input.Date >= string.datetime(utility.retrieve_string("varStartDate"))

       

      I get the following error:

       

      Filter SourceData - RULE_DESCRIPTOR-1001-F: Rule 'Output 1' failed while executing the 'filter' function:

      Output 1:5: attempt to compare nil with number

       

      If I log the "varStartDate" in the Filter Operator, the correct value displays in the log window. So something like this works just as I would expect:

       

      function filter(input)

        log.information(utility.retrieve_string("varStartDate"))

        return input.Date >= string.datetime("2014-01-01 00:00:00")

      end;


      But then changing the Filter rule to this give me the same error:


      function filter(input)

         varStartDate = utility.retrieve_string("varStartDate")

         return input.Date >= string.datetime(varStartDate)

      end;


      I originally attempted this with a utility.store_datetime. And I've tried various combinations of datatypes and "workarounds" (which is why the code I pasted above is using store_string). Nothing is working.

       

      What am I missing?

        • Re: Cannot Compare a Persistent Value

          What you are trying to do works fine in a datascript command window, so this may be an issue with the Filter operator.

           

          Try this code.

           

          function filter(input)

            varStartDate = utility.retrieve_string("varStartDate")

            log.information(varStartDate)  -- should be a string date

            vDate = string.datetime(varStartDate)

            info = vDate or "conversion didn't work"

            log.information(info) -- should be a number not the message

            return true

          end

            • Re: Cannot Compare a Persistent Value
              Paul Madden

              Thanks John. That code does produce the correct results in the log window. But still, I am unable to Filter based on these stored persistent values.

               

              More things I've been trying:

               

              decision(input.DateFound >= string.datetime(utility.retrieve_string("varStartDate")),true,false)

               

              And I also tried making making my own filter using datascript in a Multi-Transform. But everything is giving me the same error.

                • Re: Cannot Compare a Persistent Value

                  So, you are confirming that vDate is a number.  Then the conversion was successful.  What you want to do is separate the code into the initialize and filter functions.  You should be able to do what you want to do.

                   

                  varStartDate = ""

                  vDate = nil

                   

                  function initialize()

                    varStartDate = utility.retrieve_string("varStartDate")

                    vDate = string.datetime(varStartDate)

                  end

                   

                  function filter(input)

                    return input.Date>=vDate

                  end

                    • Re: Cannot Compare a Persistent Value
                      Paul Madden

                      Looks like that'll do it. Thanks!  I didn't even think about initializing first.

                        • Re: Cannot Compare a Persistent Value

                          What you were trying to do originally looked like it would have worked but the repeated calls to retrieve the persistent value probably had some sort of an unknown effect.

                           

                          Anyway, for this type of comparison, the function rule is a better choice than the expression rule as it opens up the possibility of using the initialize function as well as any helper functions you might write.  For example, you might want to initialize a datascript table with some values and only process a record if some field value is found in the table.