How to set trigger as FAILED?

[Anonymous]

Hi there,
Is there any means to set the return of a calljob or callprocess on a Trigger as an error so that the Event appears on MDM Event Manager as FAILED?
I'm getting further into my problem scenario so that you can better understand what I'm wanting to do.
Example:
After a recorde creation, which fires an UpdateReport, a trigger is called upon some conditions. The trigger calls a job (or a process that will call a job, it's the same) that will sync some data to other application. The Job tries to create some data on this 3rd party application related to the data inserted on the MDM Hub, via webservices. What if the host on which the webservice resides is down? Or someone changed the password for that webservice and the webservice fails? My job already has the capability to detect that it fail its main purpose, but there should be a way that the Job returns an error so that the Event Manager will store this Routing as FAILED. This way, after the host has came up (or the password changed to the proper value) it can be run again and the data correctly syncs on the 3rd party app. How can this be done?
Hope I've made myself clear.
Best regards,
Diogo

[Anonymous]

Yes, this is a typical "closed loop" scenario.
You could:
- Define a status element in the entity, and have the job update the status to "sync complete" or "sync failed" using a tMDMOutput. You can then setup a Browse_items_<ENTITY>#SyncFailed view so you can see those who failed easily. This view would have a where clause status = syncfailed. Also make sure you set a specific description, cause that's what is displayed in the drop down list in the web. You would finaly have a periodic job that pulls out records through a tMDMViewSearch on top of this view to try to push again the master records.
- Define a customized event by defining an entity akin to UpdateReport, then use tMDMRouteRecord at the end of your job to fire "SyncFailed" event. Then you can catch this event using regular triggers (they don't have to catch UpdateReport only, they can catch whatever document you "route" to the event manager).
Hope that makes sense.

[Anonymous]

Now, the ability to signal MDM that the trigger should fail for functional reasons is not a bad idea either. The event manager keeps all failed "routing order" (the document that results from matching an event to a trigger) in the failed queue, so you can retry them later. Dble click the event mgr in the studio, browse the failed queue, right click to retry a routing order.
There are no tMDM component to automate that by the way...
I am starting to find this idea interesting...

[Anonymous]

Well, these are some good ideas, but I really thought that there was a way to set the UpdateReport on Event Manager as failed.
Setting something on the the entity can be a problem, though... Imagine the following scenario:
Instant t:
- App1 creates MDM record, Job1 triggered by Event fails as App2 is down. Job1 sets syncfailed as true on the Entity;
Instant t + 1:
- App3 updates MDM record, Job2 triggered by Event suceeeds on updating App1. Job2 sets syncfailed as false on the Entity based on the fact that it succefully performed its purporse.
Can you see where I'm heading? The fact that the UpdateReport is stored as FAILED is a big deal, because a Job can be run again later. In the way you described your solution, there is no way a job in a 'Instant t + 2' would know that that Entity needs to be sync'ed.

[Anonymous]

Now, the ability to signal MDM that the trigger should fail for functional reasons is not a bad idea either. The event manager keeps all failed "routing order" (the document that results from matching an event to a trigger) in the failed queue, so you can retry them later. Dble click the event mgr in the studio, browse the failed queue, right click to retry a routing order.
There are no tMDM component to automate that by the way...
I am starting to find this idea interesting...

Exactly! Now you're seeing what I mean with all this talk...
I'm taking a look on eXist and MDM webservices to see if there is a way to set the Event as FAILED. As the Event is just another entry on the eXist DB and there should be a reference of the UpdateReport identifier on the UpdateReport fired.

[Anonymous]

I'm taking a look on eXist and MDM webservices to see if there is a way to set the Event as FAILED. As the Event is just another entry on the eXist DB and there should be a reference of the UpdateReport identifier on the UpdateReport fired.

Well, unfortunately there isn't any reference to the Document of the EventManager on the XML the Job receives, so I'm not able to set it as FAILED during Job execution.

<EXCHANGE>
  <REPORT>
    <UPDATE>
      <USERNAME>admin</USERNAME>
      <SOURCE>WS-App1</SOURCE>
      <TIMEINMILLIS>1291981145258</TIMEINMILLIS>
      <OPERATIONTYPE>CREATE</OPERATIONTYPE>
      <REVISIONID>null</REVISIONID>
      <DATACLUSTER>development</DATACLUSTER>
      <DATAMODEL>SGI</DATAMODEL>
      <CONCEPT>Account</CONCEPT>
      <KEY>22</KEY>
    </UPDATE>
  </REPORT>
  <ITEM>
      <ACCOUNT>
        <ID>22</ID>
        <NAME>Teste 20101210114150</NAME>      
      </ACCOUNT>
  </ITEM>
</EXCHANGE>

[Anonymous]

When a trigger has its conditions fulfilled, a routing order doc is created to keep track of that. This doc is inserted into the active routing order queue (amaltoOBJECTSActiveRoutingOrderV2 collection in eXist). If the trigger service succeeds, the routing order is moved to the completed routing order queue. If it failed, to the failed routing order queue. The routing order's status attribute changes too. I guess you could move a routing order from one eXist collection to another and change the status attribute manually. But this is at the XML persistence level, I don't think there is a web service call to do that. And you will then have cache consistency issues at the application layer.
So this must be implemented by us.
Maybe instead of a status element you could define a "log" entity in your model that keeps track of the failure. Same idea as before but you would create an actual log entry (instance of log) that describes which record failed to be pushed to the app.

[Anonymous]

Hi ctoum,
Thanks for your insight on my thoughts, but I really think that creating a "log" Entity is not the path to solve this kind of "closed loop" as you called it. I think that opting for that way beats the purpose of MDM for Data Integration.
Why would I have a separate Entity to manage Sync log if MDM has an Event Manager with status as COMPLETE, FAILED and ACTIVE? For example, when the RouteOrder fails because it can't call a process, it is set as FAILED. As far as I'm concerned, firing a Job that returns an Error is exactly the same behavior. What's the point on setting a RouteRecord as completed if the Job it called failed its purpose? My original thought on this was based on this document: http://talendforge.org/wiki/doku.php?id=mdm:calljobschemas
Output

<RESULTS>
	<ITEM>
		<ATTR>0=ok or 1=failed</ATTR>
	</ITEM>
</RESULTS>

By reading this, my tought was: well, the RouteOrder will fail if I use the tBufferOutput component to output 1. I did that, but still the RouteOrder stated as COMPLETE, even though I clearly saw on the JBoss log that the output was in accordance to what was stated in the WiKi. Well, I'm not gonna focus on this, but I think it is a feature that could be implemented. But again this is me thinking and as I'm not in the Talend team, I don't know your roadmap or design guidelines, so I may be completely wrong. But it sure does a lot of sense to me, tough.
Well, with all this writing, I had an idea: what about tJava component throwing an Exception? That may do the trick... I'm trying it!

[Anonymous]

Any news about this issue ctoum ?

[Anonymous]

Hi,
Thanks very much for this comment. It help me to think about my ideals.
Tks again and pls keep posting.