topic Data Integration with Web Serivces in Data Quality

Data Integration with Web Serivces

Anonymous — Mon, 01 Apr 2019 13:33:43 GMT

Dear All,

I have to integrate with different systems (IBM Maximo, MS Dynamics, SharePoint etc) using their REStful web services (JSON).

Each system has multiple web services with frequency of 1 minute. Total web services are 35. Two systems are exposing data on ESB and those are IBM Maximo and MS Dynamics. The other systems exposing data using JSON. The data would be fetched using web services and insert into SQL Server. The environment is based on on-premises.

Question:

1. Which talend product will fetch the web service data?

2. Free or Paid? I am interested in Free. Does it solve the problem?

3. The integration should be via ESB supported by free-ware?

4. There should be data synchronizing mechanism. If the server is stop them it need to sync it then server resume.

5. There should be queuing mechanism in place in case of multiple responses received?

Thanks,

Yasir Miraj

Re: Data Integration with Web Serivces

Anonymous — Mon, 01 Apr 2019 14:35:25 GMT

Hi @Yasir_Miraj,

I will reply to each question...

Question:

1. Which talend product will fetch the web service data? - This looks like ESB would be the best fit

2. Free or Paid? I am interested in Free. Does it solve the problem? - The free version will probably be enough. However, there is a steep learning curve for ESB development

3. The integration should be via ESB supported by free-ware? - I'm not sure what you are asking here

4. There should be data synchronizing mechanism. If the server is stop them it need to sync it then server resume. - This can be done with Talend ESB. How depends on the specific problem and the developer to be honest. You will need to a raise a specific question about this with more details.

5. There should be queuing mechanism in place in case of multiple responses received?- Again, this is a bit high level and you haven't given enough information. Talend work with many queues. ActiveMQ is an easy one that is supported.

Re: Data Integration with Web Serivces

Anonymous — Tue, 02 Apr 2019 11:32:42 GMT

Thanks rhall,

I am attaching the data flow diagram of the project. See attached. The "Integration Server" in diagram would be Talend/Other Software. The data would be exposed on IBM Maximo and MS Dynamics by using ESB. Our Integration server would be the consumer of the web service.

Secondly , i can send data back to IBM Maximo using Integration Server (Talend). This should be pushed into the external system datababse.

Does "Talend Open Studio" work for this?

Thanks,

Yasir

integration-server-diagram.png

Re: Data Integration with Web Serivces

Anonymous — Thu, 04 Apr 2019 12:44:54 GMT

From a high level, I cannot see why the Talend Open source ESB edition would not be able to meet this requirement. However, since it is going to be the backbone of your solution, you might want to consider the Enterprise Edition so that you can receive the benefits of....

Support
SVN/Git integration
Multiple developer benefits
Advanced components
Metadata management

There are other benefits, but if you are considering going down this route it is probably best to arrange to speak to our sales team.

But to answer your question, the Open Source edition will allow you to meet your requirement. It might be a little more fiddly and not as supported than if you use the Enterprise Edition

Re: Data Integration with Web Serivces

Anonymous — Sat, 06 Apr 2019 19:39:51 GMT

Many thanks rhall. I hope i got my answer.