Best approach for managing/developing larger REST projects

[xkr]

Hi there,

is there any best pracitice on how to design a bit more complex API endpoints?
I'm now at a point where I have 5 URIs with 5 complete different workflows and more to come, which is not that much honestly...

When developing a new flow I always develop them in a new and separate job, but when the job is working I copy all the components into a "main" job where the tRESTRequest resides and have to fix a ton of references...

Beside that, my "main" job now has more than 200 components and its super laggy and slow as hell and confusing as you loose overview over time.

I tried to make joblets out of my subjobs, but when running them, and there's a bug or exception or whatever, I can't see whats going on in the joblets... running in debug view is also a pain, e.g. everytime i start a run, all my breakpoints gets deleted from the previous run, which is super annoying, and i have to scroll through 100.000s LOC to find the component which may throw the error... not to mention it's super slow...

I tried to run my SubJobs with a tRunJob, but thats also not possible, as i cannot implement tRESTResponses due to the fact that the request component is in a parent job...

So yeah... do you have any tips how to design bigger projects? 

[Dave_Simo]

Hello @xkr 

You can look at the documentations below to help you in implementing the best practices to follow for API development in Talend, in any case it taught me a lot to set up robust and more or less complex Rest APIs in Talend.

https://www.linkedin.com/pulse/apis-best-practice-use-talend-power-more-vivek-varanasi?trk=portfolio...

https://community.qlik.com/t5/Official-Support-Articles/Talend-API-Designer-Best-Practices/ta-p/2151...

https://astreait.com/Salesforce-to-SAP-Integration-Using-Talend/

Best regards

[xkr]

Hi @Dave_Simo 

thanks for the links, but those are kind of more general information on how to design REST services in the first place.

My actual problem is:

You can only have one tRESTRequest in a job.
You cannot have tRESTResponses without a tRESTRequest in a Job.

The more functions a REST service has, the more it "messes up" the whole job. 
This is just half of my current main job in that project and it gets worse and slower and more unmanageable with every new function I implement. I cant even display it whole

So I'm looking for a solution to either have the flows/functions of my API in separate jobs, which would i prefer, so i can develop and debug them until they work, and then just drag&drop them into my main job, link it, add some conditions, and that's it, but I don't know if it's somehow possible to redirect the request to the spearate jobs, or split them up into joblets, but as mentioned in the first post, I don't see whats going on in the joblets when running the main job and troubleshooting them and rename everything after refactoring them into a joblet is pain in the ***

this is what I would imagine, but in this image there's only one flow from the tRESTRequest, and i don't know if they have any kind of tRESTResponses in the subjobs implemented... If so, i'd like to know how they did it 🙂

[Dave_Simo]

Hello @xkr 

"My actual problem is:

You can only have one tRESTRequest in a job."

It's normal that we can only have one tRESTRequest in a Rest API in Talend.

"You cannot have tRESTResponses without a tRESTRequest in a Job."

To avoid having to use a tRESTResponse , you need to configure your API in the tRESTRequest as below by setting it to oneway.

I would advise you to split your job (web services) into several web services, and each of the split web services will be called in the main web service via a tRestClient.

And to see what's happening in the joblets or tRunJobs, it would be better to create an error logging joblet to place in all your child processes and route the logs to a file or database, so you can more easily monitor what happened.

Best Regards