Achieving Unlimited Scale Using Talend ESB & Auto-scaling on AWS

[Anonymous]

Hi,

I have read this interesting article from Talend on using Talend ESB & Auto-scaling on AWS.

I want to utilize this architecture design for our application setup in AWS.

Here is a brief overview of our application, currently in the development phase.

• Using EC2 AWS cloud server running Ubuntu 16.04 LTS distribution.
• Everything is currently hosted on this single EC2 AWS instance:
  • Nginx (reverse proxy, web server)
  • Tomcat (application server)
  • Elastic search engine 6.x
• Front-End () application is built using Angular6 and hosted on Nginx (web server)
• Back-End service are hosted on the Tomcat 8 server
• The communication between FE and BE is via https (restful services)
• The elastic search engine is indexed through another java utility
• The communication between BE and Elasticsearch is via https (restful services)

Now, I have these following questions with regards to leveraging Unlimited Scale Using Talend ESB & Auto-scaling on AWS for our application:

• How does "Talend ESB Runtime" fit in our application landscape? If I read in the article, it says "Talend ESB runtime provides an Apache Karaf-based ESB container pre-configured to support Talend Mediation route (Apache Camel routing) and Talend Web Services Jobs (Apache CXF-based services - both REST and SOAP-based)." I am not sure, what role does this play in our situation?
• Do we need to adapt our application in order to integrate and utilize this architecture design and if yes, in what way?
• How is data synchronization supported? 
• On this article page, it says "Free Trial", I do not have any prior experience using Talend enterprise products, so my question is what are the pre-requisites to try out this infrastructure setup? At this stage, I don't want to setup a full blown industrial scale infrastructure, but want to conduct a PoC.
• Any other important point useful at this stage?

Thanks. 

[Jesperrekuh]

ESB is dead. my humble opinion

I'm not a huge fan of ESB, I love the conceptual (outdated) thoughts but I tend to lean more to the seperation of concerns design.

Could you elaborate a little on the purpose and what you are trying to achieve with this type of architecture?
Technically it would fit... so does a Mainframe... we need to have a little background so help us out.

- Whats your biggest/major 'business' challenge? 
- How is "separation of concerns" utilized now? are we talking enterprise architecture? or domain?
- Is it primary focused on statistics or data-logistics?

- Are multi co-location data-centers involved, business in Japan, Europe and South America? Any geo specific requirements?

- Data (base) storage? What type of cold-storage or active storage or (nosql) databases where (functional) data is stored ... CRUD?

- Are you considering a real time data-pipeline?

- Beside synchronization is serialization involved in your current application landscape?

Thought of :

- Microservices ? more restful api's ...  
- Docker, Kubernets ? 

- Apache Kafka, Apache AVRO ?

Honestly a hard question to answer.