Talend Log Server - elasticsearch connection issue

[radhikari]

I have a question on configuring the log server. I am getting a "Connection Failed - your elasticsearch server is down or unreachable".

I updated elasticsearch.yml and set the following:

http.cors.enabled: true

http.cors.allow-origin: "http://localhost:8080"

I used localhost since both Kibana and Elasticsearch are running on the same machine. Tac and log server are on the same box.

I restarted the log server, but I am still getting the same message.

When I try curl -sS -XGET localhost:9200, I get the following. So, it looks like Elasticsearch is running. Any ideas?

{

"status" : 200,

"name" : "Power Skrull",

"cluster_name" : "talend-log-central",

"version" : {

"number" : "1.5.2",

"build_hash" : "62ff9868b4c8a0c45860bebb259e21980778ab1c",

"build_timestamp" : "2015-04-27T09:21:06Z",

"build_snapshot" : false,

"lucene_version" : "4.10.4"

},

"tagline" : "You Know, for Search"

}

[Anonymous]

Hi,

Have you already checked your elasticsearch server in TAC to see if it is running well(green)?

Best regards

Sabrina

[radhikari]

In TAC - under Monitoring, I see that Kibana is running (has a green check mark), but I don't see a place to configure Elasticsearch.

[radhikari]

any update on this? we really need this resolved. Thank you.

[Anonymous]

Hello radhikari,

I would suggest you to replace localhost with the external facing IP.

The problem that you have is when you ping from TAC Kibana, Kibana is trying to get from your computer 'localhost' which is (unless you have installed TAC on your machine) not accessible.

Another source of issue is when your TAC is under SSL and EL is through HTTP. Tomcat will refuse the connection.
Last, could you indicate what is your version of Talend? As Talend 6.2 come with ELK version 1.x while Talend 6.3 implement ELK v5.x!

[radhikari]

We have log server and TAC running on the same host without SSL (for now).

We are using Talend 6.2.1.

I have tried using an external IP as well as "/.*/", but that didn't help.

http.cors.enabled: true

http.cors.allow-origin: "/.*/"

#http.cors.allow-origin: "http://<external_ip>:8080/"

[Anonymous]

Have you modified the logstash-talend.conf
output {
elasticsearch {
embedded => false
host => "yourhost" 
port => 9200
protocol => http
....

For the elasticsearch.yml, I will stick to http.cors.allow-origin: "http://<external_ip>:8080/"

Have you tried to access Kibana directly http://localhost:8080/kibana?
I had the issue as well in the past where the allow-origin need to be type the exact same way you will type it on your browser.
Eg. your ext ip is 1.2.3.4 but you called with a DNS like mysupertac.mycompany.com tomcat will struggle to process and it will be denied

Please tell us your progress

[radhikari]

Yes, I have modified the logstash-talend.conf to include the proper hostname.

I am using http.cors.allow-origin: "http://my_ip:8080/" in elasticsearch.yml.

I can access Kibana directly from http://my_ip:8080/kibana, but I get the following message.

Connection Failed Possibility #1: Your elasticsearch server is down or unreachable

This can be caused by a network outage, or a failure of the Elasticsearch process. If you have recently run a query that required a terms facet to be executed it is possible the process has run out of memory and stopped. Be sure to check your Elasticsearch logs for any sign of memory pressure.

Possibility #2: You are running Elasticsearch 1.4 or higher

Elasticsearch 1.4 ships with a security setting that prevents Kibana from connecting. You will need to set the following in your elasticsearch.yml:

1. http.cors.enabled: true
2. http.cors.allow-origin to the correct protocol, hostname, and port (if not 80) that your access Kibana from. Note that if you are running Kibana in a sub-url, you should exclude the sub-url path and only include the protocol, hostname and port. For example, http://mycompany.com:8080, not http://mycompany.com:8080/kibana.

[Anonymous]

Just to be sure, everything is distributed in HTTP not HTTPS?

[radhikari]

correct, everything is distributed in HTTP. We were able to finally resolve this issue. We have Talend running in AWS and port 9200 isn't open in our network. So, we had to switch to a port that was open.