I know this issue has been already discussed before, Yet I feel my question is a bit different.
I'm trying to figure out how am I to enable access to the Kibana over the self-managed AWS elastic search which I have in my AWS account.
Could be that what am I about to say here is inaccurate or complete nonsense. I am pretty novice in the whole AWS VPC wise section and to ELK stuck.
Architecture: Here is the "Architecture":
I have a VPC.
Within the VPC I have several subnets.
Each server sends it's data to the elastic search using log stash which runs on the server itself. For simplicity, let's assume I have a single server.
The elastic search https URL which can be found in the Amazon console is resolved to an internal IP within the subnet that I have defined.
Resources:
I have found the following link which suggests using one of two option:
https://aws.amazon.com/blogs/security/how-to-control-access-to-your-amazon-elasticsearch-service-domain/
Solutions:
Option 1: resource-based policy
Either to allow resource-based policy for elastic search by introducing condition which specifies a certain IP address.
This was discussed in the following thread but unfortunately did not work for me.
The proper access policy for Amazon Elastic Search Cluster
When I try to implement it in the Amazon console, Amazon notifies me that because I'm using Security group, I should resolve it by using security group.
Security group rules:
I tried to set a rule which allows my personal computer(Router) public IP to access Amazon elastic search ports or even opening all ports to my public IP.
But that didn't work out. I would be happy to get a more detailed explanation to why but I'm guessing that's because the elastic search has only internal IP and not public IP and because it is encapsulated within the VPC I am unable to access it from outside even if I define a rule for a public IP to access it.
Option 2: Using proxy
I decline to use this solution unless I have no other choice.
I'm guessing that if I set another server with public and internal IP within the same subnet and VPC as that of the elastic search, and use it as a proxy, I would then be able to access this server from the outside by defining the same rules to them it's newly created security group. Like the article suggested.
Sources:
I found out of the box solution that someone already made for this issue using a proxy server in the following link:
Using either executable or docker container.
https://github.com/abutaha/aws-es-proxy
Option 3: Other
Can you suggest another solution? Is it possible to use Amazon Load balancer or Amazon API gateway to accomplish this task?
I just need proof of concept, not something which goes into the production environment.
Bottom line:
I need to be able to access Kibana from the browser in order to be able to search elastic search indexes.
Back