How to Connect/Query to AWS Neptune instance using HTTP POST/GET request

Question

I am trying to connect to Amazon Neptune instance by sending a HTTP Post request using Neptune END Point via Fiddler. But ending up in timeout error. Can Neptune instance be connected to via HTTP request using fiddler/Postman?

Crafty

Answer 1

If you are seeing timeouts while connecting to the database, the first step would be to check if you have network connectivity to the endpoint.

Try: telnet endpoint port

If you have connectivity, you would see something like this:

Trying 172.217.5.110...
Connected to endpoint (172.217.5.110).
Escape character is '^]'

If this does work, then any HTTP client should be able to connect to your database. (CURL, POSTMAN etc)

If telnet does not work, then it is almost certain that you have not configured your EC2 Security Groups correctly. The gist of what you need to do is:

1. Create a security Group (say 'ec2') and attach that to your EC2 client instance. By default, this security group should allow outbound connections to all IPs. If that is not the case, add it.

2. Create a security Group (say 'db'). In Inbound rules, add a rule that allows inbound TCP connections to your database port, and source as the security group created in #1.

3. Now modify your Neptune Cluster, and attach 'db' to it.

4. Security Group changes propagate pretty fast, so you should be able to test this using telnet.

You may find other answers that say that you need the database and the EC2 instance to be in the same security group. That is not entirely true, it is just a special case of the steps mentioned above where instead of creating 2 security groups, you can use a single security group for both - db and the client instance. From a security and design perspective, its best if you have separate security groups for your DB and your client instances.

Once you have all the network settings correctly configured, confirm that telnet works. After that you can make HTTP requests using a client of your choice. For example:

> curl http://my-endpoint:port/status

healthy

Hope this helps.

Camphene

Answer 2

Timeout Error most commonly comes from the fact you aren't on the same VPC as your Neptune instance.
Please check this by doing an hping for example:
sudo hping3 -S -p 8182 [YOUR NEPTUNE ENDPOINT]
If hping can't ping it then you aren't on the same VPC.
So, just create a EC2 instance (ON THE SAME VPC AS YOUR NEPTUNE INSTANCE) and use it as VPN, I personally use this one setup-ipsec-vpn.
If your EC2 is using Ubuntu then just do:
wget https://git.io/vpnsetup -O vpnsetup.sh && sudo sh vpnsetup.sh
Then setup the credentials you get as a result to your computer, launch the VPN connection and retry the hping part to test the connection.

Floristic

Merely having an EC2 instance in the same VPC is not the right solution. You need to make sure that the security group associated with the DB cluster actually allows connects from your client. In most cases, the default security group allows connections from any instance in the VPC, but that's just a special case. See https://mcmap.net/q/1622697/-how-to-connect-query-to-aws-neptune-instance-using-http-post-get-request for exact details. – Camphene

Answer 3

Yes, AWS Neptune end point can be connected using postman and I am able to successfully load the TinkerPop modern graph from S3 to neptune db instance using following command.

'Content-Type: application/json' \
    http://your-neptune-endpoint:8182/loader -d '
    { 
      "source" : "s3://neptune-us-east-1/tinkerpopmodern/", 
      "format" : "csv", 
      "accessKey" : "access-key-id", 
      "secretKey" : "secret-key", 
      "region" : "us-east-1", 
      "failOnError" : "FALSE",
      "mode" : "NEW"
    }'

Adp

Passing access key and secret key as part of Bulk load is deprecated. You should first attach an IAM Role using AddRoleToCluster API and then mention the role name as an argument to bulk load. docs.aws.amazon.com/neptune/latest/userguide/… – Camphene