How to query AWS CloudSearch domain using Python boto3 library?

Question

I'm trying to use boto3 to query my CloudSearch domain using the docs as a guide: http://boto3.readthedocs.io/en/latest/reference/services/cloudsearchdomain.html#client

import boto3
import json

boto3.setup_default_session(profile_name='myprofile')
cloudsearch = boto3.client('cloudsearchdomain')

response = cloudsearch.search(
    query="(and name:'foobar')",
    queryParser='structured',
    returnFields='address',
    size=10
)
print( json.dumps(response) )

...but it fails with:

botocore.exceptions.EndpointConnectionError: Could not connect to the endpoint URL: "https://cloudsearchdomain.eu-west-1.amazonaws.com/2013-01-01/search"

But how am I supposed to set or configure the endpoint or domain that I want to connect to? I tried adding an endpoint parameter to the request, thinking maybe it was an accidental omission from the docs, but I got this error response:

Unknown parameter in input: "endpoint", must be one of: cursor, expr, facet, filterQuery, highlight, partial, query, queryOptions, queryParser, return, size, sort, start, stats

The docs say:

The endpoint for submitting Search requests is domain-specific. You submit search requests to a domain's search endpoint. To get the search endpoint for your domain, use the Amazon CloudSearch configuration service DescribeDomains action. A domain's endpoints are also displayed on the domain dashboard in the Amazon CloudSearch console.

I know what my search endpoint is, but how do I supply it?

Shelled

Answer 1

I found a post on a Google forum with the answer. You have to add the endpoint_url parameter into the client constructor e.g.

client = boto3.client('cloudsearchdomain', endpoint_url='http://...')

I hope those docs get updated, because I wasted a lot of time before I figured that out.

Shelled

On top of @Shelled 's answer, you can fetch the endpoint_url dynamically like this: cs_client = boto3.client('cloudsearch') info = cs_client.describe_domains(DomainNames=[domain_name]) endpoint_url = info['DomainStatusList'][0]['SearchService'] cs_domain_client = boto3.client('cloudsearchdomain', endpoint_url=f"https://{endpoint_url}") – Bierce

Answer 2

import boto3
client = boto3.client('cloudsearchdomain',
    aws_access_key_id= 'access-key',
    aws_secret_access_key= 'some-secret-key',
    region_name = 'us-east-1',  # your chosen region
    endpoint_url= 'cloudsearch-url'
    # endpoint_url is your Search Endpoint as defined in AWS console 
)

response = client.search(
    query='Foo', # your search string
    size = 10
)

Reference response['hits'] for returned results.

Holub