get list of tables in database using boto3

Question

I’m trying to get a list of the tables from a database in my aws data catalog. I’m trying to use boto3. I’m running the code below on aws, in a sagemaker notebook. It runs forever (like over 30 minutes) and doesn’t return any results. The test_db only has 4 tables in it. My goal is to run similar code as part of an aws glue etl job, that I would run in an edited aws etl job script. Does anyone see what the issue might be or suggest how to do this?

code:

import boto3
from pprint import pprint

glue = boto3.client('glue', region_name='us-east-2')

response = glue.get_tables(
    DatabaseName=‘test_db’
)

print(pprint(response['TableList']))

Heisser

are the API calls hitting AWS Service? You can verify this by looking at the CloudTrails event history for GetTables API calls. Also, can you verify whether the SageMaker notebook has internet access? – Visional

Answer 1

db = session.resource('dynamodb', region_name="us-east-2")
tables = list(db.tables.all())
print(tables)

resource https://boto3.amazonaws.com/v1/documentation/api/latest/guide/dynamodb.html

Etti

This answer doesn't make any sense, the question is about aws-glue which is likely athena tables, not dynamodb. – Aaberg

Answer 2

You can use boto3 with the list_table_metadata method of the Athena client as long as you know the name of your data catalog https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/athena/client/list_table_metadata.html

import boto3

client = boto3.client("athena", region_name="us-east-2")
table_metadata = client.list_table_metadata(CatalogName=`CatalogName`, DatabaseName=`test_db`)
print(table_metadata)

This will return a dictionary of metadata about your tables in the database. If you wanted to just get the name of the tables you could then use some list comprehension:

[table["Name"] for table in table_metadata['TableMetadataList']]

Hyams

Do we have to know which Catalog and Name of Database, right? – Kingsley

Correct, but you can access the list of catalogs with client.list_data_catalogs() and can access the list of databases with client.list_databases() so you don't need to have that information immediately on hand. – Hyams