Python Botocore remove unused files from data folder

Question

I have a Python app running on an OpenWrt router (Chaos Calmer) that uses the boto3 package for creating and updating AWS IoT thing shadows. The app is packaged with its own Python and dependencies (including boto3 and botocore). boto3 depends on botocore, which has a data folder with a bunch of json files for other AWS services unused by my app (S3, dynamoDB, route53, lambda, etc. etc.). Out of all the app's resources, this folder takes up the most space by far. Eventually I will need to do OTA updates over cellular to many of these routers, meaning data usage is a concern. Therefore I would like to trim out as many unused json files from the data folder as possible without breaking the code.

These are the only three boto3 and botocore imports my app uses:

from boto3 import client as boto3_client
from boto3.exceptions import ResourceNotExistsError
from botocore.exceptions import ClientError

How can I determine which json files I can get rid of?

Torsibility

Note: if you want to do controlled incremental updates, use casync to reuse unchanged parts. – Sphenogram

@Sphenogram thanks for the tip – Torsibility

Thanks. I only needed S3. So I removed all data folders, left the python modules in place and added S3 back in. Worked perfectly. Botocore went from 60mb to 1.5mb. Thats a win when you are trying to squeeze your code into a lambda. – Lizzielizzy

@Torsibility you should add an answer so I can upvote – Sweetandsour

Answer 1

Original update (1-8-19), reposted as an answer (1-13-2023): I figured out how to solve this problem. Instead of removing a handful of files that didn't look relevant, then running the app to make sure it wasn't broken, then removing more files etc. etc., I did the opposite: removed all the files, then slowly added files that looked relevant back until only the necessary files remained. In the end the data folder went from ~35 Mb to < 2 Mb 🙂 (that's a huge deal for my use case)

Torsibility