ImportError: Missing optional dependency 'S3Fs'. The S3Fs package is required to handle S3 files. Use pip or conda to install S3Fs

Question

I am using AWS Cloud9 as my IDE.

import boto3
import pandas as pd
# import s3fs
# s3_ob=boto3.resource('s3',aws_access_key_id="xxxxxxxxxx",aws_secret_access_key="xxxxxxxxxxxx")
client=boto3.client('s3')
path="s3://xxxxxx/FL_insurance_sample.csv"

df=pd.read_csv(path)
# df.head()
print(df)`

While I am able to get the output CSV file in Pycharm, when I use the same code in Cloud9 IDE on AWS I get the error mentioned in title.

I have installed the S3Fs using the pip install S3Fs and when I do "pip list" it does give me the list installed which contains S3Fs so I am confused, why am I getting this error when the module is already installed and it shows in the pip list, and I also tried uncommenting the import S3Fs and still the same error is there.

Please help me.

Ywis

Welcome Adithya. Have you tried reinstalling with pip3 install s3fs --user. I assume you are using latest version of Python 3, so you should be using pip3 instead. Furthermore, check that you have activated the conda environment correctly by checking conda env list will show you the list of environments you have and the one with * is the currently active one – Asthenopia

it shows pip 3 command not found in the bash,tried both pip3 install s3fs --user and conda env list @Asthenopia – Ywis

Answer 1

Installed the s3fs package using pip. It was showing up in pip list. But it did not show up when I did: conda list.

I resolved this problem by doing the following steps:

1. Installed the s3fs dependecy using conda install -c conda-forge s3fs
2. Restarted kernel
3. Imported s3fsand pandas
4. Read the csv file which was in my s3 bucket.

PS: It was library issue.

Dyane

I'm having the exact same issue except s3fs does show up in both pip list and conda list. Any ideas? Thanks – Heredity

try this: create a new environment altogether and then switch to it. see if this issue is fixed. most of the times the problem is with the environment. – Dyane

Thanks, actually a new env didn't do the trick, but I did need to run "conda install pip" in the env itself first, then use that env's version of pip to install s3fs. – Heredity

@SaurabhJain, I folowed exactely the solution by installing s3fs on my laptop, but i have the same erro : [ERROR] ImportError: Missing optional dependency 'fsspec'. Use pip or conda to install fsspec. – Pissed

Answer 2

pip3 install s3fs 

solved the issue for me.

Hialeah

Answer 3

You just have to install the package. Open you Command Prompt and type:
pip install fsspec

Mandamandaean

Answer 4

For me, the problem was missing aiobotocore package.

pip install aiobotocore

Wedgwood