AttributeError: module 'copy' has no attribute 'deepcopy'

Question

I'm Actually New to Python and BS4. And I Decided to create a script that will scrape a website, oscarmini.com to be precise, the code was running fine untill today when I wanted to modify it, I keep getting errors, in the little knowledge I have about Exceptions and Error, there's nothing wrong with the code it seems to be from the importation of 'bs4' module..

from bs4 import BeautifulSoup as BS
import requests
url = 'https://oscarmini.com/2018/05/techfest-2018.html'
page = requests.get(url)
soup = BS(page.text, 'lxml')
mydivs = soup.find("div", {"class": "entry-content"})
soup.find('div', id="dpsp-content-top").decompose()
print(mydivs.get_text())
input()

Below is the error message I get.

Traceback (most recent call last):
    File "C:/Users/USERNaME/Desktop/My Programs/Random/Oscarmini- 
Scrapper.py", line 1, in <module>
    from bs4 import BeautifulSoup as BS
File "C:\Users\USERNaME\AppData\Local\Programs\Python\Python36-32\lib\site-packages\bs4\__init__.py", line 35, in <module>
    import xml.etree.cElementTree as default_etree
File ":\Users\USERNaME\AppData\Local\Programs\Python\Python36-32\lib\xml\etree\cElementTree.py", line 3, in <module>
    from xml.etree.ElementTree import *
File "C:\Users\USERNaME\AppData\Local\Programs\Python\Python36-32\lib\xml\etree\ElementTree.py", line 1654, in <module>
    from _elementtree import *
AttributeError: module 'copy' has no attribute 'deepcopy'

Process finished with exit code 1

Please I really need help on this..

Patagium

Do you by chance have a file "copy.py" in your current directory? – Potion

Where am I actually looking for the "copy.py" file? Coz i can't find it in the xml or xml/etree folders – Patagium

In the folder where you have your python code, – Potion

Nope.. No such file here.. It's been working without any copy file. – Patagium

If it was working before and not working now, that means something changed. Did you update/upgrade something? Did you try reinstalling bs? – Carriole

To the best of my knowledge, i didn't update/upgrade anything.. and yes, I have unistall/upgrade and install bs4 yet same result – Patagium

Answer 1

I encountered the same problem. And I finally found that the problem is I have another script named copy.py and it shadows the original copy module.

You can show the real path for the copy module with print(copy.__file__) just before the exception occurs and see whether it is intended.

You can also list your PATHONPATH environment variable with: print(os.environ['PYTHONPATH'].split(os.pathsep)) just before the line that causes the exception, and see whether there are something unexpected.

Speleology

I had the same problem because of copy.py file. I deleted the file but still getting the error. Do you have any insight about its reason? Thanks! – Aranda

Answer 2

Make sure any copy.py file does not exists in your project working directory... like project Folder: copy.py currentOpenFile.py # when you import copy module...

Impose

Answer 3

My issue was a variation of @liyuanhe211 having a local copy.py but I had a local org.py and so this is what happened to me, when running pyspark in my case,

pyspark along the way doing import copy

but the built-in copy.py does from org.python.core import PyStringMap

So since I had a org.py with import pandas as pd , looks like along the way that also calls org.deepcopy, so it just became this circular dependency 😅.

But yea without my org.py, there, the problem goes away.

$ pyspark
Picked up JAVA_TOOL_OPTIONS: -Djavax.net.ssl.trustStoreType=KeychainStore
Python 3.10.11 (main, Apr  7 2023, 07:33:46) [Clang 14.0.0 (clang-1400.0.29.202)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
Traceback (most recent call last):
  File "/Users/hello/Downloads/2023-06-01-spark/spark-3.4.0-bin-hadoop3/python/pyspark/shell.py", line 31, in <module>
    import pyspark
  File "/Users/hello/Downloads/2023-06-01-spark/spark-3.4.0-bin-hadoop3/python/pyspark/__init__.py", line 59, in <module>
    from pyspark.rdd import RDD, RDDBarrier
  File "/Users/hello/Downloads/2023-06-01-spark/spark-3.4.0-bin-hadoop3/python/pyspark/rdd.py", line 18, in <module>
    import copy
  File "/usr/local/Cellar/[email protected]/3.10.11/Frameworks/Python.framework/Versions/3.10/lib/python3.10/copy.py", line 60, in <module>
    from org.python.core import PyStringMap
  File "/Users/hello/Repos/blah/src/org.py", line 2, in <module>
    import pandas as pd
  File "/Users/hello/.python_venvs/pandars310/lib/python3.10/site-packages/pandas/__init__.py", line 50, in <module>
    from pandas.core.api import (
  File "/Users/hello/.python_venvs/pandars310/lib/python3.10/site-packages/pandas/core/api.py", line 29, in <module>
    from pandas.core.arrays import Categorical
  File "/Users/hello/.python_venvs/pandars310/lib/python3.10/site-packages/pandas/core/arrays/__init__.py", line 20, in <module>
    from pandas.core.arrays.string_arrow import ArrowStringArray
  File "/Users/hello/.python_venvs/pandars310/lib/python3.10/site-packages/pandas/core/arrays/string_arrow.py", line 65, in <module>
    import pyarrow.compute as pc
  File "/Users/hello/.python_venvs/pandars310/lib/python3.10/site-packages/pyarrow/compute.py", line 331, in <module>
    _make_global_functions()
  File "/Users/hello/.python_venvs/pandars310/lib/python3.10/site-packages/pyarrow/compute.py", line 328, in _make_global_functions
    g[cpp_name] = g[name] = _wrap_function(name, func)
  File "/Users/hello/.python_venvs/pandars310/lib/python3.10/site-packages/pyarrow/compute.py", line 299, in _wrap_function
    return _decorate_compute_function(wrapper, name, func, options_class)
  File "/Users/hello/.python_venvs/pandars310/lib/python3.10/site-packages/pyarrow/compute.py", line 165, in _decorate_compute_function
    options_class_doc = _scrape_options_class_doc(options_class)
  File "/Users/hello/.python_venvs/pandars310/lib/python3.10/site-packages/pyarrow/compute.py", line 112, in _scrape_options_class_doc
    doc = docscrape.NumpyDocString(options_class.__doc__)
  File "/Users/hello/.python_venvs/pandars310/lib/python3.10/site-packages/pyarrow/vendored/docscrape.py", line 151, in __init__
    self._parsed_data = copy.deepcopy(self.sections)
AttributeError: partially initialized module 'copy' has no attribute 'deepcopy' (most likely due to a circular import)

Gladiolus