Should we use pandas.compat.StringIO or Python 2/3 StringIO?

Question

StringIO is the file-like string buffer object we use when reading pandas dataframe from text, e.g. "How to create a Pandas DataFrame from a string?"

Which of these two imports should we use for StringIO (within pandas)? This is a long-running question that has never been resolved over four years.

1. StringIO.StringIO (Python 2) / io.StringIO (Python 3)
   • Advantages: more stable for futureproofing code, but forces us to version-fork, e.g. see code at bottom from EmilH.
2. pandas.compat.StringIO
   • pandas.compat is a 2/3 compatibility package ("without the need for 2to3") introduced back in 0.13.0 (Jan 2014)
   • pandas.compat package is still marked 'private' as of 0.22 and no plans to make 'public' says "Warning The pandas.core, pandas.compat, and pandas.util top-level modules are considered to be PRIVATE. Stability of functionality in those modules in not guaranteed." although they essentially haven't broken since 0.13
   • pandas.compat source defines the imports builtins, StringIO/cStringIO, BytesIO, cPickle, httplib, iterator versions of range, filter, map and zip, plus other necessary elements for Python 3 compatibility - see the 0.13.0 whatsnew

Version 2/3 forking code for imports from standard (from EmilH):

import sys
if sys.version_info[0] < 3: 
    from StringIO import StringIO
else:
    from io import StringIO

# Note: but this is very much a poor-man's version of pandas.compat, which contains much much more

Note:

• pandas.compat has existed since pandas 0.13.0 (Jan 2014) as a subpackage within pandas
• it also seems to have been released as a standalone package: 0.1.0 (Jun 10, 2017) and 0.1.1 (Jun 10, 2017)

Jameson

cc: @Jeff jreback ... – Jameson

The lack of a standard causes both confusion and breakage. – Jameson

I think this is primarily opinion based. All the approaches work so use the one that you feel most comfortable with. When I answered the referenced question I used the snippet to show what to use in both Python 3 and Python 2. Today, 4 years later I'm only using Python 3 so it's a non-issue for me. Stackoverflow is probably not the place to push for a standard on this. If it's important raise an issue on the pandas issue tracker perhaps? – Iphlgenia

@EmilH: it's not opinion-based, it depends on whether the pandas developers plan to change their guidance on pandas.compat. We don't even need everything inside pandas.compat to be stable, only the identifiers I named, but in any case it has been stable since late 2015, so their warning is overly severe – Jameson

@Jameson I agree that today their warning is overly severe (at least for StringIO). But, no matter the pandas developers opinion on: Which of these two imports should we use for StringIO (within pandas)? the answer is still based on opinion. If the question was: Is there an officially recommended way to use StringIO (within pandas)? That would not be opinion based, but reading the docs the recommendation would still currently be to not use the pandas.compat (despite that being an arguably cleaner way of getting hold of StringIO). – Iphlgenia

@EmilH: it's not based on opinion, it's based on facts. Specifically, the current and future status of pandas.compat, per the pandas devel crew. (Not the doc's old 'official recommendation' and the doc on that which are clearly 3+ years out-of-date). Please look at the github links for code and issues, which I cited you. Where the current code disagrees with 4-year-old doc, ignore the doc. This wouldn't be the first time that a package's doc lagged the reality of its code, or github, by years. – Jameson

Answer 1

I know this is an old question, but I followed breadcrumbs here, so perhaps still worth answering. It's not totally definitive, but current Pandas documentation suggests using the built in StringIO rather than it's own internal methods.

For examples that use the StringIO class, make sure you import it with from io import StringIO for Python 3.

Regulation

Yes that's the answer these days. (I had meant to self-answer and close this years ago) – Jameson

Answer 2

FYI, as of pandas 0.25, StringIO was removed from pandas.compat (PR #25954), so you'll now see:

from pandas.compat import StringIO

ImportError: cannot import name 'StringIO' from 'pandas.compat'

This means the only answer is to import from the io module.

Hautbois

Thanks for the update MikeT – Jameson