How to display the subpackages of a package in python

Question

I have some third party library called a and from code examples I learned it has a subpackage b1, i.e.,

from a import b1

Is it possible to see all the subpackages of a? Package a is not pure python and it is not obvious from the file structure to tell the subpackages. I tried dir but it only shows attributes of a

import a
dir(a)

Catcall

Is this an open source library? If so, can you share a link? – Amorete

unfortunately it's a commercial one – Catcall

Answer 1

If the package author provided an explict index of the package modules, then the convention is to define a list named __all__ that contains this index. So you could do something like the following to see all the submodules / subpackages of an imported package (example prints all json submodules as determined by the package author):

import json

subs = json.__all__
print(subs)

# OUTPUT
# ['dump', 'dumps', 'load', 'loads', 'JSONDecoder', 'JSONDecodeError', 'JSONEncoder']

If the package author did not provide an index of the package modules, then it will be much more difficult to sort it out. One approach would be to use dir and then sort or filter the attributes by type in an effort to narrow down to a set likely to be submodules / subpackages. Here is an example that might be useful.

import json

def sort_by_type(t):
    return t[0].__name__

attrs = [(type(getattr(json, attr)), attr) for attr in dir(json)]
attrs.sort(key=sort_by_type)
for t, n in attrs:
    print(t, n)

# OUTPUT
# <class 'json.decoder.JSONDecoder'> _default_decoder
# <class 'json.encoder.JSONEncoder'> _default_encoder
# <class '_frozen_importlib.ModuleSpec'> __spec__
# <class '_frozen_importlib_external.SourceFileLoader'> __loader__
# <class 'dict'> __builtins__
# <class 'function'> detect_encoding
# <class 'function'> dump
# <class 'function'> dumps
# <class 'function'> load
# <class 'function'> loads
# <class 'list'> __all__
# <class 'list'> __path__
# <class 'module'> codecs
# <class 'module'> decoder
# <class 'module'> encoder
# <class 'module'> scanner
# <class 'str'> __author__
# <class 'str'> __cached__
# <class 'str'> __doc__
# <class 'str'> __file__
# <class 'str'> __name__
# <class 'str'> __package__
# <class 'str'> __version__
# <class 'type'> JSONDecodeError
# <class 'type'> JSONDecoder
# <class 'type'> JSONEncoder

Boeschen

unfortunately that package doesn't have __all__ defined – Catcall

@Catcall - in that case, it will be fairly tricky. Edited the answer with an approach that may be helpful to get you started with the investigation. – Boeschen

Answer 2

Use can use __all__ if available to list all the possible items that the python package contains.

Try the following code:

for i in sklearn.__all__:   
   try:
      y = eval('sklearn.{}'.format(i))
      for j in y.__all__:
      if 'package you want to find' in j.lower():
         print(i,' - ',j)   
   except:
     pass

Stopwatch

I get those errors only but no output not even w/o the if clause: exceptions - NotFittedError exceptions - ConvergenceWarning exceptions - DataConversionWarning exceptions - DataDimensionalityWarning exceptions - EfficiencyWarning exceptions - FitFailedWarning exceptions - SkipTestWarning exceptions - UndefinedMetricWarning exceptions - PositiveSpectrumWarning – Japheth