Perhaps as a remnant of my days with a strongly-typed language (Java), I often find myself writing functions and then forcing type checks. For example:
def orSearch(d, query):
assert (type(d) == dict)
assert (type(query) == list)
Should I keep doing this? what are the advantages to doing/not doing this?
Stop doing that.
The point of using a "dynamic" language (that is strongly typed as to values*, untyped as to variables, and late bound) is that your functions can be properly polymorphic, in that they will cope with any object which supports the interface your function relies on ("duck typing").
Python defines a number of common protocols (e.g. iterable) which different types of object may implement without being related to each other. Protocols are not per se a language feature (unlike a java interface).
The practical upshot of this is that in general, as long as you understand the types in your language, and you comment appropriately (including with docstrings, so other people also understand the types in your programme), you can generally write less code, because you don't have to code around your type system. You won't end up writing the same code for different types, just with different type declarations (even if the classes are in disjoint hierarchies), and you won't have to figure out which casts are safe and which are not, if you want to try to write just the one piece of code.
There are other languages that theoretically offer the same thing: type inferred languages. The most popular are C++ (using templates) and Haskell. In theory (and probably in practice), you can end up writing even less code, because types are resolved statically, so you won't have to write exception handlers to deal with being passed the wrong type. I find that they still require you to programme to the type system, rather than to the actual types in your programme (their type systems are theorem provers, and to be tractable, they don't analyse your whole programme). If that sounds great to you, consider using one of those languages instead of python (or ruby, smalltalk, or any variant of lisp).
Instead of type testing, in python (or any similar dynamic language) you'll want to use exceptions to catch when an object does not support a particular method. In that case, either let it go up the stack, or catch it, and raise your exception about an improper type. This type of "better to ask forgiveness than permission" coding is idiomatic python, and greatly contributes to simpler code.
* In practice. Class changes are possible in Python and Smalltalk, but rare. It's also not the same as casting in a low level language.
Update: You can use mypy to statically check your python outside of production. Annotating your code so they can check that their code is consistent lets them do that if they want; or yolo it if they want.
In most of the cases it would interfere with duck typing and with inheritance.
Inheritance: You certainly intended to write something with the effect of
assert isinstance(d, dict)
to make sure that your code also works correctly with subclasses of dict. This is similar to the usage in Java, I think. But Python has something that Java has not, namely
Duck typing: most built-in functions do not require that an object belongs to a specific class, only that it has certain member functions that behave in the right way. The for loop, e.g., does only require that the loop variable is an iterable, which means that it has the member functions __iter__() and next(), and they behave correctly.
Therefore, if you do not want to close the door to the full power of Python, do not check for specific types in your production code. (It might be useful for debugging, nevertheless.)
If you insist on adding type checking to your code, you may want to look into annotations and how they might simplify what you have to write. One of the questions on StackOverflow introduced a small, obfuscated type-checker taking advantage of annotations. Here is an example based on your question:
>>> def statictypes(a):
def b(a, b, c):
if b in a and not isinstance(c, a[b]): raise TypeError('{} should be {}, not {}'.format(b, a[b], type(c)))
return c
return __import__('functools').wraps(a)(lambda *c: b(a.__annotations__, 'return', a(*[b(a.__annotations__, *d) for d in zip(a.__code__.co_varnames, c)])))
>>> @statictypes
def orSearch(d: dict, query: dict) -> type(None):
pass
>>> orSearch({}, {})
>>> orSearch([], {})
Traceback (most recent call last):
File "<pyshell#162>", line 1, in <module>
orSearch([], {})
File "<pyshell#155>", line 5, in <lambda>
return __import__('functools').wraps(a)(lambda *c: b(a.__annotations__, 'return', a(*[b(a.__annotations__, *d) for d in zip(a.__code__.co_varnames, c)])))
File "<pyshell#155>", line 5, in <listcomp>
return __import__('functools').wraps(a)(lambda *c: b(a.__annotations__, 'return', a(*[b(a.__annotations__, *d) for d in zip(a.__code__.co_varnames, c)])))
File "<pyshell#155>", line 3, in b
if b in a and not isinstance(c, a[b]): raise TypeError('{} should be {}, not {}'.format(b, a[b], type(c)))
TypeError: d should be <class 'dict'>, not <class 'list'>
>>> orSearch({}, [])
Traceback (most recent call last):
File "<pyshell#163>", line 1, in <module>
orSearch({}, [])
File "<pyshell#155>", line 5, in <lambda>
return __import__('functools').wraps(a)(lambda *c: b(a.__annotations__, 'return', a(*[b(a.__annotations__, *d) for d in zip(a.__code__.co_varnames, c)])))
File "<pyshell#155>", line 5, in <listcomp>
return __import__('functools').wraps(a)(lambda *c: b(a.__annotations__, 'return', a(*[b(a.__annotations__, *d) for d in zip(a.__code__.co_varnames, c)])))
File "<pyshell#155>", line 3, in b
if b in a and not isinstance(c, a[b]): raise TypeError('{} should be {}, not {}'.format(b, a[b], type(c)))
TypeError: query should be <class 'dict'>, not <class 'list'>
>>>
You might look at the type-checker and wonder, "What on earth is that doing?" I decided to find out for myself and turned it into readable code. The second draft eliminated the b function (you could call it verify). The third and final draft made a few improvements and is shown down below for your use:
import functools
def statictypes(func):
template = '{} should be {}, not {}'
@functools.wraps(func)
def wrapper(*args):
for name, arg in zip(func.__code__.co_varnames, args):
klass = func.__annotations__.get(name, object)
if not isinstance(arg, klass):
raise TypeError(template.format(name, klass, type(arg)))
result = func(*args)
klass = func.__annotations__.get('return', object)
if not isinstance(result, klass):
raise TypeError(template.format('return', klass, type(result)))
return result
return wrapper
Edit:
It has been over four years since this answer was written, and a lot has changed in Python since that time. As a result of those changes and personal growth in the language, it seems beneficial to revisit the type-checking code and rewrite it to take advantage of new features and improved coding technique. Therefore, the following revision is provided that makes a few marginal improvements to the statictypes (now renamed static_types) function decorator.
#! /usr/bin/env python3
import functools
import inspect
def static_types(wrapped):
def replace(obj, old, new):
return new if obj is old else obj
signature = inspect.signature(wrapped)
parameter_values = signature.parameters.values()
parameter_names = tuple(parameter.name for parameter in parameter_values)
parameter_types = tuple(
replace(parameter.annotation, parameter.empty, object)
for parameter in parameter_values
)
return_type = replace(signature.return_annotation, signature.empty, object)
@functools.wraps(wrapped)
def wrapper(*arguments):
for argument, parameter_type, parameter_name in zip(
arguments, parameter_types, parameter_names
):
if not isinstance(argument, parameter_type):
raise TypeError(f'{parameter_name} should be of type '
f'{parameter_type.__name__}, not '
f'{type(argument).__name__}')
result = wrapped(*arguments)
if not isinstance(result, return_type):
raise TypeError(f'return should be of type '
f'{return_type.__name__}, not '
f'{type(result).__name__}')
return result
return wrapper
This is a non-idiomatic way of doing things. Typically in Python you would use try/except tests.
def orSearch(d, query):
try:
d.get(something)
except TypeError:
print("oops")
try:
foo = query[:2]
except TypeError:
print("durn")
Personally I have an aversion to asserts it seems that the programmer could see trouble coming but couldn't be bothered to think about how to handle them, the other problem is that your example will assert if either parameter is a class derived from the ones you are expecting even though such classes should work! - in your example above I would go for something like:
def orSearch(d, query):
""" Description of what your function does INCLUDING parameter types and descriptions """
result = None
if not isinstance(d, dict) or not isinstance(query, list):
print "An Error Message"
return result
...
Note type only matches if the type is exactly as expected, isinstance works for derived classes as well. e.g.:
>>> class dd(dict):
... def __init__(self):
... pass
...
>>> d1 = dict()
>>> d2 = dd()
>>> type(d1)
<type 'dict'>
>>> type(d2)
<class '__main__.dd'>
>>> type (d1) == dict
True
>>> type (d2) == dict
False
>>> isinstance(d1, dict)
True
>>> isinstance(d2, dict)
True
>>>
You could consider throwing a custom exception rather than an assert. You could even generalise even more by checking that the parameters have the methods that you need.
BTW It may be finicky of me but I always try to avoid assert in C/C++ on the grounds that if it stays in the code then someone in a few years time will make a change that ought to be caught by it, not test it well enough in debug for that to happen, (or even not test it at all), compile as deliverable, release mode, - which removes all asserts i.e. all the error checking that was done that way and now we have unreliable code and a major headache to find the problems.
I agree with Steve's approach when you need to do type checking. I don't often find the need to do type checking in Python, but there is at least one situation where I do. That is where not checking the type could return an incorrect answer that will cause an error later in computation. These kinds of errors can be difficult to track down, and I've experienced them a number of times in Python. Like you, I learned Java first, and didn't have to deal with them often.
Let's say you had a simple function that expects an array and returns the first element.
def func(arr): return arr[0]
if you call it with an array, you will get the first element of the array.
>>> func([1,2,3])
1
You will also get a response if you call it with a string or an object of any class that implements the getitem magic method.
>>> func("123")
'1'
This would give you a response, but in this case it's of the wrong type. This can happen with objects that have the same method signature. You may not discover the error until much later in computation. If you do experience this in your own code, it usually means that there was an error in prior computation, but having the check there would catch it earlier. However, if you're writing a python package for others, it's probably something you should consider regardless.
You should not incur a large performance penalty for the check, but it will make your code more difficult to read, which is a big thing in the Python world.
Two things.
First, if you're willing to spend ~$200, you can get a pretty good python IDE. I use PyCharm and have been really impressed. (It's by the same people who make ReSharper for C#.) It will analyze your code as you write it, and look for places where variables are of the wrong type (among a pile of other things).
Second:
Before I used PyCharm, I ran in to the same problem--namely, I'd forget about the specific signatures of functions I wrote. I may have found this somewhere, but maybe I wrote it (I can't remember now). But anyway it's a decorator that you can use around your function definitions that does the type checking for you.
Call it like this
@require_type('paramA', str)
@require_type('paramB', list)
@require_type('paramC', collections.Counter)
def my_func(paramA, paramB, paramC):
paramB.append(paramC[paramA].most_common())
return paramB
Anyway, here's the code of the decorator.
def require_type(my_arg, *valid_types):
'''
A simple decorator that performs type checking.
@param my_arg: string indicating argument name
@param valid_types: *list of valid types
'''
def make_wrapper(func):
if hasattr(func, 'wrapped_args'):
wrapped = getattr(func, 'wrapped_args')
else:
body = func.func_code
wrapped = list(body.co_varnames[:body.co_argcount])
try:
idx = wrapped.index(my_arg)
except ValueError:
raise(NameError, my_arg)
def wrapper(*args, **kwargs):
def fail():
all_types = ', '.join(str(typ) for typ in valid_types)
raise(TypeError, '\'%s\' was type %s, expected to be in following list: %s' % (my_arg, all_types, type(arg)))
if len(args) > idx:
arg = args[idx]
if not isinstance(arg, valid_types):
fail()
else:
if my_arg in kwargs:
arg = kwargs[my_arg]
if not isinstance(arg, valid_types):
fail()
return func(*args, **kwargs)
wrapper.wrapped_args = wrapped
return wrapper
return make_wrapper
© 2022 - 2024 — McMap. All rights reserved.
assertstatements are stripped by default, in python they aren't! This means that you are performing a lot of superfluous function calls every time. In most cases this wont be a bottleneck, but when, to execute a simple function, you have to do 5-6 function calls to check the types then it might become a bottleneck. – Intelligence