top gotchas for someone moving from a static lang (java/c#) to dynamic language like python

Question

What are the top gotchas for someone moving from a static lang (java/c#) to dynamic language like python?

It seems cool how things can be done, but renaming a method, or adding/removing parameters seems so risky!

Is the only solution to write tests for each method?

Ashbaugh

Answer 1

I would say that the number one gotcha is trying to write statically typed code in a dynamic language.

Dont hesitate to use an identifier to point to a string and then a list in self-contained sections of code

keys = 'foo bar foobar' # Imagine this coming in as an argument
keys = keys.split() # Now the semantically chose name for the argument can be 
                    # reused As the semantically chosen name for a local variable

don't hesitate to treat functions like regular values: they are. Take the following parser. Suppose that we want to treat all header tags alike and ul tags like ol tags.

class Parser(HTMLParser):
    def __init__(self, html):
        self.feed(html)

    def handle_starttag(self, tag, attrs):
        parse_method = 'parse_' + tag    
        if hasattr(self, parse_method):  
            getattr(self, parse_method)(attrs)


    def parse_list(self, attrs):
        # generic code

    def parse_header(self, attrs):
       # more generic code

    parse_h1 = parse_h2 = parse_h3 = parse_h4 = parse_h5 = parse_h6 = parse_header
    parse_ol = parse_ul = parse_list

This could be done by using less generic code in the handle_starttag method in a language like java by keeping track of which tags map to the same method but then if you decide that you want to handle div tags, you have to add that into the dispatching logic. Here you just add the method parse_div and you are good to go.

Don't typecheck! Duck-type!

def funtion(arg):
    if hasattr(arg, 'attr1') and hasattr(arg, 'attr2'):
         foo(arg):
    else:
         raise TypeError("arg must have 'attr1' and 'attr2'")

as opposed to isinstance(arg, Foo). This lets you pass in any object with attr1 and attr2. This allows you to for instance pass in a tracing class wrapped around an object for debugging purposes. You would have to modify the class to do that in Java AFAIK.

As pointed out by THC4k, another (more pythonic) way to do this is the EAPF idiom. I don't like this because I like to catch errors as early as possible. It is more efficient if you expect for the code to rarely fail though. Don't tell anybody I don't like it though our they'll stop thinking that I know how to write python. Here's an example courtesy of THC4k.

try: 
    foo(arg): 
except (AttributeError, TypeError): 
    raise InvalidArgumentError(foo, arg)

It's a tossup as to if we should be catching the AttributeError and TypeError or just let them propagate to somewhere that knows how to handle them but this is just an example so we'll let it fly.

Pubis

@THC4k. How do you say that it's stranger? Unless type(Foo) is some subclass of type that overides the __instancecheck__ attribute, isinstance depends upon location in class hierarchies. The way that I presented doesn't share this shortcoming. LBYL vs EAFP is another issue. I forgot to include the other style. – Pubis

if hasattr(arg, 'attr1') and hasattr(arg, 'attr2') is a typecheck. It checks if arg belongs to the (implicit) typeclass of "things that have a attr1 and attr2 attribute". That is more general that using isinstance, which only checks for a single type, not a whole typeclass. But those checks are often unnecessary: Just do foo(arg) right away, see what happens and catch Exceptions where it makes sense. Let Python do all the typechecks for you -- if foo(arg) makes no sense, it will throw a exception eventually. – Squinteyed

@Pubis sorry, yeah bad wording, i was rewriting the comment. – Squinteyed

@aaronasterling: I'd rather say: The try: .. except TypeError: .. belongs somewhere in foo, around the very line that uses arg in a way that depends on some properties of arg. The except there should either fix the issue with arg or raise a custom Exception (ie YouMessedUpException). – Squinteyed

@THC4k. yeah, good call. my heart wasn't in it. I'll update again. – Pubis

Or maybe as a example: try: foo(arg): except (AttributeError, TypeError): raise InvalidArgumentError(foo, arg) – Squinteyed

Answer 2

"Is the only solution to write tests for each method?"

Are you saying you didn't write tests for each method in Java?

If you wrote tests for each method in Java, then -- well -- nothing changes, does it?

renaming a method, seems so risky!

Correct. Don't do it.

adding/removing parameters seems so risky!

What? Are you talking about optional parameters? If so, then having multiple overloaded names in Java seems risky and confusing. Having optional parameters seems simpler.

---

If you search on SO for the most common Python questions, you'll find that some things are chronic questions.

• How to update the PYTHONPATH.

• Why some random floating-point calculation isn't the same as a mathematical abstraction might indicate.

• Using Python 3 and typing code from a Python 2 tutorial.

• Why Python doesn't have super-complex protected, private and public declarations.

• Why Python doesn't have an enum type.

The #1 chronic problem seems to be using mutable objects as default values for a function. Simply avoid this.

Melonymelos

I for one rarely needed to mess with PYTHONPATH. And no 3 is a language-agnostic issue with floats, every (no matter whether Java or C# or ... or even Scheme) programmer worth his salt will know it. Otherwise, agreed. – Baculiform

@delnan: You may not mess with it, but there are a surprising number of SO questions on that subject. Also, the float issue should be well-known. Yet... Amazingly... SO is full of "2.1 + 3.2 != 5.3" questions. – Melonymelos

Okay, so certain people don't get import search paths and floating point numbers - but is this because they come from a static language? Most likely not - floating points work the same in every language, and at least java seems to have something similar to PYTHONPATH, CLASSPATH (correct me if I'm wrong, I'm not a Java guy). – Baculiform

@delnan: No clue. I'm just providing some feedback based on reading a LOT of Python questions. I don't know why folks have these questions. But they do. – Melonymelos

Answer 3

Some things that struck me when first trying out Python (coming from a mainly Java background):

1. Write Pythonic code. Use idioms recommended for Python, rather than doing it the old Java/C way. This is more than just a cosmetic or dogmatic issue. Pythonic code is actually hugely faster in practice than C-like code practically all the time. As a matter of fact, IMHO a lot of the "Python is slow" notion floating around is due to the fact that inexperienced coders tried to code Java/C in Python and ended up taking a big performance hit and got the idea that Python is horribly slow. Use list comprehensions and map/filter/reduce whenever possible.

2. Get comfortable with the idea that functions are truly objects. Pass them around as callbacks, make functions return functions, learn about closures etc.

3. There are a lot of cool and almost magical stuff you can do in Python like renaming methods as you mention. These things are great to show off Python's features, but really not necessary if you don't need them. Indeed, as S. Lott pointed out, its better to avoid things that seem risky.

Farrow

Python is slower. Orders of magnitude slower. But it doesn't matter. Partly because idiomatic code is not that slow, partly because we don't do raytracers in Python. Also, the cool stuff that makes the language so slow also totally makes up for the performance loss. – Baculiform

@delnan. I've seen itertools beat pure C code by 20 seconds or so on an eighty minute task ;) – Pubis

@aaronasterling: Isn't itertools a C library? (I could be wrong) – Graphite

@Daenyth. It is a C library AKAIK but that's part of the point: idiomatic python pushes stuff off to C using comprehensions and modules like itertools as much as possible. – Pubis

@delnan: I agree, even the most optimized Python is in general slower compared to C. But it isn't really as horribly slow as some people make it out to be. I've seen Python being much faster than Java in very computationally intensive tasks. The point is, writing Java/C in Python leads to much slower code. – Farrow

@MAK: +1. And writing any dumb-old-bad-algorithm in Python will be slower than writing the same dumb-old-bad-algorithm in C. Both are bad, but C allows you to mask the badness with compiler optimizations. – Melonymelos