Real Life Benefits of Dynamic Languages?

Question

I'm exploring several possibilities for developing a new system (web application).

I'm an "old fashioned" kinda guy, object oriented in nature (converted from procedural many years ago). I played around with Python and studied a bit Ruby, but frankly I'm attracted back to using Microsoft's tools (C#, ASP.NET MVC). All this run-time typing, no compiler errors on basic stuff, etc just makes my life harder when it comes to building large complex applications.

I constantly hear people speak about the great things you can do with dynamic languages, but except for examples with dogs, cats and how quickly you can code a cool way to count things, the "industrial strength" of Visual Studio just seems to eliminate those neat small things dynamic languages offer, especially now that you have free express versions of VS and full versions available for free for start-ups.

I feel like I'm missing something here, because big applications are indeed being developed with dynamic languages, so what are those great things these languages enable you to do, when looking at large complex applications? What can make you give away the strength of VS?

Rebozo

It warms my heart that someone who's object-oriented in nature can call themselves "old-fashioned" :) I feel old now, Rax – Rienzi

@Jeremy - as I said, I converted from procedural... and feeling old is a psychological thing, I'm 67 and I don't feel old! (just kidding) – Rebozo

the actually odd thing about that formulation is that dynamic languages have been around far longer than OOP, very nearly as long as procedural – Yves

Answer 1

"how quickly you can code" so totally concerns me that I happily gave up the long, slow slog through getting things to compile.

Advantages of dynamic languages.

1. No compile, no build. Just Code and Test followed by deploy to production.

2. Immediate gratification. No time spent wringing my hands over what an API call might be. Just type it interactively in the Python >>> prompt and see what it actually does.

3. Very, very short design cycles. Rather than carefully crafting an class hierarchy with bonus interface definitions and proper abstract declarations and overrides, I can just code the classes, unit test them and be done.

4. Less code. Dynamic language introspection reduces the volume of source. I don't write this stuff in my applications; I depend on frameworks to do this for me. But the framework-based code is often very short; there are no duplicative declarations that are so common in Java, where you have to repeat things in an XML config.

5. No mysteries. As we say in the Python community: "Use the source, Luke." There's no ambiguity in what a framework does or what an API really means.

6. Absolute Flexibility. As our requirements change, we don't have to struggle with devastating changes that break the entire architecture. We can -- trivially -- make changes to a few classes because Python's Duck Typing eliminates the need to retrofit missing interface definitions where we didn't think they'd be needed. They're just aren't any; the code we didn't write is code we don't have to fix or maintain.

7. Resilience. When our actuaries have a brain-fart, we don't have to spend months figuring out how to integrate this new, more sophisticated underwriting model into the apps. Almost anything can be squeezed in. Again, this is strictly a consequence of duck typing. We're freed from force-fitting it into an architecture that couldn't anticipate a new business model.

8. Since the source is the application, the source can be it's own configuration file. We don't have XML or INI configuration files in some foreign syntax. We have configuration files in Python. The Django framework does this and we follow their lead. We have very complex mocked-up data declarations for sales demo and unit testing. The super-complex data is actually a collection of Python objects that would have come from a database -- except -- we omitted loading the database. It's simpler to just tweak the Python object constructor instead of loading a SQL database.

[BTW. After 30+ years of developing software in Cobol, Fortran, PL/I, Java, C, C++, I'm just tired of the relatively low-level hand-optimization that most compiled languages require. Years ago, I read a comment on the inefficiency of most compilers: it leads us to create elaborate build systems to work around the compiler limitations. We only need make because cc is so slow.]

---

Edit

Dynamic programming doesn't make you a genius. It just saves a lot of time. You still have to manage the learning process. Things you don't know are hard in all languages. A dynamic language gives you leverage by allowing you to proceed incrementally, uncovering one new thing at a time without having done a lot of design work only to find your assumptions were wrong.

If you want to write a lot of code based on a misunderstood API, then a dynamic language can help. You are free to write a lot of code that crashes and burns in a language: C#, VB, C++, Java or Python. You can always write code which won't work.

The compiler gives you some advance warning that the code won't work. Typically, not compiling is a big hint. However, you can still write a lot of code that compiles and fails all the unit tests. The compiler only checks syntax, not semantics.

Python can give you some advance warning that the code won't work. Typically, you can't get it to run interactively. However, you can still write a lot of code that fails all the unit tests.

Isoline

absolute freedom, frees absolutely....but it sounds a bit like a recipe for a train wreck. – Henhouse

@Kenny: Interestingly, I never mentioned "absolute freedom". We're still constrained by (a) the language (b) the architecture, (c) sensible design, and (d) having to pass extensive unit tests. Not sure where you got the "absolute freedom" quote from. I certainly didn't say it and tried not to imply it. – Isoline

"No mysteries. As we say in the Python community: Use the source, Luke." - this is another way of saying that life's too short to separate interface from implementation. Great when you're writing code, not viable if you're shipping to a third party who needs your code to be encapsulated behind some well-defined abstraction. You can just look up the source in other languages too, and conclude (for instance) that filenames can safely contain the ':' character. And crash. And burn. – Embassy

Python inherently separates interface from implementation always. That's what duck typing is. – Isoline

By "crash and burn" of course I mean, "at the next release, possibly a critical bugfix, all your tests fail and you have to re-write code that you'd have got right first time with minimal effort, if only you hadn't been told just to read the source to find out what a given function does". This has happened to me more than once. An API should be unambiguous to start with. If it is harmfully ambiguous, that ambiguity must be identified and clarified, not left as "whatever the current version does, but we'll change it in future without notice". – Embassy

"All you tests fail"? Interesting problem. Never had it, must be just lucky, I guess. Or perhaps -- by coding incrementally -- I don't commit to something until I've written one test and a few lines of code. If I waited longer, perhaps I'd have a lot of failing tests, but I try not to write code I'm not 100% confident in. – Isoline

"Python inherently separates interface from implementation always." I don't see how those 7 words of sophistry help me if I'm sent 2000 lines of code, 15 lines of API documentation, a promise of Version 1.1 in six weeks, and told "the interface is well separated from the implementation". Of course it isn't. Dynamic languages and duck typing do not assist at all in defining forward- and backward-compatible interfaces. I literally never care what the interface is to the current version of of the code - I only ever care what the common interface is between this and future versions. – Embassy

"All you tests fail? Interesting problem. Never had it, must be just lucky, I guess". Or you've never taken a new version from upstream, which changes implementation details which some smug so-and-so told you to rely on because "interface is implicit". Sure, but that is a very long way from being the whole story. – Embassy

For example, using Google's API documentation for GAE, I have not once so far had to look at the source in order to find out what the API documentation "really means". And if I ever did that, then I would expect my app to fall over with no notice at some unspecified point in the future when Google changes the undocumented behaviour. That would not be separation of interface from implementation. What Google actually does, is separation of interface from implementation, and it has nothing to do with the fact that they happen to be using Python. – Embassy

FWIW, I don't disagree with anything in your edit. I just think that if good programmers tell inexperienced programmers, "Python solves all your problems with designing good interfaces, by defining them implicitly and users just look at the source", then those inexperienced programmers will take longer to become good programmers than if they're told: "designing interfaces is important whenever other people rely on code that you might ever change without the opportunity to update their code". – Embassy

..."good programmers tell inexperienced programmers"... Don't see how that applies to this person's question. They seem to be experienced. – Isoline

True. It's what they say in the Python community that concerns me. If you tell experienced programmers "Use the source, Luke", and inexperienced programmers "read the documentation, and if it isn't good enough complain and I'll fix it", then that's fair enough. The experienced programmers will complain of their own accord, and write decent documentation of their own accord, because they know that implict interfaces aren't worth the paper they're not written on, once it comes to portability across multiple implementations/versions. – Embassy

@Steve Jessop and @Scott Lott, this is a hugely interesting thread, thx. One thing that is killing me about dynamic languages is how hard refactoring is (at least in Ruby). Yes, you can move a class and update all unit tests, but that means that... refactoring is expensive. And you're not going to spend what you saved in using duck-typing on laborious refactoring. And if you don't really take everything and refactor a few times, you either do great design every time or... you work with not-so-well-organized code. And for that, the implementation is even harder to get the interface out of. – Heterolecithal

"move a class and update all unit tests, but that means that... refactoring is expensive"? Really? Why? You might want to open a question on why you're observing this. I find that refactoring a dynamic language is no harder than a static language. – Isoline

I'd say that if you're used to compile-time typing, then you learn certain refactoring tricks that allow you to lean on the compiler. Losing those tricks on switching to a dynamic language means you have to learn new processes to perform the same refactor, which is certainly slower for newbies. I'm writing Python just now. Some changes (e.g. changing a function name) are taking slightly longer than I think they would in C++. Others (e.g. narrowing the return type of a function) become point changes--change the implementation and docs--but in C++ might generate some busy-work. – Embassy

"changing a function name are taking slightly longer"? What IDE are you using? Komodo has global search and replace. It seems pretty easy to me. That doesn't seem to be consistent with "refactoring is expensive". Maybe it's "slightly more expensive while learning". But everything is more expensive while learning. – Isoline

I agree, I think yar is probably wrong, and refactoring isn't slower in Python. It's just that the assertion "every single conceivable code change in Python is faster than the equivalent in any compile-time typed language" seems unlikely, so there are bound to be some counter-examples. Btw, a global search and replace is not necessarily correct - there could be unrelated classes having methods of the same name, which a compiler that respects type (and its corresponding IDE quick-refactor ops) would ignore naturally. – Embassy

@S.Lott: Now you're going too far man, search & replace is NOT refactoring. First it is not safe unless you do it one at a time (don't tell me you are just doing "replace all" and pray to dynamic God), second it is orders of magnitude slower than doing rename of a method with ReSharper in C#, and getting worse as your code base grows over time. – Solnit

@Piotr Owsiak: While part of that is true, the cost of refactoring in Java vs. Python doesn't seem to be an orders of magnitude. Refactoring in Java takes time to plan out to make sure all the interface definitions and other things will still compile afterwords. Python can be difficult to refactor, but Java can be difficult to even get it to compile. I'd suggest that order of magnitude isn't true. – Isoline

"No time spent wringing my hands over what an API call might be. " In my brief experience on this Earth I find that with dynamic languages you have to go look up APIs 5x more often to find out what the function signatures actually are. Even when static language APIs change, the changes are right in front of you in the code base. Changes are immediately visible right in front of you. Static typing is total magic. – Pester

Answer 2

Static typing is a form of premature optimization. It forces you to make detail decisions up front, when you may not have the knowledge to make them. It doesn't particularly help program correctness unless you create enough types to make logical sense. It makes it difficult to change data structures on the fly.

What you get out of it is a very limited amount of correctness checking: very limited because it doesn't separate ways to use ints, for example. Suppose we're dealing with rows and columns; both are probably ints, but row and column variables shouldn't be used interchangeably. You also get optimization, which can be a very useful thing, but isn't worth slowing down initial development for. You can make up for the correctness checking by writing appropriate tests.

The Common Lisp type system is good for this. All data objects know their type, and you can explicitly specify that type if you like.

An eval loop sort of execution model makes it very easy to test routines as you write them. You don't have to explicitly write tests up front (although there's nothing to stop you from doing that); you can write them and execute them on the fly (and then you can refine that into a test suite - think of it as incremental test development).

Not having a long build step makes it easier to do test-driven development, because it's a whole lot faster to run tests. You don't have to break up what you're doing every time you want to test.

When I hear people complaining about dynamic languages, I'm reminded of people complaining about version control systems without exclusive locks. It takes some people a long time to realize what they gain by moving to a modern VCS, and equally it takes some people a long time to appreciate dynamic languages.

Lahdidah

I love dynamic languages but I also love refactoring. Maybe I've got the wrong idea from the IDEs I've seen in Ruby. But if you can't work and rework your code... – Heterolecithal

The easiest I ever had it tearing algorithms out of code and substituting others was a project in Common Lisp. With the proper editor (always a necessity in CL), refactoring and reworking CL is easy. – Lahdidah

Answer 3

In general, I prefer talking about "interactive" languages rather than "dynamic" languages. When you only have an edit/compile/run cycle, any turn-around takes a long time. Well, at least on the order of "need to save, compile, check the compilation report, test-run, check test results".

With an interactive language, it's usually easy to modify a small part, then immediately test results. If your test runs still take a lomg time, you haven't won that much, but you can usually test on smaller cases. This facilitates rapid development. Once you have a known-correct implementation, this also helps optimisation, as you can develop and test your new, improved function(s) quickly and experiment with different representations or algorithms.

Kharif

Different matter - some statically typed languages (Java, at any rate) nowadays support incremental compilation and hot code replace, which makes for pretty much the same experience. – Girish

The language I mostly use this method with is Common Lisp,. I guess this means there's now a third (or maybe even fourth) axis in computer language defnitions (strongly/weakly typed, static/dynamic typing, interactive/batch and possibly scripting or not). – Kharif

Answer 4

My Experience

I've worked with both, probably about a decade with each professionally, in all.

I've anecdotally spent far more time trying to make statically typed languages understand what I want to do (probably weeks – perhaps months in total) than I've spent fixing bugs caused by dynamic type errors (perhaps an hour or two a year?).

Studies

People have tried quite hard to get evidence that one or the other is better in terms of programmer productivity, but the most recent review I've read says there's no strong evidence for either.

Totalitarian

Static type analysis is useful in some situations. I don't think it should not be inherently baked in to your language. It should be a tool within your interactive computing environment, along with your tests, your REPL, your refactors, your docs, your literate coding, etc.

Static type systems can make you think about useful things. This is especially true in a language like Haskell with Type Classes and Monads (which I confess I still don't really get). This is a good thing, but I feel it is totalitarian to believe it is always a good thing. You should think about it when appropriate. A language should not make you think about it or be aware of it from the outset of development.

Both Too Restrictive & Not Restrictive Enough

Static type systems that aren't Turing complete have limited expressivity. Why bake that in to the language using a new special domain specific language just for talking about types? Now your language sets the exact level of expressivity you have access to. Want more? You need to re-write your code or update the language. Want less? You're out of luck – use a different language.

Instead, why not use a base dynamic language to describe types and type systems when you want to? As a library. It's more expressive; more powerful (if wished); more flexible; and means a smaller base language.

Boilerplate

Statically typed languages seem to encourage boilerplate or code generation. I'm not sure if this is inherent. Perhaps a sufficiently powerful macro system would overcome it. I'm comparing the state of mocks for testing in Swift with that of objective c.

Monolithic

Statically typed languages seem to encourage monolithic applications – this is an opinion and an observation that I can't back up, but it seems to hold… They, or the tooling they come with, seems to encourage monolithic applications and thinking.

In contrast, in interactive computing environments you don't build a new application, you instead extend out the system so that it does more. The systems I know of (Lisp machines, Smalltalk, and Unix – its tools have a dynamically typed interface between them) use dynamic typing to assemble parts together.

We should be building tiny extensions to a whole, rather than setting out to build new whole applications, so this monolithic tendency, if it exists, is damaging.

Speed

The fastest dynamic tracing JIT compilers produce fast code, and they're still a fairly young technology. Still – you could also do this with a statically typed language.

Long Term

I suspect that long term, we'll end up with environments that support powerful static analysis, where you'll be able to declare types and protocols, and the system will help you to see if they are satisfied, or help show you the implied types. But you won't need to do that.

Forwarding

Answer 5

Here is parts of my answer to a previous, similar question (I know C#. Will I be more productive with Python?):

I come from a C#/.NET background myself. Started programming in .NET in abt. 2001, and at about the same time was introduced to Python. In 2001, my time spent in C# vs. in Python was about 90% C# / 10% Python. Now, the ratio is 5% C# / 95% Python. At my company, we still maintain a product line based on .NET. But all new stuff is based on Python.

We have created non-trivial applications in Python.

In my experience, what makes me more productive in Python vs. C#, is:

• It is a dynamic language. Using a dynamic language often allows you to remove whole architectural layers from your app. Pythons dynamic nature allows you to create reusable high-level abstractions in more natural and flexible (syntax-wise) way than you can in C#.
• Libraries. The standard libraries and a lot of the open-source libraries provided by the community are of high quality. The range of applications that Python is used for means that the range of libraries is wide.
• Faster development cycle. No compile step means I can test changes faster. For instance, when developing a web app, the dev server detects changes and reloads the app when the files are saved. Running a unit test from within my editor is just a keystroke away, and executes instantaneously.
• 'Easy access' to often-used features: lists, list comprehensions, generators, tuples etc.
• Less verbose syntax. You can create a WSGI-based Python web framework in fewer lines of code than your typical .NET web.config file :-)
• Good documentation. Good books.

Reactive

That's interesting as I've started learning Python (I'm a C# dev for over 5 years now) a few months ago and found out that C# 3.0 (introduced with .NET 3.5) has "borrowed" so many things from Python (and possibly other dynamic languages). C# is such a great, flexible and convenient language now and improving fast - Java seems to lag behind more and more. – Solnit

Agreed, the latest C# versions contains some interesting stuff. – Reactive

Answer 6

Interactive Shells! This is a huge productivity gain. Just launch irb/python to sit in front of interactive shell (for ruby and python respectively). You can then interactively test your classes, functions, experiment with expression (great for regex), different syntax and algorithms. This is real programmers playground and great tool for debugging too.

Just me two cents about errors:

All this run-time typing, no compiler errors on basic stuff, etc just makes my life harder when it comes to building large complex applications.

Compiler detectable errors are the kind of errors you will detect when you execute various automatic tests (unit, functional etc...) and those you should write anyway. Probably Linus Torvalds can say: Regression testing"? What's that? If it compiles, it is good, if it boots up it is perfect but he has thousands of testers that will do the job for him. Ah and using interactive shells you are getting the automatic feedback about your syntax like when you compile the application but the code gets executed in place.

Cornwallis

Some static languages with REPL ("interactive shell"): OCaml, Scala, F#, Haskell. – Mayo

Even iOS development has a REPL with the Playground. – Hypoglycemia

Answer 7

Consider the case where you have a subroutine that takes a single argument:

sub calculate( Int $x ){ ... }

Now your requirements change, and you have to deal with multiple arguments:

multi sub calculate( Int $x ){ ... }
multi sub calculate( Int @x ){
  my @ret;
  for @x -> $x {
    push @ret, calculate( $x );
  }
  return @ret;
}

Notice that there was very little change between the different versions.

Now what if you found out that you really should have been using floating point numbers:

multi sub calculate( Num $x ){ ... }
multi sub calculate( Num @x ){
  my @ret;
  for @x -> $x {
    push @ret, calculate( $x );
  }
  return @ret;
}

That was a smaller change than before, and note that any piece of code that called these subroutines would continue to work without a single change.

These examples were written in Perl6

Disvalue

Answer 8

I like static typing too. But I don't find I miss it all that much when I'm writing Python, (as long as the classes I'm using are well documented - and I'd argue that no language feature will ever save us from bad documentation). Neither do I miss most of Python's dynamic features when writing C++ (lambdas, I miss: bring on C++0x even if the syntax is horrible. And list comprehensions).

For error-detection, most "wrong type passed" and "wrong method called" errors are not subtle, so the main difference is that exceptions replace compiler errors. You have to make sure you actually execute every code path and all significant data paths, but of course you're doing that anyway for serious projects. I suspect that it's rare to be difficult to test all classes that could be assigned to a given variable. The main problem is that people used to C++ have learned to lean on the compiler, and don't try hard to avoid large classes of errors which the compiler will catch. In Python you have the option of thinking about it as you code, or waiting until the tests run to find out about it. Likewise, the C++ "automatic refactoring" operation of "change the parameters and see what fails to compile" needs some modification in Python if you want to locate all call sites.

You therefore have to make sure you're running the right subset of tests, since a full unit test run of a large Python app will take far longer than is acceptable in the "compile" phase of your current C++ code/compile/code/compile/code/compile/test cycle. But that's exactly the same problem as not wanting to rebuild everything in C++.

Static typing wins when it's actually difficult to run your code (for instance with embedded devices, or especially weird server environments you can't reproduce locally), because you catch more errors prior to the point where you're messing about with serial cables and rsync. But that's why you want an emulator regardless of what language you're writing in, and why for serious server code you have a proper test environment if developer's machines can't simulate production.

Plus it's worth remembering that a lot of C++ compiler errors and warnings aren't actually about your design, they're about the fact that there are so many different ways to write code which looks correct, but behaves completely wrong. Python programmers don't need warning that they've accidentally inserted a trigraph, or type-punned a pointer, because they don't have a psychotic preprocessor, or optimisations based on strict aliasing.

Embassy

Answer 9

I know it's been a while but the accepted answer lists properties that aren't limited to dynamically-typed languages. For example, #4 (less code) is also true of Scala (if I recall correctly). It's definitely true of Groovy (which is built on top of Java, so technically groovy is both static and dynamic). The only one I agree with is #7 (resilience)

From wikipedia,

Dynamic programming language is a ... high-level programming language which, at runtime, execute many common programming behaviors that static programming languages perform during compilation. These behaviors could include extension of the program, by adding new code, by extending objects and definitions, or by modifying the type system. Dynamic programmer may also include dynamic typing.

From my experience, the advantages of dynamic languages are less code/more efficient code and better "best" practices (like strong testing culture)... things which are not specific to dynamic languages.

Speaking of test culture, certain dynamic language enthusiasts (Ruby in particular) claim that all the tests that they write (test infection) allows them to also refactor applications easily. I tend not to buy this claim---poorly written tests (which are very, very easy to write) tend to become a maintenance nightmare.

To summarize: dynamic languages tend to make for quick product delivery, the app may or may not be easy to maintain, but are horrible for high-performance apps (statically compiled apps are way, way faster)

Peony