How to implement virtual methods in Python?

Question

I know virtual methods from PHP or Java.

How can they be implemented in Python?

Or have I to define an empty method in an abstract class and override it?

Newfoundland

Answer 1

Sure, and you don't even have to define a method in the base class. In Python methods are better than virtual - they're completely dynamic, as the typing in Python is duck typing.

class Dog:
  def say(self):
    print "hau"

class Cat:
  def say(self):
    print "meow"

pet = Dog()
pet.say() # prints "hau"
another_pet = Cat()
another_pet.say() # prints "meow"

my_pets = [pet, another_pet]
for a_pet in my_pets:
  a_pet.say()

Cat and Dog in Python don't even have to derive from a common base class to allow this behavior - you gain it for free. That said, some programmers prefer to define their class hierarchies in a more rigid way to document it better and impose some strictness of typing. This is also possible - see for example the abc standard module.

Beep

cat.say() should be another_pet.say() – Outgrow

+1 for an example. In what language do dogs say "hau" by the way? – Bum

@JeremyP: hmm, good point :-) I guess in languages where "h" is understood as making the sound like the first letter of "hippy", or of "Javier" in Spanish. – Beep

@Eli: Sorry, but I was seriously interested in the answer to the question. In English they say "woof", well they don't but that is the word we use analogous to "meow" for cats and "moo" for cows. Is "hau" Spanish then? – Bum

@Bum I know for sure that's what Polish dogs say ;) – Naumann

@Bum yes I confirm dogs say "Jau" in Spanish and when written in english would be "Hau" :) hth – Uro

@Bum In Portuguese, the onomatopoeia for the dog's barking is "au", which sounds exactly like the OP's "hau". Edit: "au" sounds like the "ow" in the English word "how". – Headroom

@JeremyP, in Polish dogs say literally "Hau" and cats say "Miau". – Dogs

I wonder how Polish and English dogs talk to each other? Do they have their own version of google translate, or do they just use body language like human tourists do? – Peursem

In Spanish dogs say "guau" or "wuau" and cats say "miau". Also roosters say "kikiriki", which baffles a lot of non spanish speakers. I heard there's a conference coming up to officialize "woofperanto", so dogs can finally communicate in this globalized world ;) – Dillondillow

Confirming that in Hebrew it's "Hau" as well (at least, this is a good transcription for האו). – Chaetopod

Answer 2

Summary of current implementation status with a focus on "blowup nicely if the method is not implemented" behavior:

methd	exception	mypy static	sphinx
raise NotImplementedError()	on method call	n	n
typing.Protocol	n	y	n
@abc.abstractmethod	on instantiation	y	y

raise NotImplementedError(): dynamic type checking

This is the recommended exception to raise on "pure virtual methods" of "abstract" base classes that don't implement a method.

https://docs.python.org/3.5/library/exceptions.html#NotImplementedError says:

This exception is derived from RuntimeError. In user defined base classes, abstract methods should raise this exception when they require derived classes to override the method.

As others said, this is mostly a documentation convention and is not required, but this way you get a more meaningful exception than a missing attribute error.

dynamic.py

class Base(object):
    def virtualMethod(self):
        raise NotImplementedError()
    def usesVirtualMethod(self):
        return self.virtualMethod() + 1

class Derived(Base):
    def virtualMethod(self):
        return 1

assert Derived().usesVirtualMethod() == 2
Base().usesVirtualMethod()

gives:

Traceback (most recent call last):
  File "./dynamic.py", line 13, in <module>
    Base().usesVirtualMethod()
  File "./dynamic.py", line 6, in usesVirtualMethod
    return self.virtualMethod() + 1
  File "./dynamic.py", line 4, in virtualMethod
    raise NotImplementedError()
NotImplementedError

typing.Protocol: static type checking (Python 3.8)

Python 3.8 added typing.Protocol which now allows us to also statically type check that a virtual method is implemented on a subclass.

protocol.py

from typing import Protocol

class Bird(Protocol):
    def fly(self) -> str:
        pass

    def peck(self) -> str:
        return 'Bird.peck'

class Pigeon(Bird):
    def fly(self):
        return 'Pigeon.fly'

    def peck(self):
        return 'Pigeon.peck'

class Parrot(Bird):
    def fly(self):
        return 'Parrot.fly'

class Dog(Bird):
    pass

pigeon = Pigeon()
assert pigeon.fly() == 'Pigeon.fly'
assert pigeon.peck() == 'Pigeon.peck'
parrot = Parrot()
assert parrot.fly() == 'Parrot.fly'
assert parrot.peck() == 'Bird.peck'
# mypy error
dog = Dog()
assert dog.fly() is None
assert dog.peck() == 'Bird.peck'

If we run this file, the asserts pass, as we didn't add any dynamic typechecking:

python protocol.py

but if we typecheck if mypy:

python -m pip install --user mypy
mypy protocol.py

we get an error as expected:

rotocol.py:31: error: Cannot instantiate abstract class "Dog" with abstract attribute "fly"  [abstract]
Found 1 error in 1 file (checked 1 source file)

It is a bit unfortunate however that the error checking only picks up the error on instantiation, and not at class definition.

typing.Protocol counts methods as abstract when their body is "empty"

I'm not sure what they count as empty, but both all of the following count as empty:

• pass
• ... ellipsis object
• raise NotImplementedError()

So the best possibility is likely:

protocol_empty.py

from typing import Protocol

class Bird(Protocol):
    def fly(self) -> None:
        raise NotImplementedError()

class Pigeon(Bird):
    def fly(self):
        return None

class Dog(Bird):
    pass

Bird().fly()
Dog().fly()

which fails as desired:

protocol_empty.py:14: error: Cannot instantiate protocol class "Bird"  [misc]
protocol_empty.py:15: error: Cannot instantiate abstract class "Dog" with abstract attribute "fly"  [abstract]
protocol_empty.py:15: note: "fly" is implicitly abstract because it has an empty function body. If it is not meant to be abstract, explicitly `return` or `return None`.
Found 2 errors in 1 file (checked 1 source file)

but if e.g. we replace the:

raise NotImplementedError()

with some random "non-empty" statement such as:

x = 1

then mypy does not count them as virtual and gives no errors.

@abc.abstractmethod: dynamic + static + documentation in one go

Previously mentioned at: https://mcmap.net/q/120781/-how-to-implement-virtual-methods-in-python but the metaclass syntax changed in Python 3 to:

class C(metaclass=abc.ABCMeta):

instead of the Python 2:

class C:
    __metaclass__=abc.ABCMeta

so now to use @abc.abstractmethod which was previously mentioned at https://mcmap.net/q/120781/-how-to-implement-virtual-methods-in-python you need:

abc_cheat.py

import abc

class C(metaclass=abc.ABCMeta):
    @abc.abstractmethod
    def m(self, i):
        pass

try:
    c = C()
except TypeError:
    pass
else:
    assert False

Vs raise NotImplementedError and Protocol:

• disadvantage: more verbose
• advantage: does all of dynamic checks, static checks and shows up on documentation (see below)

https://peps.python.org/pep-0544 does mention both approaches in passing

E.g.:

abc_bad.py

#!/usr/bin/env python

import abc

class CanFly(metaclass=abc.ABCMeta):
    '''
    doc
    '''

    @abc.abstractmethod
    def fly(self) -> str:
        '''
        doc
        '''
        pass

class Bird(CanFly):
    '''
    doc
    '''

    def fly(self):
        '''
        doc
        '''
        return 'Bird.fly'

class Dog(CanFly):
    '''
    doc
    '''
    pass

def send_mail(flyer: CanFly) -> str:
    '''
    doc
    '''
    return flyer.fly()

assert send_mail(Bird()) == 'Bird.fly'
assert send_mail(Dog()) == 'Dog.fly'

then:

mypy abc_bad.py

fails as desired with:

main.py:40: error: Cannot instantiate abstract class "Dog" with abstract attribute "fly"

Sphinx: make it show on the documentation

See: How to annotate a member as abstract in Sphinx documentation?

Of the methods mentioned above, only one shows up on the sphinx documentation output: @abc.abstractmethod.

Outro

Bibiography:

• https://peps.python.org/pep-0544 the typing.Protocol PEP
• Is it possible to make abstract classes in Python?
• What to use in replacement of an interface/protocol in python

Tested on Python 3.10.7, mypy 0.982, Ubuntu 21.10.

Extremity

Answer 3

Python methods are always virtual.

Orban

Except for dunder methods. – Carvajal

This answer is not really helping in the objective of implementing interface classes which is one of the main reason for using virtual methods. – Discriminate

Answer 4

Actually, in version 2.6 python provides something called abstract base classes and you can explicitly set virtual methods like this:

from abc import ABCMeta
from abc import abstractmethod
...
class C:
    __metaclass__ = ABCMeta
    @abstractmethod
    def my_abstract_method(self, ...):

It works very well, provided the class does not inherit from classes that already use metaclasses.

source: http://docs.python.org/2/library/abc.html

Peptide

Is there a python 3 equivalent for this directive? – Robena

@Robena metaclass syntax changed in Python 3, now you have to write class C(metaclass=abc.ABCMeta): https://mcmap.net/q/120781/-how-to-implement-virtual-methods-in-python – Extremity

Answer 5

Python methods are always virtual

like Ignacio said yet Somehow class inheritance may be a better approach to implement what you want.

class Animal:
    def __init__(self,name,legs):
        self.name = name
        self.legs = legs

    def getLegs(self):
        return "{0} has {1} legs".format(self.name, self.legs)

    def says(self):
        return "I am an unknown animal"

class Dog(Animal): # <Dog inherits from Animal here (all methods as well)

    def says(self): # <Called instead of Animal says method
        return "I am a dog named {0}".format(self.name)

    def somethingOnlyADogCanDo(self):
        return "be loyal"

formless = Animal("Animal", 0)
rover = Dog("Rover", 4) #<calls initialization method from animal

print(formless.says()) # <calls animal say method

print(rover.says()) #<calls Dog says method
print(rover.getLegs()) #<calls getLegs method from animal class

Results should be:

I am an unknown animal
I am a dog named Rover
Rover has 4 legs

Dews

Answer 6

Something like a virtual method in C++ (calling method implementation of a derived class through a reference or pointer to the base class) doesn't make sense in Python, as Python doesn't have typing. (I don't know how virtual methods work in Java and PHP though.)

But if by "virtual" you mean calling the bottom-most implementation in the inheritance hierarchy, then that's what you always get in Python, as several answers point out.

Well, almost always...

As dplamp pointed out, not all methods in Python behave like that. Dunder method don't. And I think that's a not so well known feature.

Consider this artificial example

class A:
    def prop_a(self):
        return 1
    def prop_b(self):
        return 10 * self.prop_a()

class B(A):
    def prop_a(self):
        return 2

Now

>>> B().prop_b()
20
>>> A().prob_b()
10

However, consider this one

class A:
    def __prop_a(self):
        return 1
    def prop_b(self):
        return 10 * self.__prop_a()

class B(A):
    def __prop_a(self):
        return 2

Now

>>> B().prop_b()
10
>>> A().prob_b()
10

The only thing we've changes was making prop_a() a dunder method.

A problem with the first behavior can be that you can't change the behavior of prop_a() in the derived class without impacting the behavior of prop_b(). This very nice talk by Raymond Hettinger gives an example for a use case where this is inconvenient.

Carvajal

Answer 7

Python 3.6 introduced __init_subclass__ and this let you simply do this:

class A:

    def method(self):
        '''method needs to be overwritten'''
        return NotImplemented

    def __init_subclass__(cls):
        if cls.method is A.method:
            raise NotImplementedError(
                'Subclass has not overwritten method {method}!')

The benefit of this solution is that you avoid the abc metaclass and give the user a direct imperative how to do it right. In addition to another answer here that raises NotImplementedError when calling the method. This solution is checked on runtime and not only IF the user calls the method.

Wayless

Answer 8

Virtual is not same as implementing abstract in inheriting child class.

Virtual supposed to be used with Interface contract, where you can give the code a bit extra flexibility and still gain the benefits of low-code duplication since the parent class can implement the basic function implementation that applies to 9/10 of your classes and through virtual you can override the 1/10 without needing to duplicate your functions in child-classes x,y or n because of the abstract definition would require you to do*.

I doubt there is any robust solution to support virtual in Python unluckily, as even the interfaces are not yet as part of the built-in modules (abc module, even often used to imitate interfaces, cannot really be considered as proper interface support).

*Software design quality-wise speaking it is one technique that can be used to move code towards low coupling, high cohesion quality goals, that lead to anti-spaghetti and eventually to code generation and fully generic modular coding philosophy, that is written as an generative engine, rather than custom implementation. There are many OOP programmers who claim OOP cannot be generated from the scratch due its "nature", the OOP is not supposed to be anything else than generated actually, especially if you do all of it before runtime (generate the whole runtime code in between), the better.

In case of instantiating new objects, you can enable high perf just like done in graphic rendering libraries. Generating an object blitter, to handle sudden needs for objects + on top library that handles management for such/orchestrates such real-time performance critical needs. Also, depending on your use-case, you can store them in high-perf data-structures such as Stacks, or List/Dict of Stacks incase you need to group them.

Ozonize

Answer 9

• You can just declare them implicitly like in Eli Bendersky's answer, or

• For the extra check that matching base method exists, use an older or after Python 3.12 standard @typing.override decorator. Standard one will warn only if the IDE or type checker supports PEP 698 (for example, PyCharm supports it only after v2023.3). And for some implementations, it is still optional:

from typing import override

class Base:
    def foo(self) -> str:
        return "1"

    def bar(self) -> int:
        return 2

class Derived(Base):
    # IDE inspection: Missing super method for override
    @override
    def bar(self, new_arg) -> int:
        return 4

    # IDE inspection: does not detect changed type
    @override
    def foo(self) -> int:
        return 3

    # IDE inspection: May not not warn decorator was omitted
    def bar(self) -> int:
        return 5  

• If also clear labelling of the base methods is desired, which also distinguish abstract vs virtual, my simple @virutalmethod decorator may help. Full use case is here by Action descendants.
Advantage of labelling is visible when adding another one derived class: without digs of docs you see which of the base methods were not intended to be replaced, which must be replaced to stop the class being abstract, and which virtuals were considered to be optionally extended by design. That was just an experimental wrap of @override which will give a warning if no base method is decorated as abstract or virtual, for example:

from virtual_methods import override, virutalmethod

class Root(ABC):
    @virutalmethod
    def foo(self) -> str:
        return "1"

    @abstractmethod
    def bar(self) -> int:
        return 2    

    # WARNING: missing decorator
    def baz(self) -> int:
        return 3    

# Trunk is not abstract and foo() call is already valid
class Trunk(Root):
    @override
    def bar(self) -> int:
        return 4    

class Leaf(Trunk):
    @override
    def foo(self) -> int:
        return 5

    @override
    def baz(self) -> int:
        return 6

Coastland