Operator Overloading in Python

Apr 27, 2021

As a C++ developer, I was familiar with operator overloading and happy to know that Python offered the same feature. Luckily, I hadn't been broken down by some of the poor examples of overuse that plagued the C++ world of operator overloading, like some of the folks who I've encountered online. For me, I was excited that the Python language offered the feature to (for example) add two user- defined matrices classes together with the + operator, instead of having to use an overly-verbose .add() method, like would be seen in Java.

Python, of course, added this language feature because it was practical and increased the readability of the code. As in, it made it clear what the writer of the code wanted to accomplish. However, the Python language implemented operator overloading in a way that could not be as easily abused as C++.

The way Python did this was to implement operators as dunder methods (AKA special methods with double underscores before and after). For example, when executing the code 5 + 4, Python is really executing:

int.__add__(5, 4)

To overload an operator in Python, a user-defined class needs to override the dunder method which is associated with the operator. The table is shown below. Don't worry if you don't know all of them, or understand what 'Reverse Operator' or 'In-Place Operator' is for now. Only the ones which are applicable to your class needs to be overridden (that's the benefit of Python). Reverse and In- Place operators will be covered below, as well as unary methods, which get their own table further below.

Mathematical Operators:

Bitwise Logical Operators:

Boolean Operators:

Let's focus on just the 'Standard Dunder Method' column for now. The process to override any of these methods are pretty much the same. Add a method to the user-defined class which takes self and one additional argument. The method should return the result of the operation and not change the class instance or either of the input parameters, as a best practice. This is true for the methods in the 'Reverse Operator' column as well, but may not be true for the methods in the 'In-Place Operator' column, depending on the implementation.

An example for adding two angles might look like this:

>>> class Angle:
...     def __init__(self, degrees):
...         self._degrees = degrees

...     def __repr__(self):
...         return f'Angle({self._degrees})'

...     def __add__(self, angle_to_add):
...         degrees = self._degrees + angle_to_add._degrees
...         if degrees > 360:
...             degrees -= 360
...         if degrees < 0:
...             degrees += 360
...         return Angle(degrees)
... 
>>> angle1 = Angle(270)
>>> angle2 = Angle(180)
>>> angle3 = angle1 + angle2
>>> angle3
Angle(90)

The Angle class' addition method is really simple, but it adds a step to standard addition. Instead of just adding two numbers, it also checks that the answer is between 0 and 360 degrees. If not, it wraps the answer by adding or subtracting 360 degrees.

The last line shows that this angle wrap works as expected. Instead of 270 degrees and 180 degrees becoming 450 degrees, the angle is calculated as 90 degrees.

Let's examine in detail how this worked. The __add__ dunder method takes self and angle_to_add. The self argument is actually what falls to the left of the +. The angle_to_add is what falls to the right of the + sign. You can see the newly calculated value is then determined by adding the _degrees attribute of the objects on both sides of the +. After checking that the newly calculated number of degrees is within bounds, a new object is created and returned, meeting the best practice for the __add__() method.

What if you would like to add an integer to an angle object without having to explicitly convert to an Angle? While this may not be the best practice (Python seeks to be explicit rather than implicit), it might be the right choice for your application. It also demonstrates a 'gotcha' while implementing __add__ between object types. You might think this will work:

>>> class Angle:
...     ''' Other implementation details removed for brevity'''
...     def __add__(self, int_to_add):
...         degrees = self._degrees + int_to_add
...         if degrees > 360:
...             degrees -= 360
...         if degrees < 0:
...             degrees += 360
...         return Angle(degrees)
... 
>>> angle = Angle(10)
>>> angle + 10
Angle(20)

It certainly does when the Angle is on the left side of the operand. However, placing the Angle on the right side of the operand creates a TypeError:

>>> 10 + angle
Traceback (most recent call last):
  File '', line 1, in 
TypeError: unsupported operand type(s) for +: 'int' and 'Angle'

If your application needs objects which require commutative operators (the Angle type sure does), you will need to implement the 'Reverse Operator', shown above in the operator table. For addition, this would be the rad __radd__ method. The implementation can be super easy in a lot of cases:

>>> class Angle:
...     ''' Other implementation details removed for brevity'''
...     def __radd__(self, int_to_add):
...         return self + int_to_add

The 'Reverse Operator' is invoked by Python when a user-defined class is on the right side of an operator. The arguments are a little confusing here (they certainly confused me when making this example). The left side of the operator is actually the second argument. The right side of the operator (the user defined class) is self or the first argument. The arguments are reversed from the standard operator, but make sense if thinking of it from a purely method- signature standpoint.

The __radd__ method can just delegate to the 'forward' operator __add__() in most cases.

The last operator is the 'In-Place' operator types. Many classes will not need to implement these operators. Python will automatically use the associated 'forward' operator and then assign to a new object. For example, the int class does not implement __iadd__ but += is a valid operator.

The side effect of not implementing 'In-Place' operators is that Python will create a new object after completing the new calculation. This isn't always desirable (for example, if objects are mutable or expensive to create). In those cases, it is best to implement the 'In-Place' operators so that the same objects can be modified.

Like the 'forward' and 'reverse' operators, 'In-Place' operators must take two parameters: self and the object to perform the operation with. Unlike 'forward' and 'reverse' operators, they must return self to work correctly.

An example of the 'In-Place' operator += for our Angle class:

>>> class Angle:
...    ''' Other implementation details removed for brevity'''
...     def __iadd__(self, angle_to_add):
...         self._degrees += angle_to_add._degrees
...         if self._degrees > 360:
...             self._degrees -= 360
...         if self._degrees < 0:
...             self._degrees += 360
...         return self
... 
>>> angle = Angle(10)
>>> angle += Angle(40)
>>> angle
Angle(50)

The equality operators are defined exactly the same as the math-y operators like addition, subtraction, etc. However, instead of returning an instance of the class, they return a True or False value. Like the forward and reverse operators, it is not expected that the class itself or either input parameter is changed by a equality operator.

Python also provides unary operators that didn't quite fit into the table that I placed above. There are only three:

- (__neg__): Used to negate an object
+ (__pos__): Used to reference the positive value of an object
~ (__invert__): Used to find bitwise inverse

Of course, this is where my Angle class example falls apart...

To demonstrate __neg__(), Angle will have to be satisfied with a negative _degrees value, something it wasn't satisfied with in the addition examples above. That's fine I guess...

Unary operators are fairly simple to implement from a syntax perspective. They just take a single argument self and return the representation of the object after it has been operated on. Most of the time, this will be a new object of the same type. Note that like the forward and reverse operators above, the unary operators are expected to return a new object and not modify the one being operated on.

Here's an example of __neg__() for Angle:

>>> class Angle:
...    ''' Other implementation details removed for brevity'''
...     def __neg__(self):
...         return Angle(-self._degrees)
... 
>>> angle = Angle(10)
>>> -angle
Angle(-10)

The other two unary operators can be defined similarly for your user-defined class, as applicable.

I've covered quite a bit here, and skipped over some of the more interesting operators like __pow__ and __matmul__ (introduced in Python 3.5), but I hope this served as a good primer. If I find myself using __pow__ and __matmul__ frequently in my code and come to understand and love them, I'm sure I'll write a more detailed article about them in the future.

For now, remember to only implement what is practical and what will help you clearly convey what you are trying to accomplish. That's what moved me to Python, and I hope it stays that way.