Python etc

2021-04-20 18:01:07 When something fails, usually you want to log it. Let's have a look at a small toy example:

from logging import getLogger

logger = getLogger(__name__)
channels = {}

def update_channel(slug, name):
try:
old_name = channels[slug]
except KeyError as exc:
logger.error(repr(exc))
...

update_channel('pythonetc', 'Python etc')
# Logged: KeyError('pythonetc')

This example has a few issues:

+ There is no explicit log message. So, when it fails, you can't search in the project where this log record comes from.
+ There is no traceback. When the try block execution is more complicated, we want to be able to track where exactly in the call stack the exception occurred. To achieve it, logger methods provide exc_info argument. When it is set to True, the current exception with traceback will be added to the log message.

So, this is how we can do it better:

def update_channel(slug, name):
try:
old_name = channels[slug]
except KeyError as exc:
logger.error('channel not found', exc_info=True)
...

update_channel('pythonetc', 'Python etc')
# channel not found
# Traceback (most recent call last):
# File "...", line 3, in update_channel
# old_name = channels[slug]
# KeyError: 'pythonetc'

Also, the logger provides a convenient method exception which is the same as error with exc_info=True:

logger.exception('channel not found') Open / Comment

2021-04-15 18:00:46 Most of the exceptions raised from the standard library or built-ins have a quite descriptive self-contained message:

try:
[][0]
except IndexError as e:
exc = e

exc.args
# ('list index out of range',)

However, KeyError is different: instead of a user-friendly error message it contains the key which is missed:

try:
{}[0]
except KeyError as e:
exc = e

exc.args
# (0,)

So, if you log an exception as a string, make sure you save the class name (and the traceback) as well, or at least use repr instead of str:

repr(exc)
# 'KeyError(0)' Open / Comment

2021-04-08 18:00:11 Python uses eager evaluation. When a function is called, all its arguments are evaluated from left to right and only then their results are passed into the function:

print(print(1) or 2, print(3) or 4)
# 1
# 3
# 2 4

Operators and and or are lazy, the right value is evaluated only if needed (for or if the left value is falsy, and for and if the left value is truthy):

print(1) or print(2) and print(3)
# 1
# 2

For mathematical operators, the precedence is how it is in math:

1 + 2 * 3
# 7

The most interesting case is operator ** (power) which is (supposedly, the only thing in Python which is) evaluated from right to left:

2 ** 3 ** 4 == 2 ** (3 ** 4)
# True Open / Comment

2021-04-06 18:01:44 What if we want to modify a collection inside a function but don't want these modifications to affect the caller code? Then we should explicitly copy the value.

For this purpose, all mutable built-in collections provide method .copy:

def f(v2):
v2 = v2.copy()
v2.append(2)
print(f'{v2=}')
# v2=[1, 2]
v1 = [1]
f(v1)
print(f'{v1=}')
# v1=[1]

Custom objects (and built-in collections too) can be copied using copy.copy:

import copy

class C:
pass

def f(v2: C):
v2 = copy.copy(v2)
v2.p = 2
print(f'{v2.p=}')
# v2.p=2

v1 = C()
v1.p = 1
f(v1)
print(f'{v1.p=}')
# v1.p=1

However, copy.copy copies only the object itself but not underlying objects:

v1 = [[1]]
v2 = copy.copy(v1)
v2.append(2)
v2[0].append(3)
print(f'{v1=}, {v2=}')
# v1=[[1, 3]], v2=[[1, 3], 2]

So, if you need to copy all subobjects recursively, use, copy.deepcopy:

v1 = [[1]]
v2 = copy.deepcopy(v1)
v2[0].append(2)
print(f'{v1=}, {v2=}')
# v1=[[1]], v2=[[1, 2]] Open / Comment

2021-04-01 18:00:08 In most of the programming languages (like C, PHP, Go, Rust) values can be passed into a function either as value or as reference (pointer):

+ Call by value means that the value of the variable is copied, so all modification with the argument value inside the function won't affect the original value. This is an example of how it works in Go:

package main

func f(v2 int) {
v2 = 2
println("f v2:", v2)
// Output: f v2: 2
}

func main() {
v1 := 1
f(v1)
println("main v1:", v1)
// Output: main v1: 1
}

+ Call by reference means that all modifications that are done by the function, including reassignment, will modify the original value:

package main

func f(v2 *int) {
*v2 = 2
println("f v2:", *v2)
// Output: f v2: 2
}

func main() {
v1 := 1
f(&v1)
println("main v1:", v1)
// Output: main v1: 2
}

So, which one is used in Python? Well, neither.

In Python, the caller and the function share the same value:

def f(v2: list):
v2.append(2)
print('f v2:', v2)
# f v2: [1, 2]

v1 = [1]
f(v1)
print('v1:', v1)
# v1: [1, 2]

However, the function can't replace the value (reassign the variable):

def f(v2: int):
v2 = 2
print('f v2:', v2)
# f v2: 2

v1 = 1
f(v1)
print('v1:', v1)
# v1: 1

This approach is called Call by sharing. That means the argument is always passed into a function as a copy of the pointer. So, both variables point to the same boxed object in memory but if the pointer itself is modified inside the function, it doesn't affect the caller code. Open / Comment

2021-03-30 18:01:07 PEP-526, introducing syntax for variable annotations (laded in Python 3.6), allows annotating any valid assignment target:

c.x: int = 0
c.y: int

d = {}
d['a']: int = 0
d['b']: int

The last line is the most interesting one. Adding annotations to an expression suppresses its execution:

d = {}

# fails
d[1]
# KeyError: 1

# nothing happens
d[1]: 1

Despite being a part of the PEP, it's not supported by mypy:

$ cat tmp.py
d = {}
d['a']: int
d['b']: str
reveal_type(d['a'])
reveal_type(d['b'])

$ mypy tmp.py
tmp.py:2: error: Unexpected type declaration
tmp.py:3: error: Unexpected type declaration
tmp.py:4: note: Revealed type is 'Any'
tmp.py:5: note: Revealed type is 'Any' Open / Comment

2021-03-25 18:00:07 PEP-589 (landed in Python 3.8) introduced typing.TypedDict as a way to annotate dicts:

from typing import TypedDict

class Movie(TypedDict):
name: str
year: int

movie: Movie = {
'name': 'Blade Runner',
'year': 1982,
}

It cannot have keys that aren't explicitly specified in the type:

movie: Movie = {
'name': 'Blade Runner',
'year': 1982,
'director': 'Ridley Scott', # fails type checking
}

Also, all specified keys are required by default but it can be changed by passing total=False:

movie: Movie = {} # fails type checking

class Movie2(TypedDict, total=False):
name: str
year: int

movie2: Movie2 = {} # ok Open / Comment

2021-03-23 18:00:17 Infinity has an interesting behavior on division operations. Some of them are expected, some of them are surprising. Without further talking, there is a table:

truediv (/)
| -8 | 8 | -inf | inf
-8 | 1.0 | -1.0 | 0.0 | -0.0
8 | -1.0 | 1.0 | -0.0 | 0.0
-inf | inf | -inf | nan | nan
inf | -inf | inf | nan | nan

floordiv (//)
| -8 | 8 | -inf | inf
-8 | 1 | -1 | 0.0 | -1.0
8 | -1 | 1 | -1.0 | 0.0
-inf | nan | nan | nan | nan
inf | nan | nan | nan | nan

mod (%)
| -8 | 8 | -inf | inf
-8 | 0 | 0 | -8.0 | inf
8 | 0 | 0 | -inf | 8.0
-inf | nan | nan | nan | nan
inf | nan | nan | nan | nan

The code used to generate the table:

import operator
cases = (-8, 8, float('-inf'), float('inf'))
ops = (operator.truediv, operator.floordiv, operator.mod)
for op in ops:
print(op.__name__)
row = ['{:4}'.format(x) for x in cases]
print(' ' * 6, ' | '.join(row))
for x in cases:
row = ['{:4}'.format(x)]
for y in cases:
row.append('{:4}'.format(op(x, y)))
print(' | '.join(row)) Open / Comment

2021-03-18 18:00:05 Floating point numbers in Python and most of the modern languages are implemented according to IEEE 754. The most interesting and hardcore part is "arithmetic formats" which defines a few special values:

+ inf and -inf representing infinity.
+ nan representing a special "Not a Number" value.
+ -0.0 representing "negative zero"

Negative zero is the easiest case, for all operations it considered to be the same as the positive zero:

-.0 == .0 # True
-.0 < .0 # False

Nan returns False for all comparison operations (except !=) including comparison with inf:

import math

math.nan < 10 # False
math.nan > 10 # False
math.nan < math.inf # False
math.nan > math.inf # False
math.nan == math.nan # False
math.nan != 10 # True

And all binary operations on nan return nan:

math.nan + 10 # nan
1 / math.nan # nan

You can read more about nan in previous posts:

+ https://t.me/pythonetc/561
+ https://t.me/pythonetc/597

Infinity is bigger than anything else (except nan). However, unlike in pure math, infinity is equal to infinity:

10 < math.inf # True
math.inf == math.inf # True

The sum of positive and negative infinity is nan:

-math.inf + math.inf # nan Open / Comment

2021-03-16 18:01:07 Starting Python 3.8, the interpreter warns about is comparison of literals.

Python 3.7:

>>> 0 is 0
True

Python 3.8:

>>> 0 is 0
:1: SyntaxWarning: "is" with a literal. Did you mean "=="?
True

The reason is that it is an infamous Python gotcha. While == does values comparison (which is implemented by calling __eq__ magic method, in a nutshell), is compares memory addresses of objects. It's true for ints from -5 to 256 but it won't work for ints out of this range or for objects of other types:

a = -5
a is -5 # True
a = -6
a is -6 # False
a = 256
a is 256 # True
a = 257
a is 257 # False

2.1K views15:01

Open / Comment