DenkzeitWiki

Main /

Python Idioms

Python

PythonSelf
WhitespaceInPython
PythonAbstractBaseClasses
OnlyPublicFieldsInPython
Decorators
PythonGenerators

GIL

PythonEvangelism

• Python Gotchas --eine Liste von Gefahren.
  Besonders die Hinweise für Mutable defaults for function/method arguments könnten hilfreich sein.
  
• http://zephyrfalcon.org/labs/python_pitfalls.html --10 Python pitfalls
  
  
• Python Grimoire
  
  
• Python Idioms and Efficiency
• python code guidelines
  
• AbstractionsLeak
• Python dictionary speed optimisation
  • Init key with default value
    • in fastest?
    • has_key
    • setdefault
    • get
    • try/except if key mostly not present, or in if mostly present. Exceptions are more expensive.
  • Standard Class Exceptions in Python
    • if less than 80% of the lookups are successfull, has_key() outperformes try/catch
      
  
  
• http://blog.ianbicking.org/reducing-boilerplate-code-in-init.html --Reducing boilerplate code in __init__
  treating arguments to the constructor as variable-assignments...
  
  
• Using translate
  
• Be Pythonic
  
  
• Code Like a Pythonista: Idiomatic Python
  • the presentation - referencing this page!
    
  
  
• 10 Python Optimization Tips and Issues

Exceptions

Exceptions

timeit-module

• catching(!) actually thrown exceptions (i.e. using a try/catch-block) is quite expensive in python, because CPython is implemented in C which does not have exceptions but uses return values. So catching an exception is much more expensive than throwing without catching is. See python code guidelines for an example and How "expensive" are Exceptions? for the explanation. See also slow try statements in python?.
  • the cost of setting up a try/catch-block is not that high, though
  • So raising an exception is quite cheap in python, quite in contrast to CSharp (or - generally - DotNet), where creating an Exception is very expensive due to the StackTrace being created.

python -m 'timeit' -s 'd=dict(zip(range(1000), range(1000)))' $'try: d[1999]\nexcept KeyError: pass'
100000 loops, best of 3: 8.7 usec per loop

python -m 'timeit' -s 'd=dict(zip(range(1000), range(1000)))' $'1999 in d and d[1999]'
1000000 loops, best of 3: 0.492 usec per loop

python -m 'timeit' -s 'd=dict(zip(range(1000), range(1000)))' $'try: d[0]\nexcept KeyError: pass'
1000000 loops, best of 3: 0.532 usec per loop

python -m 'timeit' -s 'd=dict(zip(range(1000), range(1000)))' $'d[0]'
1000000 loops, best of 3: 0.407 usec per loop

Definitely less readable, isn't it? Given this sort of difference, you should probably be much less concerned with raw performance and more with readability and maintainability. Only optimize when you have actually hit a performance bottleneck.[1]

• timing-information from examples in the entry how fast are exceptions of the python-faq (additionally testing in):

def keyerror(d, key):
	try:
		d[key] = d[key] + 1
	except KeyError:
		d[key] = 1

def getkey(d, key):
	if d.has_key(key):
		d[key] = d[key] + 1
	else:
		d[key] = 1

def setdefault(d, key):
	d[key] = d.setdefault(key, 0) + 1

def using_get(d, key):
	d[key] = d.get(key, 0) + 1

def using_in(d, key):
	if key in d:
		d[key] = d[key] + 1
	else:
		d[key] = 1

# d, key have to be passed in - global attributes, set up in the setup-code cannot be accessed otherwise.
def test(func_name, d, key, number):
	func = globals()[func_name]
	for x in range(number):
		func(d, key)
		key += 1

if __name__ == "__main__":
	from timeit import Timer
	
	numbers = range(500000, 5000000, 500000)
	repetitions = 5
	
	methods = ("keyerror", "getkey", "setdefault", "using_get", "using_in")
	tests = (("if key is found:", 0), ("if key is NOT found:", max(numbers)+1))

	# the setup-code is executed once per call to timeit(). perfect place to initialize the variables...
	setupcode = """d = dict(zip(range(%(keys)s), (0, ) * %(keys)s))
key = %(startvalue)s"""
	
	
	for number in numbers:
		print "doing %d iterations..." % number
		for msg, value in tests:
			print msg
			for method in methods:
				setup = "from __main__ import test\n%s" % (setupcode % {'keys':number+1, 'startvalue':value})
				t = Timer("test('%s', d, key, %s)" % (method, number), setup)
				times = t.repeat(repetitions, 1)
				print "\t%-13s: %2.4f\t%2.4f" % (method, min(times), max(times))
		print
		print

• Resulting in:

doing 500000 iterations...
if key is found:
	keyerror     : 1.6553	3.9503
	getkey       : 1.9335	2.1359
	setdefault   : 2.1095	2.7477
	using_get    : 2.3636	3.5747
	using_in     : 1.8220	3.0709
if key is NOT found:
	keyerror     : 10.1064	11.8852
	getkey       : 2.3423	3.1612
	setdefault   : 3.1672	5.2320
	using_get    : 3.2361	3.9089
	using_in     : 2.8297	3.9372
...

doing 4000000 iterations...
if key is found:
	keyerror     : 12.5921	12.6346
	getkey       : 15.6893	15.7605
	setdefault   : 16.9497	17.0034
	using_get    : 16.9623	17.0045
	using_in     : 14.0057	14.0811
if key is NOT found:
	keyerror     : 65.8745	68.0350
	getkey       : 14.0960	14.8609
	setdefault   : 17.7466	18.4614
	using_get    : 17.0413	17.8054
	using_in     : 12.3098	12.9954

• my conclusion:
  • if the key is found, exception-handling is fastest.
  • if the key is never(!) found, exc-handling is the slowest by a large margin.
  • in comes next, always. It outperforms the other mechanisms even if the key is never found (in large dicts, many accesses).
  • It seems getkey is appropriate when using small dicts/less accesses.
  • in should be used otherwise.

• How are exceptions actually implemented in assembly?
  
  
• Return from finally
  • when strange things happen...

blocks

• http://lesscode.org/2005/07/12/ruby-colored-blocks-in-python/ --Ruby-colored Blocks in Python

Blocks, however, are probably the one thing that Python will never get, due to concerns about people creating obfuscated code. That is, allowing every programmer to be a language designer means that every program can become its own language: the slippery slope that leads to Lisp. :-) One of the things that makes Python great is that it's a simple, easy-to-learn language. Sort of a "learn once, read anything" principle. Customizable syntax means that in the degenerate case there could be a new language to learn for every program you want to read. [2]

staticmethod and classmethod

Good candidates for static methods are methods that do not reference the self variable.

...class methods prepend the class reference to the argument list before calling the function. [3]

the key difference is that staticmethod doesn't have the automatic "cls" parameter [4]

new.instancemethod

• new -- Creation of runtime internal objects, the docs
• Dynamically extending APIs
• Dynamically added methods to a class
• high-performance currying with instancemethod

You can easily assign functions to classes (which are then accessible in the instances of that class), but you cannot assign functions to a single specific instance. For the latter, use new.instancemethod.

>>>def func(a, b):
   ...:     print a, b

>>>class Foo(object):
   ...:     pass

>>>foo = Foo()

>>>foo.func(4)
AttributeError: 'Foo' object has no attribute 'func'

>>>Foo.func = func

>>>foo.func(4)
<__main__.Foo object at 0x00FDEB30> 4

>>>foo.func2 = func

>>>foo.func2(4)
TypeError: func() takes exactly 2 arguments (1 given)


>>>import new

>>>foo.func2 = new.instancemethod(func, foo, Foo)

>>>foo.func2(4)
<__main__.Foo object at 0x00FDEB30> 4

raw_input & sys.stdin.readline

• http://www.freenetpages.co.uk/hp/alan.gauld/tutinput.htm
  • one can change the stream to be used with sys.stdin.readline

calling applications

• os.system
  • simple, returns returncode of called app
• os.popen[234]
  • returns process' stdin, stdout, stderr
  
  
• http://www.faqts.com/knowledge_base/view.phtml/aid/3469

Iterators

PythonGenerators

• Iterators in Action by JimBaker? @ PyCon 2007

Functions as first-class citizens

FirstClassFunctions

• Don't overuse classes in Python

imported or standalone script?

 
 if __name__ == "__main__":
     do_Whatever_Needs_To_Be_Done()
 

no Tail Recursion Elimination

TailRecursion

• http://neopythonic.blogspot.com/2009/04/tail-recursion-elimination.html
  • http://neopythonic.blogspot.com/2009/04/final-words-on-tail-calls.html
• Tail recursion decorator revisited

datetime

• Tips on using python's datetime module

super

• http://www.artima.com/weblogs/viewpost.jsp?thread=281127