David Beazley’s concurrency demo on PyCon 2015

pycon_2015David Beazley did an amazing live demo on Python Concurrency on PyCon 2015 (Montreal, last month). Lucky for us it is available on YouTube! I must say, that once I started watching the video I was hooked! I simply could not stop watching for the entire 45 minutes.

Python concurrency methods and their performance characteristics

In the beginning of the video David types in a Fibonacci socket server and shows the audience how socket programming works. That is already quite impressive (as he talks and types at the same time). Then he evaluates several approaches to Python concurrency: threads, processes, co-routines combined with the select call and finally co-routines combined with the select call and threads.

The performance characteristics of the approaches are shown by running a concurrency test and then a “requests per second” and a “response time in seconds” performance test. David clearly explains which role the GIL plays on the concurrency of co-routines and also where you see the (operating system) overhead caused by threads and processes.

In the questions he says that you should follow the best practices on concurrency and he specifically names “don’t have shared state” and “use functions”. See the talk by clicking on the video below:

Thank you, PyCon 2015, for sharing this video with us!

Best practices and the joy of programming


When I was about 16 years old (1996) I was programming object pascal (Delphi). In that language I wrote a calculator program to convert a number from decimal to binary. While implementing this functionality I found out that I could make this code generic, such that you could give it any base (2 for binary, 8 for octal, 10 for decimal and 16 for hexadecimal) and convert the numbers back and forth. This was an extremely satisfying experience and it felt like I created a thing of beauty, like I discover a secret or maybe as they call it “a programming gem”.

Since nobody uses Delphi anymore (you may want to try Lazarus though), I rewrote the code into Python for any beginner to understand.

def convert_base(s,b,nb):
	# character set
	# function to convert int to string
	def int2str(n,b):
		if n==0: return ''
		return int2str(n // b, b) + c[n % b]
	# function to convert string to int
	def str2int(s,b):
		if s=='': return 0
		return str2int(s[:-1],b) * b + c.index(s[-1:])
	# filter invalid characters
	s = ''.join([k for k in s if k in c[0:b]])
	# convert string and return
	if s=='': s = '0'
	s = int2str(str2int(s,b),nb)
	if s=='': s = '0'
	return s

I can’t imagine anyone will feel the same satisfaction reading this code as I did writing it in 1996. Let me explain why. In 1996 we did not have the Internet yet, or Stack Overflow, where all the programming questions you may dream of are already asked. You would typically learn everything from books, both the how and the why, and you could not cut corners by Googling the answer. So there was no guarantee, unlike now, that with proper research you find the most “Pythonic” implementation.

You could even turn this around. Nowadays we know so good how things “should” be done, that there is little room for innovation. If you come up with a different approach to a known problem it quickly dismissed as “wrong” and “not a best practice”. On one hand this may be a sign that the software industry is maturing. On the other hand it may be crippling, because we now have generation of programmers who have no clue why things are the way they are. They can’t argue the motives of their choices any further than it being a “best practice”.

I can think of plenty examples, but the most striking is when I was implementing “default routing” in Symfony2. It is considered a “bad practice” by the Symfony author, hence removed in the second version (and I implemented it again). Hardly any programmer I’ve met can properly argue why it makes sense that it was removed (but they all tend to have an opinion). I guess that proves the point.

10 reasons why PHP is better than Python

“There are only two kinds of languages: the ones people complain about and the ones nobody uses” – Bjarne Stroustrup,

People wonder: Did he really say that? Yes, he did. If you wonder who Bjarne Stroustrup is: He created the C++ programming language. Many people believe that Python is “pretty cool” and PHP is “really bad”. But as we all know, there is truth in the saying:

It’s a poor carpenter who blames his tools.

I believe both good and bad software can be written in any language. And I should probably also quote Joel Spolsky who calls “language wars” a “fruitless debate”. But nevertheless, if you program in PHP and run into one of these Python “evangelists”, then the following list may come in handy.

10 reasons why PHP is better than Python

  1. Python hosting, hard to find and expensive, while cheap PHP hosting is everywhere.
  2. Python cannot be mixed with HTML (needs a template library), while PHP can.
  3. Python has no proper encapsulation (private keyword), while PHP has.
  4. Python is hardly used in the real world, while something as big as Facebook is built on PHP.
  5. Python has a great community, but it is not comparable to PHP’s.
  6. Python has some books and tutorials, but PHP has way more of them.
  7. Python does not have the live docs (famous forum-like reference manual) like PHP has.
  8. Python does not have a steep learning curve, but PHP is still easier to learn.
  9. Python indentation for code blocks is prone to errors, while PHP uses curly braces.
  10. Python lexical scoping is a mess (‘global’ and ‘nonlocal’ keywords fix this), while PHP behaves as expected.

To wrap up

PHP has come a long way. Today it is a mature language that executes fairly speedy. Agreed that it has some quirky naming of it’s built-in functions, but hey.. that’s the price you pay for backwards compatibility.

Hint: Make sure to also check out this page on Python.org that is a good reference when comparing Python to PHP.


PyCon Australia 2014: conference videos online


PyCon Australia 2014 was held last week (1st – 5th August) at the Brisbane Convention & Exhibition Center.

PyCon Australia is the national conference for the Python Programming Community, bringing together professional, student and enthusiast developers with a love for developing with Python.

For all of you that did not go, most of the conference is available on YouTube (39 videos):

  1. Graphs, Networks and Python: The Power of Interconnection by Lachlan Blackhall
  2. PyBots! or how to learn to stop worrying and love coding by Anna Gerber
  3. Deploy to Android: Adventures of a Hobbyist by Brendan Scott
  4. How (not) to upgrade a platform by Edward Schofield
  5. Caching: A trip down the rabbit hole by Tom Eastman
  6. Verification: Truth in Statistics by Tennessee Leeuwenburg
  7. Record linkage: Join for real life by Rhydwyn Mcguire
  8. Command line programs for busy developers by Aaron Iles
  9. What is OpenStack? by Michael Still
  10. Software Component Architectures and circuits? by James Mills
  11. IPython parallel for distributed computing by Nathan Faggian
  12. A Fireside Chat with Simon Willison
  13. Accessibility: Myths and Delusions by Katie Cunningham
  14. Python For Every Child In Australia by Dr James R. Curran
  15. Lightning talks
  16. How to Read the Logs by Anita Kuno (HP)
  17. Serialization formats aren’t toys by Tom Eastman
  18. Django MiniConf: Lightning talks
  19. What in the World is Asyncio? by Josh Bartlett
  20. Try A Little Randomness by Larry Hastings
  21. Building Better Web APIs by HawkOwl
  22. devpi: Your One Stop Cheese Shop by Richard Jones
  23. Learning to program is hard, and how to fix that by Jackson Gatenby
  24. Lesser known data structures by Tim McNamara
  25. The Quest for the Pocket-Sized Python by Christopher Neugebauer
  26. Sounds good! by Sebastian Beswick
  27. Running Django on Docker: a workflow and code by Danielle Madeley
  28. Software Carpentry in Australia: current activity and future directions by Damien Irving
  29. The Curse of the Django Podcast by Elena Williams
  30. Grug make fire! Grug make wheel! by Russell Keith-Magee
  31. (Benford’s) Law and Order (Fraud) by Rhys Elsmore
  32. The Curse of the Django Podcast by Elena Williams
  33. Lightning talks
  34. The Curse of the Django Podcast by Elena Williams
  35. Seeing with Python by Mark Rees
  36. Descriptors: attribute access redefined by Fraser Tweedale
  37. How do debug tool bars for web applications work? by Graham Dumpleton
  38. Continuous Integration Testing for Your Database Migrations by Joshua Hesketh
  39. Closing

Have fun watching!

EuroPython 2014: full conference on YouTube


EuroPython 2014 was held last week (July 21 – 27) at the Berlin Congress Center (BCC) in Germany.

The EuroPython conference is the second largest global conference for the popular programming language Python, after the PyCon US conference in North America.

For all of you that did not go, the full conference is available on YouTube (120 videos):

  1. Welcome [20:56]
  2. pymove3D Winner Announcement [21:05]
  3. One year of Snowden, what’s next? [58:00]
  4. What can python learn from Haskell? [41:20]
  5. Lightning Talks [40:18]
  6. Amanda: A New Generation of Distributed Services Framework [24:50]
  7. Will I still be able to get a job in 2024 if I don’t do… [46:47]
  8. Cutting-edge APIs using hypermedia at BSkyB [32:30]
  9. Mobile Games to the Cloud With Python [29:45]
  10. Statistics 101 for System Administrators [29:45]
  11. The Magic of Attribute Access [26:47]
  12. Pythonista: A full-featured Python environment for iOS … [40:32]
  13. Brain Waves for Hackers [33:36]
  14. Rethinking packaging, development and deployment [40:21]
  15. The Sorry State of SSL [44:56]
  16. Traversing Mazes the pythonic way and other Algorithmic… [44:50]
  17. The Cython Compiler for Python [48:01]
  18. Message-passing concurrency for Python [42:02]
  19. Extending Python, what is the best option for me? [32:22]
  20. Marconi – OpenStack Queuing and Notification Service [24:37]
  21. Solution oriented error handling [28:19]
  22. Web Scraping in Python 101 [20:18]
  23. Jigna: a seamless Python-JS bridge to create rich HTML … [23:36]
  24. How to make a full fledged REST API with Django OAuth T… [23:59]
  25. Documenting your project with MkDocs. [22:09]
  26. VPython goes to School [25:17]
  27. Full Stack Python [25:47]
  28. Using python, LXC and linux to create a mass VM hosting… [23:09]
  29. log everything with logstash and elasticsearch [18:29]
  30. Gamers do REST [24:20]
  31. Teaching Python [28:09]
  32. PyPy status talk (a.k.a.: no no, PyPy is not dead) [31:38]
  33. pymove3D – Python moves the world – Attractive programm… [23:10]
  34. Designing NRT(NearRealTime) stream processing systems :… [36:35]
  35. GNU/Linux Hardware Emulation with Python [20:34]
  36. Lightning Talks [1:22:47]
  37. Our decentralized future [47:06]
  38. Writing multi-language documentation using Sphinx [24:46]
  39. How we switched our 800+ projects from Apache to uWSGI [28:20]
  40. Introduction to pytest [29:16]
  41. Embedding Python: Charming the Snake with C++ [28:06]
  42. SQLAlchemy Drill [28:37]
  43. Design considerations while Evaluating, Developing, Dep… [40:39]
  44. Using All These Cores: Transactional Memory in PyPy [42:01]
  45. gevent: asynchronous I/O made easy [44:01]
  46. Scaling with Ansible [40:37]
  47. Don’t fear our new robot overlords! [38:43]
  48. Systems Integration: The OpenStack success story [40:00]
  49. Performance Python for Numerical Algorithms [48:28]
  50. DevOps Risk Mitigation: Test Driven Infrastructure [47:25]
  51. Compress Me, Stupid! [35:45]
  52. Stackless: Recent advancements and future goals [44:41]
  53. Design Your Tests [25:32]
  54. Automatic code reviews [20:49]
  55. Supercharge your development environment using Docker [22:36]
  56. An HTTP request’s journey through a platform-as-a-service [24:13]
  57. 3D sensors and Python: A space odyssey [22:40]
  58. How to Setup a new Python Project [24:45]
  59. Graph Databases, a little connected tour [25:33]
  60. Identifying Bugs Before Runtime With Jedi [25:04]
  61. Python refactoring with Rope and Traad [25:02]
  62. I want to help! How to make your first contribution to … [19:14]
  63. Managing the Cloud with a Few Lines of Python [25:37]
  64. How to become an Agile company – case study [22:30]
  65. For lack of a better name(server): DNS Explained [24:54]
  66. Python in system testing [23:10]
  67. Lightning Talks [1:28:23]
  68. Advanced Uses of py.test Fixtures [26:32]
  69. Multiplatform binary packaging and distribution of your… [20:20]
  70. Conversing with people living in poverty [29:12]
  71. Eve – REST APIs for Humans™ [30:44]
  72. The Shogun Machine Learning Toolbox [19:47]
  73. RISCy Business: Development of a RNAi design and off-ta… [25:05]
  74. How Disqus is using Django as the basis of our Service … [30:59]
  75. Metaprogramming, from decorators to macros [39:03]
  76. Support Python 2 and 3 with the same code [39:58]
  77. The Return of “The Return of Peer to Peer Computing”. [39:35]
  78. Python Debugger Uncovered [31:12]
  79. Writing Awesome Command-Line Programs in Python [41:25]
  80. Elasticsearch from the bottom up [36:54]
  81. Scikit-learn to “learn them all” [40:17]
  82. How Pony ORM translates Python generators to SQL queries [44:55]
  83. Probabilistic Programming in Python [45:35]
  84. Morepath: a Python Web Framework with Super Powers [27:46]
  85. Python for Zombies: 15.000 enrolled in the first Brazil… [26:11]
  86. The inner guts of Bitbucket [28:50]
  87. Learning Chess from data [21:58]
  88. Red Hat Loves Python [23:28]
  89. Jython in practice [25:51]
  90. packaging and testing with devpi and tox [26:46]
  91. Non Sequitur: An exploration of Python’s random module [29:13]
  92. Packaging in packaging: dh-virtualenv [31:34]
  93. The Continuum Platform: Advanced Analytics and Web-base… [24:55]
  94. Farewell and Welcome Home: Python in Two Genders [31:42]
  95. Lessons learned from building Elasticsearch client [24:41]
  96. Pioneering the Future of Computing Education [25:55]
  97. Lightning Talks [1:31:25]
  98. How to become a software developer in science? [24:53]
  99. Python’s Role in Big Data Analytics: Past, Present, and… [49:45]
  100. Combining the powerful worlds of Python and R [25:54]
  101. Scientific Visualization with GR [23:19]
  102. Scalable Realtime Architectures in Python [28:31]
  103. Python Driven Company [35:12]
  104. Out-of-Core Columnar Datasets [22:32]
  105. Twisted Names: DNS Building Blocks for Python Programmers [27:51]
  106. Event discrete simulation with SimPy [25:16]
  107. Advanced Database Programming with Python [31:00]
  108. Using asyncio (aka Tulip) for home automation [26:57]
  109. Concurrent programming with Python and my little experiment [21:12]
  110. Big Data Analytics with Python using Stratosphere [20:26]
  111. Practical PyBuilder [23:01]
  112. Building Realtime Web Applications with WebRTC and Python [20:18]
  113. Ganga: an interface to the LHC computing grid [24:28]
  114. Extending Scikit-Learn with your own Regressor [25:54]
  115. Post-Mortem Debugging with Heap-Dumps [26:54]
  116. Fun with cPython memory allocator [28:36]
  117. Everything You Always Wanted to Know About Memory in Python But Were Afraid to Ask [29:04]
  118. Conference Closing [15:52]
  119. Lightning Talks [1:16:09]
  120. Sponsoring von Open Source [28:02]

If you manage to watch all of them… you deserve a medal! 😉

source: http://www.youtube.com/user/europython2014