Gael Varoquaux 11/9/2009

Writing parallel code in a readable way

Read Original

This article discusses a Python-based approach to writing parallel code that remains readable and easy to debug, particularly for embarrassingly parallel problems. The author introduces a custom tool using a 'delayed' decorator and a 'Parallel' object that allows developers to write map/reduce-style code with list comprehensions, making it look like standard procedural code. Key features include a simple switch to run code serially (n_jobs=1) for debugging, automatic pickling checks to avoid errors, and a uniform API that can encapsulate various fork/join-based parallel libraries. The tool evolved into the joblib project. This is relevant to IT/technology as it covers parallel computing, Python programming, and software development best practices.

Writing parallel code in a readable way

Comments

No comments yet

Be the first to share your thoughts!

Browser Extension

Get instant access to AllDevBlogs from your browser

Top of the Week

No top articles yet