Recommended book about parallel programming - theory & best practice?

Question

I'm looking for a book or books about multicore, multithreaded programming. The perfect book should focus on best practices and maybe include a bit of theory background. I'm not interested in a book which only describes a single library and focuses on its API.

OS actually doesn't matter.

Answer 1

This is one of only a handful of books that show you how code can be parallelized to take advantage of multi-cores. The Art of Concurrency

Answer 2

One of my old professors, Doug Lea, at SUNY Oswego wrote this book: Concurrent Programming in Java: Design Principles and Pattern. I highly recommend it, even if you don't necessarily work in Java. It has a lot of very good, generalized information that can be adapted (with a little bit of creativity) to other languages/development environments.

Answer 3

My university uses Concurrency: State Models and Java Programs in the course that discusses concurrent applications.

Answer 4

Herb Sutter has been writing an on-going, monthly column for Dr. Dobbs called Effective Concurrency. It is a series of very useful articles on concurrency theory and practice that could very well form a book.

His blog, Sutter's Mill, has links to all of the articles (22+) that have been published thus far.

Oh, yeah and its free (as in beer).

Answer 5

I like this one:

Art of Multiprocessor Programming

But haven't checked the other recommendations.

It's quite nitty-gritty, covering memory bus issues (not cache so much), lock-free algorithms, and theory. It uses Java but you can look up the original algorithms and re-implement them in C++ if you need to. If you need an introduction to threading this might not be for you but for a more serious look at how to get the most out of a shared memory, multi-processor machine it's great.

Answer 6

Technical Books for Multi-Core Software Developers is a great list of books on parallel programming that are recommended by Intel. The list includes:

• The Art of Concurrency - A Thread Monkey's Guide to Writing Parallel Applications, by Clay Breshears
• Intel® Threading Building Blocks: Outfitting C++ for Multi-core Processor Parallelism, by James Reinders
• Patterns for Parallel Programming, by Timothy G. Mattson, Beverly A. Sanders, Berna L. Massingill
• Principles of Parallel Programming, by Calvin Lin and Larry Snyder.
• Concurrent Programming on Windows, by Joe Duffy.
• Multi-Core Programming Increasing Performance through Software Multi-threading, by Shameem Akhter and Jason Roberts.
• Software Development for Embedded Multi-Core Systems, by Max Domeika
• Real World Haskell, by John Goerzen, Bryan O'Sullivan, Donald Bruce Stewart
• Using OpenMP Portable Shared Memory Parallel Programming, by Barbara Chapman, Gabriel Jost and Ruud Van Der Pas
• Programming with POSIX® Threads, by David R. Butenhof
• Win32 Multithreaded Programming, by Cohen and Woodring
• Java Concurrency in Practice, by Brian Goetz et al.
• Concurrent Programming in Java, by Douglas Lea
• Principles of Concurrent Programming, by M. Ben-Ari
• Principles of Concurrent and Distributed Programming (2nd edition), *by M. Ben-Ari
• Parallel Programming in C with MPI and OpenMP,* by Michael J. Quinn

It's available at http://softwarecommunity.intel.com/articles/eng/1567.htm with a short description of each book.

Answer 7

Consider the functional programming and unique software transactional memory combo that is Clojure; see online info and relevant chapters of Programming Clojure for more info. http://www.pragprog.com/titles/shcloj/programming-clojure

Answer 8

Try my book "The practice of parallel programming", available online at https://sourceforge.net/projects/tpopp/ . Though it doesn't go into the theory at all, it's all about the best practices. The printed version will be coming soon.