Patterns for Concurrency over Concurrent Language Features
The Multi-core Problem
I continue to see lots of posts and articles about multicore and rightly so, multicore CPUs present some real challenges for the software industry. I think by now most people get it: by default most software is written in a way that can only make use of one CPU. If I buy a new faster CPU that code will run faster, but if I buy an additional CPU or a multi-core CPU then that code will not run faster. This is because it will use only one of the available CPUs, the rest will sit idle, in fact it might even run slower as the individual cores in a multi-core CPU often run slower than a single core CPU.
At the moment this problem is masked to some degree, if you've a 2 CPU machine you probably run enough pieces of software at the same time such that, along with the operating system, both cores are kept reasonably busy. When we have 8, 16 or 32 cores the problem will be a lot more evident. I'm guessing the issue will really surface when large organizations start to upgrade their desktop machines and find out that instead of running faster, as has always been the case before, the software on those machines actually runs no faster or even slower. Nor a good return on their investment.
Evolution or Revolution
A lot of people are suggesting ways in which we get around this problem, the approaches advocated seem to fit broadly in two categories.
Evolutionary - We need to learn how to use existing threading libraries and features properly from within existing programming languages.
Revolutionary - We need new to start using different programming languages to write our code (i.e. Erlang)
What surprises me is the concentration on programming languages and language level features, I believe both approaches will fail unless we look at the architecture and patterns we use to deliver software.
Evolutionary
Lets look first at the evolutionary approach. Many people are recommending we learn how to use threads, locks, semaphores et al correctly but this is no use if you just try and bolt multithreading onto patterns you would choose for a non-concurrent design. All to often everything is designed and written ignoring the need for concurrency with the hope the bottlenecks can be tackling by adding a 'little bit' of concurrency later.
To give a concrete example consider the observer pattern, all too often this is implemented in a way that means the order that the observers are invoked is fixed. A common approach to 'bolting on' concurrency to a non-concurrent design is to service each observer on a different thread. In fact this often looks likes its working when tested on a single core machine (where the order is deterministic) but soon becomes a source of bugs once it's running on a multi-core machine and the order that the observers are called can change.
Another example is taking work from a queue, why not just have multiple threads consuming from the queue? If you've designed for this you'll be fine, but often that does not happen and it turns out that when the order items are taken from the queue is changed the outcome changes i.e the cancellation of an order overtaking the creation of that order. So the cancel is processed first and discarded (the order does not exist yet!) and then the order creation arrives.....
Concurrency is also something that is hard to compartmentalise and isolate to one part of software, the impact of concurrency tends to be pervasive. So you need to design for multi-core from the beginning and choose an architecture and patterns that create a suitable abstraction around the concurrency required for your problem domain. You may then find someone has already created or documented approaches and patterns that meet your need and you never actually need to start creating your own threads, locks, etc. This is where choosing from existing concurrency libraries comes in.
Revolutionary
So what about the revolutionary approach? I think here it's even clearer we need to change the patterns and architecture we use. I really hope people don't think that by just coding in Erlang, say, that software will magically scale to multiple cores. What's different about Erlang is that it has language features that provide really great support for a particular architectural approach. If you ignore those features and just write sequential code and design the solution as you always have you should not be surprised when it doesn't scale to many cores.
Patterns
So the evolutionary and revolutionary approaches are both valid if we adjust the architectures and the patterns we use to take account of concurrency. If we do that we'll create software that can better scale to multiple cores. If we try and ignore concurrency when making design decisions and instead expect languages or language features to just solve the problem for us we'll be no better off than we are today.
There is a reason we don't normally write assembly code directly, we have much higher level abstractions and languages. Using threads directly is not that dissimilar to assembly language, it directly exposes the underlying hardware level implementation. Therefore we should not be surprised that using threads directly is difficult and often unproductive, we need higher level abstractions for concurrency. Some languages come with features that make using certain patterns for concurrency much easier to implement than others, Erlang is the obvious example. Other languages have libraries that provide implementations of those same patterns, for example Retlang. Other people have written code that scales incredibly well across multiple CPUs in C and C++ as well. It's not about the language, it's about having a design that takes account of concurrency and then choosing patterns that provide suitable abstractions around it.
Conclusion
When it comes to software we have multiple patterns available and we try and choose the best ones for the problem at hand. We need to develop and use similar patterns to help use with concurrency and in fact many exist already. Many of the patterns described by http://www.enterpriseintegrationpatterns.com/ play equally well at the software level as they do the message level, many patterns in EDA (i.e. Event Collaboration Patterns) also lend themselves to concurrent solutions. There are plenty of other examples.
Its here at the level of patterns we should be focusing our efforts around solving the multi-core problem and not at the language level.
I continue to see lots of posts and articles about multicore and rightly so, multicore CPUs present some real challenges for the software industry. I think by now most people get it: by default most software is written in a way that can only make use of one CPU. If I buy a new faster CPU that code will run faster, but if I buy an additional CPU or a multi-core CPU then that code will not run faster. This is because it will use only one of the available CPUs, the rest will sit idle, in fact it might even run slower as the individual cores in a multi-core CPU often run slower than a single core CPU.
At the moment this problem is masked to some degree, if you've a 2 CPU machine you probably run enough pieces of software at the same time such that, along with the operating system, both cores are kept reasonably busy. When we have 8, 16 or 32 cores the problem will be a lot more evident. I'm guessing the issue will really surface when large organizations start to upgrade their desktop machines and find out that instead of running faster, as has always been the case before, the software on those machines actually runs no faster or even slower. Nor a good return on their investment.
Evolution or Revolution
A lot of people are suggesting ways in which we get around this problem, the approaches advocated seem to fit broadly in two categories.
Evolutionary - We need to learn how to use existing threading libraries and features properly from within existing programming languages.
Revolutionary - We need new to start using different programming languages to write our code (i.e. Erlang)
What surprises me is the concentration on programming languages and language level features, I believe both approaches will fail unless we look at the architecture and patterns we use to deliver software.
Evolutionary
Lets look first at the evolutionary approach. Many people are recommending we learn how to use threads, locks, semaphores et al correctly but this is no use if you just try and bolt multithreading onto patterns you would choose for a non-concurrent design. All to often everything is designed and written ignoring the need for concurrency with the hope the bottlenecks can be tackling by adding a 'little bit' of concurrency later.
To give a concrete example consider the observer pattern, all too often this is implemented in a way that means the order that the observers are invoked is fixed. A common approach to 'bolting on' concurrency to a non-concurrent design is to service each observer on a different thread. In fact this often looks likes its working when tested on a single core machine (where the order is deterministic) but soon becomes a source of bugs once it's running on a multi-core machine and the order that the observers are called can change.
Another example is taking work from a queue, why not just have multiple threads consuming from the queue? If you've designed for this you'll be fine, but often that does not happen and it turns out that when the order items are taken from the queue is changed the outcome changes i.e the cancellation of an order overtaking the creation of that order. So the cancel is processed first and discarded (the order does not exist yet!) and then the order creation arrives.....
Concurrency is also something that is hard to compartmentalise and isolate to one part of software, the impact of concurrency tends to be pervasive. So you need to design for multi-core from the beginning and choose an architecture and patterns that create a suitable abstraction around the concurrency required for your problem domain. You may then find someone has already created or documented approaches and patterns that meet your need and you never actually need to start creating your own threads, locks, etc. This is where choosing from existing concurrency libraries comes in.
Revolutionary
So what about the revolutionary approach? I think here it's even clearer we need to change the patterns and architecture we use. I really hope people don't think that by just coding in Erlang, say, that software will magically scale to multiple cores. What's different about Erlang is that it has language features that provide really great support for a particular architectural approach. If you ignore those features and just write sequential code and design the solution as you always have you should not be surprised when it doesn't scale to many cores.
Patterns
So the evolutionary and revolutionary approaches are both valid if we adjust the architectures and the patterns we use to take account of concurrency. If we do that we'll create software that can better scale to multiple cores. If we try and ignore concurrency when making design decisions and instead expect languages or language features to just solve the problem for us we'll be no better off than we are today.
There is a reason we don't normally write assembly code directly, we have much higher level abstractions and languages. Using threads directly is not that dissimilar to assembly language, it directly exposes the underlying hardware level implementation. Therefore we should not be surprised that using threads directly is difficult and often unproductive, we need higher level abstractions for concurrency. Some languages come with features that make using certain patterns for concurrency much easier to implement than others, Erlang is the obvious example. Other languages have libraries that provide implementations of those same patterns, for example Retlang. Other people have written code that scales incredibly well across multiple CPUs in C and C++ as well. It's not about the language, it's about having a design that takes account of concurrency and then choosing patterns that provide suitable abstractions around it.
Conclusion
When it comes to software we have multiple patterns available and we try and choose the best ones for the problem at hand. We need to develop and use similar patterns to help use with concurrency and in fact many exist already. Many of the patterns described by http://www.enterpriseintegrationpatterns.com/ play equally well at the software level as they do the message level, many patterns in EDA (i.e. Event Collaboration Patterns) also lend themselves to concurrent solutions. There are plenty of other examples.
Its here at the level of patterns we should be focusing our efforts around solving the multi-core problem and not at the language level.
posted by Ian Cartwright at 1:07 PM
1 Comments:
Ian,
Intersting article, thank you.
My hunch is that the multicore programming paradigm that'll prevail will be one which allows an evolutionary approach. Adopting a new language feels like a huge amount of "activation energy" to overcome.
I'd argue that what's needed is a sufficiently smooth path to multicore for all the legacy apps out there. Sure, folks will develop new apps, but for an organization that has a 5 million line C++ app, built over 10 years, it is a competitive imperative to move THAT to multicore, and soon.
For what it's worth, one resource we've recently created at Cilk Arts is an e-Book, "How to Survive the Multicore Software Revolution (Or At Least Survive the Hype)"
http://www.cilk.com/multicore-e-book/
Cheers,
ilya
Post a Comment
<< Home