The key point to write parallel programs is to treat the program as a set of many related or unrelated sub-tasks. The process that the program is divided into some independent sub-tasks and their relationship is found out is called decomposition. There are usually 3 methods to decompose a program.
1. task decomposition
Programmers have to break the problem into a set of independent sub-tasks and classify them into different kinds according to the problem itself. Then attemper them in a right order so that they won't conflict with each other.
2. data decomposition (also called data-level parallelism)
It decompose the program according to the data that those tasks deals with. A typical application is to divide the data into n parts, each of which is processed by a thread.
3. data flow decomposition
What this method cares is that how data flows among the different tasks. A typical problem is producer/consumer.