Graphics processing units have mostly been used by gamers to achieve amazing visual images on the screen. But this hardware has potential applications that don’t involve graphics at all. This is because GPUs are optimized for high frame rates which essentially requires a lot of operations to be done very fast. The math involved with data processing and technical computing is similar in regard to the huge number of operations necessary. So it makes sense to try to perform these kinds of tasks using GPUs.

The key difference between a GPU and a CPU is the extent to which parallelism is used. GPUs derive their performance gains from their effectiveness at running multiple threads simultaneously. This makes sense given the nature of their use in graphics, where different aspects of an image may be unrelated to one another. For instance, each character in a game might run on its own thread.

To extend the use of GPUs to scientific computing then, we need numerical methods that don’t rely too heavily on sequential calculations. Unfortunately, software developers are mostly familiar with algorithms that run on single threads. This is the main obstacle to effectively utilizing GPUs to solve problems in HPC. A different algorithmic approach is needed in many of the possible applications.

As an example, consider some of the computational techniques we’ve discussed in class so far. Many of them involve a succession of calculations that must be done in order. The bisection method, for instance, cannot perform an iteration without the results of the previous one. If we instead used algorithms that made use of parallelism, GPUs could possibly improve the speed with which we can obtain results in HPC.

Resources: http://www.ddj.com/hpc-high-performance-computing/206900471