Background: GPU & GPGPU
A GPU (Graphics Processing Unit) is hardware which accelerates rendering process in computer graphics. To support graphics algorithms such as transformation, rasterization, texture mapping, etc., GPU architecture is designed to be multithreaded and highly parallelized, and have large memory bandwidth. GPU architecture fully takes advantage of pipelining and parallelism. As computing power and memory bandwidth of GPU increase, GPGPU (General-Purpose GPU) approach is developed. GPGPU is used in scientific calculations, deep learning, HPC (High Performance Computing), and so on. There is another approach which integrates CPU and GPU. In this approach, GPUs no longer have their own memory or bus. This approach is called APU (Accelerated Processing Unit) by AMD, and Intel Corporation adopted similar strategy, too.