Parallel sum reduction on GPUs in CUDA

We discuss 6 ways to implement sum reduction on GPUs in CUDA. Each implementation tries to improve on the previous implementation. Note that although we discuss parallel sum reduction here, the same logic can be applied to perform reduction using any sort of associative operation (eg: sum, max, min, multiplication)