Assignment 5 - Numba

Overview

The objective of this assignment is to gain hands-on experience with new CUDA programming paradigms. This assignment will build off an example reduction implementation in Numba and extend it using CUDA Streams features.

Assignment Objectives

The objective of this assignment is for you to:

• Gain experience with Numba, which enables you to write parallel GPU algorithms entirely from Python.
• Gain experience with using CUDA Streams and how to break a problem up across multiple streams.
• Gain experience with Python Unittest

Github Classroom: https://classroom.github.com/a/6wfpcBqf

Numba Environment in Bender

On Bender, Numba currently is not installed in the base system. Therefore, to use Numba, we need to use a custom-built Apptainer environment that contain the necessary libraries and dependencies required to run Numba. To do so, we have a custom Apptainer image that needs to be loaded by running the following command:

apptainer shell --nv /singularity/cs217/cs217.sif

Once you're in this Apptainer image, you should have all the necessary libraries and dependencies to run Numba with CUDA support.

The expected output of the starter code is:

Apptainer> python numba_reduction.py 
....
----------------------------------------------------------------------
Ran 4 tests in 2.300s

OK

Code walkthrough

For this lab, we will be using Github Classroom. Please join the classroom by clicking the following link: https://classroom.github.com/a/6wfpcBqf Once you join the classroom, a private github repository will automatically be created with the starter code. Simply git clone to copy the starter code to Bender.

The Numba code is based on the sample shared memory reduction in the Numba documentation https://numba.readthedocs.io/en/stable/cuda/examples.html#shared-memory-reduction. This link walks through what the reduction implementation is doing. Note that the documentation's default implementation only uses a single block, so the starter that that you're provided was modified to run with multiple thread blocks.

For this assignment, your modifications will mainly be in the ex_reduction(self, insize) function, which acts similar to the "host code" in our previous assignments, and the array_sum(data, psum) function, which acts similar to the "kernel code" that runs on the GPU.

Unittests

This code makes use of unit testing features in Python. Specifically, the following code block checks to see if the output of your CUDA kernel matches the expected results, for each thread block:

        for i in range(nblocks):
           np.testing.assert_equal(b[i], sum(np.arange(i*nthreads,(i+1)*nthreads)))

Also, the following code blocks test your implementation against different input sizes:

    def test_reduction_256(self):
        self.ex_reduction(256)

    def test_reduction_1024(self):
        self.ex_reduction(1024)

    def test_reduction_10240(self):
        self.ex_reduction(10240)

    def test_reduction_20480(self):
        self.ex_reduction(20480)

    #def test_reduction_10241(self):
    #    self.ex_reduction(10241)

Note that the current implementation only supports input sizes which are multiples of the thread block size. The last test case, which is commented out, would result in an error because the current implementation would be incorrect.

Assignment Requirements

As part of this assignment, your goal is to modify the existing Numba Reduction implementation to utilize streams. The documentation of the Numba Streams APIs are found here: https://numba.readthedocs.io/en/stable/cuda/memory.html#streams and https://numba.readthedocs.io/en/stable/cuda-reference/host.html#stream-management.

1. The goal of your modifications should be to partition the input between streams so that kernels running in different streams can concurrently perform reduction. How your partition the input size is up to you, but the minimum requirement is that your code should perform reduction across a minimum of 2 streams.
   For an example of how Vector Add can be partitioned across Streams, refer to our CUDA Streams lecture.

2. The Unit test cases (specifically the np.testing.assert_equal) may need to be modified so that it can validate your results not just across thread blocks, but the concurrent kernels across streams as well.

Important note

Submission

1. Commit and push your completed Numba Streamed Reduction code to the Github repository. (You only need to modify kernel.cu.)
2. For this assignment, there will be no questions. Just the working code and associated unit tests.

Bonus points

If you find this assignment a bit too easy, I will give bonus points if you can generalize the implementation to support input sizes that are not multiples of the thread block size, along with additional unit test cases to test the generalization.