Occupancy calculations for CUDA kernels

http://developer.download.nvidia.com/compute/cuda/3_0/sdk/docs/CUDA_Occupancy_calculator.xls

Determining Registers Per Thread and Shared Memory Per Block

To determine the number of registers used per thread in your kernel, simply compile the kernel code using the option

--ptxas-options=-v

to nvcc. This will output information about register, local memory, shared memory, and constant memory usage for each kernel in the .cu file. Alternatively, you can compile with the -cubin option to nvcc. This will generate a .cubin file, which you can open in a text editor. Look for the code section with your kernel's name. Within the curly braces ({ ... }) for that code block, you will see a line with reg = X, where x is the number of registers used by your kernel. You can also see the amount of shared memory used as smem = Y. However, if your kernel declares any external shared memory that is allocated dynamically, you will need to add the number in the .cubin file to the amount you dynamically allocate at run time to get the correct shared memory usage.

Notes About Occupancy

Higher occupancy does not necessarily mean higher performance. If a kernel is not bandwidth bound, then increasing occupancy will not necessarily increase performance. If a kernel invocation is already running at least one thread block per multiprocessor in the GPU, and it is bottlenecked by computation and not by global memory accesses, then increasing occupancy may have no effect. In fact, making changes just to increase occupancy can have other effects, such as additional instructions, spills to local memory (which is off chip), divergent branches, etc. As with any optimization, you should experiment to see how changes affect the *wall clock time* of the kernel execution. For bandwidth bound applications, on the other hand, increasing occupancy can help better hide the latency of memory accesses, and therefore improve performance.