# Debugging a simple OpenMP application

## Related Videos
[Stepping Through Your Code with Linaro DDT](https://www.youtube.com/watch?v=I3oswXFO5GI)

----

## Outline

1. The Example: A Simple OpenMP application
2. Compiling the simple OpenMP Application
3. Launching the application to Debug with Linaro DDT
4. Controlling the OpenMP Threads with Linaro DDT
6. Inspecting Variables for Race Conditions
7. Verify the correctness of the OpenMP application

----

## 1. The Example: A Simple OpenMP application

The application we will be debugging is a classic SAXPY (Single-precision A dot X Plus Y) kernel, which is easily parallelized using OpenMP. The primary issue is that serialization is occurring by placing the `#pragma omp parallel for` on the outer repetition loop instead of the inner calculation loop. This forces only 1 thread to do the calculation, leaving all other threads idle

----

## 2. Compiling the simple OpenMP Application

To successfully debug parallel code, it is critical to compile the application with debugging symbols and without optimization, as optimization can cause the execution flow to look different from the source code.

Navigate to the examples directory:

> ```bash
> cd linaro-ddt/examples

Compile the application using a compiler that supports OpenMP (like GCC) and the necessary flags:

> ```bash
> gcc -fopenmp -O0 -g saxpy_omp.c -o saxpy_omp

----

## 3. Launching the Application to Debug with DDT

Before launching, you must set the number of threads for the OpenMP program using the OMP_NUM_THREADS environment variable. Set the number of OpenMP threads (e.g., 4 threads):
> ```bash
> export OMP_NUM_THREADS=4

Additional Environment variables need to be set to start non-mpi applications within DDT.
> ```bash
> export FORGE_MPI_INIT=main
> export FORGE_HOLD_MPI_INIT=1

Launch the application with DDT by prepending ddt to your run command:
> ```bash
> ddt --connect mpiexec -n 1 ./saxpy_omp

Run Dialog: DDT will launch the Run Dialog. Review and confirm the settings (Check that the Number of OpenMP threads is set to 4.

Launch: Click the Run button.

DDT will launch the single process and its 4 threads, immediately halting execution, allowing you to begin debugging from a known state.

----

## 4. Controlling the OpenMP Threads with Linaro DDT

Once connected, the DDT interface allows you to manage and inspect the behavior of all OpenMP threads simultaneously or individually.

Set a breakpoint inside your parallel loop (line 26 of `saxpy_omp.c`) and hit "Play".

**Thread Control**

The key difference from MPI debugging is using the Threads tab in the Process/Thread Grouping Panel.

You can click on any thread (e.g., Thread 1, Thread 2, etc.) in the list to switch the main source code view and the Variables pane to that thread's local execution context.

![image_alt](screenshots/threads.png)

**Controlling Execution**

The Run/Continue, Pause and Step controls will operate on the currently selected thread, or all threads simultaneously if the Step Threads Together toggle is active.

----

## 5. Inspecting Variables for Race Conditions

Select the "Locals" Tab:

inspect the variable `z[i]`

----

## 6. Fixing the Race Condition

What happens: OpenMP splits the iter loop iterations among threads.

Thread 0 calculates z[0] through z[400].
Thread 1 also calculates z[0] through z[400].
Thread 2 also calculates z[0] through z[400].
Thread 3 also calculates z[0] through z[400].

This is because the `#pragma omp parallel for` directive is placed on the outer loop (iter), not the inner loop (i).

> ```C
>  // BAD: Parallelizing the repetition loop
>  #pragma omp parallel for
>  for (int iter = 0; iter < r; iter++) {
>      for (int i = 0; i < n; i++) {
>          z[i] = a * x[i] + y[i]; // <--- DATA RACE
>      }
>  }

The Race: Multiple threads are writing to the exact same memory locations (z[i]) at the same time. This is a "Write-Write" race condition.

**Why does it say "PASSED CHECK"**

You might be surprised to see the code prints PASSED CHECK.

Identical Values: Because every iteration of iter performs the exact same math (a * x + y), Thread 0 and Thread 1 are fighting to write the same value to z[i].

Outcome: The final value in memory happens to be correct because the threads agree on the value, even though they are stepping on each other's toes to write it.

**Fix**

> ```C
>// We create the thread pool ONCE outside the repetition loop
>  #pragma omp parallel
>  {
>      for (int iter = 0; iter < r; iter++) {
>          // We distribute the WORK (inner loop) among the threads
>          #pragma omp for
>          for (int i = 0; i < n; i++) {
>              z[i] = a * x[i] + y[i];
>          }
>          // Implicit barrier here ensures all threads finish an iter before starting the next
>      }
>  }

----

## 7. Verify the correctness of the OpenMP application
        Run to parallel region
        Step through each Thread
        Check that each thread calculates a * x[i] + y[i]
        Check that z[i] updates after each thread completes the parallel region.