<h3>Contribution description</h3>
<p>This PR extends the IoT-LAB tooling support with the remote debugging facility provided by the testbed. The tools are wrapped inside the <code>debug</code> and <code>debug-server</code> make targets when the IOTLAB_NODE variable is used.</p>
<p>To start/stop the GDB server the nodes of an experiment, IoT-LAB cli-tools provide the <code>iotlab-node --debug-start</code> and <code>iotlab-node --debug-stop</code> commands and the GDB server can be reached on port 3333, either directly from the SSH frontend or via an SSH tunnel from the local computer.<br>
This PR supports both use case.</p>
<p>All boards exceptt firefly, zigduino and iotlab-a8-m3 can use this feature.</p>

<h3>Testing procedure</h3>
<ul>
<li>Start an experiment on IoT-LAB:</li>
</ul>
<div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="$ iotlab-experiment submit -d 120 -l saclay,m3,2 -l saclay,nrf52dk,1 -l saclay,st-lrwan1,2
$ iotlab-experiment wait
"><pre><code>$ iotlab-experiment submit -d 120 -l saclay,m3,2 -l saclay,nrf52dk,1 -l saclay,st-lrwan1,2
$ iotlab-experiment wait
</code></pre></div>
<ul>
<li>Start a debug session from your local computer:</li>
</ul>
<div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="$ BUILD_IN_DOCKER=1 make BOARD=nrf52dk -C examples/hello-world flash debug IOTLAB_NODE=auto --no-print-directory 
Launching build container using image "riot/riotbuild:latest".
docker run --rm --tty --user $(id -u) -v '/usr/share/zoneinfo/Europe/Paris:/etc/localtime:ro' -v '/work/riot/RIOT:/data/riotbuild/riotbase:delegated' -e 'RIOTBASE=/data/riotbuild/riotbase' -e 'CCACHE_BASEDIR=/data/riotbuild/riotbase' -e 'BUILD_DIR=/data/riotbuild/riotbase/build' -e 'RIOTPROJECT=/data/riotbuild/riotbase' -e 'RIOTCPU=/data/riotbuild/riotbase/cpu' -e 'RIOTBOARD=/data/riotbuild/riotbase/boards' -e 'RIOTMAKE=/data/riotbuild/riotbase/makefiles'      -e 'BOARD=nrf52dk'  -w '/data/riotbuild/riotbase/examples/hello-world/' 'riot/riotbuild:latest' make 'BOARD=nrf52dk'    
Building application "hello-world" for "nrf52dk" with MCU "nrf52".

"make" -C /data/riotbuild/riotbase/boards/nrf52dk
"make" -C /data/riotbuild/riotbase/boards/common/nrf52xxxdk
"make" -C /data/riotbuild/riotbase/core
"make" -C /data/riotbuild/riotbase/cpu/nrf52
"make" -C /data/riotbuild/riotbase/cpu/cortexm_common
"make" -C /data/riotbuild/riotbase/cpu/cortexm_common/periph
"make" -C /data/riotbuild/riotbase/cpu/nrf52/periph
"make" -C /data/riotbuild/riotbase/cpu/nrf52/vectors
"make" -C /data/riotbuild/riotbase/cpu/nrf5x_common
"make" -C /data/riotbuild/riotbase/cpu/nrf5x_common/periph
"make" -C /data/riotbuild/riotbase/drivers
"make" -C /data/riotbuild/riotbase/drivers/periph_common
"make" -C /data/riotbuild/riotbase/sys
"make" -C /data/riotbuild/riotbase/sys/auto_init
"make" -C /data/riotbuild/riotbase/sys/malloc_thread_safe
"make" -C /data/riotbuild/riotbase/sys/newlib_syscalls_default
"make" -C /data/riotbuild/riotbase/sys/stdio_uart
   text    data     bss     dec     hex filename
   8380     108    2320   10808    2a38 /data/riotbuild/riotbase/examples/hello-world/bin/nrf52dk/hello-world.elf
iotlab-node --jmespath='keys(@)[0]' --format='lambda ret: exit(int(ret))'  --list saclay,nrf52dk,1 --flash /work/riot/RIOT/examples/hello-world/bin/nrf52dk/hello-world.bin
/work/riot/RIOT/dist/testbed-support/iotlab-debug.sh "nrf52dk-1" "--list saclay,nrf52dk,1" "abadie@saclay.iot-lab.info" "" "/work/riot/RIOT/examples/hello-world/bin/nrf52dk/hello-world.elf"
Reading symbols from /work/riot/RIOT/examples/hello-world/bin/nrf52dk/hello-world.elf...
Remote debugging using localhost:3333
cortexm_init_fpu () at /data/riotbuild/riotbase/cpu/cortexm_common/include/cpu.h:104
104     /data/riotbuild/riotbase/cpu/cortexm_common/include/cpu.h: No such file or directory.
(gdb) load
Loading section .text, size 0x20bc lma 0x0
Loading section .relocate, size 0x6c lma 0x20bc
Start address 0x00000670, load size 8488
Transfer rate: 10 KB/sec, 4244 bytes/write.
(gdb) b main
Breakpoint 1 at 0xdc: file /data/riotbuild/riotbase/examples/hello-world/main.c, line 26.
Note: automatically using hardware breakpoints for read-only addresses.
(gdb) c
Continuing.

Breakpoint 1, main () at /data/riotbuild/riotbase/examples/hello-world/main.c:26
26          puts("Hello World!");
(gdb) l
21      
22      #include <stdio.h>
23      
24      int main(void)
25      {
26          puts("Hello World!");
27      
28          printf("You are running RIOT on a(n) %s board.\n", RIOT_BOARD);
29          printf("This board features a(n) %s MCU.\n", RIOT_MCU);
30      
quit) 
A debugging session is active.

        Inferior 1 [Remote target] will be detached.

EOF [assumed Y] or n) 
Detaching from program: /work/riot/RIOT/examples/hello-world/bin/nrf52dk/hello-world.elf, Remote target
Ending remote debugging.
[Inferior 1 (Remote target) detached]
"><pre><code>$ BUILD_IN_DOCKER=1 make BOARD=nrf52dk -C examples/hello-world flash debug IOTLAB_NODE=auto --no-print-directory 
Launching build container using image "riot/riotbuild:latest".
docker run --rm --tty --user $(id -u) -v '/usr/share/zoneinfo/Europe/Paris:/etc/localtime:ro' -v '/work/riot/RIOT:/data/riotbuild/riotbase:delegated' -e 'RIOTBASE=/data/riotbuild/riotbase' -e 'CCACHE_BASEDIR=/data/riotbuild/riotbase' -e 'BUILD_DIR=/data/riotbuild/riotbase/build' -e 'RIOTPROJECT=/data/riotbuild/riotbase' -e 'RIOTCPU=/data/riotbuild/riotbase/cpu' -e 'RIOTBOARD=/data/riotbuild/riotbase/boards' -e 'RIOTMAKE=/data/riotbuild/riotbase/makefiles'      -e 'BOARD=nrf52dk'  -w '/data/riotbuild/riotbase/examples/hello-world/' 'riot/riotbuild:latest' make 'BOARD=nrf52dk'    
Building application "hello-world" for "nrf52dk" with MCU "nrf52".

"make" -C /data/riotbuild/riotbase/boards/nrf52dk
"make" -C /data/riotbuild/riotbase/boards/common/nrf52xxxdk
"make" -C /data/riotbuild/riotbase/core
"make" -C /data/riotbuild/riotbase/cpu/nrf52
"make" -C /data/riotbuild/riotbase/cpu/cortexm_common
"make" -C /data/riotbuild/riotbase/cpu/cortexm_common/periph
"make" -C /data/riotbuild/riotbase/cpu/nrf52/periph
"make" -C /data/riotbuild/riotbase/cpu/nrf52/vectors
"make" -C /data/riotbuild/riotbase/cpu/nrf5x_common
"make" -C /data/riotbuild/riotbase/cpu/nrf5x_common/periph
"make" -C /data/riotbuild/riotbase/drivers
"make" -C /data/riotbuild/riotbase/drivers/periph_common
"make" -C /data/riotbuild/riotbase/sys
"make" -C /data/riotbuild/riotbase/sys/auto_init
"make" -C /data/riotbuild/riotbase/sys/malloc_thread_safe
"make" -C /data/riotbuild/riotbase/sys/newlib_syscalls_default
"make" -C /data/riotbuild/riotbase/sys/stdio_uart
   text    data     bss     dec     hex filename
   8380     108    2320   10808    2a38 /data/riotbuild/riotbase/examples/hello-world/bin/nrf52dk/hello-world.elf
iotlab-node --jmespath='keys(@)[0]' --format='lambda ret: exit(int(ret))'  --list saclay,nrf52dk,1 --flash /work/riot/RIOT/examples/hello-world/bin/nrf52dk/hello-world.bin
/work/riot/RIOT/dist/testbed-support/iotlab-debug.sh "nrf52dk-1" "--list saclay,nrf52dk,1" "abadie@saclay.iot-lab.info" "" "/work/riot/RIOT/examples/hello-world/bin/nrf52dk/hello-world.elf"
Reading symbols from /work/riot/RIOT/examples/hello-world/bin/nrf52dk/hello-world.elf...
Remote debugging using localhost:3333
cortexm_init_fpu () at /data/riotbuild/riotbase/cpu/cortexm_common/include/cpu.h:104
104     /data/riotbuild/riotbase/cpu/cortexm_common/include/cpu.h: No such file or directory.
(gdb) load
Loading section .text, size 0x20bc lma 0x0
Loading section .relocate, size 0x6c lma 0x20bc
Start address 0x00000670, load size 8488
Transfer rate: 10 KB/sec, 4244 bytes/write.
(gdb) b main
Breakpoint 1 at 0xdc: file /data/riotbuild/riotbase/examples/hello-world/main.c, line 26.
Note: automatically using hardware breakpoints for read-only addresses.
(gdb) c
Continuing.

Breakpoint 1, main () at /data/riotbuild/riotbase/examples/hello-world/main.c:26
26          puts("Hello World!");
(gdb) l
21      
22      #include <stdio.h>
23      
24      int main(void)
25      {
26          puts("Hello World!");
27      
28          printf("You are running RIOT on a(n) %s board.\n", RIOT_BOARD);
29          printf("This board features a(n) %s MCU.\n", RIOT_MCU);
30      
quit) 
A debugging session is active.

        Inferior 1 [Remote target] will be detached.

EOF [assumed Y] or n) 
Detaching from program: /work/riot/RIOT/examples/hello-world/bin/nrf52dk/hello-world.elf, Remote target
Ending remote debugging.
[Inferior 1 (Remote target) detached]
</code></pre></div>
<ul>
<li>After a debug session is finished, verify that the node can be reflashed (to verify that the GDB server is stopped):</li>
</ul>
<div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="$ BUILD_IN_DOCKER=1 make BOARD=nrf52dk -C examples/hello-world flash-only IOTLAB_NODE=auto --no-print-directory 
iotlab-node --jmespath='keys(@)[0]' --format='lambda ret: exit(int(ret))'  --list saclay,nrf52dk,1 --flash /work/riot/RIOT/examples/hello-world/bin/nrf52dk/hello-world.bin
$ echo $?
0
"><pre><code>$ BUILD_IN_DOCKER=1 make BOARD=nrf52dk -C examples/hello-world flash-only IOTLAB_NODE=auto --no-print-directory 
iotlab-node --jmespath='keys(@)[0]' --format='lambda ret: exit(int(ret))'  --list saclay,nrf52dk,1 --flash /work/riot/RIOT/examples/hello-world/bin/nrf52dk/hello-world.bin
$ echo $?
0
</code></pre></div>
<ul>
<li>Start a debug-server and in a second terminal, use gdb manually:</li>
</ul>
<div class="snippet-clipboard-content position-relative" data-snippet-clipboard-copy-content="$ make BOARD=nrf52dk -C examples/hello-world debug-server IOTLAB_NODE=auto --no-print-directory /work/riot/RIOT/dist/testbed-support/iotlab-debug.sh "nrf52dk-1" "--list saclay,nrf52dk,1" "abadie@saclay.iot-lab.info" "" "" "1"
...
$ arm-none-eabi-gdb /work/riot/RIOT/examples/hello-world/bin/nrf52dk/hello-world.elf
GNU gdb (GNU Tools for Arm Embedded Processors 9-2019-q4-major) 8.3.0.20190709-git
Copyright (C) 2019 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "--host=x86_64-linux-gnu --target=arm-none-eabi".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /work/riot/RIOT/examples/hello-world/bin/nrf52dk/hello-world.elf...
(gdb) target remote :3333
Remote debugging using :3333
cortexm_init_fpu () at /work/riot/RIOT/cpu/cortexm_common/include/cpu.h:104
104         SCB->CPACR |= (uint32_t)CORTEXM_SCB_CPACR_FPU_ACCESS_FULL;
(gdb) load
Loading section .text, size 0x20bc lma 0x0
Loading section .relocate, size 0x6c lma 0x20bc
Start address 0x670, load size 8488
Transfer rate: 10 KB/sec, 4244 bytes/write.
(gdb) b main
Breakpoint 1 at 0xdc: file /work/riot/RIOT/examples/hello-world/main.c, line 26.
(gdb) c
Continuing.
Note: automatically using hardware breakpoints for read-only addresses.

Breakpoint 1, main () at /work/riot/RIOT/examples/hello-world/main.c:26
26          puts("Hello World!");
(gdb) l
21      
22      #include <stdio.h>
23      
24      int main(void)
25      {
26          puts("Hello World!");
27      
28          printf("You are running RIOT on a(n) %s board.\n", RIOT_BOARD);
29          printf("This board features a(n) %s MCU.\n", RIOT_MCU);
30      
(gdb) quit
A debugging session is active.

        Inferior 1 [Remote target] will be detached.

Quit anyway? (y or n) EOF [assumed Y]
Detaching from program: /work/riot/RIOT/examples/hello-world/bin/nrf52dk/hello-world.elf, Remote target
Ending remote debugging.
[Inferior 1 (Remote target) detached]
"><pre><code>$ make BOARD=nrf52dk -C examples/hello-world debug-server IOTLAB_NODE=auto --no-print-directory /work/riot/RIOT/dist/testbed-support/iotlab-debug.sh "nrf52dk-1" "--list saclay,nrf52dk,1" "abadie@saclay.iot-lab.info" "" "" "1"
...
$ arm-none-eabi-gdb /work/riot/RIOT/examples/hello-world/bin/nrf52dk/hello-world.elf
GNU gdb (GNU Tools for Arm Embedded Processors 9-2019-q4-major) 8.3.0.20190709-git
Copyright (C) 2019 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "--host=x86_64-linux-gnu --target=arm-none-eabi".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /work/riot/RIOT/examples/hello-world/bin/nrf52dk/hello-world.elf...
(gdb) target remote :3333
Remote debugging using :3333
cortexm_init_fpu () at /work/riot/RIOT/cpu/cortexm_common/include/cpu.h:104
104         SCB->CPACR |= (uint32_t)CORTEXM_SCB_CPACR_FPU_ACCESS_FULL;
(gdb) load
Loading section .text, size 0x20bc lma 0x0
Loading section .relocate, size 0x6c lma 0x20bc
Start address 0x670, load size 8488
Transfer rate: 10 KB/sec, 4244 bytes/write.
(gdb) b main
Breakpoint 1 at 0xdc: file /work/riot/RIOT/examples/hello-world/main.c, line 26.
(gdb) c
Continuing.
Note: automatically using hardware breakpoints for read-only addresses.

Breakpoint 1, main () at /work/riot/RIOT/examples/hello-world/main.c:26
26          puts("Hello World!");
(gdb) l
21      
22      #include <stdio.h>
23      
24      int main(void)
25      {
26          puts("Hello World!");
27      
28          printf("You are running RIOT on a(n) %s board.\n", RIOT_BOARD);
29          printf("This board features a(n) %s MCU.\n", RIOT_MCU);
30      
(gdb) quit
A debugging session is active.

        Inferior 1 [Remote target] will be detached.

Quit anyway? (y or n) EOF [assumed Y]
Detaching from program: /work/riot/RIOT/examples/hello-world/bin/nrf52dk/hello-world.elf, Remote target
Ending remote debugging.
[Inferior 1 (Remote target) detached]
</code></pre></div>
<p>The same procedure also works from an IoT-LAB frontend (didn't try with <code>BUILD_IN_DOCKER=1</code> though).</p>

<h3>Issues/PRs references</h3>
<p>Tick the debugger item in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="347330836" data-permission-text="Title is private" data-url="https://github.com/RIOT-OS/RIOT/issues/9694" data-hovercard-type="issue" data-hovercard-url="/RIOT-OS/RIOT/issues/9694/hovercard" href="https://github.com/RIOT-OS/RIOT/issues/9694">#9694</a></p>


<hr>

<h4>You can view, comment on, or merge this pull request online at:</h4>
<p>  <a href='https://github.com/RIOT-OS/RIOT/pull/16510'>https://github.com/RIOT-OS/RIOT/pull/16510</a></p>

<h4>Commit Summary</h4>
<ul>
  <li>tools/iotlab-testbed: add support for remote debugging</li>
</ul>

<h4>File Changes</h4>
<ul>
  <li>
    <strong>A</strong>
    <a href="https://github.com/RIOT-OS/RIOT/pull/16510/files#diff-d9b3eed973f08de6f9dd22463c5fb74c13953ae03b0a94609eb554358d28458f">dist/testbed-support/iotlab-debug.sh</a>
    (67)
  </li>
  <li>
    <strong>M</strong>
    <a href="https://github.com/RIOT-OS/RIOT/pull/16510/files#diff-e6e875e6be025f3a7ba3b8464cfd2fbf8c87560c49b5469dbc5854a179730e34">dist/testbed-support/makefile.iotlab.single.inc.mk</a>
    (20)
  </li>
</ul>

<h4>Patch Links:</h4>
<ul>
  <li><a href='https://github.com/RIOT-OS/RIOT/pull/16510.patch'>https://github.com/RIOT-OS/RIOT/pull/16510.patch</a></li>
  <li><a href='https://github.com/RIOT-OS/RIOT/pull/16510.diff'>https://github.com/RIOT-OS/RIOT/pull/16510.diff</a></li>
</ul>

<p style="font-size:small;-webkit-text-size-adjust:none;color:#666;">—<br />You are receiving this because you are subscribed to this thread.<br />Reply to this email directly, <a href="https://github.com/RIOT-OS/RIOT/pull/16510">view it on GitHub</a>, or <a href="https://github.com/notifications/unsubscribe-auth/ABE7WYHHTFGHB4R5WEBVB5DTP7JKZANCNFSM45XEPSAA">unsubscribe</a>.<img src="https://github.com/notifications/beacon/ABE7WYGUZ2MKRXM6UCP3TXDTP7JKZA5CNFSM45XEPSAKYY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4NP5RTKQ.gif" height="1" width="1" alt="" /></p>
<script type="application/ld+json">[
{
"@context": "http://schema.org",
"@type": "EmailMessage",
"potentialAction": {
"@type": "ViewAction",
"target": "https://github.com/RIOT-OS/RIOT/pull/16510",
"url": "https://github.com/RIOT-OS/RIOT/pull/16510",
"name": "View Pull Request"
},
"description": "View this Pull Request on GitHub",
"publisher": {
"@type": "Organization",
"name": "GitHub",
"url": "https://github.com"
}
}
]</script>