[riot-notifications] [RIOT-OS/RIOT] make: add termdeps target (#11762)

Gaëtan Harter notifications at github.com
Thu Jul 4 11:56:21 CEST 2019


cladmi approved this pull request.

Approved the `TERMDEPS` addition to `TEST_DEPS`. It fixes running tests where the term dependencies take a long time.
As we cannot currently really test the in `murdock` part, I leave you the responsibility for this and I just test the sequence.


My local testing procedure is to define a `TERMDEPS` that takes 15 seconds on first call.

It can be done from the command line with
```
export TERMDEPS=info-debug-variable-LONG_TERM_DEPS
export LONG_TERM_DEPS='$$(test -f $(BINDIR)/termdeps || { echo termdeps; sleep 15; touch $(BINDIR)/termdeps; })'
```

Then I ran a `make clean flash test` procedure. (I assume no `-j` as I use `clean` to remove the `termdeps` instead of doing a manual removal).

<details><summary>Local success with <code>native</code> in the PR</summary>

```
RIOT_CI_BUILD=1 TERMDEPS=info-debug-variable-LONG_TERM_DEPS LONG_TERM_DEPS='$$(test -f $(BINDIR)/termdeps || { echo termdeps; sleep 15; touch $(BINDIR)/termdeps; })' make --no-print-directory -C tests/bloom_bytes/ clean flash test 
Building application "tests_bloom_bytes" for "native" with MCU "native".

   text    data     bss     dec     hex filename
  34573     648   48468   83689   146e9 /home/harter/work/git/RIOT/tests/bloom_bytes/bin/native/tests_bloom_bytes.elf
true 
termdeps

/home/harter/work/git/RIOT/tests/bloom_bytes/bin/native/tests_bloom_bytes.elf  
RIOT native interrupts/signals initialized.
LED_RED_OFF
LED_GREEN_ON
RIOT native board initialized.
RIOT native hardware initialization complete.

main(): This is RIOT! (Version: buildtest)
Testing Bloom filter.

m: 4096 k: 8

adding 512 elements took 2ms
checking 10000 elements took 26ms

267 elements probably in the filter.
9733 elements not in the filter.
0.026699 false positive rate.

All done!

```
</details>

<details><summary>Local success with <code>iotlab-m3</code> in the PR</summary>

```
RIOT_CI_BUILD=1 BOARD=iotlab-m3 TERMDEPS=info-debug-variable-LONG_TERM_DEPS LONG_TERM_DEPS='$$(test -f $(BINDIR)/termdeps || { echo termdeps; sleep 15; touch $(BINDIR)/termdeps; })' make --no-print-directory -C tests/bloom_bytes/ clean flash test 
Building application "tests_bloom_bytes" for "iotlab-m3" with MCU "stm32f1".

   text    data     bss     dec     hex filename
  14220     152    3352   17724    453c /home/harter/work/git/RIOT/tests/bloom_bytes/bin/iotlab-m3/tests_bloom_bytes.elf
/home/harter/work/git/RIOT/dist/tools/openocd/openocd.sh flash /home/harter/work/git/RIOT/tests/bloom_bytes/bin/iotlab-m3/tests_bloom_bytes.elf
### Flashing Target ###
GNU MCU Eclipse 64-bit Open On-Chip Debugger 0.10.0+dev-00462-gdd1d90111 (2019-01-18-11:37)
Licensed under GNU GPL v2
For bug reports, read
        http://openocd.org/doc/doxygen/bugs.html
Info : auto-selecting first available session transport "jtag". To override use 'transport select <transport>'.
adapter speed: 1000 kHz
adapter_nsrst_delay: 100
jtag_ntrst_delay: 100
none separate
cortex_m reset_config sysresetreq
trst_and_srst separate srst_nogate trst_push_pull srst_open_drain connect_assert_srst
Info : clock speed 1000 kHz
Info : JTAG tap: stm32f1x.cpu tap/device found: 0x3ba00477 (mfg: 0x23b (ARM Ltd.), part: 0xba00, ver: 0x3)
Info : JTAG tap: stm32f1x.bs tap/device found: 0x06414041 (mfg: 0x020 (STMicroelectronics), part: 0x6414, ver: 0x0)
Info : stm32f1x.cpu: hardware has 6 breakpoints, 4 watchpoints
Info : Listening on port 42759 for gdb connections
    TargetName         Type       Endian TapName            State       
--  ------------------ ---------- ------ ------------------ ------------
 0* stm32f1x.cpu       cortex_m   little stm32f1x.cpu       reset
Info : JTAG tap: stm32f1x.cpu tap/device found: 0x3ba00477 (mfg: 0x23b (ARM Ltd.), part: 0xba00, ver: 0x3)
Info : JTAG tap: stm32f1x.bs tap/device found: 0x06414041 (mfg: 0x020 (STMicroelectronics), part: 0x6414, ver: 0x0)
target halted due to debug-request, current mode: Thread 
xPSR: 0x01000000 pc: 0x08000460 msp: 0x20000200
auto erase enabled
Info : device id = 0x10016414
Info : flash size = 512kbytes
wrote 16384 bytes from file /home/harter/work/git/RIOT/tests/bloom_bytes/bin/iotlab-m3/tests_bloom_bytes.elf in 0.727082s (22.006 KiB/s)
verified 14372 bytes in 0.236501s (59.345 KiB/s)
Info : JTAG tap: stm32f1x.cpu tap/device found: 0x3ba00477 (mfg: 0x23b (ARM Ltd.), part: 0xba00, ver: 0x3)
Info : JTAG tap: stm32f1x.bs tap/device found: 0x06414041 (mfg: 0x020 (STMicroelectronics), part: 0x6414, ver: 0x0)
shutdown command invoked
Done flashing
termdeps

/home/harter/work/git/RIOT/dist/tools/pyterm/pyterm -p "/dev/ttyUSB1" -b "500000"
Twisted not available, please install it if you want to use pyterm's JSON capabilities
2019-07-04 11:46:54,319 - INFO # Connect to serial port /dev/ttyUSB1
Welcome to pyterm!
Type '/exit' to exit.
2019-07-04 11:46:57,765 - INFO # �main(): This is RIOT! (Version: buildtest)
2019-07-04 11:46:57,766 - INFO # Testing Bloom filter.
2019-07-04 11:46:57,766 - INFO # 
2019-07-04 11:46:57,767 - INFO # m: 4096 k: 8
2019-07-04 11:46:57,768 - INFO # 
2019-07-04 11:46:57,940 - INFO # adding 512 elements took 182ms
2019-07-04 11:46:59,444 - INFO # checking 10000 elements took 1494ms
2019-07-04 11:46:59,445 - INFO # 
2019-07-04 11:46:59,447 - INFO # 267 elements probably in the filter.
2019-07-04 11:46:59,448 - INFO # 9733 elements not in the filter.
2019-07-04 11:46:59,450 - INFO # 0.026700 false positive rate.
2019-07-04 11:46:59,450 - INFO # 
2019-07-04 11:46:59,451 - INFO # All done!
```
</details>


>From master, it fails as the `TERM_DEPS` is done during the test:

<details><summary>Failure with <code>native</code> in master</summary>

```
RIOT_CI_BUILD=1 TERMDEPS=info-debug-variable-LONG_TERM_DEPS LONG_TERM_DEPS='$$(test -f $(BINDIR)/termdeps || { echo termdeps; sleep 15; touch $(BINDIR)/termdeps; })' make --no-print-directory -C tests/bloom_bytes/ clean flash test 
Building application "tests_bloom_bytes" for "native" with MCU "native".

   text    data     bss     dec     hex filename
  34573     648   48468   83689   146e9 /home/harter/work/git/RIOT/tests/bloom_bytes/bin/native/tests_bloom_bytes.elf
true 
Timeout in expect script at "child.expect_exact("Testing Bloom filter.")" (tests/bloom_bytes/tests/01-run.py:17)

/home/harter/work/git/RIOT/tests/bloom_bytes/../../Makefile.include:592: recipe for target 'test' failed
```
</details>

<details><summary>Failure with <code>iotlab-m3</code> in master</summary>

```
RIOT_CI_BUILD=1 BOARD=iotlab-m3 TERMDEPS=info-debug-variable-LONG_TERM_DEPS LONG_TERM_DEPS='$$(test -f $(BINDIR)/termdeps || { echo termdeps; sleep 15; touch $(BINDIR)/termdeps; })' make --no-print-directory -C tests/bloom_bytes/ clean flash test 
Building application "tests_bloom_bytes" for "iotlab-m3" with MCU "stm32f1".

   text    data     bss     dec     hex filename
  14220     152    3352   17724    453c /home/harter/work/git/RIOT/tests/bloom_bytes/bin/iotlab-m3/tests_bloom_bytes.elf
/home/harter/work/git/RIOT/dist/tools/openocd/openocd.sh flash /home/harter/work/git/RIOT/tests/bloom_bytes/bin/iotlab-m3/tests_bloom_bytes.elf
### Flashing Target ###
GNU MCU Eclipse 64-bit Open On-Chip Debugger 0.10.0+dev-00462-gdd1d90111 (2019-01-18-11:37)
Licensed under GNU GPL v2
For bug reports, read
        http://openocd.org/doc/doxygen/bugs.html
Info : auto-selecting first available session transport "jtag". To override use 'transport select <transport>'.
adapter speed: 1000 kHz
adapter_nsrst_delay: 100
jtag_ntrst_delay: 100
none separate
cortex_m reset_config sysresetreq
trst_and_srst separate srst_nogate trst_push_pull srst_open_drain connect_assert_srst
Info : clock speed 1000 kHz
Info : JTAG tap: stm32f1x.cpu tap/device found: 0x3ba00477 (mfg: 0x23b (ARM Ltd.), part: 0xba00, ver: 0x3)
Info : JTAG tap: stm32f1x.bs tap/device found: 0x06414041 (mfg: 0x020 (STMicroelectronics), part: 0x6414, ver: 0x0)
Info : stm32f1x.cpu: hardware has 6 breakpoints, 4 watchpoints
Info : Listening on port 36213 for gdb connections
    TargetName         Type       Endian TapName            State       
--  ------------------ ---------- ------ ------------------ ------------
 0* stm32f1x.cpu       cortex_m   little stm32f1x.cpu       reset
Info : JTAG tap: stm32f1x.cpu tap/device found: 0x3ba00477 (mfg: 0x23b (ARM Ltd.), part: 0xba00, ver: 0x3)
Info : JTAG tap: stm32f1x.bs tap/device found: 0x06414041 (mfg: 0x020 (STMicroelectronics), part: 0x6414, ver: 0x0)
target halted due to debug-request, current mode: Thread 
xPSR: 0x01000000 pc: 0x08000460 msp: 0x20000200
auto erase enabled
Info : device id = 0x10016414
Info : flash size = 512kbytes
wrote 16384 bytes from file /home/harter/work/git/RIOT/tests/bloom_bytes/bin/iotlab-m3/tests_bloom_bytes.elf in 0.724787s (22.075 KiB/s)
verified 14372 bytes in 0.236192s (59.423 KiB/s)
Info : JTAG tap: stm32f1x.cpu tap/device found: 0x3ba00477 (mfg: 0x23b (ARM Ltd.), part: 0xba00, ver: 0x3)
Info : JTAG tap: stm32f1x.bs tap/device found: 0x06414041 (mfg: 0x020 (STMicroelectronics), part: 0x6414, ver: 0x0)
shutdown command invoked
Done flashing
Timeout in expect script at "child.expect_exact("Testing Bloom filter.")" (tests/bloom_bytes/tests/01-run.py:17)

/home/harter/work/git/RIOT/tests/bloom_bytes/../../Makefile.include:592: recipe for target 'test' failed
make: *** [test] Error 1
```
</details>



The test sequence done by the `.murdock` script


<details><summary>Separate steps with `native`</summary

```
RIOT_CI_BUILD=1 TERMDEPS=info-debug-variable-LONG_TERM_DEPS LONG_TERM_DEPS='$$(test -f $(BINDIR)/termdeps || { echo termdeps; sleep 15; touch $(BINDIR)/termdeps; })' make --no-print-directory -C tests/bloom_bytes/  clean


RIOT_CI_BUILD=1 TERMDEPS=info-debug-variable-LONG_TERM_DEPS LONG_TERM_DEPS='$$(test -f $(BINDIR)/termdeps || { echo termdeps; sleep 15; touch $(BINDIR)/termdeps; })' make --no-print-directory -C tests/bloom_bytes/  all
Building application "tests_bloom_bytes" for "native" with MCU "native".

   text    data     bss     dec     hex filename
  34573     648   48468   83689   146e9 /home/harter/work/git/RIOT/tests/bloom_bytes/bin/native/tests_bloom_bytes.elf


RIOT_CI_BUILD=1 TERMDEPS=info-debug-variable-LONG_TERM_DEPS LONG_TERM_DEPS='$$(test -f $(BINDIR)/termdeps || { echo termdeps; sleep 15; touch $(BINDIR)/termdeps; })' make --no-print-directory -C tests/bloom_bytes/  -j2 flash-only termdeps
true 
termdeps


RIOT_CI_BUILD=1 TERMDEPS=info-debug-variable-LONG_TERM_DEPS LONG_TERM_DEPS='$$(test -f $(BINDIR)/termdeps || { echo termdeps; sleep 15; touch $(BINDIR)/termdeps; })' make --no-print-directory -C tests/bloom_bytes/  test


/home/harter/work/git/RIOT/tests/bloom_bytes/bin/native/tests_bloom_bytes.elf  
RIOT native interrupts/signals initialized.
LED_RED_OFF
LED_GREEN_ON
RIOT native board initialized.
RIOT native hardware initialization complete.

main(): This is RIOT! (Version: buildtest)
Testing Bloom filter.

m: 4096 k: 8

adding 512 elements took 3ms
checking 10000 elements took 27ms

267 elements probably in the filter.
9733 elements not in the filter.
0.026699 false positive rate.

All done!
```
</details>


<details><summary>Separate steps with `iotlab-m3`</summary>
```
BOARD=iotlab-m3 RIOT_CI_BUILD=1 TERMDEPS=info-debug-variable-LONG_TERM_DEPS LONG_TERM_DEPS='$$(test -f $(BINDIR)/termdeps || { echo termdeps; sleep 15; touch $(BINDIR)/termdeps; })' make --no-print-directory -C tests/bloom_bytes/ clean


BOARD=iotlab-m3 RIOT_CI_BUILD=1 TERMDEPS=info-debug-variable-LONG_TERM_DEPS LONG_TERM_DEPS='$$(test -f $(BINDIR)/termdeps || { echo termdeps; sleep 15; touch $(BINDIR)/termdeps; })' make --no-print-directory -C tests/bloom_bytes/  all
Building application "tests_bloom_bytes" for "iotlab-m3" with MCU "stm32f1".

   text    data     bss     dec     hex filename
  14220     152    3352   17724    453c /home/harter/work/git/RIOT/tests/bloom_bytes/bin/iotlab-m3/tests_bloom_bytes.elf


BOARD=iotlab-m3 RIOT_CI_BUILD=1 TERMDEPS=info-debug-variable-LONG_TERM_DEPS LONG_TERM_DEPS='$$(test -f $(BINDIR)/termdeps || { echo termdeps; sleep 15; touch $(BINDIR)/termdeps; })' make --no-print-directory -C tests/bloom_bytes/ -j2 flash-only termdeps
/home/harter/work/git/RIOT/dist/tools/openocd/openocd.sh flash /home/harter/work/git/RIOT/tests/bloom_bytes/bin/iotlab-m3/tests_bloom_bytes.elf
### Flashing Target ###
GNU MCU Eclipse 64-bit Open On-Chip Debugger 0.10.0+dev-00462-gdd1d90111 (2019-01-18-11:37)
Licensed under GNU GPL v2
For bug reports, read
        http://openocd.org/doc/doxygen/bugs.html
Info : auto-selecting first available session transport "jtag". To override use 'transport select <transport>'.
adapter speed: 1000 kHz
adapter_nsrst_delay: 100
jtag_ntrst_delay: 100
none separate
cortex_m reset_config sysresetreq
trst_and_srst separate srst_nogate trst_push_pull srst_open_drain connect_assert_srst
Info : clock speed 1000 kHz
Info : JTAG tap: stm32f1x.cpu tap/device found: 0x3ba00477 (mfg: 0x23b (ARM Ltd.), part: 0xba00, ver: 0x3)
Info : JTAG tap: stm32f1x.bs tap/device found: 0x06414041 (mfg: 0x020 (STMicroelectronics), part: 0x6414, ver: 0x0)
Info : stm32f1x.cpu: hardware has 6 breakpoints, 4 watchpoints
Info : Listening on port 38243 for gdb connections
    TargetName         Type       Endian TapName            State       
--  ------------------ ---------- ------ ------------------ ------------
 0* stm32f1x.cpu       cortex_m   little stm32f1x.cpu       reset
Info : JTAG tap: stm32f1x.cpu tap/device found: 0x3ba00477 (mfg: 0x23b (ARM Ltd.), part: 0xba00, ver: 0x3)
Info : JTAG tap: stm32f1x.bs tap/device found: 0x06414041 (mfg: 0x020 (STMicroelectronics), part: 0x6414, ver: 0x0)
target halted due to debug-request, current mode: Thread 
xPSR: 0x01000000 pc: 0x08000460 msp: 0x20000200
auto erase enabled
Info : device id = 0x10016414
Info : flash size = 512kbytes
wrote 16384 bytes from file /home/harter/work/git/RIOT/tests/bloom_bytes/bin/iotlab-m3/tests_bloom_bytes.elf in 0.726811s (22.014 KiB/s)
verified 14372 bytes in 0.236981s (59.225 KiB/s)
Info : JTAG tap: stm32f1x.cpu tap/device found: 0x3ba00477 (mfg: 0x23b (ARM Ltd.), part: 0xba00, ver: 0x3)
Info : JTAG tap: stm32f1x.bs tap/device found: 0x06414041 (mfg: 0x020 (STMicroelectronics), part: 0x6414, ver: 0x0)
shutdown command invoked
Done flashing
termdeps


BOARD=iotlab-m3 RIOT_CI_BUILD=1 TERMDEPS=info-debug-variable-LONG_TERM_DEPS LONG_TERM_DEPS='$$(test -f $(BINDIR)/termdeps || { echo termdeps; sleep 15; touch $(BINDIR)/termdeps; })' make --no-print-directory -C tests/bloom_bytes/ test


/home/harter/work/git/RIOT/dist/tools/pyterm/pyterm -p "/dev/ttyUSB1" -b "500000"
Twisted not available, please install it if you want to use pyterm's JSON capabilities
2019-07-04 11:54:55,065 - INFO # Connect to serial port /dev/ttyUSB1
Welcome to pyterm!
Type '/exit' to exit.
2019-07-04 11:54:58,573 - INFO # �main(): This is RIOT! (Version: buildtest)
2019-07-04 11:54:58,588 - INFO # Testing Bloom filter.
2019-07-04 11:54:58,589 - INFO # 
2019-07-04 11:54:58,589 - INFO # m: 4096 k: 8
2019-07-04 11:54:58,590 - INFO # 
2019-07-04 11:54:58,764 - INFO # adding 512 elements took 182ms
2019-07-04 11:55:00,253 - INFO # checking 10000 elements took 1494ms
2019-07-04 11:55:00,253 - INFO # 
2019-07-04 11:55:00,267 - INFO # 267 elements probably in the filter.
2019-07-04 11:55:00,269 - INFO # 9733 elements not in the filter.
2019-07-04 11:55:00,270 - INFO # 0.026700 false positive rate.
2019-07-04 11:55:00,270 - INFO # 
2019-07-04 11:55:00,271 - INFO # All done!
```
</details>




-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/RIOT-OS/RIOT/pull/11762#pullrequestreview-257948176
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.riot-os.org/pipermail/notifications/attachments/20190704/96bfebbc/attachment-0001.html>


More information about the notifications mailing list