[riot-notifications] [RIOT-OS/RIOT] tests/evtimer: blacklist testing on native (#12154)

Gaëtan Harter notifications at github.com
Mon Sep 2 16:43:30 CEST 2019


### Contribution description

The test randomly fails on `native` due to timers being not accurate but
it cannot be otherwise. So better disable it than raising fake errors.

    main(): This is RIOT! (Version: buildtest)
    Testing generic evtimer
    This should list 2 items
    ev #1 offset=1000
    ev #2 offset=500
    This should list 4 items
    ev #1 offset=659
    ev #2 offset=341
    ev #3 offset=500
    ev #4 offset=2454
    Are the reception times of all 4 msgs close to the supposed values?
    At    662 ms received msg 0: "#2 supposed to be 659"
    At   1009 ms received msg 1: "#0 supposed to be 1000"
    At   1511 ms received msg 2: "#1 supposed to be 1500"

    Traceback (most recent call last):
      File "/tmp/dwq.0.3125418833043728/ef3af88c4b3615788b164464a437df5c/tests/evtimer_msg/tests/01-run.py", line 33, in <module>
        sys.exit(run(testfunc))
      File "/tmp/dwq.0.3125418833043728/ef3af88c4b3615788b164464a437df5c/dist/pythonlibs/testrunner/__init__.py", line 29, in run
        testfunc(child)
      File "/tmp/dwq.0.3125418833043728/ef3af88c4b3615788b164464a437df5c/tests/evtimer_msg/tests/01-run.py", line 26, in testfunc
        assert(actual in range(expected - ACCEPTED_ERROR, expected + ACCEPTED_ERROR))
    AssertionError

### Review procedure

The test randomly failed builds in CI, like for example 

https://ci.riot-os.org/RIOT-OS/RIOT/12150/63c475cd4da31c2abb522454460b7643d0807c70/output/compile/tests/evtimer_msg/native:llvm.txt or attached [native:llvm.txt](https://github.com/RIOT-OS/RIOT/files/3566446/native.llvm.txt)


### Testing procedure

The test is disabled for `master` in this PR:

```
BOARD=native make --no-print-directory -C tests/evtimer_msg/ info-debug-variable-TEST_ON_BOARD_ENABLED

```

In master the test was enabled

```
BOARD=native make --no-print-directory -C tests/evtimer_msg/ info-debug-variable-TEST_ON_BOARD_ENABLED
native
```

### Issues/PRs references

It made https://github.com/RIOT-OS/RIOT/pull/12150 and some other PRs from @jcarrano fail last week

You can view, comment on, or merge this pull request online at:

  https://github.com/RIOT-OS/RIOT/pull/12154

-- Commit Summary --

  * tests/evtimer: blacklist testing on native

-- File Changes --

    M tests/evtimer_msg/Makefile (3)

-- Patch Links --

https://github.com/RIOT-OS/RIOT/pull/12154.patch
https://github.com/RIOT-OS/RIOT/pull/12154.diff

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/RIOT-OS/RIOT/pull/12154
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.riot-os.org/pipermail/notifications/attachments/20190902/c92fa8f9/attachment.htm>


More information about the notifications mailing list