[riot-notifications] [RIOT-OS/RIOT] Tests/pkg_fatfs_vfs: make test fail correctly for samr21-xpro (#11389)

danpetry notifications at github.com
Thu Apr 18 15:56:22 CEST 2019

Ok thanks for explaining, yes I didn't consider the loop/multiple test output you described above. See what you're saying as well re the exception and that it's generally in line with other test philosophies, for better or worse.

The problem I was trying to solve here was that this test gives a false negative result during the release specs testing (specifically test 02). It will fail for boards where supporting hardware (an SD card) doesn't exist. I had a look at a couple of the driver tests and saw that these just expect `[FAILURE]` or the like without the test script returning any failure behaviour/information/etc to the shell. So I tried to emulate this behaviour. However, seems like this behaviour should be fixed rather than emulated.

@cladmi says he's familiar with the issue with this test and says that the proper solution is to define a makefile variable that indicates hardware-based dependencies on tests. Then the test would be excluded from being automatically run.

In summary: it doesn't look like there's a satisfactory "quick fix" for the problem. So, better to document that the tester will need to know that this test gives a false negative.

The separate problem of the long timeouts then should also be addressed "properly" and the issue above is for that.

therefore closing the PR.

You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.riot-os.org/pipermail/notifications/attachments/20190418/6556cfa8/attachment.html>

More information about the notifications mailing list