Test Driven Development with ChibiOS simulator

ChibiOS public support forum for all topics not covered by a specific support forum.

Moderators: utzig, lbednarz, tfAteba, barthess, RoccoMarco

meatball
Posts: 27
Joined: Thu May 19, 2016 4:39 pm
Has thanked: 9 times
Been thanked: 2 times

Test Driven Development with ChibiOS simulator

Postby meatball » Tue Jun 04, 2019 4:49 am

Lately, I've been reading the book about TDD by James Grenning. Has anyone implemented dual-targeting for TDD on the ChibiOS simulator?

I was thinking of trying it out, but some of the immediate complications of dual targeting are already in my way:

- Linking things like chprintf() to target my PC's standard output
- Simulating asynchronous events, such as interrupts in the RT-Posix simulator is not obvious

Suppose I want to write an events queue module for my application. The TDD method would have me write a simple test, then write code until the test passes, and then repeat until the driver is done (along with a full suite of tests).

I thought I could just do this with a compile-download-debug method, by running the tests on my STM32, but the download time is expensive, and feedback from my code changes are therefore slow. James' book argues that dual targeting allows for very quick feedback to your code changes, since your PC immediately builds and runs what you just wrote. Not only that, but that you should be able to test your ENTIRE codebase before ever getting the MCU target to run it on. This would be supremely helpful to developer's productivity.

I'm finding out that chibiOS is a little weird as far as projects go, in that the MCU is still targeted primarily for unit tests. This will be prevent the project from ever integrating into a continuous integration/delivery system, which greatly reduces the time between bugs created to bugs discovered.

Can we consider a way to expand the intent of the simulators already in place? According to this thread: viewtopic.php?f=3&t=4861 the sims are there for regression testing and code coverage.

This would require adding "mock" drivers and devices to the codebase, so that UARTs and SPIs and accelerometers and timers can ALL be easily retargeted to the developer's PC with a single, small change in a makefile.

Anyone have experience in this area? I have little experience developing on full PC platforms, but I love the ChibiOS project - so I want to know how I can help it to "keep up" with how many other projects do development, integration, and testing automatically.

Thoughts anyone?

-T

User avatar
Giovanni
Site Admin
Posts: 12177
Joined: Wed May 27, 2009 8:48 am
Location: Salerno, Italy
Has thanked: 570 times
Been thanked: 487 times
Contact:

Re: Test Driven Development with ChibiOS simulator

Postby Giovanni » Tue Jun 04, 2019 5:12 am

Hi,

We use the simulator mainly for build testing, "static" testing and code coverage of RT kernel, not much more.

The simulator suffers of a serious limitation, it is not preemptive because there are no real interrupts and it does not simulate a specific architecture, because of this it cannot detect things like race conditions and cannot replace testing done on HW.

Using an HW target it is possible to run tests quickly, just design your test code to run from RAM.

BTW, you can use chprintf() to print on stdout, just create a stream that reads/writes stdin/stdout, there no need to make a driver.

Giovanni

faisal
Posts: 249
Joined: Wed Jul 19, 2017 12:44 am
Has thanked: 35 times
Been thanked: 32 times

Re: Test Driven Development with ChibiOS simulator

Postby faisal » Wed Aug 07, 2019 5:13 pm

I would encourage you to help with the work shared in this thread: viewtopic.php?t=4861

If you can clean it up to the point where it could be 'upstreamed' it would be very beneficial.

I also contributed to those ChibiOS HAL drivers for the POSIX port. They were very useful, and we were able to test a lot of our code base in simulation. Keep in mind, these were very complex integration tests - with dozens of nodes communication with each other over CAN (simulated with linux Socket CAN), and UDP/IP encapsulated within CAN (we used the lwIP stack, with a custom CAN transport layer). At the same time, hal_serial, hal_rtc, and hal_nand were also being used. The application code was able to be built for and run on an actual target by merely changing the makefile.

Take a look at the hal_lld.c file in the posix simulator files. Giovanni simulates interrupts by polling for events in the idle thread. So, basically, whenever the idle thread is running, it is constantly polling for events and when it finds an event (for example, polling the serial port on the PC for receipt of character), it calls the appropriate simulation IRQ handler in the LLD driver. If you design the drivers properly, it should be able to cover the majority of the cases. Note, that the simulation isn't that useful for fine grained timing applications, or where true preemption is necessary. It is cooperative and non-preemptive. That said, it can be leveraged to find and eliminate many classes of bugs before even having to run it on the actual target.


Return to “General Support”

Who is online

Users browsing this forum: No registered users and 4 guests