Page 1 of 2

any other reason for "priority order violation"?

Posted: Thu Apr 06, 2017 11:34 am
by hyOzd
Hi I'm having a crash in `chSysUnlock()` function due to "priority order violation".

There were a few cases of improper use of I-class functions from lock zone in my code. I fixed those by adding `chSchRescheduleS()` right before unlocking. But problem continues. I was wondering what else might be triggering this problem? Are there any other situation that I should manually call `chSchRescheduleS`?

Re: any other reason for "priority order violation"?

Posted: Thu Apr 06, 2017 1:03 pm
by Giovanni
It would help if you could post the code delimited by that unlock.

Another possible cause is an IRQ with priority above 2 calling OS functions.

Giovanni.

Re: any other reason for "priority order violation"?

Posted: Thu Apr 06, 2017 2:17 pm
by hyOzd
Hi Giovanni, thanks for your reply.

I'm not eager to post the source code since it's quite messy : ) But I can show you a call trace of where it always crashes:

Image

As you can see above, it always crashes when waking from a specific semaphore (with msg=MSG_OK). When this happens, another thread is always in below state. Waiting on another semaphore. Interesting thing is that, below state doesn't happen as often as above one. So I'm thinking that crash happens when these events coincide.

Image

Another possible cause is an IRQ with priority above 2 calling OS functions.


Almost all interrupts have priority above 2, am I mistaken? What you mean by "OS functions", are functions that are not I-class functions, right?

Also could this be caused by me not calling chSysLockFromISR in interrupt callbacks? I've only recently learned about it :S

Re: any other reason for "priority order violation"?

Posted: Thu Apr 06, 2017 2:24 pm
by Giovanni
Stack traces are not useful.

Priorities 0 and 1 are reserved on Cortex-M port for fast interrupts which cannot call OS functions of any kind, 2 would be usable but it is used by the kernel so it is a bad idea, user priorities should start from 3.

Giovanni

Re: any other reason for "priority order violation"?

Posted: Thu Apr 06, 2017 2:34 pm
by hyOzd
Stack traces are not useful.


Ok, I will try to put together a clean code to reproduce the issue then.

When you said
priority above 2
did you actually meant "smaller number than 2"? The only interrupts I'm dealing with are peripheral interrupts timer, EXTI, SPI etc. And those all have bigger than 2 priorities defined in mcuconf.h file.

Re: any other reason for "priority order violation"?

Posted: Thu Apr 06, 2017 3:56 pm
by Giovanni
If you are using the default HAL priorities and don't have other IRQs then it should be fine, all defaults are compliant.

A minimal application to review would be great.

Giovanni

Re: any other reason for "priority order violation"?

Posted: Wed Apr 12, 2017 10:53 am
by hyOzd
Hi Giovanni, I think I've found the source of problem. I didn't observe the problem when I added `chSysLockFromISR` to my problematic ISRs. But I'm not sure if this is what fixed it.

I have been signaling semaphores from interrupt callbacks. But I did not use `chSysLockFromISR`. Different threads with different priorities have been waiting on these semaphores. I think problem was that; when two interrupts collide, order of rescheduling messes up. Because cpu switches to another interrupts callback during the processing of an ISR.

Here is a pseudo code of what I think was happening:

Code: Select all

high_priority_thread()
{
  while(true)
  {
    chBSemWait(&high_pri_sem);
    do_important_stuff();
  }
}

low_priority_thread()
{
  while(true)
  {
    chBSemWait(&low_pri_sem);  // ASSERT happens inside this call during chSysUnlock
    do_less_imporant_stuff();
  }
}

isr_callback_timer()
{
  do_stuff();
  chBSemSignalI(&high_pri_sem);  // this should wake up HIGH pri. thread
  do_some_other_stuff();
}

isr_callback_exti()
{
  chBSemSignalI(&low_pri_sem);  // this should wake up LOW pri. thread
}


Assert was fired when low priority thread was awaking from a call to `chBSemWait`. Here is what I think was happening.

1. timer interrupt occurs, isr_callback_timer() is called
2. isr_callback_timer() signals the `high_pri_sem`
3. But, before isr exits, around `do_some_other_stuff()`...
4. pin interrupt occurs, cpu switches to isr_callback_exti()
5. isr_callback_exti() signals the low_pri_sem, and at the exit does reschedule
6. cpu returns to the isr_callback_timer(), exits it and reschedules
7. since at step #5 rescheduling is done after `low_pri_sem`, cpu switches to the LOW priority thread instead of high priority thread

Can you please confirm if this is a possible scenario? I barely understand how scheduling works.

By the way I'm using STM32F407.

Re: any other reason for "priority order violation"?

Posted: Wed Apr 12, 2017 1:45 pm
by Giovanni
Hi,

It is clearly the problem, all I-class functions (with I after the name) require to be called from within lock/unlock, this includes ISRs and callbacks.

Note that you should enable the "state checker" in chconf.h it detects this kind of errors by stopping the program and setting the halt pointer to an error code.

Giovanni

Re: any other reason for "priority order violation"?

Posted: Wed Apr 12, 2017 1:58 pm
by hyOzd
Thank you. That's a load off my mind.

I'm going to enable all checks from now on : )

Re: any other reason for "priority order violation"?  Topic is solved

Posted: Thu Apr 13, 2017 12:36 pm
by hyOzd
Giovanni I'm sorry to bother you again with this. But I'm having another but similar problem now.

I've updated my chibios yesterday (r10155). It seems mailbox implementation has been recently changed. Now I get a priority violation after a call to `chMBPostS` from a lock zone. My understanding was that manual rescheduling is only required after call to an "I-class" function from a lock zone? Has this changed? Do I have to call chSchRescheduleS after S-class functions as well? Because looking at the source code of `chMBPostS` I don't see a call to reschedule. Older implementation had it.

My pseudo code:

Code: Select all

void low_pri_thread()
{
   do_stuff();
   chSysLock();
   chMBPostS(queue...); // high pri. thread is waiting on this queue
   do_some_safe_stuff();
   // should I call chSchRescheduleS??
   chSysUnlock(); // "priority viloation" ASSERT happens here
}