any other reason for "priority order violation"? Topic is solved

Discussions and support about ChibiOS/RT, the free embedded RTOS.
hyOzd
Posts: 20
Joined: Tue Jul 21, 2015 12:17 pm
Has thanked: 1 time
Been thanked: 2 times

any other reason for "priority order violation"?

Postby hyOzd » Thu Apr 06, 2017 11:34 am

Hi I'm having a crash in `chSysUnlock()` function due to "priority order violation".

There were a few cases of improper use of I-class functions from lock zone in my code. I fixed those by adding `chSchRescheduleS()` right before unlocking. But problem continues. I was wondering what else might be triggering this problem? Are there any other situation that I should manually call `chSchRescheduleS`?

User avatar
Giovanni
Site Admin
Posts: 10213
Joined: Wed May 27, 2009 8:48 am
Location: Salerno, Italy
Has thanked: 212 times
Been thanked: 197 times
Contact:

Re: any other reason for "priority order violation"?

Postby Giovanni » Thu Apr 06, 2017 1:03 pm

It would help if you could post the code delimited by that unlock.

Another possible cause is an IRQ with priority above 2 calling OS functions.

Giovanni.

hyOzd
Posts: 20
Joined: Tue Jul 21, 2015 12:17 pm
Has thanked: 1 time
Been thanked: 2 times

Re: any other reason for "priority order violation"?

Postby hyOzd » Thu Apr 06, 2017 2:17 pm

Hi Giovanni, thanks for your reply.

I'm not eager to post the source code since it's quite messy : ) But I can show you a call trace of where it always crashes:

Image

As you can see above, it always crashes when waking from a specific semaphore (with msg=MSG_OK). When this happens, another thread is always in below state. Waiting on another semaphore. Interesting thing is that, below state doesn't happen as often as above one. So I'm thinking that crash happens when these events coincide.

Image

Another possible cause is an IRQ with priority above 2 calling OS functions.


Almost all interrupts have priority above 2, am I mistaken? What you mean by "OS functions", are functions that are not I-class functions, right?

Also could this be caused by me not calling chSysLockFromISR in interrupt callbacks? I've only recently learned about it :S

User avatar
Giovanni
Site Admin
Posts: 10213
Joined: Wed May 27, 2009 8:48 am
Location: Salerno, Italy
Has thanked: 212 times
Been thanked: 197 times
Contact:

Re: any other reason for "priority order violation"?

Postby Giovanni » Thu Apr 06, 2017 2:24 pm

Stack traces are not useful.

Priorities 0 and 1 are reserved on Cortex-M port for fast interrupts which cannot call OS functions of any kind, 2 would be usable but it is used by the kernel so it is a bad idea, user priorities should start from 3.

Giovanni

hyOzd
Posts: 20
Joined: Tue Jul 21, 2015 12:17 pm
Has thanked: 1 time
Been thanked: 2 times

Re: any other reason for "priority order violation"?

Postby hyOzd » Thu Apr 06, 2017 2:34 pm

Stack traces are not useful.


Ok, I will try to put together a clean code to reproduce the issue then.

When you said
priority above 2
did you actually meant "smaller number than 2"? The only interrupts I'm dealing with are peripheral interrupts timer, EXTI, SPI etc. And those all have bigger than 2 priorities defined in mcuconf.h file.

User avatar
Giovanni
Site Admin
Posts: 10213
Joined: Wed May 27, 2009 8:48 am
Location: Salerno, Italy
Has thanked: 212 times
Been thanked: 197 times
Contact:

Re: any other reason for "priority order violation"?

Postby Giovanni » Thu Apr 06, 2017 3:56 pm

If you are using the default HAL priorities and don't have other IRQs then it should be fine, all defaults are compliant.

A minimal application to review would be great.

Giovanni

hyOzd
Posts: 20
Joined: Tue Jul 21, 2015 12:17 pm
Has thanked: 1 time
Been thanked: 2 times

Re: any other reason for "priority order violation"?

Postby hyOzd » Wed Apr 12, 2017 10:53 am

Hi Giovanni, I think I've found the source of problem. I didn't observe the problem when I added `chSysLockFromISR` to my problematic ISRs. But I'm not sure if this is what fixed it.

I have been signaling semaphores from interrupt callbacks. But I did not use `chSysLockFromISR`. Different threads with different priorities have been waiting on these semaphores. I think problem was that; when two interrupts collide, order of rescheduling messes up. Because cpu switches to another interrupts callback during the processing of an ISR.

Here is a pseudo code of what I think was happening:

Code: Select all

high_priority_thread()
{
  while(true)
  {
    chBSemWait(&high_pri_sem);
    do_important_stuff();
  }
}

low_priority_thread()
{
  while(true)
  {
    chBSemWait(&low_pri_sem);  // ASSERT happens inside this call during chSysUnlock
    do_less_imporant_stuff();
  }
}

isr_callback_timer()
{
  do_stuff();
  chBSemSignalI(&high_pri_sem);  // this should wake up HIGH pri. thread
  do_some_other_stuff();
}

isr_callback_exti()
{
  chBSemSignalI(&low_pri_sem);  // this should wake up LOW pri. thread
}


Assert was fired when low priority thread was awaking from a call to `chBSemWait`. Here is what I think was happening.

1. timer interrupt occurs, isr_callback_timer() is called
2. isr_callback_timer() signals the `high_pri_sem`
3. But, before isr exits, around `do_some_other_stuff()`...
4. pin interrupt occurs, cpu switches to isr_callback_exti()
5. isr_callback_exti() signals the low_pri_sem, and at the exit does reschedule
6. cpu returns to the isr_callback_timer(), exits it and reschedules
7. since at step #5 rescheduling is done after `low_pri_sem`, cpu switches to the LOW priority thread instead of high priority thread

Can you please confirm if this is a possible scenario? I barely understand how scheduling works.

By the way I'm using STM32F407.

User avatar
Giovanni
Site Admin
Posts: 10213
Joined: Wed May 27, 2009 8:48 am
Location: Salerno, Italy
Has thanked: 212 times
Been thanked: 197 times
Contact:

Re: any other reason for "priority order violation"?

Postby Giovanni » Wed Apr 12, 2017 1:45 pm

Hi,

It is clearly the problem, all I-class functions (with I after the name) require to be called from within lock/unlock, this includes ISRs and callbacks.

Note that you should enable the "state checker" in chconf.h it detects this kind of errors by stopping the program and setting the halt pointer to an error code.

Giovanni

hyOzd
Posts: 20
Joined: Tue Jul 21, 2015 12:17 pm
Has thanked: 1 time
Been thanked: 2 times

Re: any other reason for "priority order violation"?

Postby hyOzd » Wed Apr 12, 2017 1:58 pm

Thank you. That's a load off my mind.

I'm going to enable all checks from now on : )

hyOzd
Posts: 20
Joined: Tue Jul 21, 2015 12:17 pm
Has thanked: 1 time
Been thanked: 2 times

Re: any other reason for "priority order violation"?  Topic is solved

Postby hyOzd » Thu Apr 13, 2017 12:36 pm

Giovanni I'm sorry to bother you again with this. But I'm having another but similar problem now.

I've updated my chibios yesterday (r10155). It seems mailbox implementation has been recently changed. Now I get a priority violation after a call to `chMBPostS` from a lock zone. My understanding was that manual rescheduling is only required after call to an "I-class" function from a lock zone? Has this changed? Do I have to call chSchRescheduleS after S-class functions as well? Because looking at the source code of `chMBPostS` I don't see a call to reschedule. Older implementation had it.

My pseudo code:

Code: Select all

void low_pri_thread()
{
   do_stuff();
   chSysLock();
   chMBPostS(queue...); // high pri. thread is waiting on this queue
   do_some_safe_stuff();
   // should I call chSchRescheduleS??
   chSysUnlock(); // "priority viloation" ASSERT happens here
}


Return to “ChibiOS/RT”

Who is online

Users browsing this forum: No registered users and 1 guest