I think there is a bug in ChibiOS lwipthread / demos that causes pbuf /memp corruption.
First, low_level_input() allocates a pbuf from the pool (PBUF_POOL).
Second, the thread loop calls pbuf_free(), although admitedly it does only for packets of unknown type.
Neither pbuf_alloc() nor pbuf_free() can be safely called from other context than the LwIP worker thread unless in lwipopts.h SYS_LIGHTWEIGHT_PROT is set and/or (not sure about this) LWIP_ALLOW_MEM_FREE_FROM_OTHER_CONTEXT is also set, and none of these two options is set in lwipopts.h files in demos shipped with ChibiOS.
Either lwipopts.h needs to be modified or pbuf_alloc_callback() / pbuf_free_callback() must be used instead.
Slightly related I filed this bug some time ago:
https://sourceforge.net/p/chibios/bugs/729/
TL;DR lwipthread calls netif_add() which cannot be called from other context than LwIP worker thread. netconn_netif_add() must be used instead or netif_add() called from the callback that can be passed to tcpip_init().
LwIP pbuf/memp corruption Topic is solved
- Giovanni
- Site Admin
- Posts: 14457
- Joined: Wed May 27, 2009 8:48 am
- Location: Salerno, Italy
- Has thanked: 1076 times
- Been thanked: 922 times
- Contact:
Re: LwIP pbuf/memp corruption
Just reporting that after extensive stress testing LwIP with changes to lwipopts.h I can confirm that my occasional LwIP pbuf assertion triggers are gone. No more pbufs with invalid ref count, no more memp sanity checks panic.
- Giovanni
- Site Admin
- Posts: 14457
- Joined: Wed May 27, 2009 8:48 am
- Location: Salerno, Italy
- Has thanked: 1076 times
- Been thanked: 922 times
- Contact:
Re: LwIP pbuf/memp corruption
Giovanni wrote:Thanks, I will make changes, do you have a patch ready?
Nope, sorry. Change is trivial, just enable SYS_LIGHTWEIGHT_PROT in lwipopts.h
However first I was hoping for someone more familiary with LwIP mutual exclusion intricacies to clarify whether enabling LWIP_ALLOW_MEM_FREE_FROM_OTHER_CONTEXT is also necessary.
I've enabled both and pbuf/memp corruption is gone for good, but havent (and can't afford at the moment) to do extensive testing with just SYS_LIGHTWEIGHT_PROT enabled. The point is that according to LwIP documentation enabling LWIP_ALLOW_MEM_FREE_FROM_OTHER_CONTEXT can increase interrupt latency significantly, so it is certainly worth to find out whether it is necessary.
Also, not that I provided an alternative fix: use pbuf_alloc_callback() and pbuf_free_callback() instead of pbuf_alloc() / pbuf_free(). That would work for sure and not require any changes to lwipopts.h, but again, I would appreciate if some LwIP expert could chip in and provide advice on which is the best fix.
Re: LwIP pbuf/memp corruption
No clues about this?
Please note that it is out of the question whether this is a bug or not. ChibiOS demos, as shipped, will cause pbuf corruption due to pbuf alloc/free being done from the LwIP data pump thread (lwipthread.c) with SYS_LIGHTWEIGHT_PROT disabled.
We need someone knowledgeable about the LwIP internals to tell whether SYS_LIGHTWEIGHT_PROT enabled will suffice or LWIP_ALLOW_MEM_FREE_FROM_OTHER_CONTEXT enabled is also necessary.
My guess is that just SYS_LIGHTWEIGHT_PROT is necessary so thee is proper mutual exclusion when allocating/freeing pbuf. LWIP_ALLOW_MEM_FREE_FROM_OTHER_CONTEXT is only necessary if alloc/free is done from interrupt context or something.
Please note that it is out of the question whether this is a bug or not. ChibiOS demos, as shipped, will cause pbuf corruption due to pbuf alloc/free being done from the LwIP data pump thread (lwipthread.c) with SYS_LIGHTWEIGHT_PROT disabled.
We need someone knowledgeable about the LwIP internals to tell whether SYS_LIGHTWEIGHT_PROT enabled will suffice or LWIP_ALLOW_MEM_FREE_FROM_OTHER_CONTEXT enabled is also necessary.
My guess is that just SYS_LIGHTWEIGHT_PROT is necessary so thee is proper mutual exclusion when allocating/freeing pbuf. LWIP_ALLOW_MEM_FREE_FROM_OTHER_CONTEXT is only necessary if alloc/free is done from interrupt context or something.
- Giovanni
- Site Admin
- Posts: 14457
- Joined: Wed May 27, 2009 8:48 am
- Location: Salerno, Italy
- Has thanked: 1076 times
- Been thanked: 922 times
- Contact:
Re: LwIP pbuf/memp corruption
Hi,
The problem is that I don't know for sure, SYS_LIGHTWEIGHT_PROT should use critical zones instead of mutexes for synchronization so it should work either way because the arch layer supplies both mechanisms.
Giovanni
The problem is that I don't know for sure, SYS_LIGHTWEIGHT_PROT should use critical zones instead of mutexes for synchronization so it should work either way because the arch layer supplies both mechanisms.
Giovanni
Re: LwIP pbuf/memp corruption
iggarpe wrote:My guess is that just SYS_LIGHTWEIGHT_PROT is necessary so thee is proper mutual exclusion when allocating/freeing pbuf. LWIP_ALLOW_MEM_FREE_FROM_OTHER_CONTEXT is only necessary if alloc/free is done from interrupt context or something.
I don't claim to have any great knowledge in this area; however having been experimenting with lwIP (V2.0.2) I agree with your interpretation. I've been testing with this combination of options, mostly on a 32F429, using the sockets interface, and found it works OK. I'll try and remember to test without SYS_LIGHTWEIGHT_PROT set to see if it makes a difference.
Re: LwIP pbuf/memp corruption
Have just tested with SYS_LIGHTWEIGHT_PROT set to zero, and the system did indeed crash (although it took rather longer than I expected, given that the network interface was running close to flat out). So can confirm SYS_LIGHTWEIGHT_PROT set to 1
- Giovanni
- Site Admin
- Posts: 14457
- Joined: Wed May 27, 2009 8:48 am
- Location: Salerno, Italy
- Has thanked: 1076 times
- Been thanked: 922 times
- Contact:
Who is online
Users browsing this forum: No registered users and 72 guests