ChibiOS syscalls.c and GCC 7 Topic is solved

Report here problems in any of ChibiOS components. This forum is NOT for support.
User avatar
Giovanni
Site Admin
Posts: 14444
Joined: Wed May 27, 2009 8:48 am
Location: Salerno, Italy
Has thanked: 1074 times
Been thanked: 921 times
Contact:

Re: ChibiOS syscalls.c and GCC 7

Postby Giovanni » Wed Aug 08, 2018 2:08 pm

If this is the case then replacing malloc() and free() should be the best option, but how to disable the default implementations? we also need a realloc() equivalent (possible to do in the heap allocator, just missing).

Another option is to create a separate C heap, taking memory from core in one go.

Giovanni

steved
Posts: 823
Joined: Fri Nov 09, 2012 2:22 pm
Has thanked: 12 times
Been thanked: 135 times

Re: ChibiOS syscalls.c and GCC 7

Postby steved » Fri Aug 10, 2018 8:27 am

I replaced the reentrant versions (can't remember why!). It also allowed me to force alignment to cache lines

Code: Select all

void* _malloc_r( struct _reent* r, size_t size )
{
    (void) r;
    if (size == 0)
        return NULL;
  return chHeapAllocAligned(NULL, size, MALLOC_ALIGNMENT);
}


// This one may not be needed
void *_memalign_r(struct _reent* r, size_t alignment, size_t size)
{
  (void) r;
  if (size == 0)
      return NULL;
  return chHeapAllocAligned(NULL, size, alignment > MALLOC_ALIGNMENT ? alignment : MALLOC_ALIGNMENT);
}


void* _calloc_r( struct _reent* r, size_t nelem, size_t elem_size )
{
  (void) r;
   if (nelem == 0)
      return NULL;
   if (elem_size == 0)
      return NULL;
  void *tp = chHeapAllocAligned(NULL, nelem * elem_size, elem_size > MALLOC_ALIGNMENT ? elem_size : MALLOC_ALIGNMENT);      // Could directly call _malloc_r() instead
  if (tp == NULL)
    return tp;
  memset(tp, 0, nelem * elem_size);             // Returns zeroed memory
  return tp;
}


void _free_r( struct _reent* r, void* ptr )
{
  (void) r;
   if (ptr == NULL)
      return;            // A null pointer is allowed
   chHeapFree(ptr);
}

void* _realloc_r( struct _reent* r, void* ptr, size_t size )
{
   if (ptr == NULL)
      return _malloc_r(r, size);
   if (size == 0)
   {
      _free_r(r, ptr);
   }

   void *tp = _malloc_r(r, size);      // Get the new block
   if (tp == NULL)
   {
     r->_errno = ENOMEM;
     return NULL;                // Failure - do nothing
   }

   // Now we have to get the size of the old memory block
     heap_header_t *hp = (heap_header_t *)ptr - 1U;
     size_t oldSize = hp->used.size;
     if (oldSize > size)
       oldSize = size;             // Bytes to move
     memcpy(tp, ptr, oldSize);
     _free_r(r, ptr);              // Free the old memory block
     return tp;
}

It might be better if the heap allocator provided realloc(), but not difficult to implement.

Note: Still using GCC 6 ATM, but doubt it makes any difference to this one.

User avatar
FXCoder
Posts: 384
Joined: Sun Jun 12, 2016 4:10 am
Location: Sydney, Australia
Has thanked: 180 times
Been thanked: 130 times

Re: ChibiOS syscalls.c and GCC 7

Postby FXCoder » Sun Aug 12, 2018 3:26 am

If this is the case then replacing malloc() and free() should be the best option, but how to disable the default implementations? we also need a realloc() equivalent (possible to do in the heap allocator, just missing).


Seems that it would require a specific build of newlib with a set of intercepted calls to implement the required control.
Since the current libraries (GCC 6 or 7) aren't actually reentrant (they simply point to one _impure_data struct) then a specific C heap for managing _reent structs per thread could implement reentrancy (refer to information in arm-none-eabi/include/reent.h in newlib) and memory management.

Whilst not a small undertaking such an approach would have merit by giving ChibiOS control of Clib/newlib implementation.
The functionality would be enabled by #define making it optional for any build not needing Clib

Thoughts?

steved
Posts: 823
Joined: Fri Nov 09, 2012 2:22 pm
Has thanked: 12 times
Been thanked: 135 times

Re: ChibiOS syscalls.c and GCC 7

Postby steved » Sun Aug 12, 2018 8:39 am

The linker looks after it; since the library is the last thing to be processed, if malloc() and free() are already defined then the library routines aren't linked in.

As you say, the current newlib does nothing about reentrancy. However IIRC all the non-reentrant functions in newlib just call their reentrant equivalents, so reminplementing the reentrant versions seems logical.
Not sure about a generic management of reentrancy; I seem to recollect that the reentrancy structure can be relatively large, so a big overhead if its per-thread.

User avatar
FXCoder
Posts: 384
Joined: Sun Jun 12, 2016 4:10 am
Location: Sydney, Australia
Has thanked: 180 times
Been thanked: 130 times

Re: ChibiOS syscalls.c and GCC 7

Postby FXCoder » Sun Aug 12, 2018 9:05 am

Hi Steve,
From my testing the linker does not take care of it unfortunately.
I've tested with ChibiStudio set for GCC 6.3 and also with GCC 7 with clean builds.
Maybe I've not got something set right but I'm using ChibiStudio in a standard configuration (no GNU MCU installed in Eclipse Windows setups).
In the case of 6.3 the inclusion of syscalls.c in the project (first in $(ALLCSRC)) did not result in the ChibiOS versions of functions being linked in place of the newlib archive/lib version.
With GCC 7 the ChibiOS syscalls.c version of _sbrk_r did get linked but only because that is missing from newlib.

WRT thread based reentrancy...
I wouldn't put the _reent data into the thread but instead into the dedicated C heap Giovanni mentioned.
Perhaps a pool, within the heap... TBD.
Then the current ChibioS thread reference would be used to lookup the relevant _impure_ptr referenced data or create a new one for a newly instantiated thread.
Alternatively just keep the errno value in the thread as a simplistic implementation so that at least that is thread safe.
Anyway just some off the cuff ideas that would need to be better thought through.

Cheers,

Bob

steved
Posts: 823
Joined: Fri Nov 09, 2012 2:22 pm
Has thanked: 12 times
Been thanked: 135 times

Re: ChibiOS syscalls.c and GCC 7

Postby steved » Sun Aug 12, 2018 9:14 pm

Bob,

It can be done somehow, since I've got it working! I'll have to try and remember to look up my notes. IIRC I have all the syscalls points defined, some as null routines, some with lockups so I can be sure they aren't actually used.
Even so its a bit touchy, since there seem to be many interactions between the various allocation routines. I do remember that, having eliminated the last use of asprintf(), something was trying to access balloc()! ATM that's just 'fixed' by a dummy use of asprintf(), pending proper investigation.

User avatar
Giovanni
Site Admin
Posts: 14444
Joined: Wed May 27, 2009 8:48 am
Location: Salerno, Italy
Has thanked: 1074 times
Been thanked: 921 times
Contact:

Re: ChibiOS syscalls.c and GCC 7

Postby Giovanni » Mon Dec 31, 2018 11:01 am

bump

It is not clear to me yet what is the best way to handle this, perhaps a static area where to implement library _sbrk()?

Giovanni

faisal
Posts: 374
Joined: Wed Jul 19, 2017 12:44 am
Has thanked: 44 times
Been thanked: 60 times

Re: ChibiOS syscalls.c and GCC 7

Postby faisal » Mon Dec 31, 2018 7:17 pm

This thread could be useful:
viewtopic.php?t=3904

User avatar
Giovanni
Site Admin
Posts: 14444
Joined: Wed May 27, 2009 8:48 am
Location: Salerno, Italy
Has thanked: 1074 times
Been thanked: 921 times
Contact:

Re: ChibiOS syscalls.c and GCC 7

Postby Giovanni » Sat Feb 02, 2019 9:23 am

Maybe I found a solution for this.

It would be possible to modify the core allocator to support two separate heaps, one growing upward and the other growing downward.

The first one would be used only for implementing _sbrk(), the other would be used by ChibiOS allocators. The two would not affect each other and newlib would be happy.

(weird night, I woke at 3am with this thing in mind... it must have been a background thread for all this time)

Giovanni

User avatar
Giovanni
Site Admin
Posts: 14444
Joined: Wed May 27, 2009 8:48 am
Location: Salerno, Italy
Has thanked: 1074 times
Been thanked: 921 times
Contact:

Re: ChibiOS syscalls.c and GCC 7

Postby Giovanni » Mon Feb 04, 2019 2:03 pm

Hi,

I just committed this change in trunk, now ChibiOS allocates downward and newlib allocates upward (fun in the middle). I tried some malloc/free and it seems to work but more testing is required.

Please give it a try,I will backport this to 19.1.x for sure, if there is demand also to 18.2.x.

Giovanni


Return to “Bug Reports”

Who is online

Users browsing this forum: No registered users and 5 guests