[INFO] Results with GCC 8.2.1 vs GCC 7.3.1

[Giovanni]

Hi,

I performed some tests, the new 8.2.1 compiler works correctly but still has the bug impacting Cortex-M0.

I am not impressed, 8.2.1 scores lower that 7.3.1 except in few tests, code size is slightly larger too (test performed using -O2).

7.3.1 report:

Code: Select all


*** ChibiOS/RT Test Suite
***
*** Compiled:     Jan 26 2019 - 09:36:48
*** Platform:     STM32L4xx Ultra Low Power
*** Test Board:   STMicroelectronics STM32 Nucleo64-L476RG
*** Code Size:    42072

============================================================================
=== Test Sequence 1 (Information)
----------------------------------------------------------------------------
--- Test Case 1.1 (Port Info)
--- Architecture:                       ARMv7E-M
--- Core Variant:                       Cortex-M4
--- Compiler:                           GCC 7.3.1 20180622 (release) [ARM/embedded-7-branch revision 261907]
--- Port Info:                          Advanced kernel mode
--- Natural alignment:                  4
--- Stack alignment:                    8
--- Working area alignment:             8
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 1.2 (Kernel Info)
--- Product:                            ChibiOS/RT
--- Stable Flag:                        0
--- Version String:                     6.0.0
--- Major Number:                       6
--- Minor Number:                       0
--- Patch Number:                       0
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 1.3 (Kernel Settings)
--- CH_CFG_ST_RESOLUTION:               32
--- CH_CFG_ST_FREQUENCY:                10000
--- CH_CFG_INTERVALS_SIZE:              32
--- CH_CFG_TIME_TYPES_SIZE:             32
--- CH_CFG_ST_TIMEDELTA:                2
--- CH_CFG_TIME_QUANTUM:                0
--- CH_CFG_MEMCORE_SIZE:                0
--- CH_CFG_NO_IDLE_THREAD:              0
--- CH_CFG_OPTIMIZE_SPEED:              1
--- CH_CFG_USE_TM:                      1
--- CH_CFG_USE_REGISTRY:                1
--- CH_CFG_USE_WAITEXIT:                1
--- CH_CFG_USE_SEMAPHORES:              1
--- CH_CFG_USE_SEMAPHORES_PRIORITY:     0
--- CH_CFG_USE_MUTEXES:                 1
--- CH_CFG_USE_MUTEXES_RECURSIVE:       0
--- CH_CFG_USE_CONDVARS:                1
--- CH_CFG_USE_CONDVARS_TIMEOUT:        1
--- CH_CFG_USE_EVENTS:                  1
--- CH_CFG_USE_EVENTS_TIMEOUT:          1
--- CH_CFG_USE_MESSAGES:                1
--- CH_CFG_USE_MESSAGES_PRIORITY:       0
--- CH_CFG_USE_MAILBOXES:               1
--- CH_CFG_USE_MEMCORE:                 1
--- CH_CFG_USE_HEAP:                    1
--- CH_CFG_USE_MEMPOOLS:                1
--- CH_CFG_USE_OBJ_FIFOS:               1
--- CH_CFG_USE_DYNAMIC:                 1
--- CH_CFG_USE_FACTORY:                 1
--- CH_CFG_FACTORY_MAX_NAMES_LENGTH:    8
--- CH_CFG_FACTORY_OBJECTS_REGISTRY:    1
--- CH_CFG_FACTORY_GENERIC_BUFFERS:     1
--- CH_CFG_FACTORY_SEMAPHORES:          1
--- CH_CFG_FACTORY_MAILBOXES:           1
--- CH_CFG_FACTORY_OBJ_FIFOS:           1
--- CH_DBG_STATISTICS:                  0
--- CH_DBG_SYSTEM_STATE_CHECK:          0
--- CH_DBG_ENABLE_CHECKS:               0
--- CH_DBG_ENABLE_ASSERTS:              0
--- CH_DBG_TRACE_MASK:                  255
--- CH_DBG_TRACE_BUFFER_SIZE:           128
--- CH_DBG_ENABLE_STACK_CHECK:          0
--- CH_DBG_FILL_THREADS:                0
--- CH_DBG_THREADS_PROFILING:           0
--- Result: SUCCESS
============================================================================
=== Test Sequence 2 (System layer and port interface)
----------------------------------------------------------------------------
--- Test Case 2.1 (System integrity functionality)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 2.2 (Critical zones functionality)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 2.3 (Interrupts handling functionality)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 2.4 (System Tick Counter functionality)
--- Result: SUCCESS
============================================================================
=== Test Sequence 3 (Threads Functionality)
----------------------------------------------------------------------------
--- Test Case 3.1 (Thread Sleep functionality)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 3.2 (Ready List functionality, threads priority order)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 3.3 (Priority change test)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 3.4 (Priority change test with Priority Inheritance)
--- Result: SUCCESS
============================================================================
=== Test Sequence 4 (Suspend/Resume)
----------------------------------------------------------------------------
--- Test Case 4.1 (Suspend and Resume functionality)
--- Result: SUCCESS
============================================================================
=== Test Sequence 5 (Counter Semaphores)
----------------------------------------------------------------------------
--- Test Case 5.1 (Semaphore primitives, no state change)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 5.2 (Semaphore enqueuing test)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 5.3 (Semaphore timeout test)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 5.4 (Testing chSemAddCounterI() functionality)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 5.5 (Testing chSemWaitSignal() functionality)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 5.6 (Testing Binary Semaphores special case)
--- Result: SUCCESS
============================================================================
=== Test Sequence 6 (Mutexes, Condition Variables and Priority Inheritance)
----------------------------------------------------------------------------
--- Test Case 6.1 (Priority enqueuing test)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 6.2 (Priority return verification)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 6.3 (Repeated locks, non recursive scenario)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 6.4 (Condition Variable signal test)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 6.5 (Condition Variable broadcast test)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 6.6 (Condition Variable priority boost test)
--- Result: SUCCESS
============================================================================
=== Test Sequence 7 (Synchronous Messages)
----------------------------------------------------------------------------
--- Test Case 7.1 (Messages Server loop)
--- Result: SUCCESS
============================================================================
=== Test Sequence 8 (Event Sources and Event Flags)
----------------------------------------------------------------------------
--- Test Case 8.1 (Events registration)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 8.2 (Event Flags dispatching)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 8.3 (Events Flags wait using chEvtWaitOne())
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 8.4 (Events Flags wait using chEvtWaitAny())
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 8.5 (Events Flags wait using chEvtWaitAll())
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 8.6 (Events Flags wait timeouts)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 8.7 (Broadcasting using chEvtBroadcast())
--- Result: SUCCESS
============================================================================
=== Test Sequence 9 (Dynamic threads)
----------------------------------------------------------------------------
--- Test Case 9.1 (Threads creation from Memory Heap)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 9.2 (Threads creation from Memory Pool)
--- Result: SUCCESS
============================================================================
=== Test Sequence 10 (Benchmarks)
----------------------------------------------------------------------------
--- Test Case 10.1 (Messages performance #1)
--- Score : 388342 msgs/S, 776684 ctxswc/S
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 10.2 (Messages performance #2)
--- Score : 312494 msgs/S, 624988 ctxswc/S
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 10.3 (Messages performance #3)
--- Score : 312495 msgs/S, 624990 ctxswc/S
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 10.4 (Context Switch performance)
--- Score : 1219032 ctxswc/S
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 10.5 (Threads performance, full cycle)
--- Score : 227915 threads/S
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 10.6 (Threads performance, create/exit only)
--- Score : 295197 threads/S
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 10.7 (Mass reschedule performance)
--- Score : 99378 reschedules/S, 596268 ctxswc/S
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 10.8 (Round-Robin voluntary reschedule)
--- Score : 751160 ctxswc/S
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 10.9 (Virtual Timers set/reset performance)
--- Score : 772522 timers/S
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 10.10 (Semaphores wait/signal performance)
--- Score : 1893480 wait+signal/S
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 10.11 (Mutexes lock/unlock performance)
--- Score : 938412 lock+unlock/S
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 10.12 (RAM Footprint)
--- System: 120 bytes
--- Thread: 68 bytes
--- Timer : 20 bytes
--- Semaph: 12 bytes
--- Mutex : 16 bytes
--- CondV.: 8 bytes
--- EventS: 4 bytes
--- EventL: 20 bytes
--- MailB.: 40 bytes
--- Result: SUCCESS
----------------------------------------------------------------------------

Final result: SUCCESS

*** ChibiOS OS Library Test Suite
***
*** Compiled:     Jan 26 2019 - 09:36:48
*** Platform:     STM32L4xx Ultra Low Power
*** Test Board:   STMicroelectronics STM32 Nucleo64-L476RG

============================================================================
=== Test Sequence 1 (Mailboxes)
----------------------------------------------------------------------------
--- Test Case 1.1 (Mailbox normal API, non-blocking tests)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 1.2 (Mailbox I-Class API, non-blocking tests)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 1.3 (Mailbox timeouts)
--- Result: SUCCESS
============================================================================
=== Test Sequence 2 (Pipes)
----------------------------------------------------------------------------
--- Test Case 2.1 (Pipes normal API, non-blocking tests)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 2.2 (Pipe timeouts)
--- Result: SUCCESS
============================================================================
=== Test Sequence 3 (Memory Pools)
----------------------------------------------------------------------------
--- Test Case 3.1 (Loading and emptying a memory pool)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 3.2 (Loading and emptying a guarded memory pool without waiting)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 3.3 (Guarded Memory Pools timeout)
--- Result: SUCCESS
============================================================================
=== Test Sequence 4 (Memory Heaps)
----------------------------------------------------------------------------
--- Test Case 4.1 (Allocation and fragmentation)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 4.2 (Default Heap)
--- Result: SUCCESS
============================================================================
=== Test Sequence 5 (Objects Factory)
----------------------------------------------------------------------------
--- Test Case 5.1 (Objects Registry)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 5.2 (Dynamic Buffers Factory)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 5.3 (Dynamic Semaphores Factory)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 5.4 (Dynamic Mailboxes Factory)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 5.5 (Dynamic Objects FIFOs Factory)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 5.6 (Dynamic Pipes Factory)
--- Result: SUCCESS
----------------------------------------------------------------------------

Final result: SUCCESS


8.2.1 report:

Code: Select all


*** ChibiOS/RT Test Suite
***
*** Compiled:     Jan 26 2019 - 09:33:54
*** Platform:     STM32L4xx Ultra Low Power
*** Test Board:   STMicroelectronics STM32 Nucleo64-L476RG
*** Code Size:    42164

============================================================================
=== Test Sequence 1 (Information)
----------------------------------------------------------------------------
--- Test Case 1.1 (Port Info)
--- Architecture:                       ARMv7E-M
--- Core Variant:                       Cortex-M4
--- Compiler:                           GCC 8.2.1 20181213 (release) [gcc-8-branch revision 267074]
--- Port Info:                          Advanced kernel mode
--- Natural alignment:                  4
--- Stack alignment:                    8
--- Working area alignment:             8
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 1.2 (Kernel Info)
--- Product:                            ChibiOS/RT
--- Stable Flag:                        0
--- Version String:                     6.0.0
--- Major Number:                       6
--- Minor Number:                       0
--- Patch Number:                       0
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 1.3 (Kernel Settings)
--- CH_CFG_ST_RESOLUTION:               32
--- CH_CFG_ST_FREQUENCY:                10000
--- CH_CFG_INTERVALS_SIZE:              32
--- CH_CFG_TIME_TYPES_SIZE:             32
--- CH_CFG_ST_TIMEDELTA:                2
--- CH_CFG_TIME_QUANTUM:                0
--- CH_CFG_MEMCORE_SIZE:                0
--- CH_CFG_NO_IDLE_THREAD:              0
--- CH_CFG_OPTIMIZE_SPEED:              1
--- CH_CFG_USE_TM:                      1
--- CH_CFG_USE_REGISTRY:                1
--- CH_CFG_USE_WAITEXIT:                1
--- CH_CFG_USE_SEMAPHORES:              1
--- CH_CFG_USE_SEMAPHORES_PRIORITY:     0
--- CH_CFG_USE_MUTEXES:                 1
--- CH_CFG_USE_MUTEXES_RECURSIVE:       0
--- CH_CFG_USE_CONDVARS:                1
--- CH_CFG_USE_CONDVARS_TIMEOUT:        1
--- CH_CFG_USE_EVENTS:                  1
--- CH_CFG_USE_EVENTS_TIMEOUT:          1
--- CH_CFG_USE_MESSAGES:                1
--- CH_CFG_USE_MESSAGES_PRIORITY:       0
--- CH_CFG_USE_MAILBOXES:               1
--- CH_CFG_USE_MEMCORE:                 1
--- CH_CFG_USE_HEAP:                    1
--- CH_CFG_USE_MEMPOOLS:                1
--- CH_CFG_USE_OBJ_FIFOS:               1
--- CH_CFG_USE_DYNAMIC:                 1
--- CH_CFG_USE_FACTORY:                 1
--- CH_CFG_FACTORY_MAX_NAMES_LENGTH:    8
--- CH_CFG_FACTORY_OBJECTS_REGISTRY:    1
--- CH_CFG_FACTORY_GENERIC_BUFFERS:     1
--- CH_CFG_FACTORY_SEMAPHORES:          1
--- CH_CFG_FACTORY_MAILBOXES:           1
--- CH_CFG_FACTORY_OBJ_FIFOS:           1
--- CH_DBG_STATISTICS:                  0
--- CH_DBG_SYSTEM_STATE_CHECK:          0
--- CH_DBG_ENABLE_CHECKS:               0
--- CH_DBG_ENABLE_ASSERTS:              0
--- CH_DBG_TRACE_MASK:                  255
--- CH_DBG_TRACE_BUFFER_SIZE:           128
--- CH_DBG_ENABLE_STACK_CHECK:          0
--- CH_DBG_FILL_THREADS:                0
--- CH_DBG_THREADS_PROFILING:           0
--- Result: SUCCESS
============================================================================
=== Test Sequence 2 (System layer and port interface)
----------------------------------------------------------------------------
--- Test Case 2.1 (System integrity functionality)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 2.2 (Critical zones functionality)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 2.3 (Interrupts handling functionality)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 2.4 (System Tick Counter functionality)
--- Result: SUCCESS
============================================================================
=== Test Sequence 3 (Threads Functionality)
----------------------------------------------------------------------------
--- Test Case 3.1 (Thread Sleep functionality)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 3.2 (Ready List functionality, threads priority order)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 3.3 (Priority change test)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 3.4 (Priority change test with Priority Inheritance)
--- Result: SUCCESS
============================================================================
=== Test Sequence 4 (Suspend/Resume)
----------------------------------------------------------------------------
--- Test Case 4.1 (Suspend and Resume functionality)
--- Result: SUCCESS
============================================================================
=== Test Sequence 5 (Counter Semaphores)
----------------------------------------------------------------------------
--- Test Case 5.1 (Semaphore primitives, no state change)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 5.2 (Semaphore enqueuing test)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 5.3 (Semaphore timeout test)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 5.4 (Testing chSemAddCounterI() functionality)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 5.5 (Testing chSemWaitSignal() functionality)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 5.6 (Testing Binary Semaphores special case)
--- Result: SUCCESS
============================================================================
=== Test Sequence 6 (Mutexes, Condition Variables and Priority Inheritance)
----------------------------------------------------------------------------
--- Test Case 6.1 (Priority enqueuing test)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 6.2 (Priority return verification)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 6.3 (Repeated locks, non recursive scenario)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 6.4 (Condition Variable signal test)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 6.5 (Condition Variable broadcast test)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 6.6 (Condition Variable priority boost test)
--- Result: SUCCESS
============================================================================
=== Test Sequence 7 (Synchronous Messages)
----------------------------------------------------------------------------
--- Test Case 7.1 (Messages Server loop)
--- Result: SUCCESS
============================================================================
=== Test Sequence 8 (Event Sources and Event Flags)
----------------------------------------------------------------------------
--- Test Case 8.1 (Events registration)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 8.2 (Event Flags dispatching)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 8.3 (Events Flags wait using chEvtWaitOne())
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 8.4 (Events Flags wait using chEvtWaitAny())
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 8.5 (Events Flags wait using chEvtWaitAll())
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 8.6 (Events Flags wait timeouts)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 8.7 (Broadcasting using chEvtBroadcast())
--- Result: SUCCESS
============================================================================
=== Test Sequence 9 (Dynamic threads)
----------------------------------------------------------------------------
--- Test Case 9.1 (Threads creation from Memory Heap)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 9.2 (Threads creation from Memory Pool)
--- Result: SUCCESS
============================================================================
=== Test Sequence 10 (Benchmarks)
----------------------------------------------------------------------------
--- Test Case 10.1 (Messages performance #1)
--- Score : 388342 msgs/S, 776684 ctxswc/S
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 10.2 (Messages performance #2)
--- Score : 306508 msgs/S, 613016 ctxswc/S
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 10.3 (Messages performance #3)
--- Score : 306508 msgs/S, 613016 ctxswc/S
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 10.4 (Context Switch performance)
--- Score : 1219032 ctxswc/S
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 10.5 (Threads performance, full cycle)
--- Score : 224714 threads/S
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 10.6 (Threads performance, create/exit only)
--- Score : 296291 threads/S
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 10.7 (Mass reschedule performance)
--- Score : 96385 reschedules/S, 578310 ctxswc/S
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 10.8 (Round-Robin voluntary reschedule)
--- Score : 751160 ctxswc/S
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 10.9 (Virtual Timers set/reset performance)
--- Score : 765156 timers/S
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 10.10 (Semaphores wait/signal performance)
--- Score : 1893476 wait+signal/S
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 10.11 (Mutexes lock/unlock performance)
--- Score : 972636 lock+unlock/S
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 10.12 (RAM Footprint)
--- System: 120 bytes
--- Thread: 68 bytes
--- Timer : 20 bytes
--- Semaph: 12 bytes
--- Mutex : 16 bytes
--- CondV.: 8 bytes
--- EventS: 4 bytes
--- EventL: 20 bytes
--- MailB.: 40 bytes
--- Result: SUCCESS
----------------------------------------------------------------------------

Final result: SUCCESS

*** ChibiOS OS Library Test Suite
***
*** Compiled:     Jan 26 2019 - 09:33:54
*** Platform:     STM32L4xx Ultra Low Power
*** Test Board:   STMicroelectronics STM32 Nucleo64-L476RG

============================================================================
=== Test Sequence 1 (Mailboxes)
----------------------------------------------------------------------------
--- Test Case 1.1 (Mailbox normal API, non-blocking tests)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 1.2 (Mailbox I-Class API, non-blocking tests)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 1.3 (Mailbox timeouts)
--- Result: SUCCESS
============================================================================
=== Test Sequence 2 (Pipes)
----------------------------------------------------------------------------
--- Test Case 2.1 (Pipes normal API, non-blocking tests)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 2.2 (Pipe timeouts)
--- Result: SUCCESS
============================================================================
=== Test Sequence 3 (Memory Pools)
----------------------------------------------------------------------------
--- Test Case 3.1 (Loading and emptying a memory pool)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 3.2 (Loading and emptying a guarded memory pool without waiting)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 3.3 (Guarded Memory Pools timeout)
--- Result: SUCCESS
============================================================================
=== Test Sequence 4 (Memory Heaps)
----------------------------------------------------------------------------
--- Test Case 4.1 (Allocation and fragmentation)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 4.2 (Default Heap)
--- Result: SUCCESS
============================================================================
=== Test Sequence 5 (Objects Factory)
----------------------------------------------------------------------------
--- Test Case 5.1 (Objects Registry)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 5.2 (Dynamic Buffers Factory)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 5.3 (Dynamic Semaphores Factory)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 5.4 (Dynamic Mailboxes Factory)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 5.5 (Dynamic Objects FIFOs Factory)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 5.6 (Dynamic Pipes Factory)
--- Result: SUCCESS
----------------------------------------------------------------------------

Final result: SUCCESS


Giovanni

[wurstnase]

I like benchmarks. Yellow marked compiler are not from ARM (bleeding-edge).
Board used is a Nucleo F446RE.

[Giovanni]

Thanks for comparison.

My idea is that it is not getting better, I would expect the compiler to improve after all the works that goes into it.

It would be interesting to compare also 5.x and 6.x.

Giovanni

[alex31]

Hi,

My idea is that it is not getting better, I would expect the compiler to improve after all the works that goes into it.

Perhaps it is improving, but at other (higher) level of optimization ?

If it's easy for you, a benchmark with -Ofast would be useful.

Alexandre

[wurstnase]

I could make some more tests also with Ofast.

GCC 6 is available from ARM. But GCC 5 only for 32bit linux. I found a 5.3. 64bit.

Probably after this weekend.

[wurstnase]

Here we go:

Os

O2

Ofast

[Giovanni]

Interesting, 7.3 seems to be the best for -O2 and -Os,

Could you order from the lowest version upward? it would be interesting to see the delta over time.

This is very useful, perhaps I should include more benchmarks to the suite, the ones we have are very RTOS-centric, it would be nice to have more generic ones for different kind of workloads (memory, float etc).

Giovanni

[wurstnase]

Here is the excel file with all results. Simply copy a column C to I into B for compare.

[apmorton]

I know I am a few months late, but figured I would add my two cents anyway.

I can't speak to any arm-specific work done on the arm-none-eabi branch of gcc (or the commercial arm clang compiler), but I can offer some observations about the changes and work going into gcc/clang as a whole in the last few years.

Clang has raised the bar significantly in terms of developer experience - better error messages, better diagnostics, better tooling, etc.
The GCC contributors have responded by putting a whole lot of work into their own developer experience improvements over the last few versions.

None of that work directly translates to improvements in code size or speed, but it is equally important if you ask me - because it improves the worst part of writing code: finding your silly mistakes.

On that point alone, I would personally take a small performance or size hit just to be using the newer compilers that give me access to better diagnostics during development. Understandably not everybody can afford even a few percentage difference in size or performance. I actually have some bootloader projects that have fit on flash so closely that bumping the version number in preparation for release caused a linker overflow because the new version number was an extra few characters long.

The other place I see the majority of compiler work happening is C++ language support.

Take a look at the GCC C++ Standards Support page: https://gcc.gnu.org/projects/cxx-status.html

Since 2011 C++ development has picked up rapidly, with a new standards release being minted every 3 years.
GCC was C++11 feature complete in 4.8, and every version since then has had a major focus being support of new language/library features of subsequent C++ standards.

Like it or not, C++ is attracting a lot of compiler maintainers attention these days.
I don't see this as a bad thing personally - because a whole lot of C++ features that have landed post C++11 are incredibly useful for embedded developers (if you haven't read up on constexpr, you really should - its an absolutely amazing tool to have in your box).

Perhaps it is improving, but at other (higher) level of optimization ?

This is exactly correct - a ton of work has been done in compilers in the name of optimization - its just that a lot of it is happening for C++, and not C. I would largely expect C codegen to stay constant across compiler versions these days - short of any regressions that simply aren't caught during the development cycle because people aren't testing for them, which is what I think you are seeing here.