STM32F7 - ETH linker file

[elagil]

Hello,

I am trying to move the ethernet buffer memory to external SDRAM. In this process, I first want to move it to another region in internal RAM as a testing measure. For this purpose, I used the alternative linker file that is provided with ChibiOS (17.6.4) without modifications:

STM32F746xG_ETH.ld

instead of the normal

STM32F746xG.ld

In the standard linker file the memory region that is used by ETH is within ram3 (DTCM: org = 0x20000000, len = 64k). Now, it is in ram2 (org = 0x2004C000, len = 16k).

With this change, my ethernet application, which uses DHCP, does not successfully acquire a DHCP lease (stuck at netif_default->dhcp->state = 6 [DHCP_SELECTING]). With the normal linker script, it works fine.

Do you have an idea, why lwip works differently when moved to another memory region?

Thank you,
Adrian

[Giovanni]

Hi,

It is very easy, cache coherence is not ensured in HW and the driver does not do that for you. You need to mark the RAM area as non cacheable using MPU.

See this article: http://chibios.org/dokuwiki/doku.php?id ... _dma_guide

Note that the problem is not limited to Ethernet but all bus masters (DMAs for example).

Giovanni

[elagil]

Nice, thank you. That seems to work.

I chose 0x20030000 as my base address for Ethernet. This is the relevant section of the STM32F746xG_ETH.ld.ld file, the rest is untouched:

Code: Select all

MEMORY
{
    flash0  : org = 0x08000000, len = 1M        /* Flash as AXIM (writable) */
    flash1  : org = 0x00200000, len = 1M        /* Flash as ITCM */
    flash2  : org = 0x00000000, len = 0
    flash3  : org = 0x00000000, len = 0
    flash4  : org = 0x00000000, len = 0
    flash5  : org = 0x00000000, len = 0
    flash6  : org = 0x00000000, len = 0
    flash7  : org = 0x00000000, len = 0
    ram0    : org = 0x20010000, len = 256k      /* SRAM1 + SRAM2 */
    ram1    : org = 0x20010000, len = 128k      /* SRAM1 */
    ram2    : org = 0x20030000, len = 128k      /* SRAM2 */
    ram3    : org = 0x20000000, len = 64k       /* DTCM-RAM */
    ram4    : org = 0x00000000, len = 16k       /* ITCM-RAM */
    ram5    : org = 0x40024000, len = 4k        /* BCKP SRAM */
    ram6    : org = 0x00000000, len = 0
    ram7    : org = 0x00000000, len = 0
}


I added this in my main.c before lwipInit(), inspired by some code I found in hal_lld.c:

Code: Select all

  mpuConfigureRegion(MPU_REGION_7,
                     0x20030000, // This is the base address of the memory region
                     MPU_RASR_ATTR_AP_RW_RW |
                     MPU_RASR_ATTR_NON_CACHEABLE |
                     MPU_RASR_SIZE_128K |   // This is the size of the memory region. Base address has to be aligned to this size!
                     MPU_RASR_ENABLE);

  mpuEnable(MPU_CTRL_PRIVDEFENA);

  /* Invalidating data cache to make sure that the MPU settings are taken
     immediately.*/
  SCB_CleanInvalidateDCache();


The linker outputs these lines in the map file, so I guess that the new memory region is used correctly:

Code: Select all

.eth            0x20030000     0x2418
                0x20030000                __eth_base__ = .


DHCP works now!

[steved]

I added this in my main.c before lwipInit(), inspired by some code I found in hal_lld.c:

Code: Select all

  mpuConfigureRegion(MPU_REGION_7,
                     0x20030000, // This is the base address of the memory region
                     MPU_RASR_ATTR_AP_RW_RW |
                     MPU_RASR_ATTR_NON_CACHEABLE |
                     MPU_RASR_SIZE_128K |   // This is the size of the memory region. Base address has to be aligned to this size!
                     MPU_RASR_ENABLE);

  mpuEnable(MPU_CTRL_PRIVDEFENA);

  /* Invalidating data cache to make sure that the MPU settings are taken
     immediately.*/
  SCB_CleanInvalidateDCache();


It may be prudent to change MPU_REGION_7 to something else, to avoid conflicts between your settings and the HAL code if enabled.

IIRC there are 8 regions you can use, which are basically register sets within the MPU, in a fixed hierarchy.

So choosing MPU_REGION_3, say, will avoid potential conflict with existing Chibi code, and not affect behaviour as long as all MPU regions are non-overlapping.
(A helper function or two to assist with finding an unused region might be useful).

[elagil]

You are right, that makes sense.

Also, I found out that changing the ETH section does not actually move the relevant buffers of Lwip. I learned that there are two memory concepts used in Lwip: Heap and pools. Heap is mainly used for transmission buffers, pools for everything else. If you want to move heap to another location, make sure to set:

Code: Select all

#ifndef LWIP_RAM_HEAP_POINTER
#define LWIP_RAM_HEAP_POINTER (void *)0xC0200000UL  // free storage for heap
#endif


in lwipopts.h. A void pointer is expected for the heap pointer that Lwip uses. This memory region should not be used otherwise (e.g. linker). I chose a place in SDRAM.

If you want to move your pools as well, you need to define this first in lwipopts.h:

Code: Select all

/**
 * MEMP_SEPARATE_POOLS: if defined to 1, each pool is placed in its own array.
 * This can be used to individually change the location of each pool.
 * Default is one big array for all pools
 */
#ifndef MEMP_SEPARATE_POOLS
#define MEMP_SEPARATE_POOLS             1
#endif


Then, you can alter the pool definitions. It is suggested within the memp.c file that memory regions are declared as extern in "arch/cc.h":

Extract from memp.c:

Code: Select all

#if MEMP_SEPARATE_POOLS

/** This creates each memory pool. These are named memp_memory_XXX_base (where
 * XXX is the name of the pool defined in memp_std.h).
 * To relocate a pool, declare it as extern in cc.h. Example for GCC:
 *   extern u8_t __attribute__((section(".onchip_mem"))) memp_memory_UDP_PCB_base[];
 */
#define LWIP_MEMPOOL(name,num,size,desc) u8_t memp_memory_ ## name ## _base \
  [((num) * (MEMP_SIZE + MEMP_ALIGN_SIZE(size)))];   
#include "lwip/memp_std.h"

/** This array holds the base of each memory pool. */
static u8_t *const memp_bases[] = {
#define LWIP_MEMPOOL(name,num,size,desc) memp_memory_ ## name ## _base,   
#include "lwip/memp_std.h"
};


For names of all available (and used) memory regions, look at memp_std.h.

You can then add this to ch.h, just at the end:

Code: Select all

extern u8_t __attribute__((section(".ram7"))) memp_memory_PBUF_POOL_base[];
extern u8_t __attribute__((section(".ram7"))) memp_memory_RAW_PCB_base[];
extern u8_t __attribute__((section(".ram7"))) memp_memory_UDP_PCB_base[];
extern u8_t __attribute__((section(".ram7"))) memp_memory_TCP_PCB_base[];
extern u8_t __attribute__((section(".ram7"))) memp_memory_TCP_PCB_LISTEN_base[];
extern u8_t __attribute__((section(".ram7"))) memp_memory_TCP_SEG_base[];
extern u8_t __attribute__((section(".ram7"))) memp_memory_REASSDATA_base[];
extern u8_t __attribute__((section(".ram7"))) memp_memory_FRAG_PBUF_base[];
extern u8_t __attribute__((section(".ram7"))) memp_memory_NETBUF_base[];
extern u8_t __attribute__((section(".ram7"))) memp_memory_NETCONN_base[];
extern u8_t __attribute__((section(".ram7"))) memp_memory_FRAG_PBUF_base[];


"ram7" is a section that is defined in the .ld file. In my case, it starts at 0xC0000000.

I created an additional file for not having to change the lwip code base (lwippoolloc.h):

Code: Select all

/*
 * lwippoolloc.h
 *
 *  Created on: Apr 5, 2018
 *      Author: Useradmin
 */

#ifndef LWIPPOOLLOC_H_
#define LWIPPOOLLOC_H_

#include "arch/cc.h"
#include "lwip/opt.h"

#if MEMP_POOLS_RELOCATE && MEMP_SEPARATE_POOLS

extern u8_t __attribute__((section(".ram7"))) memp_memory_PBUF_POOL_base[];

#if LWIP_RAW
extern u8_t __attribute__((section(".ram7"))) memp_memory_RAW_PCB_base[];
#endif /* LWIP_RAW */

#if LWIP_UDP
extern u8_t __attribute__((section(".ram7"))) memp_memory_UDP_PCB_base[];
#endif /* LWIP_UDP */

#if LWIP_TCP
extern u8_t __attribute__((section(".ram7"))) memp_memory_TCP_PCB_base[];
extern u8_t __attribute__((section(".ram7"))) memp_memory_TCP_PCB_LISTEN_base[];
extern u8_t __attribute__((section(".ram7"))) memp_memory_TCP_SEG_base[];
#endif /* LWIP_TCP */

#if IP_REASSEMBLY
extern u8_t __attribute__((section(".ram7"))) memp_memory_REASSDATA_base[];
#endif /* IP_REASSEMBLY */

#if IP_FRAG && !IP_FRAG_USES_STATIC_BUF && !LWIP_NETIF_TX_SINGLE_PBUF
extern u8_t __attribute__((section(".ram7"))) memp_memory_FRAG_PBUF_base[];
#endif /* IP_FRAG && !IP_FRAG_USES_STATIC_BUF && !LWIP_NETIF_TX_SINGLE_PBUF */

#if LWIP_NETCONN
extern u8_t __attribute__((section(".ram7"))) memp_memory_NETBUF_base[];
extern u8_t __attribute__((section(".ram7"))) memp_memory_NETCONN_base[];
#endif /* LWIP_NETCONN */

extern u8_t __attribute__((section(".ram7"))) memp_memory_FRAG_PBUF_base[];

#endif

#endif /* LWIPPOOLLOC_H_ */


It is included in lwipopts.h at the very end, along with a new setting "MEMP_POOLS_RELOCATE":

Code: Select all

// ... Other code

#ifndef MEMP_POOLS_RELOCATE
#define MEMP_POOLS_RELOCATE             1
#endif

// ... Other code

#if MEMP_POOLS_RELOCATE
#include "lwippoolloc.h"
#endif

#endif /* __LWIPOPT_H__ */


Make sure to disable caching for pool and heap!