Usage of USB driver in isochronous mode (STM32)

[iggarpe]

Hi all,

I'm using ChibiOS/RT trunk on a STM32F407 based board and implementing an USB audio device. So far everything works fine except the EP1 OUT isochronous transfer. It seems every other packets is lost, or in any case the packet loss rate is exactly 50%, because wireshark tells me the host is sending 1000 packets per second and the STM32 is receiving exactly 500 per second.

(BTW, quite interestingly the host seems to be packing 8 isochronous descriptors in every URB isochronous transfer, anyone knows why?... in any case it's certain it's sending 1000/s)

I only had at hand one example of the use of the USB driver, the serial-over-USB example, and since it works quite differently closer examintation of it is not helping. This is what I do:

1- Call usbInitEndpointI on USB_EVENT_CONFIGURED.

2- Call usbPrepareReceive and usbStartReceiveI from the requests callback, when the remote host sets the audio interface alternate setting (see note 1).

3- Call usbPrepareReceve and usbStartReceiveI from the data receive callback.

It works as expected: when the interface is enabled by the host the first receive is prepared and started, and then in the data receive callback the next receive is prepared and started. The loop continues as long as the device is enabled, but the callback is being called 500 times per second while packets are being sent from the host at a 1000/s rate.

The following might be interesting to those familiar with the low level driver: while I was trying to get the thing to work I compared initialization and register values against a working audio device that I had using the ST USB library, and I noticed that when usb_lld writes the DOEPCTL1 the NAKSTS bit gets set, and that doesn't happen with the ST USB library. No idea why. I just mention it in case it's rings someone's bell.

I'm out of ideas. Any hint will be much appreciated.

EDIT 1: I just stumbled upon some interesting code in the working ST USB lib based project: the odd/even frame bit apparently needs to be toggled upon reception of each frame. I'm no USB expert, anyone thinks this might be related?.

EDIT 2: Confirmed!, isochronous transfers DO NOT WORK with the current STM32 OTG driver. I did a quick patch to the driver implementing the odd/even frame bit toggle for isochronous endpoints and now everything works like a charm, 0% packet loss. I will post to the development forum regarding this topic and hopefully will be properly fixed soon (my 10 minute hack is ugly as hell).

Note 1: in USB audio you always have two alternate interface configurations: alternate 0 has no endpoints and is selected by the host to disable the audio device. Alternate 1 is the working one and is selected by the host to enable the audio device.

[Richard Aplin]

This is an old thread but I just banged my head against a bug for several days and here's the answer....

When doing asynchronous USB audio (where you have a feedback endpoint that sends a 3-byte feedback value to the host, typically every 32ms (e.g. 1<<5 in the 'refresh' field of the endpoint descriptor) this is the problem you have...

When tx'ing data to the host on an ISO IN endpoint, the STM OTG controller will only send a queued packet when the host issues an IN on the endpoint *IF*
DIEPCTL for the endpoint has the "DIEPCTL_SEVNFRM" or "DIEPCTL_SODDFRM" (i.e. the queued frame has the ODD or EVEN bit set) to match the current USB frame number when the IN appears from the host. (USB frame number is sent by host in the SOF packet).
IF (for example) you queue up a packet with SODDFRM set (i.e. send it only on odd frame) and at the moment the PC sends an IN packet it's an even frame - the packet won't be sent (it just sits there) - the STM sends an empty packet to the host instead. errrr...thanks a lot.

When sending streaming audio (which is typically polled every frame) you need to alternate between setting SODDFRM and SEVENFRM on your packets.
If you really want something sent on the next frame you should check the value sent in the last SOF (by reading the frame number from OTG_FS_DSTS) and use the opposite, i.e. AND it with 1 and invert.
For audio streaming endpoints doesn't matter too much because the EP is polled every frame and if you get it wrong your frame is just delayed by 1ms until on the next frame ODD/EVEN matches and your data is sent.

*HOWEVER*
For the feedback endpoint, you typically set it in the descriptor to only be polled every 32ms (=32 frames, in fact you can only set power-of-two times, e.g. 4ms, 8ms, 16ms etc).
Hence, when the host issues an IN on your feedback endpoint, it will ALWAYS be either an odd or even frame (randomly determined by the host when it starts playing audio), so if you get the ODD/EVEN bit wrong on your queued packet, your endpoint will never send any feedback data!! (bangs head on nearest wall)

This is super annoying; the symptom is that about 50% of the time when you hit "play" to stream audio you see empty packets transmitted on your feedback endpoint.

The solution is rather fiddly; because you have no idea when first queuing your feedback packet whether the host is going to issue the IN on an even or odd frame; you just pick one, queue the packet, then keep checking your endpoint every SOF to see if the packet has been transmitted yet. IF your packet is still sitting in the endpoint after >32ms, then flip the odd/even bit (just set SEVENFRM or SODDFRM to the opposite of what you set when you queued the packet, you don't need to re-queue the packet itself) and wait another 32ms.

Once you find which odd/even frame phase the host is issuing the IN command on, it will (should) keep in the same phase until it shuts down the endpoint (on a windows host, typically when you stop playing audio)

I don't think there is any better way to do it than this hack. (!!!!)

That was exceedingly annoying to find. I have no idea why on earth the OTG controller implements an odd/even filter (which can't be disabled!) for ISO IN endpoints; there seems no purpose in it.

[Giovanni]

Hi,

Thanks for your analysis.

I was aware that the even/odd handling was missing from the driver, an alternate solution could be to store in the transaction structure the last even/odd bit sent and flip it each time a packet is sent, this should allow to not have to rely on the SOF frame.

Giovanni

[Richard Aplin]

Hi Giovanni,
Actually no; someone else on my team had implemented the odd/even flip-on-transmit you mention but that's not the problem I was talking about; it's more complex than that:
The odd/even bit determines which odd/even SOF frame that the STM will transmit an IN packet (if wrong, the STM ignores the IN poll from the host and replies with a zero-length packet for ISOC endpoints) - the thing is that the host computer determines when it wants to poll the device; for endpoints carrying samples this is every frame so if you're "wrong" (i.e. the next frame is an even frame and you set the ODD bit) then all that happens is the first frame is missed, but the next one is correct; after that frame is transmitted you get an endpoint interrupt, flip the odd/even, and you're in sync; no problem.
The big problem I encountered (as will anyone else doing audio with asynchronous feedback) is that the feedback endpoint (which is an IN ISOC endpoint sending a 3-byte packet which tells the host how many samples to send in the next frame, for flow rate control) is that the host polls the feedback every N frames, where N is a power of 2; typically every 32 frames (and the starting frame is effectively random) hence if the host polls the device on frame #3, the next poll is on frame #35, and so on. Hence for the feedback endpoint every IN poll generated by the host is either an odd or even frame consistently.
Therefore you get a bug where the feedback endpoint "works randomly"; if you set EVEN on all your feedback frames and the host polls on frames #1, #33, #65 etc, then you'll never send a packet (and vice versa). If you're "lucky" and you set EVEN and the host decides to start polling on frame #0, #32, etc, then it works fine.

Because there's no way to know whether the host will start polling on odd or even frames, you have to have a timeout/flip-odd-even heuristic as I describe. Again, this specifically only applies to async feedback endpoints.

[Kosyak]

Hello.

I've faced isochronous transfers too, and interested in ChibiOS support for it.

There is some information I've digged:

• ISO OUT transfer described at pp. 1343-1346 of "RM0090: STM32F405/415, STM32F407/417, STM32F427/437 and STM32F429/439 advanced ARM®-based 32-bit MCUs Reference manual Rev.9":
  

  
  Application requirements:
  1. All the application requirements for non-isochronous OUT data transfers also apply to isochronous OUT data transfers.
  2. For isochronous OUT data transfers, the transfer size and packet count fields must always be set to the number of maximum-packet-size packets that can be received in a single frame and no more. Isochronous OUT data transfers cannot span more than 1 frame.
  3. The application must read all isochronous OUT data packets from the receive FIFO (data and status) before the end of the periodic frame (EOPF interrupt in OTG_FS_GINTSTS).
  4. To receive data in the following frame, an isochronous OUT endpoint must be enabled after the EOPF (OTG_FS_GINTSTS) and before the SOF (OTG_FS_GINTSTS).
  
  Internal data flow:
  1. The internal data flow for isochronous OUT endpoints is the same as that for non-isochronous OUT endpoints, but for a few differences.
  2. When an isochronous OUT endpoint is enabled by setting the Endpoint Enable and clearing the NAK bits, the Even/Odd frame bit must also be set appropriately. The core receives data on an isochronous OUT endpoint in a particular frame only if the following condition is met:
  – EONUM (in OTG_FS_DOEPCTLx) = SOFFN[0] (in OTG_FS_DSTS)
  3. When the application completely reads an isochronous OUT data packet (data and status) from the receive FIFO, the core updates the RXDPID field in OTG_FS_DOEPTSIZx with the data PID of the last isochronous OUT data packet read from the receive FIFO.
  
  Application programming sequence:
  1. Program the OTG_FS_DOEPTSIZx register for the transfer size and the corresponding packet count
  2. Program the OTG_FS_DOEPCTLx register with the endpoint characteristics and set the Endpoint Enable, ClearNAK, and Even/Odd frame bits.
  – EPENA = 1
  – CNAK = 1
  – EONUM = (0: Even/1: Odd)
  3. Wait for the RXFLVL interrupt (in OTG_FS_GINTSTS) and empty the data packets from the receive FIFO
  – This step can be repeated many times, depending on the transfer size.
  4. The assertion of the XFRC interrupt (in OTG_FS_DOEPINTx) marks the completion of the isochronous OUT data transfer. This interrupt does not necessarily mean that the data in memory are good. This interrupt cannot always be detected for isochronous OUT transfers. Instead, the application can detect the IISOOXFRM interrupt in OTG_FS_GINTSTS.
  5. Read the OTG_FS_DOEPTSIZx register to determine the size of the received transfer and to determine the validity of the data received in the frame. The application must treat the data received in memory as valid only if one of the following conditions is met:
  – RXDPID = D0 (in OTG_FS_DOEPTSIZx) and the number of USB packets in which this payload was received = 1
  – RXDPID = D1 (in OTG_FS_DOEPTSIZx) and the number of USB packets in which this payload was received = 2
  – RXDPID = D2 (in OTG_FS_DOEPTSIZx) and the number of USB packets in which this payload was received = 3
  The number of USB packets in which this payload was received = Application programmed initial packet count – Core updated final packet count. The application can discard invalid data packets.
  

• ISO IN transfer described in pp. 1351-1355 and seems has no mention of bit odd/even toggling
• There is code in new STM32Cube firmware:
  

  Code: Select all

    if (ep->is_in == 1)
  {
   
    /* Setup code skipped */
   
    if (ep->type == EP_TYPE_ISOC)
    {
      if ((USBx_DEVICE->DSTS & ( 1 << 8 )) == 0)
      {
        USBx_INEP(ep->num)->DIEPCTL |= USB_OTG_DIEPCTL_SODDFRM;
      }
      else
      {
        USBx_INEP(ep->num)->DIEPCTL |= USB_OTG_DIEPCTL_SD0PID_SEVNFRM;
      }
    }
   
    /* EP enable, IN data in FIFO */
    USBx_INEP(ep->num)->DIEPCTL |= (USB_OTG_DIEPCTL_CNAK | USB_OTG_DIEPCTL_EPENA);
   
    if (ep->type == EP_TYPE_ISOC)
    {
      USB_WritePacket(USBx, ep->xfer_buff, ep->num, ep->xfer_len, dma);   
    }   
  }
  else /* OUT endpoint */
  {
   
    /* Setup code skipped */
   
    if (ep->type == EP_TYPE_ISOC)
    {
      if ((USBx_DEVICE->DSTS & ( 1 << 8 )) == 0)
      {
        USBx_OUTEP(ep->num)->DOEPCTL |= USB_OTG_DOEPCTL_SODDFRM;
      }
      else
      {
        USBx_OUTEP(ep->num)->DOEPCTL |= USB_OTG_DOEPCTL_SD0PID_SEVNFRM;
      }
    }
    /* EP enable */
    USBx_OUTEP(ep->num)->DOEPCTL |= (USB_OTG_DOEPCTL_CNAK | USB_OTG_DOEPCTL_EPENA);
  }

  
  As you can see, bit toggling applied to both IN and OUT xfers. Bit value depends on Frame number stored in DSTS register. Maybe, we can do the same in ChibiOS's HAL (in usb_lld_start_out()/usb_lld_start_in())?
• As for isochronous feedback, there is some solution at ST's forum:
  

  I believe the Incomplete isochronous IN transfer interrupt (OTG_FS_GINTSTS.IISOIXFR) or GINTSTS.EOPF is the supposed way by the designer of this SIE.
  
  While your firmware primes a packet to an isoc IN endpoint, the SIE should raise this interrupt, if the host wouldn't retrieve the packet in the current frame. In this interrupt, your firmware toggles even/odd setting (OTG_FS_DIEPCTLx.EONUM) of the isoc endpoint by OTG_FS_DIEPCTLx.SODDFRM/SEVNFRM, so that it matches to the next frame number.
  
  At the next frame,
  If the host sends isoc transaction to the endpoint, no incomplete interrupt occurs.
  If the host doesn't put isoc transaction, the incomplete interrupt hits again, and above even/odd tuning is applied.
  

I'm planning to play with USB ISO in next several weeks when I'll have access to hardware.

[Giovanni]

Great, let us know if you get it to work, I would like to include a demo for this, maybe an audio playback using the DAC driver.

Giovanni

[Kosyak]

I've played with STM32's USB core for week, and what I've got:

First patch (https://gist.github.com/ObKo/a08174a1a2446ea6a208) adds support for bit toggling and invalid isochronous transfers handling.
"Isochronous" in our case means that:

• 1 (2, 3) Packets sended/received every (1, 2, 3, ...) frame.
• If packet wasn't sended/received at specified time (frame) it treated as missed.

STM32 USB core has interrupts for incomplete IN/OUT isochronous transfers (IISOI(O)XFR). They're assert when:

• IN: Packet was pushed to TX FIFO, endpoint prepared for TX, but USB host hasn't sent ISOC IN request during frame OR TX FIFO was empty.
• OUT: Endpoint prepared for RX, but host hasn't sent ISOC OUT request during frame OR there was no space left in RX FIFO.

IISOI(O)XFR interrupts are asserting at end of frame (800 uS - 950 uS after SOF). Main idea is that driver should notify user code not only in case of successful transfer, but also in case of transfer failure, so application can prepare transfer for next frame.

Here is UML sequence diagrams describing my solution:

• Completed ISOC IN
  
• Incompleted ISOC IN
  
• Completed ISOC OUT
  
• Incompleted ISOC OUT
  

Although this solution works fine for USB audio, there is some limitation / small problems:

• In case of incompleted IN transfer driver flushes WHOLE TX FIFO of endpoint without any notification to user code.
• TX FIFO flushing limits FIFO size for IN endpoint to 1 packet (otherwise driver will flush packets for next frames)
• When ISOC IN transfer failed, there is only ~200 uS for resuming pump thread and pumping data to TX FIFO.
• RM for F4 says that in case of invalid isoc transfer endpoint should be disabled (write EPDIS bit to DOEPCTL) immediatly. But USB core stucks during disabling OUT endpoint in otg_isoc_out_failed_handler (EPENA bit not going low, Enpoint Disable interrupt not asserting). usbDisableEndpointsI() stucks also. As I can see, in STM32Cube firmware, endpoint never disabled, and there is even no function for that. Found similiar issue: https://my.st.com/public/STe2ecommunities/mcu/Lists/cortex_mx_stm32/Flat.aspx?RootFolder=https%3a%2f%2fmy.st.com%2fpublic%2fSTe2ecommunities%2fmcu%2fLists%2fcortex_mx_stm32%2fSTM32%20USB%20disable%20endpoint%20interrupt%20is%20not%20generated&FolderCTID=0x01200200770978C69A1141439FE559EB459D7580009C4E14902C3CDE46A77F0FFD06506F5B&currentviews=138 Maybe, there is some info in Synposys docs, but I haven't access to it.

I don't know how to fix this issues without deep modification in USB driver / API.

Second patch (https://gist.github.com/ObKo/6f85089377dfc23c8c0e) adds simple demo of new stuff. The application demonstrates the use of the STM32 USB (OTG) driver in isochronous mode. Application provides both playback and capture interfaces and implements audio loopback (echo). It uses two isochronous endpoints - for playback and capture. Data sending/receiving occurs every frame using same single buffer. For testing I'm using audacity - it can capture and playback simultaneously. The demo runs on an ST STM32F4-Discovery board.

I'm also plannig to write demo using ADC, DAC and asynchronous feedback endpoint...

[Giovanni]

Thanks for working on this, I need time to understand what you did

What kind of changes would be required to the API?

Giovanni

[Kosyak]

In case of incompleted IN transfer driver flushes WHOLE TX FIFO of endpoint without any notification to user code.

IMHO, endpoint callback for invalid isochronous transfer required, so user application can decide what to do with packet in TX FIFO (some kind of FIFO flushing should be available in API too). Also, endpoint disabling for single endpoint can be useful too.

TX FIFO flushing limits FIFO size for IN endpoint to 1 packet (otherwise driver will flush packets for next frames)

I have no idea what to do with this. I/O queues can act as additional buffer in this case.

When ISOC IN transfer failed, there is only ~200 uS for resuming pump thread and pumping data to TX FIFO.

That's a problem. By default, 200 uS = 2 systicks. Increasing pump thread priority can help in some cases. Does chibios has mechanism that forces thread resuming right after interrupt?

[Giovanni]

What is a "failed transfer"? there is not such a thing with the other EP types.

Giovanni