I2C Bus error handling

ChibiOS public support forum for all topics not covered by a specific support forum.

Moderators: RoccoMarco, lbednarz, utzig, tfAteba, barthess

User avatar
nosense
Posts: 22
Joined: Mon Oct 08, 2012 7:15 pm
Location: Czech republic
Contact:

I2C Bus error handling

Postby nosense » Mon Nov 19, 2012 11:28 pm

Hello

I am testing I2C error recovery (pulling SDA to zero externally - simulated noise) and I am using this code to read data from I2C each 500 ms:

Code: Select all

void getL3G4200GyroData(L3G4200Handle *handle, L3G4200GyroData *data){
   uint8_t reg_addr = L3G4200_GYRO_DATA;
   msg_t msg = i2cMasterTransmitTimeout(handle->i2cd, L3G4200_I2C_ADDRESS, &reg_addr, 1, (uint8_t*)data, 6, I2C_TIMEOUT);
   if (msg!=RDY_OK) {
      chprintf((BaseChannel *)&SD2, "I2C failed");
      i2cStop(&I2CD1);
      I2C1->CR1 |= I2C_CR1_SWRST;
      i2cStart(&I2CD1, &i2cCFG);
   }
}

Sometimes the i2cMasterTransmitTimeout function returns error as I expected, but sometimes the MCU gets stuck in I2C ISR. This stuck state of I2C is captured on attached debug screenshot. I think that i2c_lld_serve_event_interrupt() function should include some error handling options. In error state there is set BERR bit in I2C_SR1 register.

Reference manual says:
Bit 8 BERR: Bus error
0: No misplaced Start or Stop condition
1: Misplaced Start or Stop condition
–Set by hardware when the interface detects an SDA rising or falling edge while SCL is high,
occurring in a non-valid position during a byte transfer.
–Cleared by software writing 0, or by hardware when PE=0.


As the i2c_lld_serve_event_interrupt() function doesn't perform any action to clean this bit, the interrupt is fired again and again. ITERREN in I2C_CR2 is set and refman says:
Bit 8 ITERREN: Error interrupt enable
0: Error interrupt disabled
1: Error interrupt enabled
This interrupt is generated when:
– BERR = 1
– ARLO = 1
– AF = 1
– OVR = 1
– PECERR = 1
– TIMEOUT = 1
– SMBALERT = 1


I think that i2c error should be serviced and information should be passed back to "user space". I know that this sort of bus errors should never occur, but in real world I have to be prepared for everything and MCU freeze is not an option in any case.

I would appreciate any advice to make the I2C code "bulletproof".

Environment: ChibiOs 2.4.2, STM32F100x6
Attachments
i2c_stuck.png
i2c_stuck.png (234.96 KiB) Viewed 9327 times

User avatar
Giovanni
Site Admin
Posts: 14455
Joined: Wed May 27, 2009 8:48 am
Location: Salerno, Italy
Has thanked: 1076 times
Been thanked: 922 times
Contact:

Re: I2C Bus error handling

Postby Giovanni » Tue Nov 20, 2012 12:23 pm

Hi,

Let's wait for Barthess opinion, he is our I2C guru :)

Anyway if you have an idea on how to fix this submit a patch, even an incomplete one, passing the error to the user level would not be a problem.

Giovanni

User avatar
nosense
Posts: 22
Joined: Mon Oct 08, 2012 7:15 pm
Location: Czech republic
Contact:

Re: I2C Bus error handling

Postby nosense » Tue Nov 20, 2012 9:59 pm

I don't know how the low level I2C driver exactly works. So I can't offer any concrete solution. For my project I need rock solid I2C communication. Any glitch can lead to several damages. Therefor I will probably use only ChibiOS kernel and write own peripheral drivers. I think it will eventually take less time than digging into ChibiOS HAL to prove that everything is done correctly.

mabl
Posts: 417
Joined: Tue Dec 21, 2010 10:19 am
Location: Karlsruhe, Germany
Been thanked: 1 time
Contact:

Re: I2C Bus error handling

Postby mabl » Wed Nov 21, 2012 7:31 am

On the other hand, the ChibiOS drivers had quite some time to mature and already work around hardware bugs etc. If I were you, I'd look into making those driver more stable and validate that they work. Never underestimate the time you need to write working drivers..... :twisted:

User avatar
nosense
Posts: 22
Joined: Mon Oct 08, 2012 7:15 pm
Location: Czech republic
Contact:

Re: I2C Bus error handling

Postby nosense » Fri Nov 23, 2012 11:53 am

I agree that writing well working general drivers is a challenging task (thats probably why the STM32F1 I2C low level driver takes more than 1000 lines of code). But when I implement own driver I don't have to care about higher level interface and other things. That makes the task quite easier. I don't say that the current driver is bad. So far I have been using it without problem. But now I need to focus on reliability because any glitch can results in costly damages. In this case I simply can't trust the code that is not able to recover from bus error. But because I haven't received any feedback to problem itself there can still be some errors on my side.

But there is one another problem with improving ChibiOs drivers. I may not be allowed to release source code of my app, definitely not immediately. That is why I have to stay on stable release according to ChibiOs license. Last stable version is from July - 4 months old. I simply can't wait several months to propagate committed patches to version that I can use.

User avatar
Giovanni
Site Admin
Posts: 14455
Joined: Wed May 27, 2009 8:48 am
Location: Salerno, Italy
Has thanked: 1076 times
Been thanked: 922 times
Contact:

Re: I2C Bus error handling

Postby Giovanni » Fri Nov 23, 2012 12:07 pm

The I2C driver is so complex because it is filled of workarounds for the poor I2C in the STM32F1. It went through 3 complete rewrites. If you need extreme reliability on I2C probably a bit banged I2C would be your best option.

Giovanni

User avatar
nosense
Posts: 22
Joined: Mon Oct 08, 2012 7:15 pm
Location: Czech republic
Contact:

Re: I2C Bus error handling

Postby nosense » Fri Nov 23, 2012 8:38 pm

Ok today I have done some more digging to find a problem. I was curious why the safety timer doesn't work. After some tweaking in low level driver code I gave up and restored original file. Funny thing is that since than I can't reproduce the problem anymore. No matter how I try to abuse the I2C bus, the communication is always restored. So for now consider this whole topic to be solved. However I am still quite confused what was wrong.

Edit: Not true. Problem is still there see the later posts.
Last edited by nosense on Wed Nov 28, 2012 7:27 pm, edited 1 time in total.

User avatar
Giovanni
Site Admin
Posts: 14455
Joined: Wed May 27, 2009 8:48 am
Location: Salerno, Italy
Has thanked: 1076 times
Been thanked: 922 times
Contact:

Re: I2C Bus error handling

Postby Giovanni » Fri Nov 23, 2012 8:55 pm

Hard to tell, the timeout should release the invoking thread even in case of problems in the I2C HW. We had to add that as an extreme safety net.
BTW the STM32 I2C driver has been by far one of the more problematic because the HW quirks, don't expect the same level of complexity in most other drivers.

Giovanni

User avatar
barthess
Posts: 861
Joined: Wed Dec 08, 2010 7:55 pm
Location: Minsk, Belarus
Been thanked: 7 times

Re: I2C Bus error handling

Postby barthess » Mon Nov 26, 2012 9:21 pm

Hi nosense.
Hm... strange bug. Any way, I have never try to simulate glitches on bus.
Just a curious. How you restore you I2C slaves after lockups? Are you power down all them or have some personal switches? Or something else?

User avatar
nosense
Posts: 22
Joined: Mon Oct 08, 2012 7:15 pm
Location: Czech republic
Contact:

Re: I2C Bus error handling

Postby nosense » Tue Nov 27, 2012 3:42 pm

I actually don't do anything with slaves. When I introduce bus error then the current transaction fails. When MCU starts another I2C transaction the I2C slave responds normally (tested with L3G4200). Problem was on the MCU side which stopped responding being locked in I2C interrupt routine. During this lock the SDA and SCL were released (at 3.3V level by pull-ups) so the state of MCU wasn't influenced by slaves.
Last edited by nosense on Wed Nov 28, 2012 7:27 pm, edited 1 time in total.


Return to “General Support”

Who is online

Users browsing this forum: No registered users and 10 guests