Wednesday, 2 February 2011

Dell Poweredge 2950 Failure

Server failure
This server failed with the error "E171F PCIE Fatal Err B0 D3 F0" during power up. Upon inspection it was found that the 5/I PCIe SAS Controller card model no UCS-51 had a swollen electrolytic capacitor. This was replaced with one from an old PC motherboard, it had the same value (1500uF) and voltage rating (6.3v). After replacement the server booted and completed all it POST and began booting the OS. The server was soak tested and appeared to be operating fine. Hopefully saved an engineer call-out that probably would cost ~£500. By the way the machine was out of warranty. Below are pictures of the board and the repair.

Swollen Capacitor

Capacitor Repair
Supplemental Info A further 5 Dell servers came up with the same error and these were fixed by replacing the capacitors, bad batch of electrolytics.

27 Comments:

At 14 April 2011 at 09:08 , Blogger yutaka said...

Hello Jim-san.
Your blog makes us happy!

Same error happened at Power edge 1950.
We called DELL to get their support, but TAG Transfer from Sweden to Japan has not been finished yet, so that DELL Japan denied to support us.
By good fortune, we got your very helpful information on your blog, so power edge has been recovered.

Thanks a lot!!

Nishimura from Japan.

 
At 21 April 2011 at 15:44 , Blogger jarreau2001 said...

I have the same error on my 2950. We've got PERC 6/i. I spent a good while looking for something that looks like what you have pictured above. I found what I think is the controller but there are no capacitors on it. There's a sticker that says E2K-UCP-61-(B). I've inspected it with a magnifying glass and found nothing that looks like a bleeding or swollen capacitor.

 
At 21 April 2011 at 17:34 , Blogger jarreau2001 said...

For my fix, I reseated every device in the box, then updated BIOS and PERC 6/i firmware. All seems well at this point.

 
At 5 July 2011 at 06:51 , Blogger T said...

Already lost 15 servers with exact same error and swollen capacitor and 200 more servers to go. It seems I will be an expert in soldering when I am done with all. :-)

I just dont understand why all failing at the same time. We suspected power change but this issue is happening at two different data centers in the past two month.


All the Dell advance Support were in HighSchool when these LSI boards were designed.

 
At 28 July 2011 at 19:40 , Blogger Matt said...

Thanks for posting this, especially the pictures. I had the same error two days in a row on a Poweredge 1950. I opened up the box, and lo and behold, see the exact same bulging capacitor.

 
At 2 August 2011 at 10:48 , Blogger Ded_mazday said...

Same problem on PE 1950. Will try to replace capacitor on SAS controller.

tnx for post :)

 
At 25 January 2012 at 19:55 , Blogger Kevin Tower said...

Many thanks for this post! We've been having a rash of SAS 5i failures recently and this explains why -- they all show signs of cap swelling. Fortunately they're still under warranty, so I get to make Dell deal with it.

 
At 2 April 2012 at 17:47 , Blogger AskOtto said...

Just ran into this with a Dell 1950 II.

Dell had a similar (and huge) problem with Optiplex desktops in the mid 2000s. They sent techs out to replace the motherboards for free.

 
At 25 May 2012 at 16:47 , Blogger Brent Marnati said...

I just got this error too. I'll go check out the controller.
Thanks for sharing!

 
At 21 June 2012 at 23:07 , Blogger dthibode said...

thank you thank you thank you! I just had this same error on a poweredge 2950. I found the same capacitors in an old dell optiplex we were scrapping. rock on!

 
At 7 August 2012 at 06:27 , Blogger Heri said...

I have a Dell PE2950 with ESXi 4.0 and six 500GB drives. One VM guest that's living on the host is our office domain controller. I'm not sure of the ESX server RAID setup because we didn't setup the server, should figure this out soon. One of the hard drives failed, it's showing an amber light on the drive sled and vSphere says so in it's logs.
dell poweredge

 
At 26 October 2012 at 14:38 , Blogger peakrock said...

Thanks for putting this up...just mended my SAS 5/iR card which was giving a solid PCIe training error slot 1 by replacing the two capacitors...

 
At 15 November 2012 at 13:28 , Blogger Dordork said...

Poweredge 1950 with UCS-51, same error.(E171F
B0 D3 F0) Replaced Cap as suggested and running within 2 hours of the failure. Thanks for this Post. Saved my butt.

 
At 22 February 2013 at 23:02 , Blogger WynX said...

Brilliant article! Thanks a lot, I've got the same problem with Poweredge 2950, UCS-51.

 
At 25 February 2013 at 00:17 , Blogger WynX said...

This comment has been removed by the author.

 
At 25 February 2013 at 01:20 , Blogger WynX said...

Unfortunately the problem was not solved for me by replacing capacitators. The error did change, with now having the error:

Unable to load LSI Logic Corp. MPT BIOS
MPT BIOS Fault 10h encountered at adapter PCI(09h,03h,00h)
Press any key to continue...

It seems my LSI bios is corrupted? Have tried to reflash but the card never gets to a READY state... any ideas?

 
At 26 April 2013 at 00:12 , Blogger Gilbert Gutierrez said...

I have the same issue. I just ordered a board off ebay, but I would like to know what CAPs you used? I was looking at Nichicon HZ series and United Chemi-Con PSC series. Both of these caps are the same dimension as the original Sanyo Caps, but have lower ESR and higher ripple current. I don't know enough about capacitors and this board to know if that would be ok, or if I need the same value for ESR or ripple current.

 
At 30 April 2013 at 20:34 , Blogger jimbuzz said...

Lower ESR higher ripple current Caps should be fine in this application.

 
At 4 June 2013 at 05:59 , Blogger AdC said...

I just fixed 2 boards with this same problem. Used low-ESR 2200uF 16V, since that's all the shop had, and for power caps, can always go bigger.

Thanks a heap for posting. All boards running very nicely now.

 
At 20 June 2013 at 16:55 , Blogger jimbuzz said...

AdC good to hear you got boards fixed.

 
At 2 July 2013 at 18:06 , Blogger Clément Nussbaumer said...

It just worked perfectly ! Thank you very much !

Clément

 
At 31 July 2013 at 18:07 , Blogger betblog said...

Whaou man, so nice to share that kind of stuff !
Help very useful.

Maybe we would go for the solution to call a DELL Engineer.... This way I just need to make a report ! it's easier (and I don't have any capacitors at hand on board LNGC, Qatar !)

Bye bye and congrats for this community sharing act.

Hugo

 
At 11 December 2013 at 07:48 , Blogger Dogan Yilmaz said...

In fact I am not surprised that the problem caused by a swollen capacitor. Unfortunately I have seen lots of similar capacitor swollen problems on many different models of Dell computers. They should stop using low quality electrolytic capacitors asap.

 
At 19 December 2013 at 14:46 , Blogger chris said...

Any idead what parameters should have capacitor on perc 6i ?
C3568

http://img842.imageshack.us/img842/5736/9psb.jpg

 
At 1 February 2014 at 17:41 , OpenID ciscolinux0 said...

It has worked for me on PE 1950 Gen 2. I've replaced with the same values as original 1500uF 6.3V capacitor from an old desktop motherboard.

 
At 25 September 2014 at 10:46 , Blogger Christoph Gächter said...

Worked like a charm

Replaces with a capacitator from a HP computer mainboard

1500uF but 16V

(Voltage can be higher, but 1500uF should always stay the same!!)

 
At 17 September 2015 at 19:58 , Blogger Unknown said...

Salvou nossas vidas.... Pensávamos que era o disco SAS, mas na realidade ao substituir os capacitores funcionou de vento e polpa.
Valeu!!!!!

 

Post a Comment

Subscribe to Post Comments [Atom]

<< Home