I'm a leader not a follower!

I'm a leader not a follower!
Two Toads I rescued from the road!

Wednesday, February 2, 2011

Dell Poweredge 2950 Failure

Server failure
This server failed with the error "E171F PCIE Fatal Err B0 D3 F0" during power up. Upon inspection it was found that the 5/I PCIe SAS Controller card model no UCS-51 had a swollen electrolytic capacitor. This was replaced with one from an old PC motherboard, it had the same value (1500uF) and voltage rating (6.3v). After replacement the server booted and completed all it POST and began booting the OS. The server was soak tested and appeared to be operating fine. Hopefully saved an engineer call-out that probably would cost ~£500. By the way the machine was out of warranty. Below are pictures of the board and the repair.

Swollen Capacitor

Capacitor Repair
Supplemental Info A further 5 Dell servers came up with the same error and these were fixed by replacing the capacitors, bad batch of electrolytics.

26 comments:

  1. Hello Jim-san.
    Your blog makes us happy!

    Same error happened at Power edge 1950.
    We called DELL to get their support, but TAG Transfer from Sweden to Japan has not been finished yet, so that DELL Japan denied to support us.
    By good fortune, we got your very helpful information on your blog, so power edge has been recovered.

    Thanks a lot!!

    Nishimura from Japan.

    ReplyDelete
    Replies
    1. I have a Dell PE2950 with ESXi 4.0 and six 500GB drives. One VM guest that's living on the host is our office domain controller. I'm not sure of the ESX server RAID setup because we didn't setup the server, should figure this out soon. One of the hard drives failed, it's showing an amber light on the drive sled and vSphere says so in it's logs.
      dell poweredge

      Delete
  2. I have the same error on my 2950. We've got PERC 6/i. I spent a good while looking for something that looks like what you have pictured above. I found what I think is the controller but there are no capacitors on it. There's a sticker that says E2K-UCP-61-(B). I've inspected it with a magnifying glass and found nothing that looks like a bleeding or swollen capacitor.

    ReplyDelete
  3. For my fix, I reseated every device in the box, then updated BIOS and PERC 6/i firmware. All seems well at this point.

    ReplyDelete
  4. Already lost 15 servers with exact same error and swollen capacitor and 200 more servers to go. It seems I will be an expert in soldering when I am done with all. :-)

    I just dont understand why all failing at the same time. We suspected power change but this issue is happening at two different data centers in the past two month.


    All the Dell advance Support were in HighSchool when these LSI boards were designed.

    ReplyDelete
  5. Thanks for posting this, especially the pictures. I had the same error two days in a row on a Poweredge 1950. I opened up the box, and lo and behold, see the exact same bulging capacitor.

    ReplyDelete
  6. Same problem on PE 1950. Will try to replace capacitor on SAS controller.

    tnx for post :)

    ReplyDelete
  7. Many thanks for this post! We've been having a rash of SAS 5i failures recently and this explains why -- they all show signs of cap swelling. Fortunately they're still under warranty, so I get to make Dell deal with it.

    ReplyDelete
  8. Just ran into this with a Dell 1950 II.

    Dell had a similar (and huge) problem with Optiplex desktops in the mid 2000s. They sent techs out to replace the motherboards for free.

    ReplyDelete
  9. I just got this error too. I'll go check out the controller.
    Thanks for sharing!

    ReplyDelete
  10. thank you thank you thank you! I just had this same error on a poweredge 2950. I found the same capacitors in an old dell optiplex we were scrapping. rock on!

    ReplyDelete
  11. Thanks for putting this up...just mended my SAS 5/iR card which was giving a solid PCIe training error slot 1 by replacing the two capacitors...

    ReplyDelete
  12. Poweredge 1950 with UCS-51, same error.(E171F
    B0 D3 F0) Replaced Cap as suggested and running within 2 hours of the failure. Thanks for this Post. Saved my butt.

    ReplyDelete
  13. Brilliant article! Thanks a lot, I've got the same problem with Poweredge 2950, UCS-51.

    ReplyDelete
  14. This comment has been removed by the author.

    ReplyDelete
  15. Unfortunately the problem was not solved for me by replacing capacitators. The error did change, with now having the error:

    Unable to load LSI Logic Corp. MPT BIOS
    MPT BIOS Fault 10h encountered at adapter PCI(09h,03h,00h)
    Press any key to continue...

    It seems my LSI bios is corrupted? Have tried to reflash but the card never gets to a READY state... any ideas?

    ReplyDelete
  16. I have the same issue. I just ordered a board off ebay, but I would like to know what CAPs you used? I was looking at Nichicon HZ series and United Chemi-Con PSC series. Both of these caps are the same dimension as the original Sanyo Caps, but have lower ESR and higher ripple current. I don't know enough about capacitors and this board to know if that would be ok, or if I need the same value for ESR or ripple current.

    ReplyDelete
    Replies
    1. Lower ESR higher ripple current Caps should be fine in this application.

      Delete
  17. I just fixed 2 boards with this same problem. Used low-ESR 2200uF 16V, since that's all the shop had, and for power caps, can always go bigger.

    Thanks a heap for posting. All boards running very nicely now.

    ReplyDelete
  18. AdC good to hear you got boards fixed.

    ReplyDelete
  19. It just worked perfectly ! Thank you very much !

    Clément

    ReplyDelete
  20. Whaou man, so nice to share that kind of stuff !
    Help very useful.

    Maybe we would go for the solution to call a DELL Engineer.... This way I just need to make a report ! it's easier (and I don't have any capacitors at hand on board LNGC, Qatar !)

    Bye bye and congrats for this community sharing act.

    Hugo

    ReplyDelete
  21. In fact I am not surprised that the problem caused by a swollen capacitor. Unfortunately I have seen lots of similar capacitor swollen problems on many different models of Dell computers. They should stop using low quality electrolytic capacitors asap.

    ReplyDelete
  22. Any idead what parameters should have capacitor on perc 6i ?
    C3568

    http://img842.imageshack.us/img842/5736/9psb.jpg

    ReplyDelete
  23. It has worked for me on PE 1950 Gen 2. I've replaced with the same values as original 1500uF 6.3V capacitor from an old desktop motherboard.

    ReplyDelete
  24. Worked like a charm

    Replaces with a capacitator from a HP computer mainboard

    1500uF but 16V

    (Voltage can be higher, but 1500uF should always stay the same!!)

    ReplyDelete