Computer support is often challenging and the more challenging it gets the more enjoyable it is. At Kadence Solutions we take great delight in fixing things and making them work again. Sometimes it involves little more than replacing a failed part while other times we have to get more in-depth. That’s when the fun begins.
Recently a customer was looking to upgrade one of their servers with some faster CPU cores. This ordinarily isn’t too difficult a task and is completed without too much fanfare. In this case we ran into some complications. While removing the CPU cores from the server some dust and a few flecks of heat transfer paste fell into the socket where the CPU chips are installed. Heat transfer paste doesn’t necessarily conduct electricity and could interfere with the connection between the pins in the socket and the connection pads on the underside of the CPU. Compressed air was insufficient to remove all of the items that had fallen into the sockets. Gently coaxing the paste from the pins with a soft brush appeared to be effective but unfortunately one of the brush bristles fell out and slipped neatly underneath the pins in the socket. Slightly more vigorous attempts to remove it were unsuccessful but since the bristle was not conductive and likely to short anything out it was left in place. Unfortunately, due to the brush not being quite as soft as hoped, several of the extremely soft and fragile socket pins were now slightly bent. Reinstalling the new CPU chips into the socket and locking them in place allowed that pressure to bend the CPU pins even more, preventing the pins from touching the appropriate places on the underside of the CPU chip. As a result the server would no longer boot up.
Some additional lighting had been used inside the server case to make it easier to work but it was not enough light to allow the naked eye to detect the slight difference in positioning of the extremely small pins. As a rule, once these very small, very fragile pins are interfered with all is lost. A piece of equipment with a replacement cost of tens of thousands of dollars was reduced to e-waste in moments.
The customer was very understanding and accepted that sometimes these things happen. The server had been around for a number of years and the CPU upgrades were an attempt to squeeze just a little more life out of some older equipment.
The customer called it a day and began the process of reviewing pricing and other details for some shiny new equipment. Never one to give up I hauled the server back to my bat-cave for a closer inspection. My newly acquired microscope might just be able to help solve the problem.
When the motherboard was installed in the server case some of the pins showed slight discolouration. This was actually due to the pins being slightly bent and reflecting light differently. The pins are very small and it’s not easy to see the difference between a damaged pin and a good pin without some magnification.
With the motherboard lit up under the microscope it became even more obvious just what the issue was. With better insight into the problem and an absurdly hopeful “can-do” attitude I set to work to see if this could be repaired.
Each of the tiny pins flex slightly under the pressure applied by the CPU chip as it is locked in place from above. Since the pins flex already, adjusting the damaged pins would involve using two points of pressure to bend them ever so slightly in one direction while restricting them from flexing in another direction. Even the tiniest of precision screwdrivers in my arsenal were much too large for the task. Instead the tools to manipulate the little pins were found in odd bits and pieces laying around. A pin raided from a sewing kit, the thinnest wire exposed from within a twist-tie. Using patience and caution as accessories I started to painstakingly review each pin to see how it sat aligned with its neighbours. Only a small handful of pins needed repair. Some needed only slight coaxing to be straightened a few were bent back on themselves and needed major straightening. This motherboard was equipped with two CPU sockets, each having 2011 pins. While the pins of only one socket were believed to be damaged, it made sense to review all 4022 pins and ensure that all were in, or returned to, their original position.
As this wasn’t something I have done before I suspect that it took me much longer than would have been necessary for someone who does it frequently. Assuming there even are people who have need to tinker with these little pins on a regular basis. As mentioned ordinarily, once these sockets are compromised, the hardware gets thrown out.
For anyone who is curious, a mere 5 hours was spent peering through the eyepieces of the microscope and gently coaxing each bent pin back to its original position. All of this had me somewhat bleary-eyed but allowed me to be comfortable that each pin was ready for active service again.
A much shorter period of time was involved to put the motherboard back into the server case and re-check all of the connections. Voila! The server powered up on the first try and functioned as intended. I’m not certain yet if the customer was overjoyed to see their equipment put back online, or saddened at not having a reason to purchase a replacement. They were thrilled to know that Kadence Solutions will go above and beyond when our customers require it.
While this is not a normal task by any stretch of the imagination, at Kadence Solutions we are firm believers in repairing equipment whenever possible and learning new things at every available opportunity. Sometimes those two things go hand in hand. When both together turn out to be successful, and fun, then a day’s work was well worth it.
