Production servers should never crash. Now with that being said, the truth is sometimes they do, regardless of architecture or OS. Well, I take that back, none of our big iron machines running Z/OS or OS/390 have ever crashed, at least in my years here. Anyway, critical non redundant hardware failures are most often the cause of server crashes in our Wintel and Lintel environment, as well as our mid-range systems running AIX and AS/400.
Our second leading cause of server crashes? Human error. Yup, fat fingering a command as root or administrator, pulling the wrong hot swap memory module, wrong redundant power supply, wrong CPU module, etc. Greatest crash I've ever witnessed? Someone who brought down a 4 server $250,000 clustered DB2 solution owned by one of our very high profile customers for several hours due to, get this, disconnecting and inadvertently damaging the pretty orange (and rather delicate) fiber gigabit cables because they were "in his way." The person was adding a simple $400 RAID card to a nearby server. He was shown the door shortly after . . .
Took 3 of us over 4 hours to get everything back up and clear the back log of data that was constantly queuing up for time on that cluster. Good times for sure.
BTW, buggy code is worked out on our Dev and Q/A servers before they ever hit production. tgkprog is right, those servers don't count as much as a prod box, since their lot in life is to be kicked around, prodded, and poked with sticks anyway.
