When picking a bank for savings and CDs, most people look only at the interest rates. However, rate chasing brings some headaches with it. You're constantly applying for new accounts as you switch banks, and your money takes a few days to move from one bank to the next (you might not earn any interest during those days, which can eat away at your returns). In addition, you take your chances with customer service every time you try a new bank.
Consumerism Commentary has a nice discussion on the tradeoffs between good customer service and high rates. The consensus seems to be that customer service is more important. I agree -- you don't always need the absolute highest rate, you just need a competitive rate and some peace of mind.
See the discussion at Consumerism Commentary.
Further reading: