A small breeding pool isn't necessarily a show-stopper -- there are indications that species such as the cheetah have been reduced to small populations in the distant past, resulting in a genetic "bottleneck" -- but if good genes have been lost and bad genes retained, there seems to be a greater chance of defects. The "founder effect" limits the number of genes you've got to work with (subset of the entire species diversity) and in sexual organisms, "genetic drift" tends to eliminate some genes over time just due to random selection (you don't get to use all your sperm/eggs so some of your genes won't make it into your kids, no matter how many you have!).
I think they recently found that the effective number of breeding individuals for the California condor project is actually much lower than suspected, because some of the birds were related.
Your observation that this might create a false sense of security is right on, in my opinion ... we've had mixed success with trying to clone animals (and it gets harder when you have to rely on frozen eggs/sperm, let alone trying to extract DNA from minute tissue fragments that may have been in storage a long time).
More than a decade ago, Al Gore noted that a seed bank is next to useless if the seeds are allowed to deteriorate in storage. I've worked on an endangered species breeding project, and if anything this is even more pressing for animals. It's dicey enough trying to maintain healthy breeding populations in captivity in the hopes that you can save enough of the habitat to start re-introducing them in a decade or so. There are plenty of species which are so inbred that we're relying on getting "new blood" from wild stocks (not the way it's supposed to work ...) -- example, Goura victoria (the bane/beauty of my life!). Captive individuals are tracked through a "stud book", just like racehorses, in an effort to reduce inbreeding problems.
http://www.internationaldovesociety.com/Misc%20Species/Victoria%20Crowned%20Pigeon.htmGermplasm storage should be a last-ditch precaution and shouldn't be a substitute for habitat protection.