To approach your question more directly, Columbia is ranked higher than Washington in the US News physics ranking. I don't know what that means in terms of Astro (or in terms of anything really).
I know what you mean about the small department feel (I'm at a small Ivy too -- perhaps even the same one you're at... your description could fit someone I know... QM midterm tomorrow?), and I'd say that you're right -- that is worth something. Also, smaller programs are at a disadvantage in rankings simply because they are small and thus the volume of research they produce is also small. That doesn't mean that they don't have good professors doing excellent research who are known in their field. If that's the case, getting a good postdoc after grad school is quite possible -- I know that it's been done.
It comes down to specifics I guess. We don't know yours, and even if we did, what the heck do we know? Perhaps you should pester those profs who have been posting. To quote them:
My final message is to not rush to judgment based on US News and World Report or NRC rankings. There are many parameters that will affect your graduate experience, and reputation / pedigree effect is just one such parameters. Make sure you visit the campus and talk to faculty members and other students, go on lab tours - don't be shy, ask a lot of questions.
You need to find a perfect fit between you and the department (or more specifically, a lab or a PI that you want to work with). It's a bit like dating - physical attractiveness (ranking) is not everything - you need to find a "perfect match" in many other ways.
The other professor is right about not paying attention to ratings. I had a choice between a school that was then rated about 50th and a school that was top three (maybe top one). I chose the lower-rated school since I liked the location, the atmosphere and the friendliness of the department. It just fit better. Sure, it might have hurt a little bit in getting future jobs, but I was very happy for 5-6 years, and that made it worthwhile. It was the best decision I've ever made (and the higher rated school's admissions person was stunned....)
I've seen this misuse of the term "Ivy" tossed around a few times on this board (e.g. will uses it on "FALL 2008 acceptances").
quizivex might be interested to know that the concept of slope (as in the slope of a line) was unfamiliar and somewhat confusing to several people in an introductory econ class I took here... there are people who don't know math everywhere apparently.