I also transferred, but my experience has not been as bad as the previous poster's. I think it all depends on your situation -- that is, why you are transferring, and where you are transferring to.
I wanted to transfer for similar reasons: I had gone to the school as an undergrad and I didn't like the research prospects, and was unimpressed by the achievement of previous students. What's more, the other students who were accepted seemed mostly like slackers, and I became very self conscious about what I would accomplish, if anything, if the bar were not placed a few notches higher. So I did research into other schools, found that my suspicions were correct--that bad schools rarely graduate good physicists, and that a good adviser is crucial. So I applied to several schools, was accepted to one, and transferred. I am writing this one semester after the transfer and, while it is true that I need to retake quals, I believe I made the right decision.
IF you are unsure that this is the RIGHT decision for you, it probably doesn't matter much either way because there are no clear signs of danger. In my case it was very clear, and the decision was very easy. I simply wanted to do something I could not do at my previous school. One of my friends also applied to many schools, was accepted into a good program, but decided not to take it (despite my prodding). His theory was that your adviser matters more than the school, and he was comfortable with his adviser. I'm not so sure (see below).
You asked about the process. I found three professors that had good impressions of me and asked to write l.o.r.'s. I was first year so I didn't have an adviser, but one professor, who I had spoken with and gone on workshop trips with--a professor who knew if I didn't transfer I wanted to be in his research group--reluctantly agreed to write a letter after expressing some exacerbation that I would ask him. I don't know what the quality of his letters were. I didn't get into my first, or even second, choice schools. I did not have the time because of finals to study for the GRE so did not do as well as I would have liked. But I was accepted, and in making the decision to accept or decline the offer here is what I did: can you script, or program? I wrote a simple program that filtered google results by my own criteria. I did a google search for "school-name phd physics" and then my filter script picked out only those results in which each of those words were in close proximity--effectively returning only results for which a faculty member received a degree in physics from "school-name." My decision became a no-brainer. The school I was transferring to produced twice as many full time faculty as did my current school, and at much better schools to boot. I made sure the script produced the intuitive results one would expect by testing the filtering mechanism on Harvard, Princeton, U.Kentucky, and I can say with confidence that there is an unfortunate but undeniable amount of pedigree considered during the hiring of faculty. Harvard does produce far more tenured professors than Kentucky. This is a sad reality that I think you should factor into your decision making process (I assume that this transfers over to the private sector).
If this seems too calculating for you take a second look at last year's graduating class at your current school and ask yourself, would I like to do this? Unfortunately, I have reason to believe that those same students graduating from a different school could have much better job offers upon graduation.