The only applied math professors I talked to were at the University of Wisconsin. They recommended I take the PGRE but not the math GRE.
None of the math professors I talked to were in mathematical physics, so my advice is to talk directly to a professor in mathematical physics.
But are you sure they dont give much weight to the phy gre score. Because i'm an international student. What else did they tell you.?
Remember, I was talking to an applied math professor about applied math graduate school. And I wouldn't say that don't give much weight to the phy gre score, but I am willing to say that applied math gives less weight than a physics grad program.
Sorry my advice might not be perfectly applicable. I recommend you start a new topic about mathematical physics, you might attract more people with the most relavent experience to comment.