***I am just a student currently applying, so can't give you perfect advice.***
The best advice on this subject you can get will come from actual professors who probably don't browse this forum.
However, if you want a fellow student's opinion: I'd bet that it depends on what you want to research and where.
If you want to do mathematical physics, and are okay with not going to Harvard, then you only need to do well on the PGRE and have the minimum coursework suggested by the program. For example, a strong program like CMU (https://www.cmu.edu/physics/graduate-pr ... ssion.html
) would still suggest you take Quantum Mechanics, Laboratory, and Electronics. BUT most schools don't require/suggest specific courses, and if you are missing one or two that that a program suggests it may not be a problem.
However, if you want to study a subject that you have little coursework preparation in, indeed you may want to take a few more courses in that subject. I feel like my own lack of astro courses is hurting my applications to astronomy programs, even though I’ve taken graduate courses in advanced quantum subjects. Like me, to study astro you'd probably want GR and some other astro courses.
Finally, if you want to go to a top program, they are looking for any reason to deny you -- not in a bad way, they just receive hundreds of applicants. You could certainly get into a grad physics program as you stand. However, I don’t see MIT etc. letting in anyone who hasn’t taken the core coursework (Full semesters of advanced: Stat/Thermo, Mechanics, E&M, Quantum, and Lab) and above, unless you have outstanding research experience (likely a strong publication; your coursework isn't so important if you've actually done impressive, notable research).
If you are really concerned, try emailing a professor at your own school or, even better, a professor at another school who you could see working for as a grad student down the line.