Hi there, I just got into Caltech and Yale for Physics with a 3.74 overall GPA, though in my physics degree it's a 3.89. However perhaps this can still be helpful.
What worked for me was figuring out specifically what kind of work I wanted to do in graduate school and doing well to fill that skill set. I personally am aiming to work on hardware electronics and detector development, and so I built a skillset around that. My personal philosophy (take with a big grain of salt - I got rejected from most of the universities I applied to) was to get a lot of practical lab experience: I worked in an electronics design lab, joined a university group that allowed me to do a lot of data acquisition development, did research at a national lab with detectors.
What happens though is that you become very specialized - this is not for everyone. I personally know I want to avoid analysis work if I can. If you have a strong idea of what you want to be researching in graduate school, my advice therefore would be to focus on developing skills that you know are useful in those fields. If it's comp physics, getting a lot of experience with root, 4tran, etc. is good. You're already doing an insane amount of research which will only help you. If your research is specifically in the field you want to work in, that is also important.
Graduate courses are not a bad idea - they show an effort that you're willing to take on more challenging subjects. I wouldn't place too much weight on them however. I don't think they affect admission decisions very much.
Strong recommendation letters is another big thing. Be sure you have at least 3 people to write you good letters - if you have more than 3 who might be able to write for you, try to tailor for each university you apply to. If one of those professors used to go to that university for example.