In many schools, such as my current school and my previous schools, Math and Physics are considered very closely related fields. For example, for physics degrees, both math and physics courses count as courses "in your major/field of study".
So, I don't think the name of your degree is going to make a huge difference. It doesn't really matter if your degree says BSc Math, or BSc Physics. What does matter is the courses you took and how well you did on these courses. If you are applying to Physics PhD programs, you should make sure that you take almost all the same courses as a typical Physics major (especially the "core" upper level Physics courses, such as two semesters of EM, two of QM, and stat mech). You should also have similar numbers of upper level courses in physics or math.
If you don't do well in these courses, it will be a concern whether or not your degree says Math or Physics. Also, even if you are only applying to theoretical mathematical physics programs, your other physics courses would likely play a role in your admission decision too. Most physics departments I know would want their students to have a strong and well rounded foundation in physics, not just the tiny area of expertise. Of course, if you are working on Mathematical GR, then perhaps your Electronics Lab grade won't be weighted as much, but it will still be part of the consideration, in my opinion.