I attend a university ranked top 20 in the world. During the first semester of my freshman year, I slacked off and attained a GPA of 3.4 including a B+ in intro physics (which isn't bad I suppose). I did this mainly by cramming for a couple of days before midterms and finals. I vowed to take my academic career more seriously after that first semester, but I quickly fell into the same trap with results far worse. I failed two math courses and the E&M intro physics course, passing only General Chemistry with an A-. It was a hell of of a reality check for me - my GPA was barely above 2. I retook the two math courses I failed during the summer as well as a computer programming class (out of interest) and got an A in each of these. I self-studied the physics class I failed (properly this time) and took a supplemental exam for it, also receiving an A. Since my GPA was roughly 2.6 at that time, I had to ask departmental approval for entry into the joint honours program in mathematics and physics I was eager to take. I explained the entire situation and fortunately I was allowed entry because of my improved results.
My first year in honours I received A's across the board. My second year in the program I also received straight A's and joined a research team where I helped mainly with some tedious programming/algorithm work. In my senior year I had a 4.0 GPA my first semester, including a graduate-level course in physics, continued research, and published a paper as first author. My second semester has just concluded and I expect similar results (including another grad-level physics course). I decided not to apply to grad school this year, but to take a year off from school to boost my grad school admission chances with continued research. My GRE and PGRE scores were both in the 99th percentile, and I'm certain I can get great LOR's from two professors from research and one from the first-semester graduate level class.
However, my CGPA after all this is roughly only ~3.6 (expected), and my Physics GPA is only ~3.3, mostly courtesy of that introductory E&M class. Am I hopeless of getting into a top graduate school because of my freshman year, or will admissions committees pay little attention to it due to my success in upper division courses?
Thanks for reading (if you did). Any insight is very much appreciated.