How fruitful has this research been? I've seen people bounce around and not accomplish much of anything that is concrete. Did you get publications, make presentations, grow and mentor newer students to the group under you, etc? A person who sticks with the same group because things are rolling well and both sides benefit and are productive makes sense and isn't a negative. That one letter, if it's really good, will be the focus of your recs anyway. Exposure to multiple groups/labs is very helpful and great experience plus it's good to get multiple perspectives on a student. However, it also runs the risk of being negative too, and more experienced professor recognize this right away. Junior faculty tend to get more excited about the multi-perspective approach in my opinion. I've seen a lot of established profs go with one great rec and a gut feeling more often than not. The reason for that is simple, multiple experiences speak to potential, prolonged measured and referable success speaks to realized potential and that's the goal. It's like the adage about jobs too, always bouncing around doesn't look favorably unless in each step success was found and the move to the next step was forward and not lateral. I actually just met a undergrad interviewing with my group who advertized he had 2 and a half years of "solid" research experience with a couple different groups but when you got down to it you quickly realized he was in each position just long enough to get up to speed and then bail or not accomplish anything and get bored/frustrated then move on to the next group. In the end my report on him was that he had 2 and half years of orientation with brief periods of unsuccessful work. That a direct quote too, and he didn't make our group. If the research has been productive you'll be just fine. The groups you'll want to work for will be focused on much more important things.
You know you could always take this question to the prof you've been working with too. If he/she is really worth their weight you'll get an honest answer and a real inside perspective. Don't worry about talking about this with them either, this is something that everyone understands is a concern going forward today.