I'd say it depends on who your other potential rec writers are. If you have had other research experience with profs that know you better, choose them first, and just try to get your name on a paper from this project, that way you'll have something to show for the project other than a recommendation.
If you don't have 3 better rec prospects, (they don't have to all be research supervisors, someone who has taught you in (more than one) class is useful), then perhaps try submitting progress reports to the prof to update him on what you have done from time to time?
I had a similar situation, did an REU but only saw the prof twice, once when I arrived, once when I left. He offered to write me a letter down the road; I subconsciously laughed. He didn't know me then and certainly wouldn't know me 18 months down the road when I needed a letter. Luckily I have 3 people back home who know me well.