I agree with Goldberry, these metrics are good for big picture comparisons of cost of living, but when you are actually comparing 2 or 3 offers, you want to get specific data relevant to you. For a grad student, the biggest **variations** in cost are 1) housing/rent, 2) health insurance package, 3) cost of owning a car (if you plan to have one), and 4) cost of travelling home (or to visit other important people in your life) if you plan on doing that a lot.
So once you are looking at a few offers, it's worth spending a few hours on each school/city and finding out how much you would have to pay for these three things. For housing, look for specific apartments (whether it's in the city or part of school subsidized housing) that you would actually want to rent. See how much you have to pay for health insurance premiums and how good the coverage is. Price out car insurance for the city and vehicle registration fees, gas, etc. And check on flight costs (don't forget to include cost of getting to an airport if you are at a smaller town). Talking to students is a great way to get some of this info.
I'd take the stipend and subtract all of these costs and the remaining number is the useful number to compare. This is how much you have left to pay for other important things that don't vary very much (e.g. groceries).