Yep, the weighted average suggested by cerbere and quantum1 is the way to cut the corner (cos sometimes you can).
To clarify quantum1's example here is another with more symbols ;):
imagine rolls 1,5,3,4,2... (average 3)
each time, you count how many rolls have occurred, and get the number of...