That's indeed the crux of the problem. Only two crater ages were determined directly during the Apollo missions. So the ages are estimated from cratering of the craters - even for smaller craters - via dynamic models of cratering & cratering statistics. You get a serious problem with this method: one has to determine cratering rate in order to get the age, but this rate can only be gleaned from the cratering statistics and presumed (NB!) geophysics of the Moon over 3.5 Gyr. For small craters, cratering (that one needs to estimate the age) is mainly pockmarks left by the secondaries rather than the primaries (1000:1 to 100:1), which biases the estimates towards recent events and complicates cratering rate analyses. Finding the actual cratering rates and mechanisms for cratering that explain the observed cratering statistics is, in fact, an unsolved problem. I gave a few references below.