I believe the opposite. Knowing human nature and the way our minds function, I think it's a good thing that we don't have specific recollections of our past deeds, in as much as we won't be tempted to behave cynically in order to "do good" if we were to know the specific details for determining the reasons for erasing those previous misdeeds, and I feel this is a deliberate act on the part of our Creator. In essence, we are currently acting without vested interests on a karmic level. Yes, on some instinctive level most people realise that goodness is preferable to bad, and we somehow feel that if we follow this rule even in the most crudest of forms, we will be closer to God than if our life erred towards the other, darker end of the spectrum.
Basically, what I'm trying to say is this: if we were somehow aware of everything from our previous existences -- and therefore aware of what exactly we needed to do in order to pay off those debts -- we wouldn't do good by battling with our conscience and weighing up our choices (which is the way we function currently), but instead all our goodness wouldn't be selfless and would instead be automated; it'd be driven by a desire to do good for the sake of progression, and not because it's something we truly feel to be the correct course of action as a decision and something's that's arrived upon organically through using our budhi. The Creator is absolutely spot-on with how much he's given us non-Brahmgyanis to work with. It's all part of the plan. It is very, very difficult, though.