And I've seen players increase beyond their potential, so it doesn't really balance in the end. All it does is help players who pick late in the drafts, since we'll now be getting players that are practically equivalent to those being taken with the top picks, and it'll come down to a hidden coin flip which ones develop and which won't. A decade (in game) of these draft classes, and every team will be starting players in the mid-80s or above at every major position.
I think the drafts are fine the way they are. As someone previously mentioned it's realistic to get starting talent in the first 3 rounds. Your prediction is a bit of an over-exaggeration; projected 80 players are generally all gone halfway through round 2. In the best-case scenario (without trading), you draft 2 players each year who reach their potential. In a decade, you'll have 20 80+ rated players... except several of them will bust and it's highly unlikely you get 20 players at 20 different positions. Additionally, you'll probably lose some to free agency.
Also, from a yearly contender's point of view, a lack of talent at the end of the first round is a death knell for your future. You wouldn't be able to restock your roster, nor would your draft picks be worth anything.