I don’t think Gordon Brown did too badly for a ex-grammar school kid with one eye.
I would agree that any fast-track system left him feeling left out though. I wanted to go down the road of mathematics when I passed my 11 plus all those years ago.
I wasn’t allowed to though. I got pushed into a school specializing in providing artificers and technicians - mostly for the Chatham Dockyard which, guess what? - Got closed down the same year I left school, to find out my technical qualifications were utterly useless to me, but too late to stay on at school, and re-train! I wasn’t even “good with my hands” whilst at school. I was a better thinker than manipulator of materials…
Some of the frustrations that Brown must have experienced, in his time as PM rather than as Chancellor - I could sympathize with him. If he’d called an early election upon becoming PM - I would have voted for him, but I lost respect for him “Not being willing to make that gamble”, and ended up voting for Cleggy’s Libdems in the following 2010 election instead. “The country needs a gambler rather than a bookkeeper, I thought.” Turns out even Cleggy was no gambler though, of course. 
There’s no point being the smartest or best at anything - if people decide you’re “not to be listened to”, acting more out of one’s lack of charisma than any abilities one might excel at.
No one can doubt that Blair had a lot more Charisma than Brown. “Charisma” also makes it possible to get away with the most outrageous lies as well, of course…