The removing of the card and carrying on is falsification of a record and normally results in being warned for prosecution. You should have left the card in, done what you needed then made a written record to ‘excuse’ it. If you’d done that it wouldn’t be a trip to court.
Of course if found guilty you will almost certainly also receive a suspension from a TC. I am sure the operator has already had a visit regarding this but that may be a visit to a TC if you are found guilty.
The magistrate will only pass it onto Crown court if they feel unable to deal with it due to technicalities (they don’t understand the charge) or they feel the maximum penalty they can issue isn’t sufficient (£2500) whereas The Crown court can go for £5000 or a custodial sentence as far as I know.
I would say with your one falsification offence it’ll stay at Magistrates and if you come across as honest and it really was a mistake it’ll be £450 ish. Of course DVSA will push for more
Unfortunately drivers just don’t seem to realise that taking the card out and carrying on is far more serious than the offence they’d commit with the card still in.
I would highly recommend you get legal help. Maybe your house insurance may cover it? Some do.