Odd days:
I don’t know where the 4pm finish comes from. It will vary yard to yard as each manager can decide. Ie no overtime just time off in lieu in some places. Late afternoon is spent loading for the next day or even serving in the yard. It’s different in every yard.
I always had a rule of NEVER accepting a gate key. This way if/when they plan you to be out after the yard has closed someone still needs to be there to let you park up. This has always worked well for me.