As has bee said, if you was on a legally required break that could not legally be put off until later then your manager cannot tell you to disrupt it.
On the other hand, if you was having a break just because you felt like a break the manager has every right to ask you to return to work.