Work other company on days off

My attitude is that whatever I do in my time off, has nothing to do with the company I work for, as long as any legal constraints are observed. If I have driving/working hours available to do a shift elsewhere in my time of, than that is up to me. When did employers start telling us what we can and can not do when we are off?