Companies wouldn’t mind as much if you were all in (to use a poker analogy). If you had given up your existing job and gone on agency to get experience and had a solid few months, I bet you would get more interviews.
Nobody here can tell you what to do. Leaving a job is always a big decision. If you don’t want to leave your job, looking for a weekend day here and there may be the way forward.