You know what is right for yourself. Agency work can be consistent when you’re established and theres nothing to stop you registering with several to keep the work coming in. If your gut is telling you its wrong for you then don’t do it, family comes first and you can get straight into a permanent job, it just takes a bit longer