If you work through an agency at the same client and location all the time then this does rule out tax relief on travel expenses as it is a normal place of work.
However, no matter if through an intermediary or not, there is still the concept of a temporary workplace. It is a fairly complex area, but if you are not working continuously at a client then it can be argued that each place you go to is a temporary workplace.
An example might be an agency driver that works fairly ad-hoc and goes to many different clients, maybe for a week here and there and shifts are to a large extent confirmed the day before.
If shifts are confirmed the day before with absolutely no guarantee of any hours, then it can also be argued that you are not in continuous employment, because the agency could decide not to call you at any time. If you were to be put on a rota where you had an expectation of staying on for a period of time, this would effectively mean that you have a normal place of work and no tax relief would be possible on travel.
This is my take on it anyway. I have claimed tax back on travel for a few years. I have had the spreadsheets ready to submit to HMRC, but they have never requested them.