In a non-reactive full hydro application, at low speeds, with everything tight and good with zero play, it doesn't.
In a reactive full hydro application, it matters, reactive full hydro is basically a different kind of steering box with a fluid drag link instead of a metal rod.
If everything isn't tight and good and zero play, you'll have wobble. Right-way caster will tend to settle it. Wrong-way caster will magnify it. Speeds and harmonics can start or magnify it.
If you are driving on rear steer at speed, it will either be tight and good, or at least occasionally terrifying. The settling effects of caster at speed relative to the wobble that you'll get from things being out of whack, won't de-terrify it, because of how violent weight shifts from rear steering changes at speed are. I've had a single worn-out rod end (~1/8" of play at the rod end, translates to a wobbling inch or so of toe change at the tire) be noticeable on rear steer before, driving along, good good good shimmy good good good good wtf was that good good good good bump shimmy good good good....
If you are only ever up to trail speeds, rear caster really won't matter anyway.
Somebody has to have a service book for a GM Quadrasteer that calls out rear caster spec, I can't find it googling, but I'm thinking it was zero or very close to.