From Mises blog:
In another thread here, a commentor asks, “What, exactly, is un-libertarian about “loser pays” laws in civil suits?” This sentiment is common among libertarians who seem to assume that the “loser pays” rule is preferable, from a libertarian point of view, to a system in which each side pays its legal costs.
The push for a loser pays system seems to be a reaction to frivolous and unjust lawsuits. But such suits are inevitable in a state-run legal system, which will be less efficient than a private system–and also less just, since the relevant laws will often be unlibertarian. This means that the wrong party will often win, either because the more-inefficient and less-accurate state court system makes a factual mistake and rules for the “wrong” side; or because the law that supports the winner is unjust. Attempting to solve this problem by shifting legal fees is addressing the symptom, not the cause.
Besides, in any compulsory legal system, either the loser-pays, or winner-pays, rule can result in unjust restults. For example, Sally sues Harry for a battery that he committed. If she wins, and there is a winner-pays rule, this is unjust, since a guilty Harry ought to have to pay her damages including her legal costs in having to sue him. But suppose she loses, and there is a loser-pays rule: this is also unjust since injury is being added to injury.
In a state-run system, neither loser-pays, nor winner-pays, nor each-side-pays-his-own-fees, is just. The only just system is a voluntary, private justice system where customers of competing justice agencies agree voluntarily ahead of time to some rule–in this case, any of the fee-payment schemes is libertarian since it is agreed to by all sides.