A cashless society is a society in which banks and the government can control every aspect of your freedom, which would be a bad thing.
Cash in hand is a physical manifestation of personal freedom. You can buy what you want with privacy with no electronic record of your identity tied to the purchase. You can buy what you want without a requirement for Internet or electricity to be working for the transaction to be processed. You can even buy things that your government says are illegal for you to buy. That's true freedom.