Unfortunately cats strongly hold on to "their" territory. Therefore a simple "Keep off!" won't make them keep off forever. Especially alpha-toms with a high self esteem can be a tough nut to crack. Fortunately there are several possibilities to make cats realize that's it's your ground and that it's none of their business there.
Annoy the cat's scent
A cat's nose is able to smell scents we human beings aren't able to perceive slightly. If you want to get rid off a cat on your ground you should take advantage of that fact, because some scents that you can't smell are unbearable for a cat.
A lot of ground keepers were for example able to stop a cat siege in their garden by planting some scraredy cat plants (Plectranthus caninus). However, this simple step doesn't work with all cats since some cats seem to be impervious to these plants.
A higher success rate was reached by people who clipped aromatic smelling shrubbery that contain essential oils (such as common juniper or basil) into smaller pieces and spread it in their garden. This also works with the skin of citrus fruits like oranges, lemons and grapefruits. Those smells are often too strong for the sensitive nose of a cat which makes them avoid your garden.
Another popular and well working step is to strew black peppercorns or vinegar.
By annoying the cat's scent you as a ground owner can wonderfully mark your territory in a way that doesn't hurt a cat.
Shoo the cat off and scare her away
You can also get rid of a cats by taking advantage of their jumpiness. In unfamiliar and dangerous seeming situations a cat will do a runner pretty fast and usually won't come back until everything is back to normal again. And if a cat knows that in her favorite litter-box garden there is nothing but unfamiliar and scary situations it will most probably never come back.
Unusual harsh garden keepers sometimes betake to air rifles and bobby pistols to beware their patch from cat's dung, but a cat directly hit by rounds (even if they are no "real" ones) can lead to serious and enduring injuries (like blindness and deafness). That's why we advice you to shoo the annoying cat by other means.
Always keep this in mind: Cat's don't leave their heaps in your patches because they want to annoy you but because - from their point of view - your garden is very well suited as toilet.
Instead of air rifles you can build a noise-machine yourself from an old coke-can filled with coins and gummed up afterward.
When you see the cat on your ground, you throw the can in front of it (not on it) and the noise will scare it away immediately.
Water guns have a similar effect and can make your garden look pretty unattractive to your neighbor's cat.
Apply high fences
The average cat can jump 2 meters high without batting an eyelash. They also sense electricity with their highly sensitive whiskers. If you therefore apply an electric fence around your patches (or even all your garden) it just doesn't matter to the cat. Fences will only help in very few cases. (If you want hedgehogs, dogs and rabbits to keep off, the fence works very well, of course!)
A dog for your garden
A success quote of 95% against unwanted cats on your ground is a dog. A big, grown, cat-hating dog. Cats are in most cases afraid of such specimen and look for another garden instead.
On the other hand nobody should ever get a dog just to get rid of an annoying cat. Dogs need a lot of attention, love and care - and also a potent pack leader. A dog that's not reared can do more harm to your garden than 10 cats.
Exercise tolerance
If you are not that badly annoyed by cat's dung between your potatoes there is a nice and efficient method to get rid of all the tensions with the cat-owner:
Just apply a cat litter next to your patches. A clean cat littler will in most cases lead to a cleaner garden of yours.
Picture CC 2.0 by gagilas