Low to moderate numbers of aphids aren't usually damaging, but allowing them to continue reproducing on your plants may mean more aphids this spring. Once aphid populations explode in spring, their feeding can turn leaves yellow and stunt shoots on certain plants. Aphids can also produce a great deal of honeydew, a sticky byproduct of their feeding.
For information on how to manage aphids in many situations, consult the Pest Notes: Aphids.