Head on over to this post at the Not Quite Crunchy Parent if you'd like some ideas besides the usual candy or plastic trinket goody bag filler. The comments are full of great ideas, too. Here's my contribution:
Thanks for all the great ideas as I, too, hate typical goody bag (Chuck E. Cheese) plastic landfill filler!
For our daughter's 2nd birthday, we bought a bucket of those giant chalk sticks and gave one to every kiddo to take home-- after they colored all over the sidewalk at the park, of course. We also had bubbles for the kiddos to take home as they're a big hit with this age group.
For my son's 4th birthday, we were a little more adventurous! We got those little metal buckets and filled each one with 3 bean bags we made ourselves! The 4 year olds had fun tossing their bean bags into the buckets and maybe someone will teach them to juggle some day -- so many uses for bean bags!
Forgot to mention that the chalk and bubbles had little farm animal finger puppets (since Bug had an Old MacDonald birthday party) that were supposed to fit over them, but the 2 year olds didn't care! Other Half made the bean bags from Curious George fabric to go with the theme of Mugger's birthday party...