See, I'm with melannen on this one; I don't agree that that usage has anything to it that can be nailed down specifically. That's because that usage is almost always rooted in the particular fears and scientific misunderstandings of the person using it in that way. In practice this means anything in manufacturing (especially food manufacturing) that makes the person using it uncomfortable or uncertain ends up being called a chemical.. And what makes any given person uncomfortable can be profoundly variable, even in chemophobic circles.

Synthetic or processed, as suggested, might be closer to what people think they're talking about when in that usage, but frankly while they might think that's what they're talking about, they're not; it's much more fungible and less specific. I give you cheese, and also fermented and distilled alcoholic spirits, and ask how many people are uncomfortable with those specifically on the basis of 'chemicals'. (Despite the fact that both -- even in froofy hand-crafted settings -- are indeed highly processed with significant human intervention/chemistry involved.) And those are just two immediately off the top of my head, but there's plenty of other traditional foodstuffs with a very long history that similarly involve significant human processing and profound chemical alterations to the starting materials.