Usually, you would say "XXX (type of food) don't agree with me/upset my stomach/give me heartburn...)
We tend to describe the effect the food has on us in English and there are certain specific terms - unfortunately the only one I can think of at the moment is "lactose intolerant" (dairy products do not agree with them). Then you would say "I am lactose intolerant". I'm sure there are more like this.
Perhaps you could call them "foods that are not well tolerated" if you mean things like habanero peppers which are so HOT that few people can eat them without getting an upset stomach, or peanuts which are a common allergen.
SweetSoulSister's expression "it doesn't agree with me" is the most common polite phrase I've heard. Here are a few other options, much more casual and colloquial:
"I like (name-of-food) but it/they doesn't/don't like me."
"(name-of-food) gives me indigestion."
"I have a bad reaction to (name-of-food)."
"My stomach doesn't handle (name-of-food) well."
"I can't eat (name-of-food) anymore."