The origins of some food idioms are a piece of cake to figure out; just use your bean. Others sound so bizarre they could make you go bananas. There are so many of them, I would have to be crackers to try to list them all, but here are a few that are particularly nutty:
The apple of my eye: According to Food: A Dictionary of Literal and Nonliteral Terms, by Robert Palmatier, the Old English word for apple referred to both fruit in general and to the eyeball, which was the "fruit" of the eye. Someone who is the apple of your eye, then, is as cherished as the organ that allows you to see.
Not worth his salt: Roman soldiers were given an allowance to pay for salt, explains Mad as a Wet Hen! and Other Funny Idioms, by Marvin Terban and Giulio Maestro. If a soldier wasn't performing up to par, he wasn't worth his salary—a word that itself comes from the Latin for salt.
A red herring: According to Mad as a Wet Hen!, this expression comes from the fact that an escaped prisoner would drag a dried red herring behind them (and then, presumably, run the other direction) to leave a scent that would throw off the bloodhounds. The Glutton's Glossary, by John Ayto, however, attributes it to a former hunting practice of "pulling a pungent red herring across the trail of a hunted animal to sharpen the skill of the hounds being trained." Either way, it means a tactic intended to divert attention away from something.
Salad days: Like so many words in the English language, this expression for youthful inexperience comes from Shakespeare. In Antony and Cleopatra, Cleopatra speaks of her "salad days, when I was green in judgment; cold in blood." Back in my salad days, I imagined it referred to poor students who could only afford to eat salads.
Sowing your wild oats: According to World Wide Words, a blog written by British lexicographer Michael Quinion, this expression dates to at least the 16th century, and refers to the worthlessness of wild oats (the probable precursor to cultivated oats) as a cereal crop. Therefore, a young man who sows wild oats is—ahem—spreading seeds without purpose, or otherwise pursuing idle pastimes.
Upper crust: In olden days, the top half of a loaf a bread was considered better, and was served to the nobility. (Source: Mad as a Wet Hen!)
As for the idioms in my opening paragraph:
Piece of cake: According to The Word Detective, this may be related to the expressions "takes the cake" and "cakewalk" which both probably derive from the 19th-century African-American competitions that awarded a cake to the couple who strutted most gracefully and stylishly around it.
Use your bean: The origin of using bean as a metaphor for the head, or brain, is a little harder to track down. Palmatier's food dictionary traces it to the late 1880s. I wonder if it has to do with the brain being a little bean-shaped.
Spilling the beans: According to the Scholastic Dictionary of Idioms, by Marvin Terban, (via the Write Blog), this probably dates back to the ancient Greek method of placing black or white beans in a jar to cast votes. If someone spilled the jar of beans, the election results would be known prematurely.
Go bananas: Even the Oxford English Dictionary is trying to figure out why the tropical yellow fruit means "to go crazy," and whether this usage appeared before 1968. There seem to be a lot of ways to go crazy with food—crackers and nutty included—and they are of equally mysterious origin.
I wanted to also write about food idioms in other languages, but I don't want to bite off more than I can chew, so it will have to wait for another day...