Most of the existing metaphor studies address comprehension of nominal metaphors, like “My job is a jail”, and predicative metaphors, like “He shot down all of my arguments”. However, little attention has been given to how people comprehend adjective metaphors, such as “sweet voice”. Very few studies have focused on color adjective metaphors, like “red voice” or “black mood”, in which an adjective denoting color modifies a noun designating abstract entities like emotions. However, not a few color adjective metaphors are used in our daily life as well as in literary works. Germen poetic expressions like “blaue Klage” (blue cry) or “weiße Traurigkeit” (white sadness) indicate cognitive universality. In this paper, we examine meanings of Japanese color adjective metaphors and argue that meanings of those metaphors are not predictable from those typically associated with color adjectives, and that those metaphors evoke negative images. Based on the Abstract Performance Grammar proposed by Osgood (1980), we hypothesized meaning change patterns by the interaction between source concepts expressed by color adjectives and modified target concepts expressed by head nouns. These hypotheses were tested by psychological experiments. Materials used in the experiment were collected from the Internet. Subjects were asked to evaluate meanings of linguistic expressions by 7-point semantic differential scales (from strongly negative as -3 to strongly positive as +3). Subjects were also asked to give words associated from each expression. Results suggest color adjective metaphors tend to evoke negative images.