Recent studies have suggested that the extent to which primary task demands draw on attentional resources determines whether or not task-irrelevant emotional stimuli are processed. Another important factor that can bias task-relevant and task-irrelevant stimulus competition is the bottom-up factor of stimulus salience. Here, we investigated the effect of stimulus salience associated with a primary motion task on the processing of emotional face distractors. Faces of different emotional valences were presented within a context of randomly moving dots. Subjects had to detect short intervals of coherent motion while ignoring the background faces. Task salience was manipulated by the level of motion coherence of the dots with high motion coherence being associated with high salience. Using functional magnetic resonance imaging, we show that emotional faces, compared with neutral faces, more strongly interfered with the primary task, as reflected in significant signal decreases in task-related motion area V5/hMT+. In addition, these faces elicited significant signal increases in the left amygdala. Most importantly, task salience was found to further increase amygdala's activity when presented together with an emotional face. Our data support a more general role of the amygdala as a behavioral relevance detector, which flexibly integrates behavioral relevant, salient context information to decode the emotional content of a visual scene.