Top-down modulation is engaged during multiple stages of working memory (WM), including expectation, encoding, and maintenance. During WM maintenance period, an “incidental cue” can bring one of the two items into a privileged state and make the privileged item be recalled with higher precision, despite being irrelevant to which one to be probed as the target. With regard to the different representational states of WM, it’s unclear whether there is top-down modulation on earth sensory cortical areas. Here, We used this behavioral paradigm of “incidental cue” and event-related fMRI to investigate whether there were a privileged WM state and top-down modulation for complex stimuli including faces and natural scenes. We found that faces, not scenes, could enter into the privileged state with improved accuracy and response time of WM task. Meanwhile, cue-driven baseline activity shifts in fusiform face area (FFA) were identified by univariate analysis in the recognition of privileged faces, compared to that of non-privileged ones. In addition, the functional connectivity between FFA and right inferior frontal junction (IFJ), middle frontal gyrus (MFG), inferior frontal gyrus, right intraparietal sulcus (IPS), right precuneus and supplementary motor area was significantly enhanced, corresponding to the improved WM performance. Moreover, FFA connectivity with IFJ and IPS could predict WM improvements. These findings indicated that privileged WM state and potential top-down modulation existed for faces, but not scenes, during WM maintenance period.