The steady-state visual evoked potential (SSVEP)-based Brain-Computer Interface (BCI) has been employed in the brain-controlled wheelchair system for patients with severe dyskinesia disease. However, a long-time operation brings about users fatigue, leading to a decrease of performance of the BCI system in practical applications. The characterization of the fatigued mechanism and the improvement of the SSVEP classification accuracy remains a challenging problem of significant importance. In this letter, we first conduct SSVEP experiments to acquire the EEG signals during both normal and fatigued states. Then we develop a novel framework, which integrates the advantages of multivariate empirical mode decomposition (MEMD) and Support Vector Machine (SVM), for improving the SSVEP classification accuracy, especially during the fatigued state. The results suggest that the novel framework enables us to obtain a higher SSVEP classification accuracy compared with the method without MEMD. Furthermore, in order to reveal the fatigued behavior, we use the multivariate multiscale sample entropy (MMSE) to analyze the multi-channel EEG signals corresponding to normal and fatigued states. We interestingly find that the MMSE values in the fatigued state are lower than that in the normal state, which reflects the increase of regularity in SSVEP signals during the fatigued state. That is, a greater synchronization of neural assemblies is required to realize cognitive impairment when fatigue happens. The knowledge for the understanding of the brain fatigued behavior underlying SSVEP-based BCI experiments is gained by our analysis.