A vast literature shows that China's five largest state-owned banks (the Big Five) suffer from low cost efficiency. We offer a new explanation of this situation, by decomposing overall efficiency of Chinese banks into two parts: persistent and transient efficiency. Using the model of Kumbhakar, Lien, and Hardaker (2014) based on the stochastic frontier approach, we measure persistent and transient efficiency for a large sample of 166 Chinese banks over the period 2008–2015. We show that the lower efficiency of China's Big Five banks is almost entirely due to low persistent cost efficiency, indicating structural problems. On the contrary, the Big Five banks transient efficiency is similar to other Chinese banks, reflecting a good aptitude to minimize their costs in the short-term. Our findings support the view that major structural reforms are needed to enhance the efficiency of China's Big Five banks.