Compared with an expansion muffler with a rigid back cavity, the expansion muffler with a flexible back cavity has better low-frequency noise reduction performance. However, most of the current calculation methods for expansion mufflers with the flexible back cavity are based on transfer matrix methods under the plane wave assumption. The calculation error of such methods increases with acoustic-structure coupling, and it is difficult to obtain the modal function of the flexible structure under elastic boundary conditions with traditional calculation methods. Therefore, this paper proposed a calculation method based on the energy principle that does not rely on the plane wave assumption. The model of the muffler is decomposed into three sub-acoustic cavities, and these cavities are coupled to each other through coupling surfaces. Then, the sound pressure function of the sound field and the displacement function of the flexible structure are expanded into three-dimensional and two-dimensional Chebyshev series respectively, and the Rayleigh-Ritz method was used to solve the unknown coefficients in the Chebyshev series. The sound pressure and transmission loss of the muffler were obtained, and the correctness of the theoretical model was verified by comparing it with the FEM results. Finally, the coupling characteristics were analyzed, and the effects of boundary constraints and muffler parameters on transmission loss were studied. The results show that the expansion muffler with a flexible back cavity has a lower natural frequency and stronger low-frequency coupling effect than that with a rigid back cavity. The impact of boundary constraints on transmission loss is mainly reflected above 1000 Hz. When boundary constraints are released, the peak value of transmission loss moves to low frequencies and increases. This shift is conducive to improving sound attenuation performance. As the back cavity’s length or radius increases, the transmission loss curve moves to the low frequency, and the influence of back cavity’s radius on the acoustic performance of the muffler is more obvious. As the thickness of the flexible wall or Young's modulus decreases, the transmission loss curve will move further to the low frequency.