This study aims to understand the characteristics and variability in the β diversity of fish community and its influencing environmental factors in the Haizhou Bay, which was rich in fishery resources in history but seriously declined in recent years due to overfishing and environmental pollution. The data were collected from fishery resource surveys conducted in the Haizhou Bay and its adjacent waters from 2013 to 2017 using bottom trawl in autumn. β diversity was measured by Sørensen dissimilarity index using pairwise and multiple-site methods. Additionally, β diversity was divided into turnover and nestedness components and the results were analyzed visually by violin plots. On this basis, the generalized dissimilarity modelling (GDM) was used to analyze the relationship between β diversity and its components with environmental factors. The results of this study showed that the β diversity of fish community and its components fluctuated remarkably among survey years, the fluctuation was larger in the shallow water, less for the deep water, and remained at a relatively stable level regarding the whole sea area. The proportion of the turnover component in β diversity was basically maintained at more than 80%. The component of turnover was significantly larger than that of nestedness, implying the changes of fish community in the Haizhou Bay were dominated by turnover. The salinity and chlorophyll concentration of seawater were the main factors influencing β diversity and its turnover component, and the accumulated deviance explained rate of salinity and chlorophyll were 28.05% and 23.33%, respectively, the influence of water temperature was weak. The environmental factors selected in this study had a low explanatory rate for the nestedness component, which was only 1.52%. Our results implied that the characteristics of fish community have been driven by the environment, and the community development can yield corresponding adaptive to environmental changes. The process of changes in β diversity of fish community may be complex. For instance, salinity and chlorophyll concentration affect the prey density of fish and thus affect β diversity, however, more influencing factors have not been evaluated. Further studies on β diversity and its components are necessary to understand the mechanism of changing characteristics of fish community, and to provide scientific reference for the establishment of marine reserves as well as the optimization of marine conservation strategies.