Natural biocompatible nanomaterials such as self-assembled triterpene natural small molecule products with favorable anticancer activity show great potential for biomedical application. However, the mechanisms of their molecular self-assembled structures have not been investigated systematically, while there are still few reports of the natural active carrier for drug delivery. Herein, in this work, we further explored the molecular assembly mechanism and common regularity of tetracyclic triterpenes ergosterol, stigmasterol as well as pentacyclic triterpenes glycyrrhetinic acid and ursolic acid, which suggested that the coplanarity and orderliness of molecular arrangements which are speculated to be responsible for their self-assembly into the spherical, rod-like or lamellar nanostructure. Besides, ergosterol (ET) with better anticancer activity was chosen as a representative substance for construction of the synergistic antitumor nanodrug. By intermolecular hydrogen bonding and π-π stacking, chemotherapeutic drug paclitaxel (PTX) was encapsulated into ET-PTX NPs successfully. Then, the anti-cancer efficacy of the tumor-bearing mice was evaluated according to the protocol approved by the Experimental Animal Research Center of Harbin Medical University. The resulting nanodrug exhibited excellent biosafety and enhanced in vivo anticancer activity efficiency of 52.3%, higher than free PTX (29.4%) or ET NPs (32.5%) alone, further verifying the potential medical application value of triterpene natural products. This work provides not only a theoretical basis for exploring the self-assembly behavior of small molecule natural products, but also a promising perspective for the fabrication of active natural biocompatible nanodrug delivery systems for synergistic antitumor therapy and other biomedical applications.