Plasmon polaritons in graphene and phonon polaritons in polar materials or hyperbolic materials could enhance the near-field radiative heat transfer (NFRHT). However, the detailed investigation of the coupling mechanism of graphene plasmons and different phonon polaritons in the Reststrahlen bands of the materials is insufficient. Here we systematically investigated the NFRHT between graphene-covered polar materials or hyperbolic materials in the Reststrahlen bands. Three types of bulk materials, i.e., SiC, hBN and α-MoO3, are studied for comparisons. It is found that the coupling between graphene plasmons and surface phonon polaritons (SPhPs) in SiC could always suppress the heat transfer. While the coupling between graphene plasmons and hyperbolic phonon polaritons (HPPs) in hBN could suppress or enhance the heat transfer, depending on the chemical potential of graphene. For graphene-covered α-MoO3, it is found that the hyperbolic plasmon-phonon polaritons (HPPPs) always suppress the heat transfer, while the surface plasmon-phonon polaritons (SPPPs) could enhance or suppress the heat transfer, depending on the chemical potential of graphene and the frequencies. This work is helpful to deepen our understanding of the coupling mechanism between graphene plasmons and different kinds of phonon polaritons, and paves the way to manipulate the NFRHT of graphene-covered materials.