In this work, we clarify the underlying physics and the acceptor ionization process in the In-Mg co-doping GaN. The fundamental understandings are also applicable for other co-doping nitride systems. We show the effective acceptors ionization stems mainly from the band structure tuning effect of the 4d orbitals of In atoms. In addition, temperature dependent defect-related photoluminescence analysis was proposed to examine the energy position of Mg impurity levels, which is crucially important in the design for high p-doping. Finally, a hole concentration, which is nearly one order of magnitude higher than what is typically achievable by direct Mg-doping, was achieved.