## (a) Energy gap between the first excited state and the ground state versus the atomic transition strength *J* for different particle numbers *N*

**Figure 4.** (a) Energy gap between the first excited state and the ground state versus the atomic transition strength *J* for different particle numbers *N*. (b) The finite-size scaling behaviours of Δ*J* = *J _{N}* −

*J*

_{c}with δ

_{G}= 2/3 and (c) Δ

*E*

_{min}with μ = 1/3 for different atomic interaction \widetilde{U}, where the discrete points denote the numerical results for finite particle numbers

*N*= [130, 270], and the solid lines are the fitting functions. (d) The finite-size scaling analysis of the energy gap Δ

*E*with \widetilde{U}=2g for different system sizes

*N*= 130, 150, 170, 190, 210, where ν = 2/3, J_N\sim (3-\sqrt{2})/2+2.425N^{-2/3} and Δ

*E*

_{min}~ 1.704

*N*

^{−1/3}. The parameter is δ

_{b}= −2

*g*.

**Abstract**

The quantum phase transition in an atom–molecule conversion system with atomic transition between two hyperfine states is studied. In the mean-field approximation, we give the phase diagram in which the phase boundary only depends on the atomic transition strength and the atom–molecule energy detuning, but not on the atomic interactions. Such a phase boundary is further confirmed by calculating the fidelity of the ground state and the energy gap between the first excited state and the ground state. As a comparison to the mean-field results, we also study the quantum phase transition with the full quantum method where the phase boundary depends on the particle number. Analysing the finite-size scaling behaviours of the energy gap, the fidelity susceptibility and the first-order derivative of entanglement entropy with respect to the atomic transition strength, we obtain their critical exponents by numerical calculation and show that in the thermodynamic limit, one can obtain the same phase boundary as in the mean-field approximation. Our results show a new way to manipulate the quantum phase transition by regulating the atomic transition strength with the intensity of the laser.