## Magnetic order of the itinerant model at *U* = 5 as a function of γ ∈ [0, 0.25] for different values of α = 1/3, 1/6, 1/4, 1/5 (from top to bottom) obtained within RDMFT

**Figure 6.** Magnetic order of the itinerant model at *U* = 5 as a function of γ ∈ [0, 0.25] for different values of α = 1/3, 1/6, 1/4, 1/5 (from top to bottom) obtained within RDMFT. We generally find Néel order around γ = 0 and collinear order around γ = 0.25. In between for 0.05 γ 0.20, we observe different kinds of spiral order with periodicities of three, four, six, eight, and ten sites due to a competition of a Dzyaloshinskii–Moriya (DM) and an anisotropic XXZ Néel type spin interaction, see equation (4). Phase boundaries are indicated by solid vertical lines. Black dots denote the actual values of γ that we simulated using RDMFT. The magnetization diagrams show the direction of magnetic order in *S _{y}*–

*S*plane. The background colour indicates the magnetization amplitude

_{z}*S*

_{tot}(

*x*). The magnetization diagrams are extracted from the specific parameter point that is indicated by the dashed circle. If, due to quantum fluctuations, the classical spin Hamiltonian exhibits a different phase boundary than the itinerant model, we denote the phase boundary of the spin Hamiltonian by a vertical dashed line with the label

*H*

_{spin}.

**Abstract**

Motivated by the recent progress in engineering artificial non-Abelian gauge fields for ultracold fermions in optical lattices, we investigate the time-reversal-invariant Hofstadter–Hubbard model. We include an additional staggered lattice potential and an artificial Rashba-type spin–orbit coupling term available in experiment. Without interactions, the system can be either a (semi)-metal, a normal or a topological insulator, and we present the non-Abelian generalization of the Hofstadter butterfly. Using a combination of real-space dynamical mean-field theory (RDMFT), analytical arguments, and Monte-Carlo simulations we study the effect of strong on-site interactions. We determine the interacting phase diagram, and discuss a scenario of an interaction-induced transition from a normal to a topological insulator. At half-filling and large interactions, the system is described by a quantum spin Hamiltonian, which exhibits exotic magnetic order due to the interplay of Rashba-type spin–orbit coupling and the artificial time-reversal-invariant magnetic field term. We determine the magnetic phase diagram: both for the itinerant model using RDMFT and for the corresponding spin model in the classical limit using Monte-Carlo simulations.