Spontaneous CP violation in the D₅-symmetric four-Higgs-doublet models

With the 2012 announcement by the Large Hadron Collider (LHC) [1, 2] of a Higgs boson with a mass of approximately 125 GeV, the Standard Model (SM) of particle physics received decisive experimental confirmation. Owing to these numerous experimental confirmations, the SM has become the most successful theory for describing the behavior and interactions of elementary particles. However, many puzzles remain beyond the explanatory power of the SM. These include, for example, that the strength of CP violation in the SM is insufficient to generate the matter-antimatter asymmetry of the universe [3, 4], and that neutrino oscillation experiments have established that neutrinos have tiny masses [5], whereas in the SM neutrinos are strictly massless.

In the SM, there is only one Higgs doublet. For a long time, there has been no strong experimental evidence indicating that only one type of Higgs boson exists in nature. As a result, extensions of the SM Higgs sector—namely, multi-Higgs-doublet models (MHDMs)—have emerged as an important research direction in studying new physics beyond the SM [6−17]. One important motivation for studying MHDMs is that explicit or spontaneous CP violation in their scalar potentials can provide additional sources of CP violation [18−32], thereby helping to address the matter-antimatter asymmetry of the universe. The study of MHDMs traces back to the two-Higgs-doublet model (2HDM) proposed by T. D. Lee in 1973 [33], in which spontaneous CP violation (SCPV) originating from the relative phase of the vacuum expectation values (vevs) provides an additional source of CP violation. Subsequently, in 1976, S. Weinberg introduced the three-Higgs-doublet model (3HDM), providing a natural theoretical explanation for the milliweak origin of CP violation [34]. To date, the 2HDM and 3HDM have been extensively studied, with numerous related results continuing to emerge [35−52]. However, studies of the four-Higgs-doublet model (4HDM) remain relatively scarce [53].

In existing 4HDM studies, such models are typically employed to explain the hierarchical structure of the masses and the CKM mixing matrix for the three generations of fermions, as well as CP violation [54−56]. They have also been used to account for dark matter within supersymmetric extensions of the SM [57], and to construct new neutrino models in grand unified theories [58]. A primary reason for the limited exploration of the 4HDM is that MHDMs typically involve numerous free parameters. Compared with the 3HDM, the additional Higgs doublet in the 4HDM further increases the number of free parameters, which in turn renders the analysis of the vacuum structure, phenomenology, and other aspects of the model more complex and challenging.

By introducing appropriate discrete symmetries, the number of free parameters in a model can be effectively reduced. In this work, we impose the $ D_5 $ symmetry group on the 4HDM. Compared with other symmetries, the advantage of $ D_5 $ lies in its ability to significantly reduce the number of free parameters in the 4HDM, leaving only 16 real parameters. This number is comparable to that of certain 3HDMs (e.g., the $ Z_2 $ 3HDM [44], which also has 16 real parameters), thereby substantially reducing the complexity of studying the model.

Previous studies employing $ D_5 $ symmetry have often aimed to explain fermion masses and mixing [59−64], with one of the earliest applications being a four-quark model [59], in which three Higgs doublets were introduced. Later, C. Hagedorn et al. [61] attempted to implement $ D_5 $ symmetry in the 3HDM, but found that its scalar potential exhibits an accidental $ U(1) $ symmetry that would lead to phenomenologically unacceptable massless Goldstone bosons. Building on this, they successfully introduced $ D_5 $ symmetry into the 4HDM and proved that the scalar potential of this model has no accidental symmetries. However, their work only demonstrated that spontaneous CP violation is not allowed for completely arbitrary vev parameters; they neither investigated the specific vev configurations that might lead to SCPV nor analyzed the full vacuum structure of the model. In general, a thorough study of the full vacuum structure is crucial for any successful MHDM. In this paper, we investigate the full neutral vacuum structure of the 4HDM with $ D_5 $ symmetry. We systematically analyze all possible real and complex vacua, focusing on the possibility of spontaneous CP breaking. Finally, we provide a complete list of the complex vacua that lead to spontaneous CP violation.

The remainder of this paper is organized as follows. Section 2 introduces the non-Abelian discrete group $ D_5 $ and incorporates it into the 4HDM, thereby constructing the scalar potential of the model. Section 3 systematically summarizes the procedure for solving the neutral vacuum structure. Section 4 analyzes the possible real vacua in the $ D_5 $ 4HDM and presents the conditions for the positive definiteness of the Hessian. Section 5 analyzes the possible complex vacua in the $ D_5 $ 4HDM, investigates whether they can lead to spontaneous CP violation, and discusses the corresponding positive-definiteness conditions. Section 6 discusses the residual $ Z_2 $ symmetry. Section 7 discusses the relationship between real and complex vacua. Finally, Section 8 summarizes our conclusions.

The dihedral group $ D_5 $ is the symmetry group of a regular pentagon. It is isomorphic to $ Z_5 \rtimes Z_2 $ and has ten elements, denoted by $ a^{m}b^{k} $ with $ m = 0, 1, 2, 3, 4 $ and $ k = 0, 1 $, where the generators a and b satisfy $ a^5 = e $, $ b^2 = e $, and $ bab = a^{-1} $. $ D_5 $ has four irreducible representations: two singlets, $ {\bf{1}} $ and $ {\bf{1}}' $, and two doublets, $ {\bf{2}} $ and $ {\bf{2}}' $. The generators for the singlets of $ D_5 $ are given in [65].

We next consider the complex representation of $ D_5 $. We choose a complex basis in which the generators of the doublet representation of $ D_5 $ are given by [65]

When the two doublets, $ x=(x_1 , x_2)^\intercal $ and $ y=(y_1 , y_2)^\intercal $, are in the doublet representation of $ D_5 $, we denote them by:

then

In $ D_5 $, in the basis of Eqs. (1) and (2), the Clebsch-Gordan coefficients for all tensor products are as follows. The tensor product of the singlet w with the singlet z yields

The singlet w, together with the doublet $ x=(x_1, x_2)^\intercal $, gives

where $ \bar{{\bf{2}}}={\bf{2}}, {\bf{2}}' $. The two doublets $ x=(x_1,x_2)^\intercal $ and $ y=(y_1,y_2)^\intercal $ yield

In the 4HDM, there are four scalar fields $ \phi_1 $, $ \phi_2 $, $ \phi_3 $, and $ \phi_4 $, all of which are $ SU(2) $ doublets. We introduce a $ D_5 $ symmetry into the model and construct the four-component vector $ \vec{\Phi} = (\phi_1,\phi_2,\phi_3,\phi_4)^T $ in the internal space, comprising the Higgs-field components. We assume that $ (\phi_1,\phi_2) $ transforms as a doublet under $ D_5 $ and $ (\phi_3,\phi_4) $ forms another doublet. The transformations of the scalar fields $ \phi_i $ under the $ D_5 $ symmetry are then given by:

The most general scalar potential of the $ D_5 $-symmetric 4HDM, consistent with the complex representation given in Eq. (2), consists of quadratic and quartic terms:

Using the tensor products given in Eqs. (5)–(7) for the complex representation, we compute $ \Phi^\dagger \otimes \Phi $ and extract the combinations of $ \phi^\dagger_i\phi_j $ that transform as the invariant singlet representation $ {\bf{1}} $. Thus, the quadratic part of the $ D_5 $-invariant potential is given by

By evaluating the tensor product $ ({\Phi^\dagger}\otimes{\Phi})\otimes({\Phi^\dagger}\otimes{\Phi}) $, we obtain the quartic part of the $ D_5 $-invariant potential:

We now introduce an alternative notation for the scalar potential. Expanding the first six terms in Eq. (11) and combining like terms yields new contributions whose coefficients are combinations of the original parameters. We define these combinations as new parameters:

In terms of the new parameters, the invariant quartic part of the potential can be written as:

It is straightforward to verify that Eq. (13) is equivalent to Eq. (11). In this work, we adopt the notation in Eq. (13). The dihedral group $ D_5 $ satisfies $ D_5 \simeq Z_5 \rtimes Z_2 $, and its generators are:

It should be emphasized that if two field configurations $ (\phi_1, \phi_2, \phi_3,\phi_4 ) $ and $ (\phi_1', \phi_2', \phi_3',\phi_4') $ are related by a $ Z_5 $ or $ Z_2 $ transformation within $ D_5 $ (i.e., one can be obtained from the other by applying such a symmetry operation), then, owing to the invariance of the scalar potential under $ D_5 $ transformations acting on the Higgs fields, the two configurations are physically equivalent. Hence, it suffices to consider only one representative from each equivalence class in the vacuum analysis.

We require all 16 unknown parameters in the potential to be real, implying that there is no explicit CP violation in the potential. This allows us to study the scalar sector under CP conservation and to investigate spontaneous CP violation induced by complex vevs.

In Ref. [61], an alternative notation was adopted to construct the general $ D_5 $-symmetric 4HDM scalar potential, and it was shown that this potential contains no accidental symmetries. We now demonstrate that the scalar potential in the notation used for the $ D_5 $ 4HDM in this work likewise contains no accidental symmetries. We assume that the fields $ \phi_i $ ($ i = 1, 2, 3, 4 $) transform as follows:

Then the parameter terms $ \mu^2_{1,2} $ and $ \lambda_{k} $ $ (k=1,2,...,10) $ all preserve the full $ U(1)^4 $ symmetry. When the condition $ \sigma_1=\sigma_3+\sigma_4-\sigma_2 $ is satisfied, each of the $ \lambda_{11} $ and $ \lambda_{12} $ terms individually breaks the $ U(1)^4 $ symmetry down to the same $ U(1)^3 $ symmetry. Meanwhile, $ \lambda_{13} $ and $ \lambda_{14} $ are constrained, respectively, by the conditions $ 2\sigma_2=\sigma_1+ \sigma_3, 2\sigma_1=\sigma_2+\sigma_4 $ and $ 2\sigma_3=\sigma_1+\sigma_4, 2\sigma_4=\sigma_2+\sigma_3 $, which can further break $ U(1)^3 $ to $ U(1)^2 $. In other words, none of the potential parameter terms is invariant only under the $ U(1)_Y $ gauge symmetry; each must also respect at least one additional $ U(1) $ symmetry. Furthermore, only when $ \lambda_{13} \neq 0 $ and $ \lambda_{14} \neq 0 $, and under the condition $ \sigma_1=\sigma_2=\sigma_3=\sigma_4 $, can the $ U(1)^4 $ symmetry be broken to the $ U(1)_Y $ gauge symmetry. In other words, even if the potential contains both the $ \lambda_{11} $ and $ \lambda_{12} $ terms together with only the $ \lambda_{13} $ term (or only the $ \lambda_{14} $ term), it still remains invariant under $ U(1)^2 $. Therefore, when $ \lambda_{13}=0 $ or $ \lambda_{14}=0 $, an accidental $ U(1) $ symmetry inevitably appears in the scalar potential in addition to the gauge symmetry $ U(1)_Y $. This leads to serious consequences: first, in the vacuum structure, the corresponding vevs are no longer minima of the scalar potential of the $ D_5 $ 4HDM; second, phenomenologically unacceptable massless Goldstone bosons emerge in the scalar mass spectrum. Consequently, in a 4HDM with an exact $ D_5 $ symmetry, it is necessary to satisfy:

This is the most fundamental prerequisite for the model's validity.

To guarantee a stable minimum in the $ D_5 $ 4HDM, the scalar potential must be bounded from below (BFB); that is, there must be no direction in field space along which the potential tends to negative infinity. Since this paper focuses on the neutral vacuum structure, we consider only the bounded-from-below conditions along neutral directions (BFB-n). To ensure the stability of the $ D_5 $ 4HDM potential, we examine all possible directions in which the fields $ \phi_1 $, $ \phi_2 $, $ \phi_3 $, and $ \phi_4 $ tend to infinity. The quadratic parameters $ \mu^2_{1} $ and $ \mu^2_{2} $ in the potential must be positive. To derive the necessary conditions for the quartic parameters $ \lambda_i $, a straightforward approach is to study the behavior of the potential along specific field directions—namely, for any direction in field space along which $ \phi_i \to \infty $, one requires that $ V_4 \gt 0 $[42]. Following the methods of Refs. [66−69] for deriving the BFB conditions in the 2HDM and 3HDM, we derive a set of necessary and sufficient BFB-n conditions for the $ D_5 $ 4HDM:

where

and the necessary conditions:

where

These conditions ensure the stability of the scalar potential along the neutral directions.

After spontaneous symmetry breaking (SSB), we express the complex scalar field as an $ SU(2) $ doublet as follows:

where $ u_i = v_i e^{i\theta_i} $, with real moduli $ v_i $ and arbitrary phases $ \theta_i $, and $ \sqrt{v_1^2+v_2^2+v_3^2+v_4^2}=v_w=246 $ GeV. When all phases $ \theta_i $ vanish, the vacuum is real. Since a global $ U(1) $ rotation can absorb one overall phase, we fix $ \theta_4 = 0 $ without loss of generality. If, after this rotation, any of the remaining phases $ \theta_i $ is nonzero, the vacuum is complex.

The purpose of this paper is to identify all possible complex vacuum states in the $ D_5 $ 4HDM that can give rise to SCPV. A vacuum state corresponds to a specific configuration of vacuum expectation values (vevs) for four complex Higgs doublets $ \phi_i $ $ (i = 1, 2, 3, 4) $, which can be written as $ (v_1e^{i\theta_1}, v_2e^{i\theta_2}, v_3e^{i\theta_3}, v_4)/\sqrt{2} $. We determine these states by minimizing the scalar potential with respect to $ v_i $ and $ \theta_i $. For notational simplicity, we omit the $ \sqrt{2} $ factor in the text but include it in all calculations.

We further classify all possible vacuum solutions obtained from the minimization of the potential according to the following definitions:

1. Normal vacua: The vacuum expectation values (vevs) of all four doublets are nonzero ($ v_i \neq 0 $), so each Higgs doublet participates in electroweak symmetry breaking.

2. Inert vacua: At least one doublet has a vanishing vacuum expectation value (VEV), $ v_i = 0 $. The corresponding doublet does not participate in electroweak symmetry breaking, thereby defining an inert model.

Notice that, unlike transforming to the Higgs basis, inert vacua are identified in the original $ D_5 $-symmetric basis, in which the symmetry remains manifest. In the Higgs basis, by contrast, the four doublets are recombined so that all nonzero vevs are rotated into a single doublet, artificially yielding a representation in which the other three vevs vanish. This basis change merely alters the field coordinates, rendering the original symmetry non-manifest; it does not correspond to a different physical vacuum. For a detailed definition and discussion, see Ref. [42].

Our first step is to determine all stationary points of the scalar potential associated with the neutral vacuum, i.e., the vevs that satisfy the first-derivative conditions of the potential. We consider the potential $ V(u_i, u_i^\ast) $ and solve for the vevs by imposing the stationarity conditions:

These two conditions are complex conjugates of one another and therefore equivalent; hence, it suffices to use only one of them. Choosing the latter yields the following four stationarity conditions:

Since the five parameter pairs ($ \lambda_3, \lambda_7 $), ($ \lambda_4, \lambda_8 $), ($ \lambda_5, \lambda_9 $), ($ \lambda_6, \lambda_{10} $), and ($ \lambda_{11}, \lambda_{12} $) enter the stationarity conditions only through their sums, there are effectively 11 independent parameters out of the original 16. To make the equations more concise, we introduce the following abbreviations:

Although the stationarity conditions above are presented in complex form, they can equivalently be expressed in terms of the real parameters $ v_i $ and $ \theta_i $:

After fixing $ \theta_4=0 $, the seven stationarity conditions are given in Appendix B.

Subsequently, to ensure that the vevs obtained by solving the stationarity conditions correspond to local minima of the scalar potential, we construct the Hessian with respect to the VEV parameters:

and we require it to be positive definite. For real vacua, the Hessian is given by:

Finally, after obtaining all possible vevs that satisfy the model conditions, we determine whether each complex vacuum state leads to spontaneous CP violation.

When all $ \theta_i = 0 $, the vacuum is real, and the vevs can be denoted as:

$ \bullet $ Normal vacua

We now discuss the Normal vacua in the real case, in which all $ v_i $ are nonzero. Substituting Eq. (31) into the stationarity conditions (23)–(26), we obtain the following four expressions for $ \mu^2_1 $ and $ \mu^2_2 $ in terms of the quartic couplings:

Note that equations (32) and (33) do not hold when $ v_1 = 0 $ and $ v_2 = 0 $, respectively. Similarly, equations (34) and (35) do not hold when $ v_3 = 0 $ and $ v_4 = 0 $, respectively. Since the two expressions for $ \mu_1^2 $ in (32) and (33) must be mutually consistent, and likewise for $ \mu_2^2 $ in (34) and (35), we obtain the following two consistency conditions:

First, consider the most general real vacuum ($ v_1, v_2, v_3, v_4 $). We find that, provided the BFB conditions (17) and (19) hold, it is possible to satisfy the two consistency conditions (36a) and (36b) by appropriately choosing the remaining potential parameters $ \lambda_1 $, $ \lambda_2 $, $ \bar\lambda_3 $, $ \bar\lambda_4 $, $ \bar\lambda_5 $, $ \bar\lambda_6 $, and $ \bar\lambda_{11} $ so that $ \lambda_{13} \neq 0 $ and $ \lambda_{14} \neq 0 $. Therefore, the vacuum $ (v_1, v_2, v_3, v_4) $ constitutes a solution of the model, subject to the constraints on the potential parameters imposed by Eqs. (32), (34), (36a), and (36b).

Next, we consider special cases in which some of the moduli $ |v_i| $ of the vacuum $ (v_1, v_2, v_3, v_4) $ are equal, to determine whether these equalities can simplify the constraints on the potential parameters or reduce their number. The following cases are considered:

$ (1) $ When $ \left |v_1 \right |=\left |v_2 \right | $ and $ \left |v_3 \right |=\left |v_4 \right | $, the vevs can take four forms: $ (v_1, v_1, v_3, v_3) $, $ (v_1, -v_1, -v_3, v_3) $, $ (v_1, -v_1, v_3, v_3) $, and $ (v_1, v_1, -v_3, v_3) $. However, the last two imply $ \lambda_{13}=0 $ and $ \lambda_{14}=0 $, which do not satisfy the model requirements. Thus, only the first two forms are acceptable. For simplicity, we denote them uniformly as the vacuum $ (v_1,\pm v_1,\pm v_3,v_3) $, since both sides of the constraint Eqs. (36a) and (36b) vanish identically; consequently, the number of effective constraints on the parameters is reduced from four to two.

$ (2) $ When $ \left |v_1 \right |=\left |v_3 \right | $ and $ \left |v_2 \right |=\left |v_4 \right | $, the vevs can be expressed in four forms: $ (v_1, v_2, v_1, v_2) $, $ (v_1, -v_2, -v_1, v_2) $, $ (v_1, -v_2, v_1, v_2) $, and $ (v_1, v_2, -v_1, v_2) $. None of these forms imply $ \lambda_{13}=0 $ or $ \lambda_{14}=0 $, and, while they significantly simplify the constraints, their number remains four. We choose the first two forms for discussion, which we denote collectively as the vacuum $ (v_1, \pm v_2, \pm v_1, v_2) $.

$ (3) $ When $ \left |v_1 \right |=\left |v_4 \right | $ and $ \left |v_2 \right |=\left |v_3 \right | $, this case is similar to the one above. We select two forms for discussion, denoted collectively as the vacuum $ (v_1, \pm v_2, v_2, \pm v_1) $.

$ (4) $ When any three moduli $ |v_i| $ are equal, there are four cases up to permutation. Taking $ |v_1| = |v_2| = |v_3| = v $ as an example, the vevs can be expressed in four forms: $ (v,v,v,v_4) $, $ (v,-v,-v,v_4) $, $ (v,-v,v,v_4) $, and $ (v,v,-v,v_4) $. None of these forms imply $ \lambda_{13}=0 $ or $ \lambda_{14}=0 $. Since there are numerous permutations of such special vacua, and since the constraints on the potential parameters are neither significantly simplified nor reduced in number, and the same holds for the complex vacua discussed later, we do not further elaborate on these special vacua in this paper.

$ (5) $ When $ |v_1| = |v_2| = |v_3| = |v_4| = v $ and $ \lambda_{13} \neq 0 $, $ \lambda_{14} \neq 0 $, the vevs reduce to two forms, $ (v,v,v,v) $ and $ (v,-v,-v,v) $, which we denote uniformly as $ (v,\pm v,\pm v,v) $. In this case, the two constraints on the potential parameters can be obtained directly from those for the vacuum $ (v_1,\pm v_1,\pm v_3,v_3) $ by setting $ v_1 = v_3 = v $, but their number is not further reduced. The same holds for the complex vacua discussed later, so we do not further elaborate on this special vacuum in this paper.

The constraints on the potential parameters for the first three special vacua discussed above are summarized in Table 1.

$ \bullet $ Inert vacua

We now discuss the inert vacua among the real vacua. First, we consider the case in which only one of the $ v_i $ is zero. Taking $ v_1 = 0 $ as an example, the vevs are given by $ (0, v_2, v_3, v_4) $. In this case, Eq. (32) no longer applies; instead, we use the original stationarity condition $ \frac{\partial V}{\partial v_1} = 0 $ to obtain:

Furthermore, the expression for $ \mu_1^2 $ (33) is automatically consistent, while the two expressions for $ \mu_2^2 $ (34) and (35) still need to satisfy the consistency condition (36b). We find that this vacuum does not lead to $ \lambda_{13}=0 $ or $ \lambda_{14}=0 $ under these constraints and is a solution of the model. For other real vacua in which only one of the $ v_i $ is zero, the situation is similar by permutation of the indices, and all such vacua satisfy the requirements of the model. Moreover, we have examined special cases where some of the $ |v_i| $ are equal. We find that although the constraints can be simplified, their number is not reduced, and the same holds for the complex vacua discussed later; hence, we do not elaborate further on these special vacua in this paper.

Secondly, we consider the case where two of the $ v_i $ are zero. Taking $ v_1 = v_2 = 0 $ as an example, the vevs can be denoted as $ (0, 0, v_3, v_4) $. In this case, Eqs. (32) and (33) no longer hold; instead, the original stationarity conditions $ \dfrac{\partial V}{\partial v_1} = 0 $ and $ \dfrac{\partial V}{\partial v_2} = 0 $ are used to obtain:

Further solving yields $ \lambda_{14}=0 $, so this vacuum fails to satisfy the model requirements. By explicit calculation, we find that for all real vacua in which two of the $ v_i $ are zero, the stationarity conditions $ \dfrac{\partial V}{\partial v_i}=0 $ inevitably imply that at least one of $ \lambda_{13}=0 $ or $ \lambda_{14}=0 $. Therefore, in this case, no vacuum satisfies the model requirements.

Finally, we consider the case in which three of the $ v_i $ are zero. Taking $ v_2 = v_3 = v_4 = 0 $ as an example, the vevs are $ (v, 0, 0, 0) $. In this case, Eq. (32) yields the following unique constraint:

It does not imply $ \lambda_{13}=0 $ or $ \lambda_{14}=0 $, so this vacuum is a valid solution of the model. For other real vacua in which three of the $ v_i $ vanish, the situation is similar, and all such vacua satisfy the requirements of the model.

In Table 1, we summarize the complete list of all possible real vacua and the constraints on the parameter space that admit each solution. For convenience in comparing real and complex vacua, we classify each real vacuum in the table using the notation R-X-Yz. Here, R denotes a real vacuum, X is either N (normal vacuum) or I (inert vacuum), Y gives the number of constraints on the potential parameters, and z distinguishes cases with identical values of X and Y.

We now discuss each of the real vacua listed in Table 1.

● Vacuum R-N-2 contains two forms: $ (v_1,v_1,v_3,v_3) $ and $ (v_1,-v_1,-v_3,v_3) $, with $ v_1^2 + v_3^2 = \frac{v^2_{w}}{2} $. After SSB, the former preserves a residual $ Z_2 $ symmetry of the potential, whereas the latter completely breaks the $ D_5 $ symmetry.

● Vacuum R-N-4a is the most general real vacuum and completely breaks the $ D_5 $ symmetry.

● The Vacua R-N-4b and R-N-4c are related by a permutation of the Higgs fields, satisfy $ v_1^2 + v_2^2 = \frac{v^2_{w}}{2} $, and completely break the $ D_5 $ symmetry.

● The Vacua R-I-1a and R-I-1b are also related by a permutation of the Higgs fields, satisfy $ v = v_{w} $, and completely break the $ D_5 $ symmetry. In these two vacua, because only one Higgs doublet acquires a non-zero vev, there are no flavor-changing neutral currents (FCNCs).

● The Vacua R-I-4a, R-I-4b, R-I-4c, and R-I-4d satisfy, respectively, $ \sqrt{v_1^2+v_2^2+v_3^2}=v_{w} $, $ \sqrt{v_1^2+v_2^2+v_4^2} = v_w $, $ \sqrt{v_1^2 + v_3^2 + v_4^2} = v_w $, and $ \sqrt{v^2_2 + v^2_3 + v^2_4} = v_{w} $. All four vacua completely break the $ D_5 $ symmetry.

Relationships among real vacua To provide a more intuitive analysis of the real vacua listed in Table 1, we further classify them into the following three types:

$ (1) $ The first type comprises the Normal vacua R-N-2, R-N-4a, R-N-4b, and R-N-4c. Vacuum R-N-4a is the most general among them, as the other three can be obtained by assigning specific values to the parameters $ v_i $ in R-N-4a.

$ (2) $ The second type comprises the Inert vacua R-I-1a and R-I-1b, with three of the $ v_i $ set to zero.

$ (3) $ The third type comprises the Inert vacua R-I-4a, R-I-4b, R-I-4c, and R-I-4d, with only one of the $ v_i $ set to zero.

We now discuss the conditions for the positive definiteness of the Hessians (30) associated with these real vacua.

For the vacuum R-N-2 to be a local minimum of the potential, its Hessian must be positive definite, which requires that all four leading principal minors be positive. Accordingly, the potential parameters must satisfy the following four conditions:

where

For Vacuum R-I-1a, positive definiteness requires that:

The positive-definiteness conditions for the Hessians that ensure the remaining vacua in Table 1 are local minima of the scalar potential are presented in Appendix C.

Before studying complex vacua, we first discuss how to determine whether a given vacuum leads to CP violation. By definition, in spontaneous CP violation induced by the vacuum, the Lagrangian before spontaneous symmetry breaking (SSB) is explicitly CP-conserving, which requires all parameters in the scalar potential to be real. In that case, if the vacuum after SSB is not CP-invariant, this indicates spontaneous CP violation. More precisely, if the Lagrangian is invariant under a CP transformation, yet there exists no transformation that can be physically identified with CP and that leaves both the vacuum and the Lagrangian invariant, then CP is spontaneously violated. In the SM, there is only one Higgs doublet, and its CP transformation is conventionally defined as $ \phi(t, \vec{x}) \to \phi^\ast(t, -\vec{x}) $, which requires all parameters to be real for the potential to be CP-invariant. Hence, no spontaneous CP violation arises from the scalar sector. In an extension of the SM, T. D. Lee [33] constructed the two-Higgs-doublet model (2HDM) and proposed an early mechanism for spontaneous CP violation. We consider an MHDM consisting of n $ SU(2)_L \times U(1)_Y $ Higgs doublets. One must consider the most general CP transformation that leaves the kinetic terms of the Lagrangian invariant, in order to accommodate all possible symmetries of the Lagrangian under which the Higgs doublets transform nontrivially. Hence, the CP transformation for n Higgs doublets can be written as

where $ U_{ij} $ denotes an arbitrary $ n \times n $ unitary matrix acting on the Higgs doublets. Assuming that the vacuum is invariant under this CP transformation, i.e.,

By combining Eqs. (43) and (44), the following relation can be obtained [70]:

This means that if, in such a vacuum, none of the CP symmetries allowed by the Lagrangian satisfy Eq. (45), then the vacuum is not CP invariant; i.e., there is spontaneous CP violation.

We now study the complex vacua, i.e.

$ \bullet $ Normal vacua

We now discuss the Normal vacua in the complex case, in which all $ v_i $ are non-zero. Substituting Eq. (46) into the stationarity conditions (23)–(26) yields the following four expressions for $ \mu^2_1 $ and $ \mu^2_2 $:

Each of the four equations above contains both a real and an imaginary part; therefore, we have eight constraint equations. Analogous to the real vacua in Section 4.1, the four equations (47), (48), (49), and (50) are not applicable when $ v_1 = 0 $, $ v_2 = 0 $, $ v_3 = 0 $, and $ v_4 = 0 $, respectively. Since the two expressions for $ \mu_1^2 $ in (47) and (48) must be consistent with each other, and likewise for $ \mu_2^2 $ in (49) and (50), we obtain the following two consistency conditions:

By simultaneously solving the equations obtained from the imaginary parts of Eqs. (47)–(50), we obtain the following three independent constraints:

First, consider the most general complex vacuum ($ v_1e^{i\theta_1}, v_2e^{i\theta_2}, v_3e^{i\theta_3}, v_4 $), whose vevs comprise four arbitrary moduli $ v_1, v_2, v_3, v_4 $ and three arbitrary phases $ \theta_1, \theta_2, \theta_3 $. Solving Eqs. (52a) and (52b), we find that this vacuum necessarily implies $ \lambda_{13}=0 $ and $ \lambda_{14}=0 $. Therefore, a vacuum in which both the parameters $ v_i $ and $ \theta_i $ can simultaneously take arbitrary values does not satisfy the conditions of the model. Further calculations show that only if the three phases $ \theta_1 $, $ \theta_2 $, and $ \theta_3 $ are constrained—i.e., they must satisfy a specific relation—can one avoid $ \lambda_{13}=0 $ and $ \lambda_{14}=0 $.

Imposing $ \lambda_{13} \neq 0 $ and $ \lambda_{14} \neq 0 $, we keep the four moduli $ v_1, v_2, v_3, v_4 $ arbitrary but lift the arbitrariness of the three phases $ \theta_1, \theta_2, \theta_3 $. From Eqs. (52a) and (52b) we obtain:

Further solving yields the corresponding values of the three phases $ \theta_1 $, $ \theta_2 $, and $ \theta_3 $. Substituting these values into Eqs. (47)–(51b) and (52c) gives four constraints on the potential parameters. This vacuum is denoted as C-N-4a in Table 2.

Next, we consider cases in which some of the moduli $ |v_i| $ are equal among the complex vevs. Under the conditions that $ \lambda_{13} \neq 0 $ and $ \lambda_{14} \neq 0 $, solving Eqs. (51a) and (51b) yields the following solutions:

and

Substituting Eq. (54) into Eqs. (52a) and (52b) yields the following:

By combining Eqs. (55) and (56) and solving them, we obtain the values of the three phases $ \theta_1 $, $ \theta_2 $, and $ \theta_3 $. Substituting these phase values, together with Eq. (54), into Eqs. (47)–(50) and (52c) then yields constraints on the potential parameters. These vacua are denoted as C-N-2, C-N-3a, C-N-3b, C-N-3c, and C-N-3d in Table 2.

$ \bullet $ Inert vacua

We now discuss the Inert vacua among the complex vacua. First, we consider the case in which only one of the $ v_i $ is zero. For example, taking $ v_1=0 $, the vevs are written as $ (0, v_2e^{i\theta_2}, v_3e^{i\theta_3}, v_4) $. In this case, Eq. (47) no longer applies; instead, we use the original stationarity condition (23). Furthermore, the expression for $ \mu_1^2 $ in (48) remains automatically consistent, whereas the two expressions for $ \mu_2^2 $, (49) and (50), must still satisfy the consistency condition (51b). Assuming $ \lambda_{13} \neq 0 $ and $ \lambda_{14} \neq 0 $, solving these equations readily yields the values of $ \theta_2 $ and $ \theta_3 $ as:

Since $ e^{im\pi} = \pm 1 $ and $ v_2, v_3, v_4 $ are arbitrary non-zero real values, we can absorb the factor $ e^{im\pi} $ into the sign of $ v_3 $. Therefore, this vacuum can be written as $ (0, v_2e^{\frac{in\pi}{5}}, v_3e^{-\frac{in\pi}{5}}, v_4) $, which is a solution of the model. For other complex vacua in which exactly one of the $ v_i $ is zero, analogous configurations arise by permuting indices, and all satisfy the model requirements.

Second, we consider the case in which two of the $ v_i $ are zero. This outcome is consistent with the results for the real vacua. For all complex vacua with two vanishing $ v_i $, substituting them into the four stationarity conditions (23)–(26) inevitably leads to at least one of $ \lambda_{13}=0 $ or $ \lambda_{14}=0 $. Thus, in this case, there is no vacuum that satisfies the model requirements.

Finally, we consider the case in which three of the $ v_i $ are zero. Since a global rotation of the vacuum can always eliminate one phase, the vacuum is actually real in this case. This case has already been discussed in Section 4.1 on real vacua.

In Table 2, we summarize the complete list of complex vacua and explicitly indicate whether each vacuum exhibits spontaneous CP violation, marked as “yes” or “no”. We also use the notation C-X-Yz to classify each complex vacuum. For vacua with a single unknown phase θ (originally $ \theta_i $), we drop the subscript for brevity. Additionally, the constraints on the parameter space admitting each complex vacuum solution are summarized in Table 3.

We now discuss each complex vacuum listed in Table 2 and explain why it does or does not lead to spontaneous CP violation.

● The vacuum C-N-2 satisfies $ v_1^2 + v_3^2 = \dfrac{v^2_{w}}{2} $. When the integer n is divisible by 5 (i.e., $ 5 \mid n $), the phases become trivial and this vacuum becomes the real vacuum R-N-2. When n is not divisible by 5 (i.e., $ 5 \nmid n $), both forms of this vacuum completely break the $ D_5 $ symmetry and exhibit three calculable, nontrivial phases. These phases are entirely determined by the $ D_5 $ symmetry of the potential, are independent of its coupling constants, and cannot be rotated away. Because there exist two matrices $ U_1 $ and $ U_2 $ that satisfy the constraints of Eq. (45), namely,

Since either one alone suffices to preclude spontaneous CP violation, there is no spontaneous CP violation in this vacuum. Moreover, $ U_2 $ is also a symmetry of the potential. This indicates that a complex vacuum does not necessarily lead to spontaneous CP violation.

● Vacuum C-N-3a satisfies $ v_1^2 + v_3^2 = \dfrac{v^2_{w}}{2} $. When the integer n is divisible by $ 5 $ (i.e., $ 5 \mid n $), it reduces to Vacuum C-N-3b, leaving a residual $ Z_2 $ symmetry. When n is not divisible by 5 (i.e., $ 5 \nmid n $), both forms of this vacuum completely break the $ D_5 $ symmetry. This vacuum allows for three non-trivial phases. Two of them can be determined as functions of $ \bar\lambda_{11} $, $ \lambda_{13} $, and $ \lambda_{14} $, while the third one is independent of the coupling constants, as shown in Table 3. We have found that when θ satisfies

This vacuum exhibits spontaneous CP violation. If $ \theta=\dfrac{n\pi}{5} $, Vacuum C-N-3a reduces to Vacuum C-N-2, and spontaneous CP violation no longer occurs. Condition (59) is necessary and sufficient for C-N-3a to exhibit spontaneous CP violation.

● Vacuum C-N-3b can be viewed as a special case of C-N-3a when $ n=0 $. It contains $ (v_1e^{i\theta}, v_1e^{i\theta}, v_3, v_3) $ and $ (v_1e^{i\theta}, -v_1e^{i\theta}, -v_3, v_3) $. The former preserves a residual $ Z_2 $ symmetry, whereas the latter completely breaks the $ D_5 $ symmetry. This vacuum admits two identical non-trivial phases, which can be determined as functions of $ \bar\lambda_{11} $, $ \lambda_{13} $, and $ \lambda_{14} $, as shown in Table 3. We have found that when θ satisfies

This vacuum violates CP spontaneously.

● Vacuum C-N-3c satisfies $ v_1^2 + v_3^2 = \dfrac{v^2_{w}}{2} $. When $ n=0 $, it reduces to Vacuum C-N-3d, which leaves a residual $ Z_2 $ symmetry. When n is a non-zero integer, both forms of this vacuum completely break the $ D_5 $ symmetry and exhibit three calculable nontrivial phases. As in the case of C-N-2, these phases are entirely determined by the $ D_5 $ symmetry of the potential, are independent of its coupling constants, and cannot be rotated away. However, this vacuum violates CP spontaneously. Moreover, Vacuum C-N-3c can be seen as a special case of C-N-3a when $ \theta= (\dfrac{1}{2}+\dfrac{n}{5})\pi $.

● Vacuum C-N-3d can be seen as a special case of C-N-3c when $ n=0 $. It contains $ (iv_1, iv_1, v_3, v_3) $ and $ (iv_1, -iv_1, -v_3, v_3) $. The former leaves a residual $ Z_2 $ symmetry, while the latter completely breaks the $ D_5 $ symmetry. This vacuum exhibits two identical, calculable, nontrivial phases, which are entirely determined by the $ D_5 $ symmetry of the potential, are independent of its coupling constants, and cannot be rotated away. This vacuum violates CP spontaneously.

● Vacuum C-N-4a completely breaks the $ D_5 $ symmetry. When $ 5 \mid n $, its phases become trivial and this vacuum becomes the real vacuum R-N-4a. When $ 5 \nmid n $, this vacuum exhibits three calculable nontrivial phases, which are entirely determined by the $ D_5 $ symmetry of the potential, are independent of its coupling constants, and cannot be rotated away. Due to the existence of a matrix U that satisfies the constraint of Eq. (45), namely

There is no spontaneous CP violation in this vacuum.

● The vacua C-N-4b and C-N-4c are related by a permutation of the Higgs fields, satisfy $ v_1^2 + v_2^2 = \dfrac{v^2_{w}}{2} $, and completely break $ D_5 $. They can be seen as special cases of C-N-4a where some of the moduli $ |v_i| $ become equal. Since there exists a matrix U (given in Eq. (61)) that satisfies Eq. (45), CP is not spontaneously violated in these vacua either.

● Vacuum C-N-4d can also be seen as a special case of C-N-4a where some of the moduli $ |v_i| $ become equal, but it actually has only two constraints on the potential parameters. It is related to C-N-2 by a permutation of the Higgs fields, and the two vacua impose almost the same constraints. To emphasize that it can be obtained from C-N-4a and to distinguish it from C-N-2, the notation of this vacuum does not reflect the number of constraints. C-N-4d can be analyzed by reasoning analogous to that for C-N-2. Due to the existence of two matrices U (given in Eq. (61)) and $ U' $ that satisfy Eq. (45), namely

Since either one alone suffices to preclude spontaneous CP violation, there is no spontaneous CP violation in this vacuum. Moreover, $ U' $ is also a symmetry of the potential.

● Vacua C-I-4a, C-I-4b, C-I-4c, and C-I-4d completely break the $ D_5 $ symmetry. When $ 5 \mid n $, their phases become trivial, and these vacua reduce to the real vacua R-I-4a, R-I-4b, R-I-4c, and R-I-4d, respectively. When $ 5 \nmid n $, each such vacuum exhibits two calculable nontrivial phases, which are entirely determined by the $ D_5 $ symmetry of the potential, independent of its coupling constants, and cannot be rotated away. However, these four vacua do not lead to spontaneous CP violation due to the existence of the matrices $ U_1 = \text{diag}\left(\eta^2,\eta,1,\eta^3\right) $, $ U_2 = \text{diag}\left(\eta,\eta^2,\eta^3,1\right) $, $ U_3 = \text{diag}\left(\eta^2,\eta^4,\eta,1\right) $, and $ U_4 = \text{diag}\left(\eta^3,\eta,\eta^4,1\right) $, respectively, where $ \eta=e^{\frac{2in\pi}{5}} $, each of which satisfies the constraint of Eq. (45).

Relationships among complex vacua As summarized in Table 2, all complex vacua can be further classified into three types:

$ (1) $ The first type consists of the Normal vacua with SCPV: C-N-3a, C-N-3b, C-N-3c, and C-N-3d. Vacuum C-N-3a can be regarded as a more general case of the other three vacua, since each can be obtained by assigning specific values to its parameters $ v_i $, θ, and n.

$ (2) $ The second type consists of the Normal vacua without SCPV: C-N-2, C-N-4a, C-N-4b, C-N-4c, and C-N-4d. Vacuum C-N-4a can be regarded as a more general case encompassing C-N-4b, C-N-4c, and C-N-4d, since each can be obtained by assigning specific values to its parameters $ v_i $.

$ (3) $ The third type consists of the Inert vacua without SCPV: C-I-4a, C-I-4b, C-I-4c, and C-I-4d.

To be local minima of the scalar potential, these complex vacua must satisfy the positive-definiteness conditions for their Hessians (29). A detailed discussion is provided in Appendix C.

Among the vacua summarized in this work, three vevs preserve a residual $ Z_2 $ symmetry of the scalar potential after SSB:

It was shown in Ref. [71] that in an N-Higgs-doublet model, if the vev leaves a nontrivial residual symmetry (apart from the baryon number symmetry $ U(1)_B $), this leads to unphysical quark masses and an unphysical CKM matrix. The consequences include: (i) a block-diagonal CKM matrix; (ii) degenerate quark masses; (iii) massless quarks. Therefore, to avoid such unphysical quark masses and mixings, the vev must completely break the flavour symmetry of the potential.

One way to proceed is to add new interaction terms to the scalar potential that explicitly break $ D_5 $ symmetry. In this case, the model is no longer an exactly $ D_5 $-symmetric 4HDM, but rather a modified model. Consequently, the stationarity conditions and the corresponding vacuum solutions will also change, thereby affecting the complex vacua that might lead to spontaneous CP violation. We plan to investigate this type of model in future work.

In our $ D_5 $ 4HDM, complex vacua are characterized by four moduli and three phases; their vevs are obtained by solving seven stationarity conditions. In contrast, real vacua are characterized by four moduli and satisfy three stationarity conditions. In the discussion of complex vacua in Section 5.2, when $ 5\mid n $, some complex vacua become real. We are interested in the relationship between these two types of vacua. In addition, a key question arises: which vacuum is deeper—the complex or the real?

Following Ref. [27], let $ \mathcal{C}(\text{C-X-Yz}) $ denote the set of constraints (see Table 3) satisfied by a specific complex vacuum, and let $ \mathcal{C}(\text{R-X}' \text{-Y}'\text{z}') $ denote the set of constraints (see Table 1) satisfied by a specific real vacuum. Then, if there exists a real limit of the vacuum (possibly up to a sign) in which C-X-Yz coincides in specification with R-X$ ' $-Y$ ' $z$ ' $, and the constraints are compatible, i.e., $ \mathcal{C}(\text{C-X-Yz})\subset \mathcal{C}(\text{R-X}' \text{-Y}'\text{z}') $, we may regard the real vacuum R-X$ ' $-Y$ ' $z$ ' $ as the “origin” of the specific complex vacuum C-X-Yz. We list the real vacua that satisfy these requirements in Table 4.

Except for Vacua C-N-3c and C-N-3d, which cannot be transformed into real vacua, all other complex vacua can be regarded as complex generalizations of corresponding real vacua. There are two main differences between real and complex vacua. First, the number of constraints for complex vacua is generally no less than that for real vacua, implying tighter restrictions on certain potential parameters in the complex case. Second, the Hessian’s positive-definiteness conditions differ: a real vacuum requires its four leading principal minors to be positive, whereas a complex vacuum requires seven. Given their more intricate structure, we provide explicit expressions only for the first four minors; the remaining ones must be verified numerically or by other means. In summary, complex vacua involve more phase degrees of freedom and are therefore subject to significantly more restrictive constraints than real vacua.

To compare the potential depths of two vacua, one must first determine whether they can coexist as local minima. This requires identifying a region in parameter space where both the potential constraints and the positive-definiteness conditions are satisfied simultaneously. In the $ D_5 $ 4HDM, establishing such coexistence is a formidable challenge, given the complexity of the vacuum structure, the large number of parameters, and the intricacy of the positive-definiteness conditions. If such a parameter region exists, the two vacua can both be local minima, and one may compare their depths by evaluating $\Delta V = V_{\text{R}} - V_{\text{C}}$, which generally depends on the chosen parameters. Comparing real and complex vacua is a prerequisite for determining the global vacuum minimum, which is crucial for subsequent phenomenological analyses.

In this paper, we construct a four-Higgs-doublet model (4HDM) based on $ D_5 $ symmetry, which exhibits extremely rich phenomenology. Its scalar potential is described by 16 parameters, and the physical spectrum consists of 3 pairs of charged Higgs bosons and 7 neutral Higgs bosons, corresponding to 13 physical scalars after the 3 Goldstone modes are absorbed by the $ W^\pm $ and Z bosons. In addition, the model offers possibilities for CP violation and baryogenesis.

The $ D_5 $ 4HDM also possesses a rich vacuum structure. Assuming explicit CP conservation in the scalar potential, we comprehensively analyze its full neutral vacuum structure, systematically identify all possible real and complex vacuum solutions, and study the conditions for them to be local minima of the potential, namely the stationarity conditions and the positive-definiteness of the Hessian. In contrast to previous studies that mostly focused on real vacua, this work emphasizes complex vacua, aiming to assess whether the CP symmetry of the scalar potential can be spontaneously broken. We demonstrate that among the identified complex vacuum solutions (as shown in Table 2), several specific complex vacua can spontaneously break CP symmetry. Among them, Vacuum C-N-3a, i.e., $ (v_1e^{i\theta},\pm v_1e^{i(\theta+\frac{n\pi}{5})}, \pm v_3e^{\frac{3in\pi}{5}},v_3) $, is the most general complex vacuum that can lead to spontaneous CP violation. Moreover, comparing the real and complex vacua reveals that the latter can be regarded as complex generalizations of certain real vacua and impose more restrictive constraints on the potential parameters.

In summary, the $ D_5 $ 4HDM provides a natural and viable possibility for CP violation in the study of physics beyond the Standard Model, and our detailed analysis of its neutral vacuum structure lays a solid foundation for subsequent research. This work primarily focuses on the scalar sector, and future efforts will be dedicated to applying these vacua to the Yukawa sector to investigate fermion masses and mixing, thereby achieving a complete construction of the model.

For Vacuum C-N-4a, the following four constraints apply:

The stationarity conditions for the moduli in Eq. (28) can be written as:

The stationarity conditions for the phases can be written as

For R-N-4a, positive definiteness requires:

where

For R-N-4b, the conditions for positive definiteness can be derived directly from those for R-N-4a by replacing the parameters $ (v_1, v_2, v_3, v_4) $ with $ (v_1, \pm v_2, \pm v_1, v_2) $. The case of R-N-4c can be treated similarly.

For R-I-1b, the conditions for positive definiteness are:

For R-I-4a, the conditions for positive definiteness are identical to those for R-N-4a given in (72), except that the elements $ a_{ij} $ therein are replaced by:

For R-I-4b, the conditions for positive definiteness are identical to those for R-N-4a given in (72), except that the elements $ a_{ij} $ are replaced by:

For R-I-4c, the positive-definiteness conditions are identical to those for R-N-4a given in (72), except that the elements $ a_{ij} $ therein are replaced by:

For R-I-4d, the positive-definiteness conditions are identical to those for R-N-4a given in (72), except that the elements $ a_{ij} $ therein are replaced by:

For complex vacua, positive-definiteness of the Hessian (29) requires that all seven leading principal minors be strictly positive. Because of the complexity of the structure, we can explicitly provide only the first four conditions.

For C-N-2, the requirements that its first four leading principal minors of the Hessian be positive are identical to the four positive-definiteness conditions for R-N-2 given in (40), except that the elements $ a_{ij} $ therein are replaced by:

For C-N-3a, the first four conditions for positive definiteness are identical to those for R-N-2 given in (40), but the elements $ a_{ij} $ therein are replaced by:

For C-N-3b, the first four positive definiteness conditions are identical to those for R-N-2, given in (40), except that the elements $ a_{ij} $ are replaced by:

For C-N-3c, the first four conditions for positive definiteness are identical to those for R-N-2, as given in (40), except that the elements $ a_{ij} $ are replaced by:

For C-N-3d, the positive definiteness conditions are identical to those for C-N-3c.

For C-N-4a, the first four positive definiteness conditions are identical to those for R-N-4a given in (72), except that the elements $ a_{ij} $ are replaced by:

For C-N-4b, the positive-definiteness conditions follow directly from those for C-N-4a by replacing the parameters $ (v_1, v_2, v_3, v_4) $ with $ (v_1, \pm v_2, \pm v_1, v_2) $. The case of C-N-4c can be treated similarly.

For C-N-4d, the first four positive-definiteness conditions are identical to those for R-N-2 given in (40), except that the elements $ a_{ij} $ therein are replaced by:

For C-I-4a, the first four conditions for positive definiteness are identical to those for R-N-4a given in (72), except that the elements $ a_{ij} $ therein are replaced by:

For C-I-4b, the first four conditions for positive definiteness are identical to those for R-N-4a given in (72), except that the elements $ a_{ij} $ therein are replaced by:

For C-I-4c, the first four conditions for positive definiteness are identical to those for R-N-4a given in (72), except that the elements $ a_{ij} $ are replaced by:

For C-I-4d, the first four conditions for positive definiteness are identical to those for R-N-4a given in (72), except that the elements $ a_{ij} $ are replaced by: