Search for the Standard Model Higgs boson pair production and new physics phenomena in the $b\bar{b} \tau_{had}^{+}\tau_{had}^{-}$ final state with the ATLAS detector

A search for the Standard Model (SM) Higgs boson pair production and new physics phenomena in the $b\bar{b}\tau_{had}^+ \tau_{had}^-$ final state is presented. The search uses 36.1 ${fb^{-1}}$ of $pp$ collisions data at a centre-of-mass energy of 13 TeV provided by the Large Hadron Collider (LHC) and recorded by the ATLAS experiment in 2015 and 2016. The observed (expected) 95% confidence level (CL) upper limit on the non-resonant di-Higgs production cross section is 16.4 (17.4) times the predicted SM cross section. The ratios of the Higgs boson self-coupling and of the top-quark Yukawa coupling to their SM expectations ($k_{\lambda}=\lambda_{HHH}/\lambda_{HHH}^{SM}$ and $k_t=y_t/y_t^{SM}$) are constrained at 95\% confidence level respectively to $-9<k_\lambda<17$ and $-1.7<k_t<2$. The results of the $bb\tau_{had}\tau_{had}$ channel are statistically combined with the results of the $bb\tau_{lep}\tau_{had}$ channel and then further combined with the results of the $bbbb$ and $bb\gamma\gamma$ di-Higgs decay channels. The combined observed (expected) limit on the non-resonant di-Higgs production cross section corresponds to 6.7 (10.4) times the SM expectation and the triple Higgs self-coupling is constrained to $-5<k_\lambda<12.1$. Exclusion limits are also set on resonant Higgs boson pair production, probing a model with an extended Higgs sector based on two doublets, and a Randall-Sundrum bulk graviton model. In addition, a search for pair production of third generation scalar leptoquarks is performed in the $b\tau_{had}b\tau_{had}$ channel and combined with the $b\tau_{lep}\tau_{had}$ channel, setting limits on the leptoquark mass.