Mathematics Question from JEE Main 2023 · 30 January, Shift 2

Let $\lambda \in \mathbb{R}$, $\mathbf{a} = \lambda i + 2j - 3k$, $\mathbf{b} = i - \lambda j + 2k$. If $\left( \left(\mathbf{a} + \mathbf{b}\right) \times \left(\mathbf{a} \times \mathbf{b}\right) \right) \times \left(\mathbf{a} - \mathbf{b}\right) = 8i - 40j - 24k$, then $|\lambda \left(\mathbf{a} + \mathbf{b}\right) \times \left(\mathbf{a} - \mathbf{b}\right)|^2$ is equal to ......