The Cauchy-Schwarz inequality is a popular inequality which is used for vast number for applications. The most common form is as follows:
$latex \bigg(\sum_{i=1}^n x_iy_i \bigg)^2 \leq \bigg(\sum_{i=1}^n x_i^2\bigg) \bigg(\sum_{i=1}^n y_i^2\bigg) $ for $latex x_i,y_i \in \mathbb{R}; i=1,\ldots,n$.
This can be easily proved with the aid of Jensen inequality over the convex function ($latex -log(\cdot)$ to be specific). Moreover, above inequality can be presented in a different form, using the definition of the norm.
$latex x^Ty \leq \|x\|_2 \|y\|_2$ for any $latex x,y \in \mathbb{R}^n$.
Norm:
for any $latex x \in \mathbb{R}^n$ and $latex p \geq 1$, $latex p$-norm of $latex x$ is given by,
$latex \|x\|_p = \sqrt[p]{ \sum_{i=1}^n |x_i|^p }$,
where $latex |u| = \{u \mbox{ for } u\geq 0, -u \mbox{ for } u< 0\}$.
However, above inequality is generalized for any norm
as follows:
$latex x^Ty \leq \|x\|\|y\|_{\star}$ for all $latex x,y \in \mathbb{R}^n$.
where $latex \|\cdot\|_{\star}$ is the dual norm.
Dual norm:
The dual norm of the norm $latex \|\cdot\|$ is,
$latex \|x\|_{\star} = \{\max(x^Ty) | \|y\| \leq 1\}$
.
Let us prove the generalized inequality using the definition of the dual norm.
$latex \|y\|_{\star} = \{\max(y^Tz) | \|z\| \leq 1\} \implies y^Tz \leq \|y\|_{\star}, \|z\| \leq 1$
$latex \implies y^Tz^\prime \leq \|y\|_{\star}, \|z^\prime\| = 1$
$latex \implies y^T\frac{x}{\|x\|} \leq \|y\|_{\star}$ since $latex \|z^\prime\|=\bigg\|\frac{x}{\|x\|}\bigg\|=\frac{\|x\|}{\|x\|}=1$
$latex \implies y^Tx \leq \|x\|\|y\|_{\star}$ since $latex \|x\| \geq 0$.
$latex \implies x^Ty \leq \|x\|\|y\|_{\star}$
pretty nice, isn't it?
Still we need to match the generalized version to the commonly used version. For that, let us try to find an expression for the dual norm of the $latex p$-norm. Let $latex \|\cdot\|_q$ be the dual of $latex \|\cdot\|_p$. Using the generalized inequality and the definition of the dual norm, we need to solve a maximization problem for $latex x\succeq 0$ as follows.
$latex x^Ty \leq \|x\|_p\|y\|_q \implies \|y\|_q = \max_x\bigg(\frac{y^Tx}{\|x\|_p}\bigg)$
$latex \|y\|_q = \max_x\bigg(\frac{\sum x_iy_i}{\sqrt[p]{\sum x_i^p}}\bigg) =\frac{\sum u_iy_i}{\sqrt[p]{\sum u_i^p}}$
where
$latex \nabla_x \bigg(\frac{\sum x_iy_i}{\sqrt[p]{\sum x_i^p}}\bigg)\bigg|_{x=u}=0$
$latex \implies \sqrt[p]{\sum u_i^p}y_i = (\sum u_iy_i)(\sum u_i^p)^{1/p-1}u_i^{p-1}$ for all $latex i = 1,\ldots,n$
$latex \implies \frac{y_i}{u_i^{p-1}} = \frac{\sum u_iy_i}{\sum u_i^p} = \frac{1}{\lambda}$ for all $latex i = 1,\ldots,n$
$latex \implies u_i = \sqrt[p-1]{ \lambda y_i }$ for all $latex i = 1,\ldots,n$ with $latex \lambda = \frac{\sum u_i^p}{\sum u_iy_i}$.
Substituting the result on the previous expression,
$latex \|y\|_q = \frac{\sum u_iy_i}{\sqrt[p]{\sum u_i^p}} = \Big({\sum u_iy_i}\Big)^{(1-1/p)}\frac{\sqrt[p]{\sum u_iy_i}}{\sqrt[p]{\sum u_i^p}}$
$latex \|y\|_q = \frac{\sqrt[(1-1/p)]{\sum u_iy_i}}{\sqrt[p]{\lambda}} = \sqrt[(1-1/p)]{\sum \frac{u_iy_i}{ \sqrt[(p-1)]{\lambda} } } = \sqrt[(p-1)/p]{ \sum y_i^{p/(p-1)} }$.
This proves that the dual norm of $latex \|\cdot\|_p$ is $latex \|\cdot\|_q$ with
$latex \frac{1}{p}+\frac{1}{q}=1$.
When $latex p=2 \implies q=2$. This simplifies the generalized inequality to the common expression.
