Cauchy–Schwarz inequality for Norm and Dual Norm:

The Cauchy-Schwarz inequality is a popular inequality which is used for vast number for applications. The most common form is as follows:

$latex \bigg(\sum_{i=1}^n x_iy_i \bigg)^2 \leq \bigg(\sum_{i=1}^n x_i^2\bigg) \bigg(\sum_{i=1}^n y_i^2\bigg) $  for $latex x_i,y_i \in \mathbb{R}; i=1,\ldots,n$.

This can be easily proved with the aid of Jensen inequality over the convex function ($latex -log(\cdot)$ to be specific). Moreover, above inequality can be presented in a different form, using the definition of the norm.

$latex x^Ty \leq \|x\|_2 \|y\|_2$  for any $latex x,y \in \mathbb{R}^n$.

Norm:

for any $latex x \in \mathbb{R}^n$ and $latex p \geq 1$, $latex p$-norm of $latex x$ is given by,

$latex \|x\|_p = \sqrt[p]{ \sum_{i=1}^n |x_i|^p }$,

where $latex |u| = \{u \mbox{ for } u\geq 0, -u \mbox{ for } u< 0\}$.

However, above inequality is generalized for any norm as follows:

$latex x^Ty \leq \|x\|\|y\|_{\star}$  for all  $latex x,y \in \mathbb{R}^n$.

where $latex \|\cdot\|_{\star}$ is the dual norm.

Dual norm:

The dual norm of the norm $latex \|\cdot\|$ is,

$latex \|x\|_{\star} = \{\max(x^Ty) | \|y\| \leq 1\}$ .

 Let us prove the generalized inequality using the definition of the dual norm.

          $latex \|y\|_{\star} = \{\max(y^Tz) | \|z\| \leq 1\} \implies y^Tz \leq \|y\|_{\star}, \|z\| \leq 1$

          $latex \implies y^Tz^\prime \leq \|y\|_{\star}, \|z^\prime\| = 1$

          $latex \implies y^T\frac{x}{\|x\|} \leq \|y\|_{\star}$  since $latex \|z^\prime\|=\bigg\|\frac{x}{\|x\|}\bigg\|=\frac{\|x\|}{\|x\|}=1$

          $latex \implies y^Tx \leq \|x\|\|y\|_{\star}$  since $latex \|x\| \geq 0$.

          $latex \implies x^Ty \leq \|x\|\|y\|_{\star}$

pretty nice, isn't it?

Still we need to match the generalized version to the commonly used version. For that, let us try to find an expression for the dual norm of the $latex p$-norm. Let $latex \|\cdot\|_q$ be the dual of $latex \|\cdot\|_p$. Using the generalized inequality and the definition of the dual norm, we need to solve a maximization problem for $latex x\succeq 0$ as follows.

$latex x^Ty \leq \|x\|_p\|y\|_q \implies \|y\|_q = \max_x\bigg(\frac{y^Tx}{\|x\|_p}\bigg)$

$latex \|y\|_q = \max_x\bigg(\frac{\sum x_iy_i}{\sqrt[p]{\sum x_i^p}}\bigg) =\frac{\sum u_iy_i}{\sqrt[p]{\sum u_i^p}}$

where

          $latex \nabla_x \bigg(\frac{\sum x_iy_i}{\sqrt[p]{\sum x_i^p}}\bigg)\bigg|_{x=u}=0$

          $latex \implies \sqrt[p]{\sum u_i^p}y_i = (\sum u_iy_i)(\sum u_i^p)^{1/p-1}u_i^{p-1}$  for all $latex i = 1,\ldots,n$

          $latex \implies \frac{y_i}{u_i^{p-1}} = \frac{\sum u_iy_i}{\sum u_i^p} = \frac{1}{\lambda}$  for all $latex i = 1,\ldots,n$

          $latex \implies u_i = \sqrt[p-1]{ \lambda y_i }$  for all $latex i = 1,\ldots,n$  with $latex \lambda = \frac{\sum u_i^p}{\sum u_iy_i}$.

 Substituting the  result on the previous expression,

          $latex \|y\|_q = \frac{\sum u_iy_i}{\sqrt[p]{\sum u_i^p}} = \Big({\sum u_iy_i}\Big)^{(1-1/p)}\frac{\sqrt[p]{\sum u_iy_i}}{\sqrt[p]{\sum u_i^p}}$

          $latex \|y\|_q = \frac{\sqrt[(1-1/p)]{\sum u_iy_i}}{\sqrt[p]{\lambda}} = \sqrt[(1-1/p)]{\sum \frac{u_iy_i}{ \sqrt[(p-1)]{\lambda} } } = \sqrt[(p-1)/p]{ \sum y_i^{p/(p-1)} }$.

This proves that the dual norm of $latex \|\cdot\|_p$  is $latex \|\cdot\|_q$ with 

$latex \frac{1}{p}+\frac{1}{q}=1$.

When $latex p=2 \implies q=2$. This simplifies the generalized inequality to the common expression.

Freezing the action: Splashes and Flows

The captures made using high speed photography techniques, which are used to freeze the time, are truely amazing and inspiering. The spilt of a second is hard to experience from the naked-eye (actually it might not that hard to see nice frozen moments in a disco party). Therefore, most photographers  keen to capture the exact moment, exact action which can be admired by almost all the people. There is a vast collection of such photos and tutorials on above technique is available in the internet. Inspired by their work, I decided to work on mine and share my experience. May be you have known this before, or even mastered it. Yet I just want to share this with you.

The equipment I used are,

• Nikon D3100 camera.
• Tamron AF 18-270mm f/3.5-6.3 Di II VC LD Aspherical (IF) MACRO lens.
• Nissin Di622 Mark II speedlight.
• 30 cm $latex \times$ 30 cm softbox made by myself.

I know they are not the best, but they are more than enough for get the job done. At the moment I am using D3100 built-in popup flash to trigger the off-camera Nissin speedlight.

Splashes:
The setup is illustrated in the  figure below.

• Use a colored paper or wall as the background. It will add a nice flavor to the water.
• The glass is filled with water and I used to drop grapes to make the splashes. Water will be spilling all around. Therefore, make sure to use this setup in a proper place.
• Take the speedlight with the softbox close to the glass as close as possible. Moreover, make sure to cover them with a water proof transparent sheet. You do not want mess them with the water.
• Set the camera with a safe distance which avoid the spilling water.
• For camera settings, I used Manual mode with,
• Shutter speed: flash sync speed (1/200 for D3100). If you have high speed sync capability, you may try it. I have no experience with it.
• Apeture: make it small (f-8 to f-11). Henceforth, you can make sure that everything is focused.
• ISO: some lower values (100-200).
• focus: auto focusing at the begining. Then switched to manul in order to avoid refocusing issues under low light.
• Try to use a remote trigger to operate the camera. It will avoid shaking the camera. If you do not have a remote trigger or it just got skrewed like mine, you can use the self timer.

Drop the objects. Shoot them several times. Results will be rewarding :)

Flows:
Here, we need a nice white background. Therefore, I am using my speedlight to light the background (wall). From the camera settings, I overexpose the background and make it looks like a white (almost) back drop. The setup is as follows:

Side view

Top view

The rest of the steps are quite similar to 'Splashes' scenario.

• Camera setting follows the same. However, the aperture needs to be adjusted such that the background is overexposed and produces bright white pane.
• I used colored water to have a contrast between the flow and the background.
• Start pouring the water, and shoot them.
• Finally, grab the glass from the top and lift it up, take one photo of its base. This is used to add the base during the post-processing.

Water drops:

This is captured about a year ago when I do not have the speedlight. I used camera's built-in pop-up flash as the light source. It lights up the background and, using a black cover I cut out the direct light on the water container. It ensured that the water drops are illuminated by the reflected light from the wall which avoids unnecessary bright reflections.

The rest of my shots can be found here. Just want to say that "practise makes perfect".

Courtesy of: SnapKnot

Some EPS figures for communications diagrams.

TeX is a type setting system which is widely used among the scientific writers. The capability of producing complex mathematical expressions accurately is the main reason for such popularity. Although working on TeX environment is not comfortable as working on well-known word processors coming from MS-Office, iWork, Open-Office, etc., once you get used to it, it becomes more easy to work around with it. Moreover, the ability of producing high-quality outputs, especially vector based documents, will be able to hold you with the TeX environment (for most cases).

One major issue of working with TeX is the need of eps or pdf type figures whenever you need to produce loss-less documents. There is number of applications which can be used to produce the vector graphics, but not everybody going to happy with working on them. On the other hand, applications such as Visio and Smart Draw are not only providing vast collection of arts for diagrams and flow charts, but also provide the facility to produce impressive diagrams even for novices. Thus, such applications are very popular among everybody.

However, the main limitation of such applications is to convert such diagrams to TeX compatible formats without losing their qualities. I have tried working on both Visio and Smart Draw to produce high resolution figures and diagrams, yet once the file conversion takes place (save as pdf, save as eps, jpg/png to eps), diagrams looks (slightly) blurred in the final TeX output. Another disadvantage is when creating flow diagrams; the equations written in Visio/Smart Draw are not neat as the rest of the equations in the TeX.

Therefore, I moved to Inkscape and Ipe for all kind of drawings which are required for my work. It was/is a pain working on such environments, yet the outputs are so blessed – and I am happy about the struggle with Ipe and Inkscape. A flow chart containing equations accurately and a diagram with loss-less vector format are shown below. Ipe needs additional adjustments in order to obtain the fonts and symbols according to the IEEE format. I did include that tip here.

      

- Flow diagram with equations -

      

- Vector graphic based figure -

I was managed to reproduce few of shapes related to communications in Ipe. The original shapes are coming from Visio. It was quite a time consuming process to produce them all + I was not familiar with all those tools. However, I manage to produce them with a certain level of satisfaction, and I thought to share them with you.

The EPS figures of above shape can be found here. The Ipe drawing project can be downloaded from here.

I will add more shapes whenever I produce.

Nice fonts for text and equations in IPE.

Ipe uses the TeX interpreter. Therefore, it is a nice solution for producing vector based graphics along the equations with accurate format. However, the default formatting may not be the best. Especially for the IEEE based documents, it would be more convenient to use the standard fonts for both text and equations.

In order to obtain such nice format, you just need to follow few more steps.

1. In Ipe, access the document properties via ‘Edit’ menu. Alternately, you can use the key combo: ctrl + shift + P (for windows) or cmd + shift + P (for mac).
2. In Ipe document properties dialog window, add following lines under the section named ‘Latex preamble’.
• \usepackage{amsmath}
• \usepackage[T1]{fontenc}
• \usepackage{times}
• \usepackage{dsfont}
3. Click Ok and compile your document (ctrl + L).

There you go. The sweet IEEE format.

Let's find Euler's number - e

The Euler’s number is another mathematical constant which is an irrational number similar to $latex \pi$. In addition, this is the base of the natural logarithm. Few of the interesting things related to this number are as follows;

• $latex \displaystyle e=\lim_{n\rightarrow\infty} \bigg(1+\frac{1}{n}\bigg)^n$.
• $latex \displaystyle e = \sum_{n=0}^\infty {\frac{1}{n!}}$.
• When $latex f(x)=e^x$, the first derivation of the function (gradient of $latex f$ at point $latex x$) is $latex f^1(x)=e^x$. Moreover, for any integer $latex n$ (positive for differentiation and negative for integration neglecting the arbitrary constant), the n-th derivation is $latex f^n(x)=e^x$.
• As $latex x\rightarrow\infty$, $latex e^x\rightarrow\infty$ and as $latex x\rightarrow-\infty$, $latex e^x\rightarrow0$. And $latex e^0=1$ which is an obvious fact. The function $latex e^{-x}$ is the reflection of $latex e^x$ around the y-axis.

Today we are going to use a simple technique to estimate the value of ‘$latex e$’ (You probably can search the value from the internet easily). We are going to use the plots in the figure below.

We have $latex y=e^{-x}$ curve and it is bounded by two step functions. In the step functions, the width of each column (step size) is $latex \frac{1}{n}$, i.e. the unit length of x-axis is divided in to '$latex n$' segments. 

Let's look at the two overlapped columns starting from point '$latex x$'.

• Height of the tall column is $latex e^{-x}$.
• Height of the short column is $latex e^{-(x+\frac{1}{n})}$.
• Width of both columns are $latex \frac{1}{n}$.

Therefore, the surface area of the tall column is $latex \frac{1}{n}\times{e^{-x}}$ and for the short column it is $latex \frac{1}{n}\times{e^{-(x+1/n)}}=\frac{e^{-1/n}}{n}\times{e^{-x}}$.

With the knowledge of calculus, we can say that the area under the curve '$latex y$' within the width of those columns is $latex \int_x^{(x+1/n)}e^{-x}dx$. 

We can observe that the area under the curve is larger than the area of short column, but smaller than the area of the tall column. Thus,

$latex \frac{e^{-1/n}}{n}{e^{-x}}<\int_x^{(x+1/n)}e^{-x}dx<\frac{1}{n}{e^{-x}}$

Now, let us add together all the areas starting from $latex x=1$ to the infinity by the step size of $latex \Delta{x}={1}/{n}$,

$latex \displaystyle \sum_{x=1,\Delta{x}=\frac{1}{n}}^{\infty}\frac{e^{-1/n}}{n}{e^{-x}}<\displaystyle\sum_{x=1,\Delta{x}=\frac{1}{n}}^{\infty}\int_x^{(x+1/n)}e^{-x}dx <\displaystyle\sum_{x=1,\Delta{x}=\frac{1}{n}}^{\infty}\frac{1}{n}{e^{-x}}$

$latex \frac{e^{-1/n}}{n}\displaystyle\sum_{x=1,\Delta{x}=\frac{1}{n}}^{\infty}{e^{-x}}<\int_1^{\infty}e^{-x}dx<\frac{1}{n}\displaystyle\sum_{x=1,\Delta{x}=\frac{1}{n}}^{\infty}{e^{-x}}$

If you look at the infinite summations, you may notice that both of them are geometric progressions. Right series has $latex \frac{e^{-1}}{n}$ as the first term and for the left series it is $latex \frac{e^{-1}e^{-1/n}}{n}$ and, both of them have $latex e^{-1/n}$ as the common ratio (it is less than 1 which we take by the fact of that '$latex e$' is larger than 1).

Thus, by the knowledge of infinite sum of geometric series and integration, we can re-write above inequalities as below.

$latex \frac{(e^{-1}e^{-1/n})/n}{1-e^{-1/n}}<e^{-1}<\frac{(e^{-1})/n}{1-e^{-1/n}}$

$latex \frac{e^{-1/n}}{n}<1-e^{-1/n}<\frac{1}{n}$

$latex \frac{1}{n}<e^{1/n}-1<\frac{e^{1/n}}{n}$

By considering left-middle terms and middle-right terms as separate inequalities, we can simplify above furthermore and later combine it as follows;

$latex \frac{n+1}{n}<e^{1/n}<\frac{n}{n-1}$

$latex (\frac{n+1}{n})^n<e<(\frac{n}{n-1})^n$

There it is. By using any positive integer for '$latex n$', we can estimate the value of '$latex e$'.

Examples: 

1. Let $latex n=8$. Then,

$latex (\frac{8+1}{8})^4<e<(\frac{8}{8-1})^8$

$latex 2.56578<e<2.91028$

However, this approach converges very slowly. Let's say that we need to find the exact answer for $latex k$ decimal points. That implies $latex (\frac{n}{n-1})^n-(\frac{n+1}{n})^n<10^{-k}$ need to be satisfied. Now we are going to simplify this.

$latex (\frac{n}{n-1})^n-(\frac{n+1}{n})^n<10^{-k}$

$latex (\frac{n^2}{n^2-1})^n-1<(\frac{n}{n+1})^n10^{-k}$

We know that as '$latex n$' increases the lower bound increases. Therefore, the reciprocal decreases. From above example we can guess that the lower bound never reaches to 3 which means the reciprocal never drops below 1/3. So above inequality is simplified with that assumption ( $latex \forall{n\in\mathbb{Z}^+},(\frac{n}{n+1})^n<\frac{1}{3}$ ).

Therefore if we can find an '$latex n$' which satisfy $latex (\frac{n^2}{n^2-1})^n-1<\frac{1}{3}10^{-k}$ for given '$latex k$', using that '$latex n$' we can estimate the value of '$latex e$' for '$latex k$' decimal points correctly.

Examples: 

1. Let $latex n=100$. Then $latex 3[(\frac{n^2}{n^2-1})^n-1]=0.03015<10^{-1}$. Therefore, $latex (\frac{100+1}{100})^{100}=2.7048$ gives the estimation for one decimal point, which is $latex e\simeq2.7$.
2. Let $latex n=1000$. Then $latex 3[(\frac{n^2}{n^2-1})^n-1]=0.0030015<10^{-2}$. Therefore, $latex (\frac{100+1}{100})^{100}=2.7169$ gives the estimation for two decimal point, which is $latex e\simeq2.71$.
3. Let $latex n=10000$. Then $latex 3[(\frac{n^2}{n^2-1})^n-1]=0.000300015<10^{-3}$. Therefore, $latex (\frac{1000+1}{1000})^{1000}=2.71814$ gives the estimation for three decimal point, which is $latex e\simeq2.718$.

Hope you got the idea. And have fun with calculating to large number of decimal points.

From wikipedia I got it for 50 decimal points, which is

e = 2.71828182845904523536028747135266249775724709369995.

The infinite sum of 1/xⁿ

”Convergence of series” is a quite interesting topic during high school mathematics. A popular such series is $latex \sum_{x\in\mathbb{Z}^+}(\frac{1}{x^n})$ where $latex n$ is a positive integer. When $latex n>1$, there were lots of examples to determine the convergence and finding the sum up to general $latex N$ terms.

However, it is not true for $latex n=1$ and I always wondered how it happens. Therefore, I thought to share the proof with a simple graphical approach as follows.

Let’s look at the figure.

We have two curves: $latex y_1=\frac{1}{x^n}$ and $latex y_2=\frac{1}{(x+1)^n}$. In between them, I drew a bar-graph and you may notice that the height of each column is equivalent to $latex y=\frac{1}{{\lfloor x\rfloor}^n}$ where $latex \lfloor x\rfloor$ is the closest lower integer of $latex x$.
( Eg: $latex \lfloor 2.5 \rfloor = \lfloor 2 \rfloor = 2$ )

Now, if we try to calculate the area of $latex k$-th column, it will be $latex \frac{1}{k^n}\times 1=\frac{1}{k^n}\$. Therefore, what will be the sum of area of all columns? Yes, it is our nice buddy series, $latex \sum_{x=1}^{\infty}(\frac{1}{x^n})$. Pretty interesting, isn’t it?

Now look at the area under each curve for positive $latex x$ values.

We can see that each column is popping out a little bit from the curve $latex y_2$. Therefore, we can say that the area under curve $latex y_2$ is less than the sum area of columns.

And it is the opposite for curve $latex y_1$. Every column is underneath the curve and thus its area is larger than the sum area of columns. We know that $latex y_1$ tends to infinity when it is close to 0. Thus, we modify our argument like this. For $latex x>1$, we can say that the area under $latex y_1$ is larger than the sum area of columns excluding the first column. The area of the 1^st column is 1. Then, we can say the sum area of columns is less than the area under the curve $latex y_1$ for $latex x>1$ plus 1.

(Area under $latex y_2$ for $latex x>0$ ) < $latex \sum_{x=1}^{\infty}(\frac{1}{x^n})$ < (1 + Area under $latex y_1$ for $latex x>1$ )

The guys who play with calculus know that the area under a given curve $latex f(x)$ between the interval $latex [a,b]$ is $latex \int_a^b|f(x)|dx$. Therefore, we can make above relation in a mathematical form as following; 

$latex \int_0^\infty\frac{1}{(x+1)^n}dx < \sum_{x>0}(\frac{1}{x^n}) <1 + \int_1^\infty\frac{1}{x^n}dx$,

where $latex x \in \mathbb{Z}^+$.

Let’s solve above relations.

• For $latex n=1$,

$latex \int_0^\infty\frac{1}{(x+1)}dx < \sum_{x>0}(\frac{1}{x}) < 1 +\int_1^\infty\frac{1}{x}dx$

$latex \ln(x+1)|_0^\infty < \sum_{x>0}(\frac{1}{x}) <1 + \ln(x)|_1^\infty$

$latex \infty < \sum_{x>0}(\frac{1}{x}) < \infty$

Yeah, $latex \sum_{x>0}(\frac{1}{x})$ is not going to converge.

• For $latex n > 1$,

$latex \int_0^\infty\frac{1}{(x+1)^n}dx < \sum_{x>0}(\frac{1}{x^n}) <1 + \int_1^\infty\frac{1}{x^n}dx$

$latex \frac{1}{(1-n)(x+1)^{n-1}}|_0^\infty < \sum_{x>0}(\frac{1}{x^n}) < 1 + \frac{1}{(1-n)x^{n-1}}|_1^\infty$
 $latex \frac{1}{(n-1)} < \sum_{x > 0}(\frac{1}{x^n}) < \frac{n}{(n-1)} $

As $latex n\rightarrow\infty$,

$latex 0 < \sum_{x > 0}(\frac{1}{x^n}) < 1 $

Series converges.

Furthermore, for $latex n < 1$ and $latex \forall x \in \mathbb{Z}^+$, 

$latex \frac{1}{x} <\frac{1}{x^n} $

$latex \sum_{x>0}(\frac{1}{x} )<\sum_{x>0}(\frac{1}{x^n})$

          $latex \infty < \sum_{x>0}(\frac{1}{x^n})$

Therefore, we can conclude that,

• For $latex n > 1$ ; $latex \sum_{x=1}^{\infty}(\frac{1}{x^n})$ converges.
• For $latex n \le 1$ ; $latex \sum_{x=1}^{\infty}(\frac{1}{x^n})$ diverges.