Proving the chain and product rules (technical)

The chain and product rules can be easily generated from using the approximate form of the derivatives with $\Delta$s.

Why the chain rule works

Just like most thinks in math, it's better to not just take a result as given and memorize it, but rather to figure out why it looks that way. Let's see if we can do it.

The chain rule is the rule you use when you have a function of a function — something like when $y = f(g(x))$.

The basic idea of a derivative is the way a function changes when its argument changes. If we have a function $f(x)$, if we have a small change, $\Delta x$, in the value of $x$, we expect to have a small change in the value of $f$. The ratio of those small changes (which might not be small) is defined as the derivative:

$$\frac{df}{dx} \approx \frac{\Delta f}{\Delta x}$$

In our case, since it's $g$ that's a function of $x$ let's apply Eq. (2) to that to get

$$\frac{dg}{dx} \approx \frac{\Delta g}{\Delta x}.$$

In our case, $f$ is a function of $g$, so if we apply Eq. (2) to $f(g)$ we get

$$\frac{df}{dg} \approx \frac{\Delta f}{\Delta g}$$

But what we want is $df/dx$ so

$$\frac{df}{dx} \approx \frac{\Delta f}{\Delta x} = \frac{\Delta f}{\Delta g} \frac{\Delta g}{\Delta x}$$

Letting the $\Delta$s get small, we can replace the ratio of the deltas with the derivatives:

$$\frac{df}{dx} \approx \ \frac{\Delta f}{\Delta g} \frac{\Delta g}{\Delta x} \approx \frac{df}{dg} \frac{dg}{dx}. $$

So if you forget the chain rule, you can easily figure it out by going to the delta form of the ratio and cancelling the $\Delta g$s. (Don't tell your math prof I told you to do this!)

Why the product rule works

We use the product rule iwhen we have the product of two functions: something like $f(x)g(x).$

We can figure out where the product rule comes from using deltas again. This time, let's look at how $f$ and $g$ each change when $x$ changes. Let's assume that $x$ is just a number (we're now doing math) and write

$$f(x + \Delta x) \approx f(x) + \Delta f.$$

Then using Eq. (2), we have

$$f(x + \Delta x) \approx f(x) + \frac{df}{dx}\Delta x.$$

Using Equation (15) for both $f$ and $g$ we get

$$f(x + \Delta x) g(x + \Delta x) \approx (f(x) + \frac{df}{dx}\Delta x) (g(x) + \frac{dg}{dx}\Delta x)$$

Multiplying out, we get a term in $\Delta x^2$. Since $\Delta x$ is supposed to be very small (we're working with numbers here, not measurements!), $\Delta x^2$ is even smaller and we'll ignore it. (For example, if $(\Delta x)^2$ is 10^-4, then  $(\Delta x)^2$ is 10^-8.)

The result is

$$f(x + \Delta x) g(x + \Delta x) \approx f(x)g(x) + \frac{df}{dx}g(x)\Delta x + f(x) \frac{dg}{dx}\Delta x)$$

so 

$$\Delta (fg) \approx f(x)g(x) + \frac{df}{dx}g(x)\Delta x + f(x) \frac{dg}{dx}\Delta x$$

and dividing by $\Delta x$, gives

$$\frac{d}{dx}(fg) \approx \frac{\Delta (fg)}{dx} = \frac{df}{dx}g(x) + f(x) \frac{dg}{dx} $$

the product rule as claimed. Again, it's pretty straightforward to work it out using deltas.

Joe Redish 5/27/19