I’m working through Andrej Karpathy’s awesome introduction to neural networks and the backpropagation algorithm, and am trying to differentiate the sigmoid function:

$ $ \sigma(x) = \frac{1}{(1+e)^{-x}} $ $

My understanding is the quotient rule holds that given an equation $ y=\frac{t}{b}$ the derivative of the function should be:

$ $ y’ = \frac{t’b – tb’}{b^{2}}$ $

After applying the quotient rule to the sigmoid function, I thought the unsimplified result would be:

$ $ \frac{-e^{-x}}{(1 + e^{-x})^{2}} $ $

Because $ t’$ is 0, which I thought would yield $ 0 – tb’ $ in the numerator, or $ -e^{-x}$ . However, Karpathy’s unsimplified derivative looks like:

$ $ \frac{e^{-x}}{(1 + e^{-x})^{2}} $ $

Does anyone know why the negative sign in the numerator gets dropped? I’d be very grateful for any help others can offer with this question!