I made this short video and notebook to explain two classic goto activation functions. I’d highly appreciate constructive feedback!

Video , Notebook , And images (it won’t render images on GitHub)

PS – the “previous video” will be published soon…

EDIT: the code is attached for your reference:

# coding: utf-8  # ### Intro to Deep Learning: Activation Functions  # In[1]:   import numpy as np #NumPy is the fundamental package for scientific computing in Python.   # In[6]:   def Sigmoid(x):     """     Sigmoid will transform data into a probability between o and 1, based on which     a neuron will be activated (or not).     """     return 1/ (1+np.exp(-x))   # In[2]:   def Sigmoid_derivative(x):     """     merely the derivative of the sigmoid, to be used for gradient descent     """     return (x*(1-x))   # ![The Sigmoid](Sigmoid.png)  # In[3]:   def ReLU(x):     """     This is the golden standard nowadays, for computational reasons.     since computation is within the essence when considering deep learning, let's find     the best way to write this function:     https://stackoverflow.com/questions/32109319/how-to-implement-the-relu-function-in-numpy     """ #     return min(o,x) #     x * (x > 0) #     (abs(x) + x) / 2 #     x[x<0]=0 This or the following:     return np.maximum(x, 0, x)   # In[5]:   def ReLU_derivative(x,epslion): #this is not part of the video, left as an exercise for the student.     if x <=0:         return 1     else:         return epslion   # In[ ]:   #building on the computational hack we've learned, let's use this time the "fancy slicing": def better_relu_de(x, epsilon)     x[x<=0] = epsilon     x[x>0] = 1     return x   # ![The ReLU](ReLU.png)  # ![ReLU's Derivative:](ReLU_derivative.png)  # ### Resources: #  # Deriving sigmoid: https://beckernick.github.io/sigmoid-derivative-neural-network/ #