14. Secure and Private AI : Part 1 - Tensors in PyTorch-Exercises¶
[1]:
import torch
[2]:
def activation(x):
""" Sigmoid activation function
Arguments
---------
x: torch.Tensor
"""
return 1/(1+torch.exp(-x))
[3]:
### Generate some data
torch.manual_seed(7) # Set the random seed so things are predictable
# Features are 3 random normal variables
features = torch.randn((1, 5))
# True weights for our data, random normal variables again
weights = torch.randn_like(features)
# and a true bias term
bias = torch.randn((1, 1))
[4]:
print(features)
tensor([[-0.1468, 0.7861, 0.9468, -1.1143, 1.6908]])
[5]:
print(weights)
tensor([[-0.8948, -0.3556, 1.2324, 0.1382, -1.6822]])
[6]:
print(bias)
tensor([[0.3177]])
14.1. Exercise:¶
Calculate the output of the network with input features features, weights weights, and bias bias. Similar to Numpy, PyTorch has a torch.sum() function, as well as a .sum() method on tensors, for taking sums. Use the function activation defined above as the activation function.
[7]:
y = activation(torch.sum(features * weights) + bias)
[8]:
print (y)
tensor([[0.1595]])
[9]:
#does not work because the size of features and weights aren't allowing the multiplication
# uncomment to test it torch.mm(features, weights)
[10]:
features.shape
[10]:
torch.Size([1, 5])
[11]:
weights.shape
[11]:
torch.Size([1, 5])
[12]:
bias.shape
[12]:
torch.Size([1, 1])
14.2. Exercise:¶
Calculate the output of our little network using matrix multiplication. Reshape weights to have five rows and one column with something like weights.view(5, 1).
[13]:
activation(torch.mm(features, weights.view(5,1))+bias)
[13]:
tensor([[0.1595]])
[14]:
features.shape
[14]:
torch.Size([1, 5])
[15]:
### Generate some data
torch.manual_seed(7) # Set the random seed so things are predictable
# Features are 3 random normal variables
features = torch.randn((1, 3))
# Define the size of each layer in our network
n_input = features.shape[1] # Number of input units, must match number of input features
n_hidden = 2 # Number of hidden units
n_output = 1 # Number of output units
# Weights for inputs to hidden layer
W1 = torch.randn(n_input, n_hidden)
# Weights for hidden layer to output layer
W2 = torch.randn(n_hidden, n_output)
# and bias terms for hidden and output layers
B1 = torch.randn((1, n_hidden))
B2 = torch.randn((1, n_output))
print(W1)
print(W2)
print(B1)
print(B2)
tensor([[-1.1143, 1.6908],
[-0.8948, -0.3556],
[ 1.2324, 0.1382]])
tensor([[-1.6822],
[ 0.3177]])
tensor([[0.1328, 0.1373]])
tensor([[0.2405]])
14.3. Exercise:¶
Calculate the output for this multi-layer network using the weights W1 & W2, and the biases, B1 & B2.
[16]:
h = activation(torch.mm(features, W1) + B1)
output = activation(torch.mm(h, W2) + B2)
print(output)
tensor([[0.3171]])
[17]:
import numpy as np
a = np.random.rand(4,3)
a
[17]:
array([[0.98843161, 0.91529327, 0.87711017],
[0.03036671, 0.87467354, 0.38863948],
[0.15188762, 0.30653778, 0.87092012],
[0.65136106, 0.59484865, 0.13040916]])
[18]:
# To create a tensor from a Numpy array, use torch.from_numpy()
b = torch.from_numpy(a)
b
[18]:
tensor([[0.9884, 0.9153, 0.8771],
[0.0304, 0.8747, 0.3886],
[0.1519, 0.3065, 0.8709],
[0.6514, 0.5948, 0.1304]], dtype=torch.float64)
[19]:
# To convert a tensor to a Numpy array, use the .numpy() method
b.numpy()
[19]:
array([[0.98843161, 0.91529327, 0.87711017],
[0.03036671, 0.87467354, 0.38863948],
[0.15188762, 0.30653778, 0.87092012],
[0.65136106, 0.59484865, 0.13040916]])
[20]:
# Multiply PyTorch Tensor by 2, in place
b.mul_(2)
[20]:
tensor([[1.9769, 1.8306, 1.7542],
[0.0607, 1.7493, 0.7773],
[0.3038, 0.6131, 1.7418],
[1.3027, 1.1897, 0.2608]], dtype=torch.float64)
[21]:
# Numpy array matches new values from Tensor
a
[21]:
array([[1.97686322, 1.83058654, 1.75422035],
[0.06073342, 1.74934708, 0.77727897],
[0.30377524, 0.61307556, 1.74184025],
[1.30272212, 1.18969731, 0.26081832]])