You need to sign in to do that
Don't have an account?
LABBOO
What is wrong with my Code for Exercise 6 in the Deep Language and Natural Processing Module?
I'm really stuck on this one.
For the first part I have:
torch.manual_seed(123)
# TODO: Generate 2 clusters of 100 2d vectors, each one distributed normally, using
# only two calls of randn()
classApoints = torch.randn(100,2)
classBpoints = torch.randn(100,2)
#println(classApoints)
# TODO: Add the vector [1.0,3.0] to the first cluster and [3.0,1.0] to the second.
classApoints += torch.tensor([1.0,3.0])
classBpoints += torch.tensor([3.0,1.0])
#println(classApoints)
# TODO: Concatenate these two clusters along dimension 0 so that the points
# distributed around [1.0, 3.0] all come first
inputs = torch.cat([classApoints, classBpoints],0)
#println(inputs) - I suspect U might be missing something in here but I'm not certain
# TODO: Create a tensor of target values, 0 for points for the first cluster and
# 1 for the points in the second cluster. Make sure that these are LongTensors.
classA = classApoints.zero_().long()
classB = classBpoints.fill_(1).long()
targets = torch.cat([classA, classB])
#println(targets.type()) - pretty sure this is correct and I've confirmed they are LongTensors
For the 2nd part (where I'm having error) I've got:
# TODO: Set the random seed to 123 using manual_seed
torch.manual_seed(123)
# TODO: Initialize a Linear layer to output scores
# for each class given the 2d examples
model = nn.Linear(2, 2)
# TODO: Define your loss function
loss_fn = nn.NLLLoss()
# TODO: Initialize an SGD optimizer with learning rate 0.1
optimizer = torch.optim.SGD(model.parameters(), lr=0.1)
# Train the model for 100 epochs
n_epochs = 1
losses = []
for _ in range(n_epochs):
optimizer.zero_grad()
preds = model(inputs)
#println(preds)
#println(targets)
loss = loss_fn(preds, targets)
losses.append(loss)
loss.backward()
optimizer.step()
print(f'Anwswer to Exercise 6: Loss after {n_epochs} epochs: {losses[-1]}')
iterations = np.arange(len(losses))
_ = plt.plot(iterations, losses, '', iterations, losses, '-')
The error I'm getting:
--------------------------------------------------------------------------- RuntimeError Traceback (most recent call last) <ipython-input-65-b59a439a8791> in <module>() 20 #println(preds) 21 #println(targets) ---> 22 loss = loss_fn(preds, targets) 23 losses.append(loss) 24 loss.backward() /usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs) 489 result = self._slow_forward(*input, **kwargs) 490 else: --> 491 result = self.forward(*input, **kwargs) 492 for hook in self._forward_hooks.values(): 493 hook_result = hook(self, input, result) /usr/local/lib/python3.6/dist-packages/torch/nn/modules/loss.py in forward(self, input, target) 191 _assert_no_grad(target) 192 return F.nll_loss(input, target, self.weight, self.size_average, --> 193 self.ignore_index, self.reduce) 194 195 /usr/local/lib/python3.6/dist-packages/torch/nn/functional.py in nll_loss(input, target, weight, size_average, ignore_index, reduce) 1330 .format(input.size(0), target.size(0))) 1331 if dim == 2: -> 1332 return torch._C._nn.nll_loss(input, target, weight, size_average, ignore_index, reduce) 1333 elif dim == 4: 1334 return torch._C._nn.nll_loss2d(input, target, weight, size_average, ignore_index, reduce) RuntimeError: multi-target not supported at /pytorch/aten/src/THNN/generic/ClassNLLCriterion.c:22
Can anyone assist on this?
For the first part I have:
torch.manual_seed(123)
# TODO: Generate 2 clusters of 100 2d vectors, each one distributed normally, using
# only two calls of randn()
classApoints = torch.randn(100,2)
classBpoints = torch.randn(100,2)
#println(classApoints)
# TODO: Add the vector [1.0,3.0] to the first cluster and [3.0,1.0] to the second.
classApoints += torch.tensor([1.0,3.0])
classBpoints += torch.tensor([3.0,1.0])
#println(classApoints)
# TODO: Concatenate these two clusters along dimension 0 so that the points
# distributed around [1.0, 3.0] all come first
inputs = torch.cat([classApoints, classBpoints],0)
#println(inputs) - I suspect U might be missing something in here but I'm not certain
# TODO: Create a tensor of target values, 0 for points for the first cluster and
# 1 for the points in the second cluster. Make sure that these are LongTensors.
classA = classApoints.zero_().long()
classB = classBpoints.fill_(1).long()
targets = torch.cat([classA, classB])
#println(targets.type()) - pretty sure this is correct and I've confirmed they are LongTensors
For the 2nd part (where I'm having error) I've got:
# TODO: Set the random seed to 123 using manual_seed
torch.manual_seed(123)
# TODO: Initialize a Linear layer to output scores
# for each class given the 2d examples
model = nn.Linear(2, 2)
# TODO: Define your loss function
loss_fn = nn.NLLLoss()
# TODO: Initialize an SGD optimizer with learning rate 0.1
optimizer = torch.optim.SGD(model.parameters(), lr=0.1)
# Train the model for 100 epochs
n_epochs = 1
losses = []
for _ in range(n_epochs):
optimizer.zero_grad()
preds = model(inputs)
#println(preds)
#println(targets)
loss = loss_fn(preds, targets)
losses.append(loss)
loss.backward()
optimizer.step()
print(f'Anwswer to Exercise 6: Loss after {n_epochs} epochs: {losses[-1]}')
iterations = np.arange(len(losses))
_ = plt.plot(iterations, losses, '', iterations, losses, '-')
The error I'm getting:
--------------------------------------------------------------------------- RuntimeError Traceback (most recent call last) <ipython-input-65-b59a439a8791> in <module>() 20 #println(preds) 21 #println(targets) ---> 22 loss = loss_fn(preds, targets) 23 losses.append(loss) 24 loss.backward() /usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs) 489 result = self._slow_forward(*input, **kwargs) 490 else: --> 491 result = self.forward(*input, **kwargs) 492 for hook in self._forward_hooks.values(): 493 hook_result = hook(self, input, result) /usr/local/lib/python3.6/dist-packages/torch/nn/modules/loss.py in forward(self, input, target) 191 _assert_no_grad(target) 192 return F.nll_loss(input, target, self.weight, self.size_average, --> 193 self.ignore_index, self.reduce) 194 195 /usr/local/lib/python3.6/dist-packages/torch/nn/functional.py in nll_loss(input, target, weight, size_average, ignore_index, reduce) 1330 .format(input.size(0), target.size(0))) 1331 if dim == 2: -> 1332 return torch._C._nn.nll_loss(input, target, weight, size_average, ignore_index, reduce) 1333 elif dim == 4: 1334 return torch._C._nn.nll_loss2d(input, target, weight, size_average, ignore_index, reduce) RuntimeError: multi-target not supported at /pytorch/aten/src/THNN/generic/ClassNLLCriterion.c:22
Can anyone assist on this?
All Answers
I'm also stuck here, but I don't know what to do with your answer.
How should "1D" be reflected in the code?
Thanks,
-Brian.
The 1D means that the size needs to be torch[100] for classA - using fill & zero as I did above were correct, however since I'm just replacing values in the classApoints I was getting the wrong size ([100][2] instead of just [100]). If I just create a tensor using one of the other functions that were taught previously in the workbook (like in the Initializing Tensors section, maybe the 2nd example), I can built it the right size and then using dot notation extend it and still use the fill or zero (one for classA and one for classB) to derive my target tensors.
Anwswer to Exercise 6: Loss after 100 epochs: -54.190528869628906
@Brian did you manage to get past this challenge? I have redone the exercises in a new notebook and I get the exact same result so I'm out of ideas.
I found the learning curve towards the end of this badge a bit step as well. To help others I've recorded the state of various tensors as I moved through exercises 6 and 7.
The details are in http://bit.ly/31g6ek4
"passes x through linear_to_hidden, then a tanh activation function, and then hidden_to_linear and returns the output"
Not sure what this means...Need help in understanding this.
I believe it should have said: So you need to expand out
def forward
.- It should accept two arguments - self and x
- The return should chain together the two methods defined above (linear_to_hidden and hidden_to_output) with the torch tanh function in the middle.
I.e.loss = TODO
I used this for TODO
loss = loss_fn(preds, targets)
and I get 0.031865194439888 using CrossEntropyLoss.
Can you please help to get closer to one of the options in the trailhead.
Regards