Before the seminar, write code for a backprop network and test it on this problem:
A mystery system has two inputs, a and b, and an output x. Your network must be trained to predict x given a and b.
In fact, the system's output x is just (a-b)2 but pretend you don't know this!
We can generate some training data:
|
Input a |
Input b |
Output x |
|---|---|---|
|
0.000 |
0.000 |
0.000 |
|
0.000 |
1.000 |
1.000 |
|
1.000 |
0.000 |
1.000 |
|
1.000 |
1.000 |
0.000 |
So your network has two inputs and one output; try having two hidden nodes for this problem.
Write the code so that you can easily change the number of inputs, hidden nodes, and output nodes.
After training, how does your network generalise to other inputs? Has it captured the hidden relationship x=(a-b)2 well?
If you have time, see if adding momentum during training helps: it often does. And what happens if you use different numbers of hidden nodes?
The eagle-eyed will have spotted that our training data is the exclusive-OR function: this is good for debugging our code because we know it is not linearly separable, so demands hidden nodes and a way to train them.
Bring your code to the seminar, with whatever problems, questions, suggestions, or observations you'd like to discuss.
Have fun! Do this now and the assignment will be easy.