1. In this problem, you are going to implement a network that learns (or memorizes) the multiplication

table (“¸carpım tablosu” in Turkish). Using MATLAB, design/implement a three-layer feedforward neural network that multiplies two integers in the range {1,2,...,10}. Use the backpropagation learning algorithm. You are free to design the network as you like, as long as it is capable of multiplying

two integers between 1 and 10. Alternatively, you can refer to my suggestions below.

• Use the hyperbolic tangent function as the activation function. • Use 20 neurons in the input layer. Neurons 1:10 will represent the ﬁrst integer, and neurons 11:20 will represent the second integer. Use a binary input scheme (i.e., -1’s and 1’s). For example, if

you are multiplying 3 and 8, 3rd neuron in the ﬁrst set and 8th neuron in the second set will be

1, others will be -1. In other words, neurons 3 and 18 will be 1 and others will be -1. If you are

multiplying 9 and 5, neurons 9 and 15 will be 1 and others will be -1.

• Use 100 neurons in the output layer and use a binary desired output scheme. For example, if you are multiplying 3 and 8, the desired output vector will have 1 in the 24th entry and -1 in

other entries. If you are multiplying 9 and 5, the desired output will have 1 in the 45th entry

and -1 in the other entries.

Choose an appropriate number of hidden neurons, an appropriate learning rate, an appropriate error

threshold, and/or an appropriate number for maximum number of epochs to stop the iterations. Train

the network using all 100 possible multiplications and report how the error decreases with the number

of epochs. Does your network converge? In how many epochs? Then test the network using all 100

possible multiplications and demonstrate that it works. How many of the 100 possible multiplications

does your network calculate correctly?

2. Do you think the problem above can be solved using a single-layer perceptron (i.e., no hidden

layer)? Explain why or why not.

3. Explain the role of the learning rate in the backpropagation algorithm. What would happen if it

is too large? Too small?

4. An improvement to the backpropagation algorithm is to use a “momentum term” in the weight

update equations. Brieﬂy research and explain what the momentum factor is, and how and why it is

used in the weight update equations.

5. Explain the diﬀerence between synchronous update and asynchronous update in a Hopﬁeld network.

Which one do you think is advantageous? Why?

6. Explain the role of the variance σ2 of the Gaussian function used in Kohonen’s self-organizing

maps. Explain why it needs to be decreased gradually with the number of epochs.

7. Suppose, in a Kohonen’s self-organizing neural network, we use a “Mexican hat function” instead

of the Gaussian. Such a function is plotted below. Explain how using this function would aﬀect the

execution of the algorithm. Do you think this is an improvement to the original algorithm? Why?

2018-03-05
Electrical engineering