注册 登录  
 加关注
   显示下一条  |  关闭
温馨提示!由于新浪微博认证机制调整,您的新浪微博帐号绑定已过期,请重新绑定!立即重新绑定新浪微博》  |  关闭

widebright的个人空间

// 编程和生活

 
 
 

日志

 
 

机器学习导论 第四周作业 神经网络的backpropagation algorithm  

2011-11-15 00:50:39|  分类: 程序设计 |  标签: |举报 |字号 订阅

  下载LOFTER 我的照片书  |

       这个是上周那个字符识别的,训练神经网络的theta参数的backpropagation algorithm算法的实现。

其实这神经网络还挺复杂的,老实说英语水平差的很,视频都不知道老师在讲什么,就看老师视频上一边画啊画,然后他图示做的挺好的,大概可以蒙到一点,机器学习导论 第四周作业 神经网络的backpropagation algorithm - widebright - widebright的个人空间。算法有很多细节地方不是很理解,我就知道按照人家文档套公式吧。可能人家练习的框架,其实已经实现了很多其他相关的辅助函数了,也没时间去仔细学习理解一下。可能有什么好的书参考一下就不错吧。

        另外视频上,举了一个神经网路的很有意思 的应用,那个无人驾驶的汽车就是用神经网络来做的,这周的最后视频可以看到车自动驾驶。好像那里看过说那个讲课的教授也有参加Google的汽车无人驾驶项目的。看来这神经网络还是比较强大啊。

        这两天睡的都挺晚的,新公司比较远,又不是像以前那么方便的弹性上班时间了,早上7点半就要起来刚快刷牙,挤公车去。路上最快也要1个小时了,感觉不是很适应。老是感觉睡眠不足,以后早点睡吧。



==============================================================

function [J grad] = nnCostFunction(nn_params, ...

                                   input_layer_size, ...

                                   hidden_layer_size, ...

                                   num_labels, ...

                                   X, y, lambda)

%NNCOSTFUNCTION Implements the neural network cost function for a two layer

%neural network which performs classification

%   [J grad] = NNCOSTFUNCTON(nn_params, hidden_layer_size, num_labels, ...

%   X, y, lambda) computes the cost and gradient of the neural network. The

%   parameters for the neural network are "unrolled" into the vector

%   nn_params and need to be converted back into the weight matrices. 

%   The returned parameter grad should be a "unrolled" vector of the

%   partial derivatives of the neural network.

%


% Reshape nn_params back into the parameters Theta1 and Theta2, the weight matrices

% for our 2 layer neural network

Theta1 = reshape(nn_params(1:hidden_layer_size * (input_layer_size + 1)), ...

                 hidden_layer_size, (input_layer_size + 1));


Theta2 = reshape(nn_params((1 + (hidden_layer_size * (input_layer_size + 1))):end), ...

                 num_labels, (hidden_layer_size + 1));


% Setup some useful variables

m = size(X, 1);

         

% You need to return the following variables correctly 

J = 0;

Theta1_grad = zeros(size(Theta1));

Theta2_grad = zeros(size(Theta2));


% ====================== YOUR CODE HERE ======================

% Instructions: You should complete the code by working through the

%               following parts.

%

% Part 1: Feedforward the neural network and return the cost in the

%         variable J. After implementing Part 1, you can verify that your

%         cost function computation is correct by verifying the cost

%         computed in ex4.m

%

% Part 2: Implement the backpropagation algorithm to compute the gradients

%         Theta1_grad and Theta2_grad. You should return the partial derivatives of

%         the cost function with respect to Theta1 and Theta2 in Theta1_grad and

%         Theta2_grad, respectively. After implementing Part 2, you can check

%         that your implementation is correct by running checkNNGradients

%

%         Note: The vector y passed into the function is a vector of labels

%               containing values from 1..K. You need to map this vector into a 

%               binary vector of 1's and 0's to be used with the neural network

%               cost function.

%

%         Hint: We recommend implementing backpropagation using a for-loop

%               over the training examples if you are implementing it for the 

%               first time.

%

% Part 3: Implement regularization with the cost function and gradients.

%

%         Hint: You can implement this around the code for

%               backpropagation. That is, you can compute the gradients for

%               the regularization separately and then add them to Theta1_grad

%               and Theta2_grad from Part 2.

%



a1 = [ones(m, 1) X];


z2 = Theta1 * a1' ;

a2 = sigmoid(z2);

a2 = [ones(m, 1) a2'];


z3 = Theta2 * a2';

a3 = sigmoid(z3);


yk = zeros(num_labels, m); 

for i=1:m,

  yk(y(i),i) = 1;

end


%size(yk)   = 10行 x 5000列

%size(a3)   = 5000 x 10


J = (1/m) * sum ( sum (  (-yk) .* log(a3)  -  (1-yk) .* log(1-a3) ));


%Note that you should not be regularizing the terms that correspond to

%the bias. For the matrices Theta1 and Theta2, this corresponds to the first

%column of each matrix.

t1 = Theta1(:,2:size(Theta1,2));

t2 = Theta2(:,2:size(Theta2,2));


J = J +  (lambda/(2 * m)) * (sum(sum(t1 .^2)) + sum(sum(t2 .^2)));



% -------------------------------------------------------------



X = [ones(m, 1) X];  % add bias 给矩阵最前面增加一个全部为1的一列


for t=1:m    %遍历每一个训练数据


%1. feedforward

a1 = X(t,:); %取出每一行数据, a1 是单行矩阵?z2 = Theta1 * a1';

a2 = sigmoid(z2);

a2 = [1 ; a2]; % a2是一个单列矩阵,在最前面增加1

z3 = Theta2 * a2;

a3 = sigmoid(z3);  % a3 就是 h(theta)


%2.  For each output unit k in layer 3 (the output layer)

delta_3 = a3 - yk(:,t); % y(k) trick - getting columns of t element

%3. For the hidden layer l = 2,

z2=[1; z2]; %add bias

delta_2 = (Theta2' * delta_3) .* sigmoidGradient(z2);


% skipping sigma2(0) 

delta_2 = delta_2(2:end); 


%4. Accumulate the gradient from this example using the following formula

Theta2_grad = Theta2_grad + delta_3 * a2'; %a2是单列矩阵,需要转换成一行才和delta3的单列矩阵相乘

Theta1_grad = Theta1_grad + delta_2 * a1;  %a1 是单行的矩阵,不用转置


end


%5. Obtain the (unregularized) gradient for the neural network cost func-

% tion by dividing the accumulated gradients by 1/m

Theta1_grad = Theta1_grad ./ m;

Theta2_grad = Theta2_grad ./ m;




% Regularized Neural Networks

Theta1 = Theta1(:,2:end);

Theta2 = Theta2(:,2:end);


Theta1 = [zeros(size(Theta1,1),1) Theta1];

Theta2 = [zeros(size(Theta2,1),1) Theta2];


Theta1_grad = Theta1_grad + (lambda/m) * Theta1;

Theta2_grad = Theta2_grad + (lambda/m) * Theta2;


% =========================================================================






% Unroll gradients

grad = [Theta1_grad(:) ; Theta2_grad(:)];



end



  评论这张
 
阅读(1501)| 评论(0)
推荐 转载

历史上的今天

评论

<#--最新日志,群博日志--> <#--推荐日志--> <#--引用记录--> <#--博主推荐--> <#--随机阅读--> <#--首页推荐--> <#--历史上的今天--> <#--被推荐日志--> <#--上一篇,下一篇--> <#-- 热度 --> <#-- 网易新闻广告 --> <#--右边模块结构--> <#--评论模块结构--> <#--引用模块结构--> <#--博主发起的投票-->
 
 
 
 
 
 
 
 
 
 
 
 
 
 

页脚

网易公司版权所有 ©1997-2017