查看ML的源代码 ML 0

=Cost Function损失函数=
Squared error function/Mean squared function均方误差: <math>J(&theta;)=\frac{1}{2m}\sum_{i=1}^m(h_&theta;(x_i)-y_i)^2</math>
Cross entropy交叉熵: <math>J(&theta;)=-\frac{1}{m}\sum_{i=1}^m[y^{(i)}*logh_&theta;(x^{(i)})+(1-y^{(i)})*log(1-h_&theta;(x^{(i)}))]</math>

=Gradient Descent梯度下降=
<math>&theta;_j:=&theta;_j+&alpha;\frac{&part;}{&part;&theta;_j}J(&theta;)</math>
对于线性模型，其损失函数为均方误差，故有：
<math>&alpha;\frac{&part;}{&part;&theta;_j}J(&theta;)= \frac{&part;}{&part;&theta;_j}(\frac{1}{2m}\sum_{i=1}^m(h_&theta;(x_i)-y_i)^2)</math>
:<math>= \frac{1}{2m}\frac{&part;}{&part;&theta;_j}(\sum_{i=1}^m(h_&theta;(x_i)-y_i)^2)</math>
:<math>= \frac{1}{2m}\sum_{i=1}^m( \frac{&part;}{&part;&theta;_j}(h_&theta;(x_i)-y_i)^2 )</math>
:<math>= \frac{1}{m}\sum_{i=1}^m( (h_&theta;(x_i)-y_i) \frac{&part;}{&part;&theta;_j}h_&theta;(x_i) )  //链式求导法式</math>
:<math>= \frac{1}{m}\sum_{i=1}^m( (h_&theta;(x_i)-y_i) \frac{&part;}{&part;&theta;_j}x_i&theta; ) </math>
:<math>= \frac{1}{m}\sum_{i=1}^m( (h_&theta;(x_i)-y_i) \frac{&part;}{&part;&theta;_j}\sum_{k=0}^nx_{ik}&theta;_k ) </math>
对于j>=1:
:<math>= \frac{1}{m}\sum_{i=1}^m( (h_&theta;(x_i)-y_i) x_{ij} ) </math>
:<math>= \frac{1}{2m}\frac{&part;}{&part;&theta;_j} \sum_{i=1}^m(x_i&theta;-y_i)^2 </math>
:<math>= \frac{1}{m}\frac{&part;}{&part;&theta;_j} \sum_{i=1}^mx_{ij}&theta;_j //链式求导法式</math>