查看ML的源代码 ML 0

=Week1=
==Cost Function损失函数==
Squared error function/Mean squared function均方误差: <math>J(&theta;)=\frac{1}{2m}\sum_{i=1}^m(h_&theta;(x_i)-y_i)^2</math>
Cross entropy交叉熵: <math>J(&theta;)=-\frac{1}{m}\sum_{i=1}^m[y^{(i)}*logh_&theta;(x^{(i)})+(1-y^{(i)})*log(1-h_&theta;(x^{(i)}))]</math>

==Gradient Descent梯度下降==
<math>&theta;_j:=&theta;_j+&alpha;\frac{&part;}{&part;&theta;_j}J(&theta;)</math>
对于线性模型，其损失函数为均方误差，故有（这里输入训练数据x为m*n矩阵, 线性参数<math>&theta;</math>为n*1，<math>x_i</math>代表训练矩阵中的第i行，<math>x_{ik}</math>代表第i行第k列）：
<math>\frac{&part;}{&part;&theta;_j}J(&theta;)= \frac{&part;}{&part;&theta;_j}(\frac{1}{2m}\sum_{i=1}^m(h_&theta;(x_i)-y_i)^2)</math>
:<math>= \frac{1}{2m}\frac{&part;}{&part;&theta;_j}(\sum_{i=1}^m(h_&theta;(x_i)-y_i)^2)</math>
:<math>= \frac{1}{2m}\sum_{i=1}^m( \frac{&part;}{&part;&theta;_j}(h_&theta;(x_i)-y_i)^2 )</math>
:<math>= \frac{1}{m}\sum_{i=1}^m( (h_&theta;(x_i)-y_i) \frac{&part;}{&part;&theta;_j}h_&theta;(x_i) )  //链式求导法式</math>
:<math>= \frac{1}{m}\sum_{i=1}^m( (h_&theta;(x_i)-y_i) \frac{&part;}{&part;&theta;_j}x_i&theta; ) </math>
:<math>= \frac{1}{m}\sum_{i=1}^m( (h_&theta;(x_i)-y_i) \frac{&part;}{&part;&theta;_j}\sum_{k=0}^{n-1}x_{ik}&theta;_k ) </math>
对于j>=1:
:<math>= \frac{1}{m}\sum_{i=1}^m( (h_&theta;(x_i)-y_i) x_{ij} ) </math>
:<math>= \frac{1}{m} (h_&theta;(x)-y) x_{j}  </math>

=Week2=
==multivariate linear regression==
<math>h_&theta;(x) = &theta;_0x_0 + &theta;_1x_1 + &theta;_2x_2 + ... + &theta;_nx_n = &theta;^Tx</math>
其中，
<math>
x=\begin{vmatrix}
x_0  \\
x_1 \\
x_2 \\
... \\
x_n
\end{vmatrix}
 = \begin{vmatrix}
x_0^{(0)} & x_0^{(1)} & x_0^{(2)} & ... & x_0^{(m)} \\
x_1^{(0)} & x_1^{(1)} & x_1^{(2)} & ... & x_1^{(m)} \\
... & ... & ... & ... & ...\\
x_n^{(0)} & x_m^{(1)} & x_m^{(2)} & ... & x_n^{(m)} \\
\end{vmatrix}
, 
&theta;=\begin{vmatrix}
&theta;_0 \\
&theta;_1\\
&theta;_2\\
...\\
&theta;_n
\end{vmatrix}
</math>
:m为训练数据组数，n为特征个数（通常，为了方便处理，会令<math>x_0^{(i)}=1, i=0,1,2,...,m）</math>。

==Feature Scaling & Standard Normalization==
<math>
x_i := \frac{x_i-&mu;_i}{s_i}
</math>
其中，<math>&mu;_i</math>是第i个特征数据x_i的均值，而 <math>s_i</math>则要视情况而定：
:*Feature Scaling:<math>s_i</math>为<math>x_i</math>中最大值与最小值的差（max-min）；
:*Standard Normalization:<math>s_i</math>为<math>x_i</math>中数据标准差（standard deviation）。