# 第二门课 改善深层神经网络：超参数调试、正则化以及优化(Improving Deep Neural Networks:Hyperparameter tuning, Regularization and

- [第一周：深度学习的实用层面(Practical aspects of Deep Learning)](https://baozoulin.gitbook.io/neural-networks-and-deep-learning/di-er-men-ke-gai-shan-shen-ceng-shen-jing-wang-luo-chao-can-shu-tiao-shi-zheng-ze-hua-yi-ji-you-hua/improving-deep-neural-networks/practical-aspects-of-deep-learning.md)
- [1.1 训练，验证，测试集（Train / Dev / Test sets）](https://baozoulin.gitbook.io/neural-networks-and-deep-learning/di-er-men-ke-gai-shan-shen-ceng-shen-jing-wang-luo-chao-can-shu-tiao-shi-zheng-ze-hua-yi-ji-you-hua/improving-deep-neural-networks/practical-aspects-of-deep-learning/11-xun-lian-ff0c-yan-zheng-ff0c-ceshi-ji-ff08-train-dev-test-sets.md)
- [1.2 偏差，方差（Bias /Variance）](https://baozoulin.gitbook.io/neural-networks-and-deep-learning/di-er-men-ke-gai-shan-shen-ceng-shen-jing-wang-luo-chao-can-shu-tiao-shi-zheng-ze-hua-yi-ji-you-hua/improving-deep-neural-networks/practical-aspects-of-deep-learning/12-pian-cha-ff0c-fang-cha-ff08-bias-variance.md)
- [1.3 机器学习基础（Basic Recipe for Machine Learning）](https://baozoulin.gitbook.io/neural-networks-and-deep-learning/di-er-men-ke-gai-shan-shen-ceng-shen-jing-wang-luo-chao-can-shu-tiao-shi-zheng-ze-hua-yi-ji-you-hua/improving-deep-neural-networks/practical-aspects-of-deep-learning/13-ji-qi-xue-xi-ji-chu-ff08-basic-recipe-for-machine-learning.md)
- [1.4 正则化（Regularization）](https://baozoulin.gitbook.io/neural-networks-and-deep-learning/di-er-men-ke-gai-shan-shen-ceng-shen-jing-wang-luo-chao-can-shu-tiao-shi-zheng-ze-hua-yi-ji-you-hua/improving-deep-neural-networks/practical-aspects-of-deep-learning/14-zheng-ze-hua-ff08-regularization.md)
- [1.5 为什么正则化有利于预防过拟合呢？（Why regularization reduces overfitting?）](https://baozoulin.gitbook.io/neural-networks-and-deep-learning/di-er-men-ke-gai-shan-shen-ceng-shen-jing-wang-luo-chao-can-shu-tiao-shi-zheng-ze-hua-yi-ji-you-hua/improving-deep-neural-networks/practical-aspects-of-deep-learning/15-wei-shi-yao-zheng-ze-hua-you-li-yu-yu-fang-guo-ni-he-ni-ff1f-ff08-why-regularization-reduces-over.md)
- [1.6 dropout 正则化（Dropout Regularization）](https://baozoulin.gitbook.io/neural-networks-and-deep-learning/di-er-men-ke-gai-shan-shen-ceng-shen-jing-wang-luo-chao-can-shu-tiao-shi-zheng-ze-hua-yi-ji-you-hua/improving-deep-neural-networks/practical-aspects-of-deep-learning/16-dropout-zheng-ze-hua-ff08-dropout-regularization.md)
- [1.7 理解 dropout（Understanding Dropout）](https://baozoulin.gitbook.io/neural-networks-and-deep-learning/di-er-men-ke-gai-shan-shen-ceng-shen-jing-wang-luo-chao-can-shu-tiao-shi-zheng-ze-hua-yi-ji-you-hua/improving-deep-neural-networks/practical-aspects-of-deep-learning/17-li-jie-dropout-understanding-dropout.md)
- [1.8 其他正则化方法（Other regularization methods）](https://baozoulin.gitbook.io/neural-networks-and-deep-learning/di-er-men-ke-gai-shan-shen-ceng-shen-jing-wang-luo-chao-can-shu-tiao-shi-zheng-ze-hua-yi-ji-you-hua/improving-deep-neural-networks/practical-aspects-of-deep-learning/18-qi-ta-zheng-ze-hua-fang-fa-ff08-other-regularization-methods.md)
- [1.9 归一化输入（Normalizing inputs）](https://baozoulin.gitbook.io/neural-networks-and-deep-learning/di-er-men-ke-gai-shan-shen-ceng-shen-jing-wang-luo-chao-can-shu-tiao-shi-zheng-ze-hua-yi-ji-you-hua/improving-deep-neural-networks/practical-aspects-of-deep-learning/19-gui-yi-hua-shu-ru-ff08-normalizing-inputs.md)
- [1.10 梯度消失/梯度爆炸（Vanishing / Exploding gradients）](https://baozoulin.gitbook.io/neural-networks-and-deep-learning/di-er-men-ke-gai-shan-shen-ceng-shen-jing-wang-luo-chao-can-shu-tiao-shi-zheng-ze-hua-yi-ji-you-hua/improving-deep-neural-networks/practical-aspects-of-deep-learning/110-ti-du-xiao-5931-ti-du-bao-zha-ff08-vanishing-exploding-gradients.md)
- [1.11 神经网络的权重初始化（Weight Initialization for Deep Networks）](https://baozoulin.gitbook.io/neural-networks-and-deep-learning/di-er-men-ke-gai-shan-shen-ceng-shen-jing-wang-luo-chao-can-shu-tiao-shi-zheng-ze-hua-yi-ji-you-hua/improving-deep-neural-networks/practical-aspects-of-deep-learning/111-shen-jing-wang-luo-de-quan-zhong-chu-shi-hua-ff08-weight-initialization-for-deep-networks.md)
- [1.12 梯度的数值逼近（Numerical approximation of gradients）](https://baozoulin.gitbook.io/neural-networks-and-deep-learning/di-er-men-ke-gai-shan-shen-ceng-shen-jing-wang-luo-chao-can-shu-tiao-shi-zheng-ze-hua-yi-ji-you-hua/improving-deep-neural-networks/practical-aspects-of-deep-learning/112-ti-du-de-shu-zhi-bi-jin-ff08-numerical-approximation-of-gradients.md)
- [1.13 梯度检验（Gradient checking）](https://baozoulin.gitbook.io/neural-networks-and-deep-learning/di-er-men-ke-gai-shan-shen-ceng-shen-jing-wang-luo-chao-can-shu-tiao-shi-zheng-ze-hua-yi-ji-you-hua/improving-deep-neural-networks/practical-aspects-of-deep-learning/113-ti-du-jian-yan-ff08-gradient-checking.md)
- [1.14 梯度检验应用的注意事项（Gradient Checking Implementation Notes）](https://baozoulin.gitbook.io/neural-networks-and-deep-learning/di-er-men-ke-gai-shan-shen-ceng-shen-jing-wang-luo-chao-can-shu-tiao-shi-zheng-ze-hua-yi-ji-you-hua/improving-deep-neural-networks/practical-aspects-of-deep-learning/114-ti-du-jian-yan-ying-yong-de-zhu-yi-shi-xiang-ff08-gradient-checking-implementation-notes.md)
- [Initialization](https://baozoulin.gitbook.io/neural-networks-and-deep-learning/di-er-men-ke-gai-shan-shen-ceng-shen-jing-wang-luo-chao-can-shu-tiao-shi-zheng-ze-hua-yi-ji-you-hua/improving-deep-neural-networks/practical-aspects-of-deep-learning/dai-ma.md)
- [Gradient Checking](https://baozoulin.gitbook.io/neural-networks-and-deep-learning/di-er-men-ke-gai-shan-shen-ceng-shen-jing-wang-luo-chao-can-shu-tiao-shi-zheng-ze-hua-yi-ji-you-hua/improving-deep-neural-networks/practical-aspects-of-deep-learning/gradient-checking.md)
- [Regularization](https://baozoulin.gitbook.io/neural-networks-and-deep-learning/di-er-men-ke-gai-shan-shen-ceng-shen-jing-wang-luo-chao-can-shu-tiao-shi-zheng-ze-hua-yi-ji-you-hua/improving-deep-neural-networks/practical-aspects-of-deep-learning/regularization.md)
- [reg\_utils.py](https://baozoulin.gitbook.io/neural-networks-and-deep-learning/di-er-men-ke-gai-shan-shen-ceng-shen-jing-wang-luo-chao-can-shu-tiao-shi-zheng-ze-hua-yi-ji-you-hua/improving-deep-neural-networks/practical-aspects-of-deep-learning/regutils-py.md)
- [testCases.py](https://baozoulin.gitbook.io/neural-networks-and-deep-learning/di-er-men-ke-gai-shan-shen-ceng-shen-jing-wang-luo-chao-can-shu-tiao-shi-zheng-ze-hua-yi-ji-you-hua/improving-deep-neural-networks/practical-aspects-of-deep-learning/testcasespy.md)
- [第二周：优化算法 (Optimization algorithms)](https://baozoulin.gitbook.io/neural-networks-and-deep-learning/di-er-men-ke-gai-shan-shen-ceng-shen-jing-wang-luo-chao-can-shu-tiao-shi-zheng-ze-hua-yi-ji-you-hua/improving-deep-neural-networks/optimization-algorithms.md)
- [2.1 Mini-batch 梯度下降（Mini-batch gradient descent）](https://baozoulin.gitbook.io/neural-networks-and-deep-learning/di-er-men-ke-gai-shan-shen-ceng-shen-jing-wang-luo-chao-can-shu-tiao-shi-zheng-ze-hua-yi-ji-you-hua/improving-deep-neural-networks/optimization-algorithms/21-mini-batch-ti-du-xia-jiang-ff08-mini-batch-gradient-descent.md)
- [2.2 理解 mini-batch 梯度下降法（Understanding mini-batch gradient descent）](https://baozoulin.gitbook.io/neural-networks-and-deep-learning/di-er-men-ke-gai-shan-shen-ceng-shen-jing-wang-luo-chao-can-shu-tiao-shi-zheng-ze-hua-yi-ji-you-hua/improving-deep-neural-networks/optimization-algorithms/22-li-jie-mini-batch-ti-du-xia-jiang-fa-ff08-understanding-mini-batch-gradient-descent.md)
- [2.3 指数加权平均数（Exponentially weighted averages）](https://baozoulin.gitbook.io/neural-networks-and-deep-learning/di-er-men-ke-gai-shan-shen-ceng-shen-jing-wang-luo-chao-can-shu-tiao-shi-zheng-ze-hua-yi-ji-you-hua/improving-deep-neural-networks/optimization-algorithms/23-zhi-shu-jia-quan-ping-jun-shu-ff08-exponentially-weighted-averages.md)
- [2.4 理解指数加权平均数（Understanding exponentially weighted averages ）](https://baozoulin.gitbook.io/neural-networks-and-deep-learning/di-er-men-ke-gai-shan-shen-ceng-shen-jing-wang-luo-chao-can-shu-tiao-shi-zheng-ze-hua-yi-ji-you-hua/improving-deep-neural-networks/optimization-algorithms/24-li-jie-zhi-shu-jiaquanping-jun-shu-ff08-understandingexponentially-weighted-averages.md)
- [2.5 指 数 加 权 平 均 的 偏 差 修 正 （ Bias correction in exponentially weighted averages ）](https://baozoulin.gitbook.io/neural-networks-and-deep-learning/di-er-men-ke-gai-shan-shen-ceng-shen-jing-wang-luo-chao-can-shu-tiao-shi-zheng-ze-hua-yi-ji-you-hua/improving-deep-neural-networks/optimization-algorithms/25-zhi-shu-jia-quan-ping-jun-de-pian-cha-xiu-zheng-bias-correction-in-exponentially-weighted-average.md)
- [2.6 动量梯度下降法（Gradient descent with Momentum ）](https://baozoulin.gitbook.io/neural-networks-and-deep-learning/di-er-men-ke-gai-shan-shen-ceng-shen-jing-wang-luo-chao-can-shu-tiao-shi-zheng-ze-hua-yi-ji-you-hua/improving-deep-neural-networks/optimization-algorithms/26-dong-liang-tidu-xia-jiang-fa-ff08-gradient-descent-with-momentum.md)
- [2.7 RMSprop( root mean square prop)](https://baozoulin.gitbook.io/neural-networks-and-deep-learning/di-er-men-ke-gai-shan-shen-ceng-shen-jing-wang-luo-chao-can-shu-tiao-shi-zheng-ze-hua-yi-ji-you-hua/improving-deep-neural-networks/optimization-algorithms/27-rmsprop.md)
- [2.8 Adam 优化算法(Adam optimization algorithm)](https://baozoulin.gitbook.io/neural-networks-and-deep-learning/di-er-men-ke-gai-shan-shen-ceng-shen-jing-wang-luo-chao-can-shu-tiao-shi-zheng-ze-hua-yi-ji-you-hua/improving-deep-neural-networks/optimization-algorithms/28-adam-you-hua-suan-6cd528-adam-optimization-algorithm.md)
- [2.9 学习率衰减(Learning rate decay)](https://baozoulin.gitbook.io/neural-networks-and-deep-learning/di-er-men-ke-gai-shan-shen-ceng-shen-jing-wang-luo-chao-can-shu-tiao-shi-zheng-ze-hua-yi-ji-you-hua/improving-deep-neural-networks/optimization-algorithms/29-xue-xi-lv-shuai-51cf28-learning-rate-decay.md)
- [2.10 局部最优的问题(The problem of local optima)](https://baozoulin.gitbook.io/neural-networks-and-deep-learning/di-er-men-ke-gai-shan-shen-ceng-shen-jing-wang-luo-chao-can-shu-tiao-shi-zheng-ze-hua-yi-ji-you-hua/improving-deep-neural-networks/optimization-algorithms/210-ju-bu-zui-you-de-wen-989828-the-problem-of-local-optima.md)
- [Optimization](https://baozoulin.gitbook.io/neural-networks-and-deep-learning/di-er-men-ke-gai-shan-shen-ceng-shen-jing-wang-luo-chao-can-shu-tiao-shi-zheng-ze-hua-yi-ji-you-hua/improving-deep-neural-networks/optimization-algorithms/optimization-methods.md)
- [opt\_utils.py](https://baozoulin.gitbook.io/neural-networks-and-deep-learning/di-er-men-ke-gai-shan-shen-ceng-shen-jing-wang-luo-chao-can-shu-tiao-shi-zheng-ze-hua-yi-ji-you-hua/improving-deep-neural-networks/optimization-algorithms/optutils-py.md)
- [testCases.py](https://baozoulin.gitbook.io/neural-networks-and-deep-learning/di-er-men-ke-gai-shan-shen-ceng-shen-jing-wang-luo-chao-can-shu-tiao-shi-zheng-ze-hua-yi-ji-you-hua/improving-deep-neural-networks/optimization-algorithms/testcasespy.md)
- [第 三 周 超 参 数 调 试 、 Batch 正 则 化 和 程 序 框 架 （Hyperparameter tuning）](https://baozoulin.gitbook.io/neural-networks-and-deep-learning/di-er-men-ke-gai-shan-shen-ceng-shen-jing-wang-luo-chao-can-shu-tiao-shi-zheng-ze-hua-yi-ji-you-hua/improving-deep-neural-networks/hyperparameter-tuning.md)
- [3.1 调试处理（Tuning process）](https://baozoulin.gitbook.io/neural-networks-and-deep-learning/di-er-men-ke-gai-shan-shen-ceng-shen-jing-wang-luo-chao-can-shu-tiao-shi-zheng-ze-hua-yi-ji-you-hua/improving-deep-neural-networks/hyperparameter-tuning/31-diao-shi-chu-li-ff08-tuning-process.md)
- [3.2 为超参数选择合适的范围（Using an appropriate scale to pick hyperparameters）](https://baozoulin.gitbook.io/neural-networks-and-deep-learning/di-er-men-ke-gai-shan-shen-ceng-shen-jing-wang-luo-chao-can-shu-tiao-shi-zheng-ze-hua-yi-ji-you-hua/improving-deep-neural-networks/hyperparameter-tuning/32-wei-chao-can-shu-xuan-ze-he-shi-de-fan-wei-ff08-using-an-appropriate-scale-to-pick-hyperparameter.md)
- [3.3 超参数训练的实践： Pandas VS Caviar（Hyperparameters tuning in practice: Pandas vs. Caviar）](https://baozoulin.gitbook.io/neural-networks-and-deep-learning/di-er-men-ke-gai-shan-shen-ceng-shen-jing-wang-luo-chao-can-shu-tiao-shi-zheng-ze-hua-yi-ji-you-hua/improving-deep-neural-networks/hyperparameter-tuning/33-chao-can-shu-xun-lian-de-shi-jian-ff1a-pandas-vs-caviar-hyperparameterstuning-in-practice-pandas.md)
- [3.4 归一化网络的激活函数（ Normalizing activations in a network）](https://baozoulin.gitbook.io/neural-networks-and-deep-learning/di-er-men-ke-gai-shan-shen-ceng-shen-jing-wang-luo-chao-can-shu-tiao-shi-zheng-ze-hua-yi-ji-you-hua/improving-deep-neural-networks/hyperparameter-tuning/34-gui-yi-hua-wang-luo-de-ji-huo-han-shu-ff08-normalizing-activations-in-a-network.md)
- [3.5 将 Batch Norm 拟合进神经网络（Fitting Batch Norm into a neural network）](https://baozoulin.gitbook.io/neural-networks-and-deep-learning/di-er-men-ke-gai-shan-shen-ceng-shen-jing-wang-luo-chao-can-shu-tiao-shi-zheng-ze-hua-yi-ji-you-hua/improving-deep-neural-networks/hyperparameter-tuning/35-jiang-batch-norm-ni-he-jin-shen-jing-wang-luo-ff08-fitting-batch-norm-into-a-neural-network.md)
- [3.6 Batch Norm 为什么奏效？（Why does Batch Norm work?）](https://baozoulin.gitbook.io/neural-networks-and-deep-learning/di-er-men-ke-gai-shan-shen-ceng-shen-jing-wang-luo-chao-can-shu-tiao-shi-zheng-ze-hua-yi-ji-you-hua/improving-deep-neural-networks/hyperparameter-tuning/36-batch-norm-wei-shi-yao-zou-xiao-ff1f-ff08-why-does-batch-norm-work.md)
- [3.7 测试时的 Batch Norm（Batch Norm at test time）](https://baozoulin.gitbook.io/neural-networks-and-deep-learning/di-er-men-ke-gai-shan-shen-ceng-shen-jing-wang-luo-chao-can-shu-tiao-shi-zheng-ze-hua-yi-ji-you-hua/improving-deep-neural-networks/hyperparameter-tuning/37-ce-shi-shi-de-batch-norm-batch-norm-at-test-time.md)
- [3.8 Softmax 回归（Softmax regression）](https://baozoulin.gitbook.io/neural-networks-and-deep-learning/di-er-men-ke-gai-shan-shen-ceng-shen-jing-wang-luo-chao-can-shu-tiao-shi-zheng-ze-hua-yi-ji-you-hua/improving-deep-neural-networks/hyperparameter-tuning/38-softmax-hui-gui-ff08-softmax-regression.md)
- [3.9 训练一个 Softmax 分类器（Training a Softmax classifier）](https://baozoulin.gitbook.io/neural-networks-and-deep-learning/di-er-men-ke-gai-shan-shen-ceng-shen-jing-wang-luo-chao-can-shu-tiao-shi-zheng-ze-hua-yi-ji-you-hua/improving-deep-neural-networks/hyperparameter-tuning/39-xun-lian-yi-ge-softmax-fen-lei-qi-ff08-training-a-softmax-classifier.md)
- [tensorflow tutorial](https://baozoulin.gitbook.io/neural-networks-and-deep-learning/di-er-men-ke-gai-shan-shen-ceng-shen-jing-wang-luo-chao-can-shu-tiao-shi-zheng-ze-hua-yi-ji-you-hua/improving-deep-neural-networks/hyperparameter-tuning/tensorflow-tutorial.md)
- [improv\_utils.py](https://baozoulin.gitbook.io/neural-networks-and-deep-learning/di-er-men-ke-gai-shan-shen-ceng-shen-jing-wang-luo-chao-can-shu-tiao-shi-zheng-ze-hua-yi-ji-you-hua/improving-deep-neural-networks/hyperparameter-tuning/improvutils-py.md)
- [tf\_utils.py](https://baozoulin.gitbook.io/neural-networks-and-deep-learning/di-er-men-ke-gai-shan-shen-ceng-shen-jing-wang-luo-chao-can-shu-tiao-shi-zheng-ze-hua-yi-ji-you-hua/improving-deep-neural-networks/hyperparameter-tuning/tfutils-py.md)


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://baozoulin.gitbook.io/neural-networks-and-deep-learning/di-er-men-ke-gai-shan-shen-ceng-shen-jing-wang-luo-chao-can-shu-tiao-shi-zheng-ze-hua-yi-ji-you-hua/improving-deep-neural-networks.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.