# 激励函数

![](https://morvanzhou.github.io/static/results/ML-intro/active3.png)

**AF** 就是指的**激励函数**，是一个**非线性函数**. 比如说**relu**, **sigmoid**, **tanh**.

**激励函数**运行时激活神经网络中某一部分神经元，将激活信息向后传入下一层的神经系统

套在原有的结果之上, 强行把原有的线性结果给扭曲. 使输出结果 y 也有了非线性的特征

**激励函数**必须是可以微分的, 因为在 **backpropagation** **误差反向传递**的时候, 只有这些可微分的激励函数才能把误差传递回去.

## 常用选择

![](https://morvanzhou.github.io/static/results/ML-intro/active4.png)

当**神经网络层**只有两三层, 不是很多的时候, 对于**隐藏层**, 使用任意的**激励函数**, 随便掰弯是可以的, 不会有特别大的影响.

当使用特别多层的**神经网络**, 在掰弯的时候, 不得随意选择利器. 因为会涉及到**梯度爆炸**, **梯度消失**的问题.

在少量层结构中, 可以尝试很多种不同的激励函数.

在**卷积神经网络 Convolutional neural networks** 的**卷积层**中, 推荐的**激励函数**是 **relu**

在**循环神经网络中 recurrent neural networks**, 推荐的是 **tanh** 或者是 **relu**

```python
import tensorflow as tf
import numpy as np
import matplotlib.pyplot as plt

# fake data
x = np.linspace(-5, 5, 200)     # x data, shape=(100, 1)

# following are popular activation functions
y_relu = tf.nn.relu(x)
y_sigmoid = tf.nn.sigmoid(x)
y_tanh = tf.nn.tanh(x)
y_softplus = tf.nn.softplus(x)
# y_softmax = tf.nn.softmax(x)  softmax is a special kind of activation function, it is about probability

sess = tf.Session()
y_relu, y_sigmoid, y_tanh, y_softplus = sess.run([y_relu, y_sigmoid, y_tanh, y_softplus])

# plt to visualize these activation function
plt.figure(1, figsize=(8, 6))
plt.subplot(221)
plt.plot(x, y_relu, c='red', label='relu')
plt.ylim((-1, 5))
plt.legend(loc='best')

plt.subplot(222)
plt.plot(x, y_sigmoid, c='red', label='sigmoid')
plt.ylim((-0.2, 1.2))
plt.legend(loc='best')

plt.subplot(223)
plt.plot(x, y_tanh, c='red', label='tanh')
plt.ylim((-1.2, 1.2))
plt.legend(loc='best')

plt.subplot(224)
plt.plot(x, y_softplus, c='red', label='softplus')
plt.ylim((-0.2, 6))
plt.legend(loc='best')

plt.show()
```

![](/files/-Le0cKUpyWms-hWrbBuj)


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://baozoulin.gitbook.io/tensorflow/chapter1/ji-li-han-shu.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.