ICode9

精准搜索请尝试: 精确搜索
首页 > 其他分享> 文章详细

【Tensorflow快速上手】手写数字识别现成代码每行都有注释、自定义损失函数、自定义训练步骤train_step、早停预防过拟合

2021-08-04 19:30:44  阅读:219  来源: 互联网

标签:04 自定义 08 09 step train 2021 0.0000 tensorflow


TensorFlow 是一个端到端开源机器学习平台,由谷歌大脑开发和维护。TensorFlow对深度学习常用的数学方法和数据操作进行了封装,使用者利用TensorFlow可以更加方便快速地搭建和训练神经网络。
可以将许多层次加入一个普通的Python列表然后作为参数传入模型来构造顺序结构的神经网络,如果想搭建如GoogLeNet一样存在分支的网络这可以使用tensorflow.keras.layers.concatenate实现,将多个层次构成的列表作为参数inputs传入方法可以做到。
本文用LeNet进行数字识别,LeNet结构简单易于理解和实现但涉及到的知识一点都不少,是入手深度学习框架的不错的选择。LeNet-5是最早的卷积神经网络之一。论文提出的卷积层、池化层的概念,也提到本文所使用的MNIST数据集。MNIST数据集已经集成进了Tensorflow框架,可以通过调用一个方法直接使用。

诶哟图丢了

环境如何配置

Cuda compilation tools, release 11.2, V11.2.152
Build cuda_11.2.r11.2/compiler.29618528_0
Python 3.8.10
tensorflow-gpu 2.5.0

代码hello_ world

#!~/.conda/envs python
# -*- coding: utf-8 -*-

import functools  # 来源Python标准库的包,用于处理函数和可调用对象的工具

import numpy as np  # Numpy是常用数学包,提供了很多数学计算方法

import tensorflow.keras as keras  # https://keras.io/zh/ 可看作Tensorflow的一个子包,但也可单独使用
import tensorflow.keras.optimizers as opts  # 引入优化器
from tensorflow import (  # 提倡在导入大量模块和方法时加括号
    GradientTape,  # 自动求导类,求导之前得先监听变量,用其监听的变量所做的各种操作来确定这个导该怎么求
    Tensor,  # 描述矩阵的类,调试中可以看到其不同实例的名字互相关联
    clip_by_value,  # 用来限制变量取值范围的方法,给这个变量设置上下限
    convert_to_tensor,  # 将列表或者numpy.ndarray转换为Tensor
    float32,  # Tensorflow提供32位浮点数
    reshape # 对Tensor形状进行变更
)
from tensorflow import math as tfmath  # 框架内进行一些数学运算,用的是自己封装的类和方法
from tensorflow import reduce_max  # 求Tensor矩阵中值最大的一个元素
from tensorflow.keras.layers import (AvgPool2D, Conv2D, Dense, Flatten, InputLayer)  # 导入所需层
from tensorflow.keras.losses import BinaryCrossentropy  # 框架自带交叉熵损失函数,这次我们不用这个,我们自己来写
from tensorflow.python.keras.callbacks import Callback  # 每轮拟合都调用一次的回调函数类的基类
from tensorflow.python.keras.engine import data_adapter # 将不同的输入数据转换为tf.dataset


np.set_printoptions(suppress=True, threshold=np.inf)  # 设置不用科学计数法
SHAPE = (-1, 28, 28, 1) #神经网络输入形状,-1代表自动设置长度

(x, y), (x_t, y_t) = keras.datasets.mnist.load_data() # 读数据,读取出来的数据形状为((n,28,28,1),(n,10)),((m,28,28,1),(m,10))
x = reshape(x, SHAPE) # 改成(-1, 28, 28, 1)是要描述图片的通道数,1就是灰度图片,3应该是RGB彩色图片
x_t = reshape(x_t, SHAPE) # 同上
y = [[1 if j == y[i] else 0 for j in range(10)] for i in range(len(y))] # 这里y的值就是0~9,这不利于计算,提前将1个值处理成10个值
y_t = [[1 if j == y_t[i] else 0 for j in range(10)] for i in range(len(y_t))]) # 同上

# 取一定量的数据减少内存占用,否则就溢出
rands = np.random.randint(0, len(x), (2000,))  # 随机一些序号
x = [x[i] for i in range(len(x)) if i in rands]  # 按照序号获取列表中的一些数值
y = [y[i] for i in range(len(y)) if i in rands]  # 按照序号获取列表中的一些数值
x = convert_to_tensor(x)  # 将列表转换为Tensor对象
y = convert_to_tensor(y, dtype=float32)  # 将列表转换为Tensor对象并设置Tensor对象的dtype

# dtype描述了对象内部存储的数据类型,这个Tensor中的每一个值都转化为这个类型

print('x.shape = %s, y.shape = %s' % (x.shape, y.shape))  # 打印数据形状

# 如果y的第0维度与输出层神经元数目不同就会影响损失函数工作,除非在损失函数中将y转换成正确的形状

EPOCHS = 500  # 最大拟合500轮
NETWORK = [
    InputLayer(input_shape=SHAPE[1:], name="input"),  # 输入层,确定第一个隐层的权重矩阵尺寸
    Conv2D( 
        filters=32,  # 卷积核数目,有许多神经元对输入做一致的操作,不妨让它们的权重矩阵就也可以同步变化,换种说法:共享权重
        kernel_size=(3, 3),  # 卷积核大小,就是权重矩阵的shape
        activation='relu',  # 激励函数的名字一定要写对
        padding='same',  # 这个参数描述卷积层是否对输入进行填充,默认是'valid'(不填充),具体怎么填充是框架里已经做好的事情
        name="Conv_1"  # 给这一层起个名字
    ),
    AvgPool2D(  # 平均池化层,对输入的几个相邻值合并成一个值(求平均)来降维
        pool_size=(2, 2),  # 描述“窗口”大小,对多少值求平均,这里取了2行2列4个值求平均
        strides=None,  # “窗口”的步长,为None时就自动让“窗口”永不重合
        padding='valid',  # 无填充
        name="Pool_1"  # 名字
    ),
    Conv2D(filters=64, kernel_size=(3, 3), activation='relu', padding='same'), name="Conv_2" # 同一网络中层的名字要互不相同
    AvgPool2D(name="Pool_2"), # 输出的shape应该是(None,7,7,64)
    Flatten(), # 平展层用来改变矩阵形状,其没有可训练参数,仅仅将输入的矩阵处理成全连接层所接受的形状
    Dense(units=64, activation='relu', name="Dense_1"), # 经过平展层后输入应该是(None,3136)
    Dense(units=10, activation='sigmoid', name="Dense_2") # 每一个神经元用上一层的64个值判断输入是否是自己的输出的类别
]

model = keras.Sequential(layers=NETWORK, name='LeNet')  # 将层顺序连接形成神经网络模型


def step(self, data):  # 定义训练步骤方法,模型每轮拟合都进行

    print('data type: %s' % type(data))  # data是tuple类型的
    print(data)  # data里面有两个Tensor对象
    
    x, y, _ = data_adapter.unpack_x_y_sample_weight(data)  # 把元组拆成单个值
    print(x.shape)  # 这里有一行输出,在训练过程中并不会显示出来
    with GradientTape(persistent=True, watch_accessed_variables=False) as tape:
        tape.watch(self.trainable_variables)  # 监控网络参数进行过的运算
        y_hat = self(x, training=True)  # 得出模型输出
        loss = self.loss(y, y_hat)  # 计算模型损失
    grads = tape.gradient(loss, self.trainable_variables)  # GradientTape 自动求导
    self.optimizer.apply_gradients(  # 优化网络参数
        zip(grads, self.trainable_variables) # 函数用于将可迭代的对象作为参数,将对象中对应的元素打包成一个个元组,然后返回由这些元组组成的列表。
    )

    accuracy = tfmath.reduce_mean((y_hat*y)+((1-y_hat)*(1-y))) # 计算正确率,不算也行,但是早停需要用所以就进行计算
    return {'loss': loss, 'accuracy': accuracy} # 返回值是字典


class Callback_(Callback):  # 定义回调类
    def on_epoch_end(self, epoch, logs={}):  # self就是Model对象的指针
        acc = logs.get('accuracy')  # logs是step的返回值
        if acc > 0.97:  # 如果能达到97%的训练误差就停止训练
            self.model.stop_training = True  # 设置停止训练


def log(v: Tensor):  # 定义一个求对数的方法
    return tfmath.log(clip_by_value(v, 1e-8, reduce_max(v)))  # 限制v的取值防止出现nan


def binary_crossentropy_(y, y_hat):  # 自定义的交叉熵损失函数
    loss = log(y_hat) * y + (log(1 - y_hat) * (1-y)) # L_i=log_e(hat{y}_i)y_i+log_e(1-hat{y}_i)(1-y_i)
    loss = -tfmath.reduce_mean(loss)  # (1/N)∑_{i}L_i
    return loss  # 返回损失值


opt = opts.Nadam(learning_rate=.0001)  # 定义优化器
model.compile(  # 模型训练相关配置
    optimizer=opt,  # 指定优化器
    loss=binary_crossentropy_,  # 使用我们上面定义的损失函数
    # loss=BinaryCrossentropy(from_logits=True), # 使用框架自带的损失函数
    metrics=['accuracy']  # 定义模型评估指标,名字和上面一致
)


model.train_step = functools.partial(step, model)  # partial负责把model赋给step的self参数
model.fit(x, y, epochs=EPOCHS, callbacks=[Callback_()])  # 填入数据和标签,定义回调列表
model.summary()  # 打印网络结构

rands = np.random.randint(0, len(y_t), (30,))  # 取30个测试数据进行打印
for index in rands: # 遍历30个序号
    y_hat = model.predict(convert_to_tensor([x_t[index]]))  # 非训练下获取网络输出
    mi = np.argmax(y_t[index])  # 算出第index个测试数据是什么数字
    mti = np.argmax(y_hat[0])  # 算出模型计算出第index个测试数据是什么数字
    print('[%s]::%s' % (', '.join(['%s%.04f\033[0m' % (('\033[32m'if i == mi else '\033[31m') if i == mti else '', y_hat[0][i]) for i in range(len((y_hat[0])))]), mi))  # 打印输出

控制台

2021-08-04 09:05:56.073437: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcudart.so.11.0
2021-08-04 09:05:58.276934: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcuda.so.1
2021-08-04 09:05:58.312833: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-04 09:05:58.313281: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1733] Found device 0 with properties: 
pciBusID: 0000:01:00.0 name: NVIDIA GeForce GTX 1050 Ti computeCapability: 6.1
coreClock: 1.62GHz coreCount: 6 deviceMemorySize: 3.95GiB deviceMemoryBandwidth: 104.43GiB/s
2021-08-04 09:05:58.313349: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcudart.so.11.0
2021-08-04 09:05:58.332447: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcublas.so.11
2021-08-04 09:05:58.332597: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcublasLt.so.11
2021-08-04 09:05:58.342695: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcufft.so.10
2021-08-04 09:05:58.347238: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcurand.so.10
2021-08-04 09:05:58.352323: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcusolver.so.11
2021-08-04 09:05:58.357109: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcusparse.so.11
2021-08-04 09:05:58.358519: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcudnn.so.8
2021-08-04 09:05:58.358676: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-04 09:05:58.359252: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-04 09:05:58.360137: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1871] Adding visible gpu devices: 0
2021-08-04 09:05:58.361005: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2021-08-04 09:05:58.361972: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-04 09:05:58.362384: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1733] Found device 0 with properties: 
pciBusID: 0000:01:00.0 name: NVIDIA GeForce GTX 1050 Ti computeCapability: 6.1
coreClock: 1.62GHz coreCount: 6 deviceMemorySize: 3.95GiB deviceMemoryBandwidth: 104.43GiB/s
2021-08-04 09:05:58.362536: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-04 09:05:58.362903: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-04 09:05:58.363168: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1871] Adding visible gpu devices: 0
2021-08-04 09:05:58.363670: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcudart.so.11.0
2021-08-04 09:05:59.532060: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1258] Device interconnect StreamExecutor with strength 1 edge matrix:
2021-08-04 09:05:59.532138: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1264]      0 
2021-08-04 09:05:59.532165: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1277] 0:   N 
2021-08-04 09:05:59.532560: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-04 09:05:59.533604: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-04 09:05:59.534597: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-04 09:05:59.535497: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1418] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 2896 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce GTX 1050 Ti, pci bus id: 0000:01:00.0, compute capability: 6.1)
x.shape = (1965, 28, 28, 1), y.shape = (1965, 10)
2021-08-04 09:06:01.174218: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:176] None of the MLIR Optimization Passes are enabled (registered 2)
2021-08-04 09:06:01.196600: I tensorflow/core/platform/profile_utils/cpu_utils.cc:114] CPU Frequency: 2299965000 Hz
Epoch 1/500
data type: <class 'tuple'>
(<tf.Tensor 'IteratorGetNext:0' shape=(None, 28, 28, 1) dtype=uint8>, <tf.Tensor 'IteratorGetNext:1' shape=(None, 10) dtype=float32>)
(None, 28, 28, 1)
data type: <class 'tuple'>
(<tf.Tensor 'IteratorGetNext:0' shape=(None, 28, 28, 1) dtype=uint8>, <tf.Tensor 'IteratorGetNext:1' shape=(None, 10) dtype=float32>)
(None, 28, 28, 1)
2021-08-04 09:06:01.823192: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcudnn.so.8
2021-08-04 09:06:02.627137: I tensorflow/stream_executor/cuda/cuda_dnn.cc:359] Loaded cuDNN version 8101
2021-08-04 09:06:03.810120: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcublas.so.11
2021-08-04 09:06:04.736986: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcublasLt.so.11
62/62 [==============================] - 4s 5ms/step - loss: 0.4304 - accuracy: 0.8350
Epoch 2/500
62/62 [==============================] - 0s 4ms/step - loss: 0.1316 - accuracy: 0.9237
Epoch 3/500
62/62 [==============================] - 0s 4ms/step - loss: 0.0837 - accuracy: 0.9487
Model: "LeNet"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
Conv_1 (Conv2D)              (None, 28, 28, 32)        320       
_________________________________________________________________
Pool_1 (AveragePooling2D)    (None, 14, 14, 32)        0         
_________________________________________________________________
Conv_2 (Conv2D)              (None, 14, 14, 64)        18496     
_________________________________________________________________
Pool_2 (AveragePooling2D)    (None, 7, 7, 64)          0         
_________________________________________________________________
flatten (Flatten)            (None, 3136)              0         
_________________________________________________________________
Dense_1 (Dense)              (None, 64)                200768    
_________________________________________________________________
Dense_2 (Dense)              (None, 10)                650       
=================================================================
Total params: 220,234
Trainable params: 220,234
Non-trainable params: 0
_________________________________________________________________
[0.0044, 0.1280, 0.0010, 0.8811, 0.0001, 0.2038, 0.0064, 0.0054, 0.0051, 0.0099]::3
[0.0001, 0.0000, 0.0001, 0.0008, 0.0056, 0.0001, 0.0000, 0.1023, 0.0061, 0.6481]::9
[0.0001, 0.0000, 0.9795, 0.0185, 0.0305, 0.0000, 0.0010, 0.0003, 0.0001, 0.0873]::2
[0.0161, 0.0018, 0.0000, 0.0052, 0.0000, 0.0058, 0.0000, 0.9935, 0.0004, 0.0060]::7
[0.0040, 0.9984, 0.0287, 0.0160, 0.0410, 0.0005, 0.0075, 0.0049, 0.0087, 0.0022]::1
[0.0893, 0.1610, 0.0002, 0.2568, 0.0014, 0.8967, 0.0010, 0.0109, 0.0177, 0.0115]::5
[0.0028, 0.0003, 0.0004, 0.2713, 0.0008, 0.8692, 0.0105, 0.0001, 0.0055, 0.0221]::5
[0.0014, 0.0012, 0.0216, 0.0056, 0.0007, 0.0193, 0.0005, 0.0007, 0.8914, 0.0003]::8
[0.0061, 0.9460, 0.0034, 0.0133, 0.0300, 0.0054, 0.0115, 0.0127, 0.0174, 0.0053]::1
[0.0028, 0.0010, 0.0003, 0.9993, 0.0000, 0.0172, 0.0000, 0.0008, 0.0090, 0.0004]::3
[0.9639, 0.0001, 0.0032, 0.0000, 0.0049, 0.1168, 0.0001, 0.0000, 0.0016, 0.0001]::0
[0.0001, 0.0033, 0.0523, 0.1046, 0.0034, 0.6914, 0.0008, 0.0004, 0.0016, 0.0005]::3
[0.0004, 0.0003, 0.0002, 0.0000, 0.9466, 0.0015, 0.0010, 0.0205, 0.0126, 0.0938]::4
[0.0105, 0.0015, 0.0085, 0.0000, 0.0357, 0.0018, 0.0015, 0.0200, 0.0046, 0.5125]::9
[0.0166, 0.9857, 0.0070, 0.0126, 0.0316, 0.0035, 0.0089, 0.0130, 0.0227, 0.0047]::1
[0.8887, 0.0017, 0.0005, 0.0003, 0.0023, 0.0275, 0.0003, 0.0053, 0.0046, 0.0181]::0
[0.0037, 0.0614, 0.0001, 0.0011, 0.0011, 0.0364, 0.0002, 0.9784, 0.0013, 0.0498]::7
[0.0002, 0.0008, 0.0013, 0.0000, 0.0028, 0.0069, 0.5261, 0.0000, 0.0002, 0.0022]::6
[0.0001, 0.0087, 0.0025, 0.0002, 0.0661, 0.0011, 0.0017, 0.4763, 0.0150, 0.0057]::7
[0.0144, 0.0000, 0.0007, 0.0091, 0.0000, 0.2172, 0.0046, 0.0038, 0.0680, 0.0001]::5
[0.0004, 0.0013, 0.0149, 0.0000, 0.0067, 0.0162, 0.9983, 0.0026, 0.0000, 0.0034]::6
[0.0022, 0.3633, 0.0997, 0.0494, 0.0009, 0.0017, 0.0013, 0.0129, 0.1099, 0.0021]::2
[0.0019, 0.0000, 0.0000, 0.0000, 0.0090, 0.0001, 0.0003, 0.0263, 0.0016, 0.2663]::9
[0.0003, 0.0003, 0.0000, 0.0000, 0.0885, 0.0004, 0.0001, 0.0871, 0.0227, 0.7294]::9
[0.9997, 0.0092, 0.0007, 0.0000, 0.0075, 0.0057, 0.0000, 0.0001, 0.1147, 0.0031]::0
[0.8477, 0.0006, 0.0023, 0.0000, 0.0007, 0.0003, 0.0131, 0.0013, 0.0001, 0.0027]::0
[0.0299, 0.0032, 0.0000, 0.0001, 0.0001, 0.0130, 0.0000, 0.9598, 0.0039, 0.0085]::7
[0.0004, 0.0000, 0.9995, 0.0233, 0.0001, 0.0000, 0.0000, 0.0000, 0.0009, 0.0000]::2
[0.0008, 0.0116, 0.0121, 0.6877, 0.0000, 0.2762, 0.0910, 0.0002, 0.0004, 0.0003]::3
[0.0000, 0.6402, 0.0112, 0.1691, 0.0003, 0.0108, 0.4377, 0.0317, 0.0018, 0.0021]::1

标签:04,自定义,08,09,step,train,2021,0.0000,tensorflow
来源: https://blog.csdn.net/dscn15848078969/article/details/119344442

本站声明: 1. iCode9 技术分享网(下文简称本站)提供的所有内容,仅供技术学习、探讨和分享;
2. 关于本站的所有留言、评论、转载及引用,纯属内容发起人的个人观点,与本站观点和立场无关;
3. 关于本站的所有言论和文字,纯属内容发起人的个人观点,与本站观点和立场无关;
4. 本站文章均是网友提供,不完全保证技术分享内容的完整性、准确性、时效性、风险性和版权归属;如您发现该文章侵犯了您的权益,可联系我们第一时间进行删除;
5. 本站为非盈利性的个人网站,所有内容不会用来进行牟利,也不会利用任何形式的广告来间接获益,纯粹是为了广大技术爱好者提供技术内容和技术思想的分享性交流网站。

专注分享技术,共同学习,共同进步。侵权联系[81616952@qq.com]

Copyright (C)ICode9.com, All Rights Reserved.

ICode9版权所有