标签：acc 10 val loss 调参 step ReLU Epoch 1000

本文在调参记录9的基础上，在数据增强部分添加了shear_range = 30，测试Adaptively Parametric ReLU（APReLU）激活函数在Cifar10图像集上的效果。

Keras里ImageDataGenerator的用法见如下网址：
https://fairyonice.github.io/Learn-about-ImageDataGenerator.html

深度残差网络+自适应参数化ReLU激活函数（调参记录9）
https://blog.csdn.net/dangqing1988/article/details/105688144

自适应参数化ReLU激活函数的基本原理见下图：

Keras程序如下：

  1 #!/usr/bin/env python3
  2 # -*- coding: utf-8 -*-
  3 """
  4 Created on Tue Apr 14 04:17:45 2020
  5 Implemented using TensorFlow 1.0.1 and Keras 2.2.1
  6 
  7 Minghang Zhao, Shisheng Zhong, Xuyun Fu, Baoping Tang, Shaojiang Dong, Michael Pecht,
  8 Deep Residual Networks with Adaptively Parametric Rectifier Linear Units for Fault Diagnosis, 
  9 IEEE Transactions on Industrial Electronics, 2020,  DOI: 10.1109/TIE.2020.2972458 
 10 
 11 @author: Minghang Zhao
 12 """
 13 
 14 from __future__ import print_function
 15 import keras
 16 import numpy as np
 17 from keras.datasets import cifar10
 18 from keras.layers import Dense, Conv2D, BatchNormalization, Activation, Minimum
 19 from keras.layers import AveragePooling2D, Input, GlobalAveragePooling2D, Concatenate, Reshape
 20 from keras.regularizers import l2
 21 from keras import backend as K
 22 from keras.models import Model
 23 from keras import optimizers
 24 from keras.preprocessing.image import ImageDataGenerator
 25 from keras.callbacks import LearningRateScheduler
 26 K.set_learning_phase(1)
 27 
 28 # The data, split between train and test sets
 29 (x_train, y_train), (x_test, y_test) = cifar10.load_data()
 30 
 31 # Noised data
 32 x_train = x_train.astype('float32') / 255.
 33 x_test = x_test.astype('float32') / 255.
 34 x_test = x_test-np.mean(x_train)
 35 x_train = x_train-np.mean(x_train)
 36 print('x_train shape:', x_train.shape)
 37 print(x_train.shape[0], 'train samples')
 38 print(x_test.shape[0], 'test samples')
 39 
 40 # convert class vectors to binary class matrices
 41 y_train = keras.utils.to_categorical(y_train, 10)
 42 y_test = keras.utils.to_categorical(y_test, 10)
 43 
 44 # Schedule the learning rate, multiply 0.1 every 300 epoches
 45 def scheduler(epoch):
 46     if epoch % 300 == 0 and epoch != 0:
 47         lr = K.get_value(model.optimizer.lr)
 48         K.set_value(model.optimizer.lr, lr * 0.1)
 49         print("lr changed to {}".format(lr * 0.1))
 50     return K.get_value(model.optimizer.lr)
 51 
 52 # An adaptively parametric rectifier linear unit (APReLU)
 53 def aprelu(inputs):
 54     # get the number of channels
 55     channels = inputs.get_shape().as_list()[-1]
 56     # get a zero feature map
 57     zeros_input = keras.layers.subtract([inputs, inputs])
 58     # get a feature map with only positive features
 59     pos_input = Activation('relu')(inputs)
 60     # get a feature map with only negative features
 61     neg_input = Minimum()([inputs,zeros_input])
 62     # define a network to obtain the scaling coefficients
 63     scales_p = GlobalAveragePooling2D()(pos_input)
 64     scales_n = GlobalAveragePooling2D()(neg_input)
 65     scales = Concatenate()([scales_n, scales_p])
 66     scales = Dense(channels, activation='linear', kernel_initializer='he_normal', kernel_regularizer=l2(1e-4))(scales)
 67     scales = BatchNormalization(momentum=0.9, gamma_regularizer=l2(1e-4))(scales)
 68     scales = Activation('relu')(scales)
 69     scales = Dense(channels, activation='linear', kernel_initializer='he_normal', kernel_regularizer=l2(1e-4))(scales)
 70     scales = BatchNormalization(momentum=0.9, gamma_regularizer=l2(1e-4))(scales)
 71     scales = Activation('sigmoid')(scales)
 72     scales = Reshape((1,1,channels))(scales)
 73     # apply a paramtetric relu
 74     neg_part = keras.layers.multiply([scales, neg_input])
 75     return keras.layers.add([pos_input, neg_part])
 76 
 77 # Residual Block
 78 def residual_block(incoming, nb_blocks, out_channels, downsample=False,
 79                    downsample_strides=2):
 80     
 81     residual = incoming
 82     in_channels = incoming.get_shape().as_list()[-1]
 83     
 84     for i in range(nb_blocks):
 85         
 86         identity = residual
 87         
 88         if not downsample:
 89             downsample_strides = 1
 90         
 91         residual = BatchNormalization(momentum=0.9, gamma_regularizer=l2(1e-4))(residual)
 92         residual = aprelu(residual)
 93         residual = Conv2D(out_channels, 3, strides=(downsample_strides, downsample_strides), 
 94                           padding='same', kernel_initializer='he_normal', 
 95                           kernel_regularizer=l2(1e-4))(residual)
 96         
 97         residual = BatchNormalization(momentum=0.9, gamma_regularizer=l2(1e-4))(residual)
 98         residual = aprelu(residual)
 99         residual = Conv2D(out_channels, 3, padding='same', kernel_initializer='he_normal', 
100                           kernel_regularizer=l2(1e-4))(residual)
101         
102         # Downsampling
103         if downsample_strides > 1:
104             identity = AveragePooling2D(pool_size=(1,1), strides=(2,2))(identity)
105             
106         # Zero_padding to match channels
107         if in_channels != out_channels:
108             zeros_identity = keras.layers.subtract([identity, identity])
109             identity = keras.layers.concatenate([identity, zeros_identity])
110             in_channels = out_channels
111         
112         residual = keras.layers.add([residual, identity])
113     
114     return residual
115 
116 
117 # define and train a model
118 inputs = Input(shape=(32, 32, 3))
119 net = Conv2D(16, 3, padding='same', kernel_initializer='he_normal', kernel_regularizer=l2(1e-4))(inputs)
120 net = residual_block(net, 9, 16, downsample=False)
121 net = residual_block(net, 1, 32, downsample=True)
122 net = residual_block(net, 8, 32, downsample=False)
123 net = residual_block(net, 1, 64, downsample=True)
124 net = residual_block(net, 8, 64, downsample=False)
125 net = BatchNormalization(momentum=0.9, gamma_regularizer=l2(1e-4))(net)
126 net = Activation('relu')(net)
127 net = GlobalAveragePooling2D()(net)
128 outputs = Dense(10, activation='softmax', kernel_initializer='he_normal', kernel_regularizer=l2(1e-4))(net)
129 model = Model(inputs=inputs, outputs=outputs)
130 sgd = optimizers.SGD(lr=0.1, decay=0., momentum=0.9, nesterov=True)
131 model.compile(loss='categorical_crossentropy', optimizer=sgd, metrics=['accuracy'])
132 
133 # data augmentation
134 datagen = ImageDataGenerator(
135     # randomly rotate images in the range (deg 0 to 180)
136     rotation_range=30,
137     # shear angle in counter-clockwise direction in degrees
138     shear_range = 30,
139     # randomly flip images
140     horizontal_flip=True,
141     # randomly shift images horizontally
142     width_shift_range=0.125,
143     # randomly shift images vertically
144     height_shift_range=0.125)
145 
146 reduce_lr = LearningRateScheduler(scheduler)
147 # fit the model on the batches generated by datagen.flow().
148 model.fit_generator(datagen.flow(x_train, y_train, batch_size=100),
149                     validation_data=(x_test, y_test), epochs=1000, 
150                     verbose=1, callbacks=[reduce_lr], workers=4)
151 
152 # get results
153 K.set_learning_phase(0)
154 DRSN_train_score = model.evaluate(x_train, y_train, batch_size=100, verbose=0)
155 print('Train loss:', DRSN_train_score[0])
156 print('Train accuracy:', DRSN_train_score[1])
157 DRSN_test_score = model.evaluate(x_test, y_test, batch_size=100, verbose=0)
158 print('Test loss:', DRSN_test_score[0])
159 print('Test accuracy:', DRSN_test_score[1])

实验结果如下：

   1 x_train shape: (50000, 32, 32, 3)
   2 50000 train samples
   3 10000 test samples
   4 Epoch 1/1000
   5 113s 225ms/step - loss: 3.2549 - acc: 0.4158 - val_loss: 2.7729 - val_acc: 0.5394
   6 Epoch 2/1000
   7 68s 137ms/step - loss: 2.6403 - acc: 0.5484 - val_loss: 2.3416 - val_acc: 0.6117
   8 Epoch 3/1000
   9 69s 138ms/step - loss: 2.2763 - acc: 0.6049 - val_loss: 2.0151 - val_acc: 0.6705
  10 Epoch 4/1000
  11 69s 137ms/step - loss: 2.0062 - acc: 0.6393 - val_loss: 1.8055 - val_acc: 0.6907
  12 Epoch 5/1000
  13 69s 137ms/step - loss: 1.7997 - acc: 0.6673 - val_loss: 1.6339 - val_acc: 0.7058
  14 Epoch 6/1000
  15 69s 138ms/step - loss: 1.6338 - acc: 0.6849 - val_loss: 1.4391 - val_acc: 0.7345
  16 Epoch 7/1000
  17 69s 138ms/step - loss: 1.4911 - acc: 0.7032 - val_loss: 1.3495 - val_acc: 0.7435
  18 Epoch 8/1000
  19 69s 138ms/step - loss: 1.3733 - acc: 0.7196 - val_loss: 1.2311 - val_acc: 0.7668
  20 Epoch 9/1000
  21 68s 137ms/step - loss: 1.2893 - acc: 0.7308 - val_loss: 1.1543 - val_acc: 0.7741
  22 Epoch 10/1000
  23 68s 137ms/step - loss: 1.2164 - acc: 0.7402 - val_loss: 1.0974 - val_acc: 0.7761
  24 Epoch 11/1000
  25 69s 137ms/step - loss: 1.1580 - acc: 0.7470 - val_loss: 1.0477 - val_acc: 0.7835
  26 Epoch 12/1000
  27 69s 137ms/step - loss: 1.1127 - acc: 0.7519 - val_loss: 1.0269 - val_acc: 0.7813
  28 Epoch 13/1000
  29 69s 138ms/step - loss: 1.0713 - acc: 0.7598 - val_loss: 0.9656 - val_acc: 0.7996
  30 Epoch 14/1000
  31 68s 136ms/step - loss: 1.0369 - acc: 0.7664 - val_loss: 0.9576 - val_acc: 0.7929
  32 Epoch 15/1000
  33 68s 135ms/step - loss: 1.0158 - acc: 0.7677 - val_loss: 0.9189 - val_acc: 0.8064
  34 Epoch 16/1000
  35 68s 135ms/step - loss: 0.9948 - acc: 0.7733 - val_loss: 0.9198 - val_acc: 0.8022
  36 Epoch 17/1000
  37 68s 136ms/step - loss: 0.9720 - acc: 0.7775 - val_loss: 0.9267 - val_acc: 0.7954
  38 Epoch 18/1000
  39 68s 135ms/step - loss: 0.9548 - acc: 0.7813 - val_loss: 0.8897 - val_acc: 0.8043
  40 Epoch 19/1000
  41 68s 135ms/step - loss: 0.9446 - acc: 0.7847 - val_loss: 0.8642 - val_acc: 0.8104
  42 Epoch 20/1000
  43 68s 135ms/step - loss: 0.9290 - acc: 0.7873 - val_loss: 0.8666 - val_acc: 0.8119
  44 Epoch 21/1000
  45 68s 135ms/step - loss: 0.9131 - acc: 0.7913 - val_loss: 0.8433 - val_acc: 0.8202
  46 Epoch 22/1000
  47 68s 135ms/step - loss: 0.9099 - acc: 0.7912 - val_loss: 0.8735 - val_acc: 0.8077
  48 Epoch 23/1000
  49 67s 135ms/step - loss: 0.9000 - acc: 0.7956 - val_loss: 0.8418 - val_acc: 0.8150
  50 Epoch 24/1000
  51 68s 135ms/step - loss: 0.8962 - acc: 0.7966 - val_loss: 0.8452 - val_acc: 0.8181
  52 Epoch 25/1000
  53 68s 135ms/step - loss: 0.8874 - acc: 0.7994 - val_loss: 0.8209 - val_acc: 0.8242
  54 Epoch 26/1000
  55 68s 136ms/step - loss: 0.8810 - acc: 0.8021 - val_loss: 0.8378 - val_acc: 0.8202
  56 Epoch 27/1000
  57 68s 135ms/step - loss: 0.8764 - acc: 0.8026 - val_loss: 0.8474 - val_acc: 0.8173
  58 Epoch 28/1000
  59 67s 135ms/step - loss: 0.8706 - acc: 0.8040 - val_loss: 0.8239 - val_acc: 0.8230
  60 Epoch 29/1000
  61 68s 135ms/step - loss: 0.8655 - acc: 0.8075 - val_loss: 0.8163 - val_acc: 0.8244
  62 Epoch 30/1000
  63 68s 135ms/step - loss: 0.8600 - acc: 0.8074 - val_loss: 0.8065 - val_acc: 0.8288
  64 Epoch 31/1000
  65 68s 135ms/step - loss: 0.8544 - acc: 0.8113 - val_loss: 0.8080 - val_acc: 0.8306
  66 Epoch 32/1000
  67 68s 135ms/step - loss: 0.8510 - acc: 0.8121 - val_loss: 0.8152 - val_acc: 0.8304
  68 Epoch 33/1000
  69 68s 135ms/step - loss: 0.8464 - acc: 0.8142 - val_loss: 0.7827 - val_acc: 0.8387
  70 Epoch 34/1000
  71 68s 135ms/step - loss: 0.8429 - acc: 0.8166 - val_loss: 0.7738 - val_acc: 0.8453
  72 Epoch 35/1000
  73 68s 135ms/step - loss: 0.8366 - acc: 0.8160 - val_loss: 0.7855 - val_acc: 0.8388
  74 Epoch 36/1000
  75 68s 135ms/step - loss: 0.8352 - acc: 0.8191 - val_loss: 0.7651 - val_acc: 0.8468
  76 Epoch 37/1000
  77 68s 135ms/step - loss: 0.8292 - acc: 0.8212 - val_loss: 0.7620 - val_acc: 0.8470
  78 Epoch 38/1000
  79 68s 135ms/step - loss: 0.8319 - acc: 0.8208 - val_loss: 0.7890 - val_acc: 0.8376
  80 Epoch 39/1000
  81 68s 136ms/step - loss: 0.8239 - acc: 0.8256 - val_loss: 0.7870 - val_acc: 0.8370
  82 Epoch 40/1000
  83 68s 135ms/step - loss: 0.8266 - acc: 0.8216 - val_loss: 0.7975 - val_acc: 0.8331
  84 Epoch 41/1000
  85 68s 135ms/step - loss: 0.8209 - acc: 0.8239 - val_loss: 0.7982 - val_acc: 0.8334
  86 Epoch 42/1000
  87 68s 135ms/step - loss: 0.8135 - acc: 0.8276 - val_loss: 0.7722 - val_acc: 0.8427
  88 Epoch 43/1000
  89 68s 135ms/step - loss: 0.8115 - acc: 0.8280 - val_loss: 0.7658 - val_acc: 0.8430
  90 Epoch 44/1000
  91 67s 135ms/step - loss: 0.8166 - acc: 0.8259 - val_loss: 0.7388 - val_acc: 0.8559
  92 Epoch 45/1000
  93 67s 135ms/step - loss: 0.8108 - acc: 0.8293 - val_loss: 0.7728 - val_acc: 0.8436
  94 Epoch 46/1000
  95 68s 135ms/step - loss: 0.8046 - acc: 0.8303 - val_loss: 0.7684 - val_acc: 0.8434
  96 Epoch 47/1000
  97 68s 136ms/step - loss: 0.8055 - acc: 0.8322 - val_loss: 0.7478 - val_acc: 0.8511
  98 Epoch 48/1000
  99 68s 135ms/step - loss: 0.8100 - acc: 0.8290 - val_loss: 0.7644 - val_acc: 0.8445
 100 Epoch 49/1000
 101 68s 135ms/step - loss: 0.8027 - acc: 0.8325 - val_loss: 0.7449 - val_acc: 0.8545
 102 Epoch 50/1000
 103 67s 135ms/step - loss: 0.8052 - acc: 0.8299 - val_loss: 0.7941 - val_acc: 0.8377
 104 Epoch 51/1000
 105 68s 135ms/step - loss: 0.7969 - acc: 0.8339 - val_loss: 0.7617 - val_acc: 0.8481
 106 Epoch 52/1000
 107 68s 135ms/step - loss: 0.7989 - acc: 0.8335 - val_loss: 0.7559 - val_acc: 0.8550
 108 Epoch 53/1000
 109 68s 136ms/step - loss: 0.7927 - acc: 0.8353 - val_loss: 0.7482 - val_acc: 0.8536
 110 Epoch 54/1000
 111 68s 135ms/step - loss: 0.7931 - acc: 0.8365 - val_loss: 0.7405 - val_acc: 0.8570
 112 Epoch 55/1000
 113 68s 135ms/step - loss: 0.7933 - acc: 0.8372 - val_loss: 0.7541 - val_acc: 0.8535
 114 Epoch 56/1000
 115 68s 135ms/step - loss: 0.7887 - acc: 0.8389 - val_loss: 0.7805 - val_acc: 0.8436
 116 Epoch 57/1000
 117 68s 135ms/step - loss: 0.7877 - acc: 0.8385 - val_loss: 0.7304 - val_acc: 0.8617
 118 Epoch 58/1000
 119 68s 135ms/step - loss: 0.7836 - acc: 0.8404 - val_loss: 0.7630 - val_acc: 0.8480
 120 Epoch 59/1000
 121 68s 135ms/step - loss: 0.7859 - acc: 0.8394 - val_loss: 0.7369 - val_acc: 0.8568
 122 Epoch 60/1000
 123 68s 135ms/step - loss: 0.7864 - acc: 0.8376 - val_loss: 0.7606 - val_acc: 0.8492
 124 Epoch 61/1000
 125 68s 135ms/step - loss: 0.7827 - acc: 0.8401 - val_loss: 0.7497 - val_acc: 0.8524
 126 Epoch 62/1000
 127 68s 135ms/step - loss: 0.7804 - acc: 0.8427 - val_loss: 0.7526 - val_acc: 0.8559
 128 Epoch 63/1000
 129 68s 135ms/step - loss: 0.7766 - acc: 0.8435 - val_loss: 0.7448 - val_acc: 0.8586
 130 Epoch 64/1000
 131 68s 135ms/step - loss: 0.7792 - acc: 0.8419 - val_loss: 0.7605 - val_acc: 0.8511
 132 Epoch 65/1000
 133 68s 135ms/step - loss: 0.7790 - acc: 0.8435 - val_loss: 0.7330 - val_acc: 0.8551
 134 Epoch 66/1000
 135 68s 135ms/step - loss: 0.7748 - acc: 0.8435 - val_loss: 0.7528 - val_acc: 0.8543
 136 Epoch 67/1000
 137 68s 135ms/step - loss: 0.7733 - acc: 0.8452 - val_loss: 0.7330 - val_acc: 0.8585
 138 Epoch 68/1000
 139 68s 135ms/step - loss: 0.7759 - acc: 0.8438 - val_loss: 0.7497 - val_acc: 0.8520
 140 Epoch 69/1000
 141 68s 135ms/step - loss: 0.7680 - acc: 0.8466 - val_loss: 0.7422 - val_acc: 0.8606
 142 Epoch 70/1000
 143 68s 135ms/step - loss: 0.7662 - acc: 0.8473 - val_loss: 0.7185 - val_acc: 0.8633
 144 Epoch 71/1000
 145 68s 135ms/step - loss: 0.7658 - acc: 0.8467 - val_loss: 0.7170 - val_acc: 0.8657
 146 Epoch 72/1000
 147 68s 135ms/step - loss: 0.7681 - acc: 0.8464 - val_loss: 0.7325 - val_acc: 0.8600
 148 Epoch 73/1000
 149 68s 135ms/step - loss: 0.7658 - acc: 0.8477 - val_loss: 0.7109 - val_acc: 0.8662
 150 Epoch 74/1000
 151 68s 135ms/step - loss: 0.7616 - acc: 0.8499 - val_loss: 0.7028 - val_acc: 0.8733
 152 Epoch 75/1000
 153 68s 135ms/step - loss: 0.7621 - acc: 0.8482 - val_loss: 0.7178 - val_acc: 0.8639
 154 Epoch 76/1000
 155 68s 135ms/step - loss: 0.7606 - acc: 0.8496 - val_loss: 0.7096 - val_acc: 0.8674
 156 Epoch 77/1000
 157 68s 135ms/step - loss: 0.7590 - acc: 0.8500 - val_loss: 0.7340 - val_acc: 0.8598
 158 Epoch 78/1000
 159 68s 135ms/step - loss: 0.7639 - acc: 0.8475 - val_loss: 0.7212 - val_acc: 0.8655
 160 Epoch 79/1000
 161 68s 135ms/step - loss: 0.7613 - acc: 0.8477 - val_loss: 0.7171 - val_acc: 0.8702
 162 Epoch 80/1000
 163 67s 135ms/step - loss: 0.7562 - acc: 0.8518 - val_loss: 0.7336 - val_acc: 0.8594
 164 Epoch 81/1000
 165 68s 136ms/step - loss: 0.7532 - acc: 0.8515 - val_loss: 0.7229 - val_acc: 0.8607
 166 Epoch 82/1000
 167 68s 135ms/step - loss: 0.7511 - acc: 0.8541 - val_loss: 0.7062 - val_acc: 0.8688
 168 Epoch 83/1000
 169 68s 135ms/step - loss: 0.7510 - acc: 0.8530 - val_loss: 0.6977 - val_acc: 0.8746
 170 Epoch 84/1000
 171 68s 135ms/step - loss: 0.7562 - acc: 0.8524 - val_loss: 0.7319 - val_acc: 0.8595
 172 Epoch 85/1000
 173 67s 135ms/step - loss: 0.7527 - acc: 0.8530 - val_loss: 0.7161 - val_acc: 0.8660
 174 Epoch 86/1000
 175 67s 135ms/step - loss: 0.7523 - acc: 0.8524 - val_loss: 0.7244 - val_acc: 0.8654
 176 Epoch 87/1000
 177 67s 135ms/step - loss: 0.7505 - acc: 0.8532 - val_loss: 0.7192 - val_acc: 0.8636
 178 Epoch 88/1000
 179 68s 135ms/step - loss: 0.7528 - acc: 0.8516 - val_loss: 0.7316 - val_acc: 0.8645
 180 Epoch 89/1000
 181 68s 135ms/step - loss: 0.7480 - acc: 0.8557 - val_loss: 0.7289 - val_acc: 0.8638
 182 Epoch 90/1000
 183 68s 135ms/step - loss: 0.7435 - acc: 0.8550 - val_loss: 0.7020 - val_acc: 0.8763
 184 Epoch 91/1000
 185 68s 135ms/step - loss: 0.7466 - acc: 0.8563 - val_loss: 0.6977 - val_acc: 0.8750
 186 Epoch 92/1000
 187 68s 135ms/step - loss: 0.7438 - acc: 0.8561 - val_loss: 0.7171 - val_acc: 0.8643
 188 Epoch 93/1000
 189 67s 135ms/step - loss: 0.7438 - acc: 0.8564 - val_loss: 0.7189 - val_acc: 0.8687
 190 Epoch 94/1000
 191 68s 135ms/step - loss: 0.7442 - acc: 0.8566 - val_loss: 0.7072 - val_acc: 0.8685
 192 Epoch 95/1000
 193 68s 135ms/step - loss: 0.7468 - acc: 0.8569 - val_loss: 0.7547 - val_acc: 0.8560
 194 Epoch 96/1000
 195 68s 135ms/step - loss: 0.7468 - acc: 0.8547 - val_loss: 0.7080 - val_acc: 0.8699
 196 Epoch 97/1000
 197 68s 135ms/step - loss: 0.7455 - acc: 0.8559 - val_loss: 0.7020 - val_acc: 0.8711
 198 Epoch 98/1000
 199 68s 135ms/step - loss: 0.7427 - acc: 0.8544 - val_loss: 0.7352 - val_acc: 0.8610
 200 Epoch 99/1000
 201 68s 136ms/step - loss: 0.7424 - acc: 0.8567 - val_loss: 0.7480 - val_acc: 0.8583
 202 Epoch 100/1000
 203 68s 135ms/step - loss: 0.7397 - acc: 0.8579 - val_loss: 0.7151 - val_acc: 0.8650
 204 Epoch 101/1000
 205 68s 135ms/step - loss: 0.7447 - acc: 0.8568 - val_loss: 0.7235 - val_acc: 0.8659
 206 Epoch 102/1000
 207 68s 135ms/step - loss: 0.7367 - acc: 0.8598 - val_loss: 0.7229 - val_acc: 0.8623
 208 Epoch 103/1000
 209 67s 135ms/step - loss: 0.7371 - acc: 0.8586 - val_loss: 0.6899 - val_acc: 0.8769
 210 Epoch 104/1000
 211 68s 135ms/step - loss: 0.7401 - acc: 0.8567 - val_loss: 0.7273 - val_acc: 0.8616
 212 Epoch 105/1000
 213 68s 135ms/step - loss: 0.7382 - acc: 0.8578 - val_loss: 0.7089 - val_acc: 0.8682
 214 Epoch 106/1000
 215 68s 135ms/step - loss: 0.7386 - acc: 0.8580 - val_loss: 0.7158 - val_acc: 0.8659
 216 Epoch 107/1000
 217 67s 135ms/step - loss: 0.7361 - acc: 0.8584 - val_loss: 0.7147 - val_acc: 0.8701
 218 Epoch 108/1000
 219 67s 135ms/step - loss: 0.7408 - acc: 0.8580 - val_loss: 0.7083 - val_acc: 0.8686
 220 Epoch 109/1000
 221 68s 135ms/step - loss: 0.7362 - acc: 0.8599 - val_loss: 0.7096 - val_acc: 0.8703
 222 Epoch 110/1000
 223 67s 135ms/step - loss: 0.7335 - acc: 0.8600 - val_loss: 0.7148 - val_acc: 0.8683
 224 Epoch 111/1000
 225 67s 135ms/step - loss: 0.7334 - acc: 0.8626 - val_loss: 0.7050 - val_acc: 0.8741
 226 Epoch 112/1000
 227 68s 135ms/step - loss: 0.7360 - acc: 0.8586 - val_loss: 0.7150 - val_acc: 0.8682
 228 Epoch 113/1000
 229 68s 136ms/step - loss: 0.7371 - acc: 0.8583 - val_loss: 0.7447 - val_acc: 0.8583
 230 Epoch 114/1000
 231 68s 135ms/step - loss: 0.7352 - acc: 0.8599 - val_loss: 0.6937 - val_acc: 0.8755
 232 Epoch 115/1000
 233 68s 135ms/step - loss: 0.7314 - acc: 0.8604 - val_loss: 0.7140 - val_acc: 0.8684
 234 Epoch 116/1000
 235 68s 135ms/step - loss: 0.7333 - acc: 0.8607 - val_loss: 0.7305 - val_acc: 0.8686
 236 Epoch 117/1000
 237 68s 135ms/step - loss: 0.7277 - acc: 0.8617 - val_loss: 0.7002 - val_acc: 0.8719
 238 Epoch 118/1000
 239 68s 135ms/step - loss: 0.7356 - acc: 0.8580 - val_loss: 0.6926 - val_acc: 0.8763
 240 Epoch 119/1000
 241 68s 135ms/step - loss: 0.7244 - acc: 0.8642 - val_loss: 0.7079 - val_acc: 0.8669
 242 Epoch 120/1000
 243 68s 136ms/step - loss: 0.7302 - acc: 0.8613 - val_loss: 0.7113 - val_acc: 0.8695
 244 Epoch 121/1000
 245 68s 135ms/step - loss: 0.7340 - acc: 0.8608 - val_loss: 0.7415 - val_acc: 0.8554
 246 Epoch 122/1000
 247 68s 135ms/step - loss: 0.7304 - acc: 0.8608 - val_loss: 0.6978 - val_acc: 0.8760
 248 Epoch 123/1000
 249 68s 135ms/step - loss: 0.7263 - acc: 0.8630 - val_loss: 0.6974 - val_acc: 0.8734
 250 Epoch 124/1000
 251 68s 135ms/step - loss: 0.7261 - acc: 0.8625 - val_loss: 0.7109 - val_acc: 0.8715
 252 Epoch 125/1000
 253 67s 135ms/step - loss: 0.7313 - acc: 0.8623 - val_loss: 0.6946 - val_acc: 0.8745
 254 Epoch 126/1000
 255 67s 135ms/step - loss: 0.7277 - acc: 0.8620 - val_loss: 0.7178 - val_acc: 0.8685
 256 Epoch 127/1000
 257 68s 135ms/step - loss: 0.7231 - acc: 0.8653 - val_loss: 0.6999 - val_acc: 0.8762
 258 Epoch 128/1000
 259 68s 135ms/step - loss: 0.7252 - acc: 0.8635 - val_loss: 0.7009 - val_acc: 0.8718
 260 Epoch 129/1000
 261 68s 135ms/step - loss: 0.7284 - acc: 0.8626 - val_loss: 0.7148 - val_acc: 0.8682
 262 Epoch 130/1000
 263 68s 135ms/step - loss: 0.7236 - acc: 0.8646 - val_loss: 0.6945 - val_acc: 0.8746
 264 Epoch 131/1000
 265 68s 135ms/step - loss: 0.7203 - acc: 0.8653 - val_loss: 0.7002 - val_acc: 0.8705
 266 Epoch 132/1000
 267 68s 135ms/step - loss: 0.7248 - acc: 0.8626 - val_loss: 0.7097 - val_acc: 0.8718
 268 Epoch 133/1000
 269 67s 135ms/step - loss: 0.7190 - acc: 0.8660 - val_loss: 0.6993 - val_acc: 0.8722
 270 Epoch 134/1000
 271 68s 136ms/step - loss: 0.7206 - acc: 0.8645 - val_loss: 0.7042 - val_acc: 0.8763
 272 Epoch 135/1000
 273 68s 135ms/step - loss: 0.7248 - acc: 0.8637 - val_loss: 0.6742 - val_acc: 0.8844
 274 Epoch 136/1000
 275 68s 135ms/step - loss: 0.7181 - acc: 0.8650 - val_loss: 0.6972 - val_acc: 0.8721
 276 Epoch 137/1000
 277 67s 135ms/step - loss: 0.7170 - acc: 0.8667 - val_loss: 0.7270 - val_acc: 0.8642
 278 Epoch 138/1000
 279 68s 135ms/step - loss: 0.7209 - acc: 0.8649 - val_loss: 0.7107 - val_acc: 0.8687
 280 Epoch 139/1000
 281 68s 136ms/step - loss: 0.7195 - acc: 0.8652 - val_loss: 0.6993 - val_acc: 0.8752
 282 Epoch 140/1000
 283 68s 135ms/step - loss: 0.7229 - acc: 0.8647 - val_loss: 0.6949 - val_acc: 0.8800
 284 Epoch 141/1000
 285 67s 135ms/step - loss: 0.7154 - acc: 0.8674 - val_loss: 0.6828 - val_acc: 0.8780
 286 Epoch 142/1000
 287 67s 135ms/step - loss: 0.7146 - acc: 0.8675 - val_loss: 0.6799 - val_acc: 0.8818
 288 Epoch 143/1000
 289 68s 135ms/step - loss: 0.7131 - acc: 0.8679 - val_loss: 0.7237 - val_acc: 0.8655
 290 Epoch 144/1000
 291 68s 135ms/step - loss: 0.7167 - acc: 0.8662 - val_loss: 0.7140 - val_acc: 0.8696
 292 Epoch 145/1000
 293 68s 136ms/step - loss: 0.7131 - acc: 0.8677 - val_loss: 0.7086 - val_acc: 0.8696
 294 Epoch 146/1000
 295 67s 135ms/step - loss: 0.7184 - acc: 0.8665 - val_loss: 0.7058 - val_acc: 0.8729
 296 Epoch 147/1000
 297 68s 135ms/step - loss: 0.7179 - acc: 0.8654 - val_loss: 0.7021 - val_acc: 0.8741
 298 Epoch 148/1000
 299 67s 135ms/step - loss: 0.7176 - acc: 0.8671 - val_loss: 0.6892 - val_acc: 0.8795
 300 Epoch 149/1000
 301 68s 135ms/step - loss: 0.7123 - acc: 0.8685 - val_loss: 0.7027 - val_acc: 0.8700
 302 Epoch 150/1000
 303 68s 136ms/step - loss: 0.7146 - acc: 0.8671 - val_loss: 0.6926 - val_acc: 0.8755
 304 Epoch 151/1000
 305 68s 135ms/step - loss: 0.7122 - acc: 0.8651 - val_loss: 0.7179 - val_acc: 0.8685
 306 Epoch 152/1000
 307 68s 136ms/step - loss: 0.7149 - acc: 0.8675 - val_loss: 0.7136 - val_acc: 0.8690
 308 Epoch 153/1000
 309 68s 135ms/step - loss: 0.7141 - acc: 0.8669 - val_loss: 0.7193 - val_acc: 0.8672
 310 Epoch 154/1000
 311 68s 136ms/step - loss: 0.7084 - acc: 0.8684 - val_loss: 0.6779 - val_acc: 0.8826
 312 Epoch 155/1000
 313 67s 135ms/step - loss: 0.7143 - acc: 0.8671 - val_loss: 0.7092 - val_acc: 0.8685
 314 Epoch 156/1000
 315 68s 136ms/step - loss: 0.7118 - acc: 0.8674 - val_loss: 0.7010 - val_acc: 0.8732
 316 Epoch 157/1000
 317 69s 138ms/step - loss: 0.7126 - acc: 0.8677 - val_loss: 0.6918 - val_acc: 0.8766
 318 Epoch 158/1000
 319 68s 137ms/step - loss: 0.7064 - acc: 0.8701 - val_loss: 0.7253 - val_acc: 0.8636
 320 Epoch 159/1000
 321 68s 137ms/step - loss: 0.7107 - acc: 0.8674 - val_loss: 0.7008 - val_acc: 0.8745
 322 Epoch 160/1000
 323 68s 137ms/step - loss: 0.7097 - acc: 0.8698 - val_loss: 0.6922 - val_acc: 0.8771
 324 Epoch 161/1000
 325 68s 137ms/step - loss: 0.7091 - acc: 0.8675 - val_loss: 0.6786 - val_acc: 0.8813
 326 Epoch 162/1000
 327 69s 138ms/step - loss: 0.7117 - acc: 0.8680 - val_loss: 0.7017 - val_acc: 0.8740
 328 Epoch 163/1000
 329 69s 137ms/step - loss: 0.7110 - acc: 0.8681 - val_loss: 0.6862 - val_acc: 0.8800
 330 Epoch 164/1000
 331 68s 137ms/step - loss: 0.7099 - acc: 0.8693 - val_loss: 0.7053 - val_acc: 0.8709
 332 Epoch 165/1000
 333 69s 138ms/step - loss: 0.7104 - acc: 0.8694 - val_loss: 0.6846 - val_acc: 0.8828
 334 Epoch 166/1000
 335 68s 136ms/step - loss: 0.7078 - acc: 0.8715 - val_loss: 0.6968 - val_acc: 0.8749
 336 Epoch 167/1000
 337 68s 136ms/step - loss: 0.7076 - acc: 0.8719 - val_loss: 0.6872 - val_acc: 0.8782
 338 Epoch 168/1000
 339 68s 136ms/step - loss: 0.7099 - acc: 0.8679 - val_loss: 0.6928 - val_acc: 0.8755
 340 Epoch 169/1000
 341 68s 136ms/step - loss: 0.7101 - acc: 0.8678 - val_loss: 0.6947 - val_acc: 0.8786
 342 Epoch 170/1000
 343 68s 137ms/step - loss: 0.7097 - acc: 0.8717 - val_loss: 0.6886 - val_acc: 0.8789
 344 Epoch 171/1000
 345 69s 137ms/step - loss: 0.7070 - acc: 0.8702 - val_loss: 0.6878 - val_acc: 0.8793
 346 Epoch 172/1000
 347 69s 137ms/step - loss: 0.7117 - acc: 0.8679 - val_loss: 0.6783 - val_acc: 0.8836
 348 Epoch 173/1000
 349 68s 137ms/step - loss: 0.7102 - acc: 0.8687 - val_loss: 0.6709 - val_acc: 0.8865
 350 Epoch 174/1000
 351 69s 137ms/step - loss: 0.7038 - acc: 0.8717 - val_loss: 0.6839 - val_acc: 0.8804
 352 Epoch 175/1000
 353 68s 137ms/step - loss: 0.7062 - acc: 0.8713 - val_loss: 0.6934 - val_acc: 0.8780
 354 Epoch 176/1000
 355 68s 137ms/step - loss: 0.7092 - acc: 0.8684 - val_loss: 0.7045 - val_acc: 0.8737
 356 Epoch 177/1000
 357 68s 137ms/step - loss: 0.7048 - acc: 0.8703 - val_loss: 0.6935 - val_acc: 0.8764
 358 Epoch 178/1000
 359 68s 137ms/step - loss: 0.7056 - acc: 0.8713 - val_loss: 0.6825 - val_acc: 0.8800
 360 Epoch 179/1000
 361 69s 137ms/step - loss: 0.7027 - acc: 0.8722 - val_loss: 0.6860 - val_acc: 0.8812
 362 Epoch 180/1000
 363 67s 135ms/step - loss: 0.7056 - acc: 0.8699 - val_loss: 0.6882 - val_acc: 0.8762
 364 Epoch 181/1000
 365 67s 135ms/step - loss: 0.6974 - acc: 0.8745 - val_loss: 0.7030 - val_acc: 0.8704
 366 Epoch 182/1000
 367 67s 135ms/step - loss: 0.7028 - acc: 0.8714 - val_loss: 0.6754 - val_acc: 0.8860
 368 Epoch 183/1000
 369 67s 135ms/step - loss: 0.7022 - acc: 0.8715 - val_loss: 0.6635 - val_acc: 0.8842
 370 Epoch 184/1000
 371 68s 136ms/step - loss: 0.7034 - acc: 0.8704 - val_loss: 0.6905 - val_acc: 0.8762
 372 Epoch 185/1000
 373 67s 135ms/step - loss: 0.7058 - acc: 0.8709 - val_loss: 0.7066 - val_acc: 0.8740
 374 Epoch 186/1000
 375 67s 135ms/step - loss: 0.7016 - acc: 0.8726 - val_loss: 0.6842 - val_acc: 0.8784
 376 Epoch 187/1000
 377 67s 135ms/step - loss: 0.6999 - acc: 0.8719 - val_loss: 0.7051 - val_acc: 0.8731
 378 Epoch 188/1000
 379 67s 135ms/step - loss: 0.7026 - acc: 0.8710 - val_loss: 0.6811 - val_acc: 0.8811
 380 Epoch 189/1000
 381 68s 135ms/step - loss: 0.7040 - acc: 0.8711 - val_loss: 0.6794 - val_acc: 0.8786
 382 Epoch 190/1000
 383 67s 135ms/step - loss: 0.7004 - acc: 0.8728 - val_loss: 0.6594 - val_acc: 0.8916
 384 Epoch 191/1000
 385 68s 136ms/step - loss: 0.6982 - acc: 0.8747 - val_loss: 0.6616 - val_acc: 0.8850
 386 Epoch 192/1000
 387 68s 135ms/step - loss: 0.7036 - acc: 0.8718 - val_loss: 0.6959 - val_acc: 0.8730
 388 Epoch 193/1000
 389 67s 135ms/step - loss: 0.7017 - acc: 0.8708 - val_loss: 0.6671 - val_acc: 0.8862
 390 Epoch 194/1000
 391 67s 135ms/step - loss: 0.6982 - acc: 0.8738 - val_loss: 0.6885 - val_acc: 0.8790
 392 Epoch 195/1000
 393 68s 136ms/step - loss: 0.6996 - acc: 0.8714 - val_loss: 0.6892 - val_acc: 0.8770
 394 Epoch 196/1000
 395 68s 136ms/step - loss: 0.7026 - acc: 0.8706 - val_loss: 0.6824 - val_acc: 0.8792
 396 Epoch 197/1000
 397 68s 136ms/step - loss: 0.7061 - acc: 0.8695 - val_loss: 0.6893 - val_acc: 0.8793
 398 Epoch 198/1000
 399 68s 135ms/step - loss: 0.7023 - acc: 0.8714 - val_loss: 0.6797 - val_acc: 0.8819
 400 Epoch 199/1000
 401 67s 135ms/step - loss: 0.7021 - acc: 0.8726 - val_loss: 0.6969 - val_acc: 0.8754
 402 Epoch 200/1000
 403 68s 136ms/step - loss: 0.7023 - acc: 0.8711 - val_loss: 0.6922 - val_acc: 0.8758
 404 Epoch 201/1000
 405 68s 135ms/step - loss: 0.7050 - acc: 0.8705 - val_loss: 0.6879 - val_acc: 0.8792
 406 Epoch 202/1000
 407 68s 135ms/step - loss: 0.7012 - acc: 0.8713 - val_loss: 0.6756 - val_acc: 0.8845
 408 Epoch 203/1000
 409 68s 136ms/step - loss: 0.7021 - acc: 0.8726 - val_loss: 0.6542 - val_acc: 0.8904
 410 Epoch 204/1000
 411 68s 136ms/step - loss: 0.6981 - acc: 0.8741 - val_loss: 0.7060 - val_acc: 0.8739
 412 Epoch 205/1000
 413 68s 135ms/step - loss: 0.7008 - acc: 0.8718 - val_loss: 0.6938 - val_acc: 0.8741
 414 Epoch 206/1000
 415 68s 136ms/step - loss: 0.6974 - acc: 0.8725 - val_loss: 0.6786 - val_acc: 0.8833
 416 Epoch 207/1000
 417 67s 135ms/step - loss: 0.6938 - acc: 0.8739 - val_loss: 0.6928 - val_acc: 0.8750
 418 Epoch 208/1000
 419 68s 135ms/step - loss: 0.7075 - acc: 0.8690 - val_loss: 0.6770 - val_acc: 0.8806
 420 Epoch 209/1000
 421 68s 136ms/step - loss: 0.6978 - acc: 0.8723 - val_loss: 0.6913 - val_acc: 0.8812
 422 Epoch 210/1000
 423 67s 135ms/step - loss: 0.6974 - acc: 0.8727 - val_loss: 0.6764 - val_acc: 0.8827
 424 Epoch 211/1000
 425 68s 136ms/step - loss: 0.6998 - acc: 0.8724 - val_loss: 0.7139 - val_acc: 0.8700
 426 Epoch 212/1000
 427 68s 136ms/step - loss: 0.6975 - acc: 0.8740 - val_loss: 0.6851 - val_acc: 0.8805
 428 Epoch 213/1000
 429 68s 136ms/step - loss: 0.7032 - acc: 0.8704 - val_loss: 0.7101 - val_acc: 0.8712
 430 Epoch 214/1000
 431 68s 135ms/step - loss: 0.6979 - acc: 0.8732 - val_loss: 0.7108 - val_acc: 0.8756
 432 Epoch 215/1000
 433 68s 136ms/step - loss: 0.6986 - acc: 0.8749 - val_loss: 0.7092 - val_acc: 0.8701
 434 Epoch 216/1000
 435 68s 135ms/step - loss: 0.6921 - acc: 0.8757 - val_loss: 0.6868 - val_acc: 0.8792
 436 Epoch 217/1000
 437 68s 135ms/step - loss: 0.6930 - acc: 0.8755 - val_loss: 0.7097 - val_acc: 0.8721
 438 Epoch 218/1000
 439 68s 135ms/step - loss: 0.7039 - acc: 0.8713 - val_loss: 0.6901 - val_acc: 0.8789
 440 Epoch 219/1000
 441 68s 136ms/step - loss: 0.6931 - acc: 0.8757 - val_loss: 0.6927 - val_acc: 0.8793
 442 Epoch 220/1000
 443 68s 136ms/step - loss: 0.6991 - acc: 0.8734 - val_loss: 0.6946 - val_acc: 0.8777
 444 Epoch 221/1000
 445 68s 135ms/step - loss: 0.6950 - acc: 0.8739 - val_loss: 0.6740 - val_acc: 0.8819
 446 Epoch 222/1000
 447 68s 136ms/step - loss: 0.6970 - acc: 0.8739 - val_loss: 0.6847 - val_acc: 0.8787
 448 Epoch 223/1000
 449 68s 136ms/step - loss: 0.7006 - acc: 0.8719 - val_loss: 0.7139 - val_acc: 0.8707
 450 Epoch 224/1000
 451 68s 135ms/step - loss: 0.7014 - acc: 0.8720 - val_loss: 0.6901 - val_acc: 0.8768
 452 Epoch 225/1000
 453 68s 136ms/step - loss: 0.6981 - acc: 0.8739 - val_loss: 0.6758 - val_acc: 0.8853
 454 Epoch 226/1000
 455 68s 136ms/step - loss: 0.6921 - acc: 0.8739 - val_loss: 0.6743 - val_acc: 0.8844
 456 Epoch 227/1000
 457 68s 136ms/step - loss: 0.6983 - acc: 0.8734 - val_loss: 0.7031 - val_acc: 0.8736
 458 Epoch 228/1000
 459 68s 136ms/step - loss: 0.6954 - acc: 0.8734 - val_loss: 0.7181 - val_acc: 0.8663
 460 Epoch 229/1000
 461 68s 136ms/step - loss: 0.6893 - acc: 0.8759 - val_loss: 0.6982 - val_acc: 0.8740
 462 Epoch 230/1000
 463 68s 135ms/step - loss: 0.6964 - acc: 0.8736 - val_loss: 0.6927 - val_acc: 0.8748
 464 Epoch 231/1000
 465 68s 136ms/step - loss: 0.6987 - acc: 0.8742 - val_loss: 0.6898 - val_acc: 0.8772
 466 Epoch 232/1000
 467 67s 135ms/step - loss: 0.6980 - acc: 0.8731 - val_loss: 0.6862 - val_acc: 0.8810
 468 Epoch 233/1000
 469 68s 136ms/step - loss: 0.6975 - acc: 0.8749 - val_loss: 0.6987 - val_acc: 0.8783
 470 Epoch 234/1000
 471 67s 135ms/step - loss: 0.6892 - acc: 0.8778 - val_loss: 0.6902 - val_acc: 0.8773
 472 Epoch 235/1000
 473 68s 135ms/step - loss: 0.6925 - acc: 0.8762 - val_loss: 0.6787 - val_acc: 0.8799
 474 Epoch 236/1000
 475 68s 136ms/step - loss: 0.6954 - acc: 0.8735 - val_loss: 0.6910 - val_acc: 0.8797
 476 Epoch 237/1000
 477 68s 136ms/step - loss: 0.6963 - acc: 0.8746 - val_loss: 0.6886 - val_acc: 0.8785
 478 Epoch 238/1000
 479 68s 136ms/step - loss: 0.6950 - acc: 0.8764 - val_loss: 0.7008 - val_acc: 0.8766
 480 Epoch 239/1000
 481 67s 135ms/step - loss: 0.6969 - acc: 0.8749 - val_loss: 0.7100 - val_acc: 0.8736
 482 Epoch 240/1000
 483 68s 136ms/step - loss: 0.6905 - acc: 0.8757 - val_loss: 0.6971 - val_acc: 0.8733
 484 Epoch 241/1000
 485 68s 136ms/step - loss: 0.6912 - acc: 0.8740 - val_loss: 0.6809 - val_acc: 0.8805
 486 Epoch 242/1000
 487 68s 136ms/step - loss: 0.6949 - acc: 0.8729 - val_loss: 0.6903 - val_acc: 0.8760
 488 Epoch 243/1000
 489 69s 138ms/step - loss: 0.6938 - acc: 0.8753 - val_loss: 0.6809 - val_acc: 0.8823
 490 Epoch 244/1000
 491 68s 136ms/step - loss: 0.6912 - acc: 0.8754 - val_loss: 0.6700 - val_acc: 0.8829
 492 Epoch 245/1000
 493 69s 137ms/step - loss: 0.6939 - acc: 0.8766 - val_loss: 0.6691 - val_acc: 0.8847
 494 Epoch 246/1000
 495 69s 138ms/step - loss: 0.6885 - acc: 0.8756 - val_loss: 0.7018 - val_acc: 0.8782
 496 Epoch 247/1000
 497 69s 138ms/step - loss: 0.6916 - acc: 0.8766 - val_loss: 0.6896 - val_acc: 0.8789
 498 Epoch 248/1000
 499 69s 138ms/step - loss: 0.6918 - acc: 0.8751 - val_loss: 0.7025 - val_acc: 0.8735
 500 Epoch 249/1000
 501 69s 138ms/step - loss: 0.6944 - acc: 0.8756 - val_loss: 0.6754 - val_acc: 0.8811
 502 Epoch 250/1000
 503 69s 138ms/step - loss: 0.6845 - acc: 0.8766 - val_loss: 0.6937 - val_acc: 0.8776
 504 Epoch 251/1000
 505 69s 138ms/step - loss: 0.6915 - acc: 0.8753 - val_loss: 0.6944 - val_acc: 0.8773
 506 Epoch 252/1000
 507 69s 138ms/step - loss: 0.6923 - acc: 0.8751 - val_loss: 0.6830 - val_acc: 0.8790
 508 Epoch 253/1000
 509 69s 138ms/step - loss: 0.6889 - acc: 0.8756 - val_loss: 0.7251 - val_acc: 0.8658
 510 Epoch 254/1000
 511 69s 137ms/step - loss: 0.6963 - acc: 0.8741 - val_loss: 0.6919 - val_acc: 0.8777
 512 Epoch 255/1000
 513 69s 137ms/step - loss: 0.6920 - acc: 0.8759 - val_loss: 0.7098 - val_acc: 0.8706
 514 Epoch 256/1000
 515 69s 138ms/step - loss: 0.6896 - acc: 0.8750 - val_loss: 0.6964 - val_acc: 0.8772
 516 Epoch 257/1000
 517 69s 137ms/step - loss: 0.6891 - acc: 0.8757 - val_loss: 0.6604 - val_acc: 0.8871
 518 Epoch 258/1000
 519 69s 137ms/step - loss: 0.6932 - acc: 0.8753 - val_loss: 0.6820 - val_acc: 0.8803
 520 Epoch 259/1000
 521 69s 138ms/step - loss: 0.6863 - acc: 0.8767 - val_loss: 0.7197 - val_acc: 0.8710
 522 Epoch 260/1000
 523 68s 137ms/step - loss: 0.6900 - acc: 0.8762 - val_loss: 0.6588 - val_acc: 0.8907
 524 Epoch 261/1000
 525 68s 137ms/step - loss: 0.6912 - acc: 0.8750 - val_loss: 0.6815 - val_acc: 0.8833
 526 Epoch 262/1000
 527 69s 137ms/step - loss: 0.6893 - acc: 0.8750 - val_loss: 0.6795 - val_acc: 0.8831
 528 Epoch 263/1000
 529 69s 137ms/step - loss: 0.6920 - acc: 0.8755 - val_loss: 0.6830 - val_acc: 0.8822
 530 Epoch 264/1000
 531 68s 137ms/step - loss: 0.6924 - acc: 0.8743 - val_loss: 0.6989 - val_acc: 0.8762
 532 Epoch 265/1000
 533 69s 138ms/step - loss: 0.6905 - acc: 0.8763 - val_loss: 0.6836 - val_acc: 0.8806
 534 Epoch 266/1000
 535 69s 137ms/step - loss: 0.6883 - acc: 0.8759 - val_loss: 0.6731 - val_acc: 0.8855
 536 Epoch 267/1000
 537 69s 138ms/step - loss: 0.6852 - acc: 0.8768 - val_loss: 0.6898 - val_acc: 0.8794
 538 Epoch 268/1000
 539 69s 138ms/step - loss: 0.6901 - acc: 0.8754 - val_loss: 0.6825 - val_acc: 0.8804
 540 Epoch 269/1000
 541 69s 138ms/step - loss: 0.6929 - acc: 0.8742 - val_loss: 0.6916 - val_acc: 0.8745
 542 Epoch 270/1000
 543 69s 138ms/step - loss: 0.6870 - acc: 0.8772 - val_loss: 0.6772 - val_acc: 0.8804
 544 Epoch 271/1000
 545 68s 137ms/step - loss: 0.6891 - acc: 0.8761 - val_loss: 0.6891 - val_acc: 0.8756
 546 Epoch 272/1000
 547 68s 137ms/step - loss: 0.6863 - acc: 0.8768 - val_loss: 0.6851 - val_acc: 0.8813
 548 Epoch 273/1000
 549 68s 137ms/step - loss: 0.6891 - acc: 0.8765 - val_loss: 0.6921 - val_acc: 0.8776
 550 Epoch 274/1000
 551 69s 138ms/step - loss: 0.6853 - acc: 0.8779 - val_loss: 0.6785 - val_acc: 0.8820
 552 Epoch 275/1000
 553 69s 138ms/step - loss: 0.6879 - acc: 0.8767 - val_loss: 0.6994 - val_acc: 0.8728
 554 Epoch 276/1000
 555 69s 138ms/step - loss: 0.6869 - acc: 0.8767 - val_loss: 0.6949 - val_acc: 0.8732
 556 Epoch 277/1000
 557 69s 137ms/step - loss: 0.6796 - acc: 0.8793 - val_loss: 0.6813 - val_acc: 0.8820
 558 Epoch 278/1000
 559 69s 138ms/step - loss: 0.6909 - acc: 0.8746 - val_loss: 0.6745 - val_acc: 0.8841
 560 Epoch 279/1000
 561 69s 138ms/step - loss: 0.6837 - acc: 0.8777 - val_loss: 0.6951 - val_acc: 0.8761
 562 Epoch 280/1000
 563 69s 137ms/step - loss: 0.6882 - acc: 0.8769 - val_loss: 0.6828 - val_acc: 0.8805
 564 Epoch 281/1000
 565 69s 138ms/step - loss: 0.6909 - acc: 0.8767 - val_loss: 0.6801 - val_acc: 0.8836
 566 Epoch 282/1000
 567 69s 137ms/step - loss: 0.6890 - acc: 0.8743 - val_loss: 0.6931 - val_acc: 0.8757
 568 Epoch 283/1000
 569 69s 138ms/step - loss: 0.6871 - acc: 0.8772 - val_loss: 0.6791 - val_acc: 0.8837
 570 Epoch 284/1000
 571 69s 138ms/step - loss: 0.6846 - acc: 0.8796 - val_loss: 0.7228 - val_acc: 0.8674
 572 Epoch 285/1000
 573 69s 138ms/step - loss: 0.6857 - acc: 0.8792 - val_loss: 0.7068 - val_acc: 0.8735
 574 Epoch 286/1000
 575 69s 138ms/step - loss: 0.6891 - acc: 0.8759 - val_loss: 0.7089 - val_acc: 0.8735
 576 Epoch 287/1000
 577 69s 137ms/step - loss: 0.6927 - acc: 0.8770 - val_loss: 0.6755 - val_acc: 0.8823
 578 Epoch 288/1000
 579 69s 137ms/step - loss: 0.6878 - acc: 0.8770 - val_loss: 0.6939 - val_acc: 0.8761
 580 Epoch 289/1000
 581 68s 137ms/step - loss: 0.6858 - acc: 0.8795 - val_loss: 0.6844 - val_acc: 0.8829
 582 Epoch 290/1000
 583 69s 137ms/step - loss: 0.6901 - acc: 0.8774 - val_loss: 0.6603 - val_acc: 0.8877
 584 Epoch 291/1000
 585 69s 138ms/step - loss: 0.6827 - acc: 0.8805 - val_loss: 0.6700 - val_acc: 0.8877
 586 Epoch 292/1000
 587 69s 137ms/step - loss: 0.6875 - acc: 0.8770 - val_loss: 0.6843 - val_acc: 0.8802
 588 Epoch 293/1000
 589 69s 138ms/step - loss: 0.6861 - acc: 0.8795 - val_loss: 0.6889 - val_acc: 0.8812
 590 Epoch 294/1000
 591 68s 137ms/step - loss: 0.6896 - acc: 0.8759 - val_loss: 0.6688 - val_acc: 0.8874
 592 Epoch 295/1000
 593 69s 138ms/step - loss: 0.6792 - acc: 0.8805 - val_loss: 0.6813 - val_acc: 0.8802
 594 Epoch 296/1000
 595 69s 138ms/step - loss: 0.6946 - acc: 0.8733 - val_loss: 0.6697 - val_acc: 0.8858
 596 Epoch 297/1000
 597 69s 138ms/step - loss: 0.6887 - acc: 0.8755 - val_loss: 0.6707 - val_acc: 0.8848
 598 Epoch 298/1000
 599 69s 138ms/step - loss: 0.6875 - acc: 0.8765 - val_loss: 0.7025 - val_acc: 0.8718
 600 Epoch 299/1000
 601 69s 137ms/step - loss: 0.6853 - acc: 0.8789 - val_loss: 0.6842 - val_acc: 0.8805
 602 Epoch 300/1000
 603 69s 138ms/step - loss: 0.6806 - acc: 0.8809 - val_loss: 0.6948 - val_acc: 0.8809
 604 Epoch 301/1000
 605 lr changed to 0.010000000149011612
 606 69s 138ms/step - loss: 0.5763 - acc: 0.9142 - val_loss: 0.5780 - val_acc: 0.9169
 607 Epoch 302/1000
 608 69s 138ms/step - loss: 0.5127 - acc: 0.9355 - val_loss: 0.5618 - val_acc: 0.9209
 609 Epoch 303/1000
 610 68s 137ms/step - loss: 0.4950 - acc: 0.9401 - val_loss: 0.5561 - val_acc: 0.9223
 611 Epoch 304/1000
 612 68s 137ms/step - loss: 0.4744 - acc: 0.9449 - val_loss: 0.5485 - val_acc: 0.9229
 613 Epoch 305/1000
 614 68s 137ms/step - loss: 0.4602 - acc: 0.9489 - val_loss: 0.5469 - val_acc: 0.9206
 615 Epoch 306/1000
 616 69s 137ms/step - loss: 0.4533 - acc: 0.9479 - val_loss: 0.5368 - val_acc: 0.9209
 617 Epoch 307/1000
 618 69s 137ms/step - loss: 0.4463 - acc: 0.9498 - val_loss: 0.5294 - val_acc: 0.9230
 619 Epoch 308/1000
 620 69s 137ms/step - loss: 0.4371 - acc: 0.9508 - val_loss: 0.5304 - val_acc: 0.9228
 621 Epoch 309/1000
 622 69s 137ms/step - loss: 0.4276 - acc: 0.9515 - val_loss: 0.5217 - val_acc: 0.9236
 623 Epoch 310/1000
 624 68s 136ms/step - loss: 0.4185 - acc: 0.9542 - val_loss: 0.5202 - val_acc: 0.9235
 625 Epoch 311/1000
 626 69s 138ms/step - loss: 0.4079 - acc: 0.9563 - val_loss: 0.5213 - val_acc: 0.9224
 627 Epoch 312/1000
 628 69s 137ms/step - loss: 0.4028 - acc: 0.9559 - val_loss: 0.5149 - val_acc: 0.9241
 629 Epoch 313/1000
 630 68s 136ms/step - loss: 0.3940 - acc: 0.9582 - val_loss: 0.5182 - val_acc: 0.9229
 631 Epoch 314/1000
 632 69s 138ms/step - loss: 0.3913 - acc: 0.9584 - val_loss: 0.5063 - val_acc: 0.9222
 633 Epoch 315/1000
 634 69s 138ms/step - loss: 0.3815 - acc: 0.9599 - val_loss: 0.5065 - val_acc: 0.9242
 635 Epoch 316/1000
 636 69s 138ms/step - loss: 0.3779 - acc: 0.9596 - val_loss: 0.5105 - val_acc: 0.9197
 637 Epoch 317/1000
 638 69s 138ms/step - loss: 0.3734 - acc: 0.9607 - val_loss: 0.4951 - val_acc: 0.9242
 639 Epoch 318/1000
 640 69s 138ms/step - loss: 0.3668 - acc: 0.9608 - val_loss: 0.4984 - val_acc: 0.9226
 641 Epoch 319/1000
 642 68s 137ms/step - loss: 0.3600 - acc: 0.9628 - val_loss: 0.5003 - val_acc: 0.9195
 643 Epoch 320/1000
 644 68s 137ms/step - loss: 0.3562 - acc: 0.9622 - val_loss: 0.4927 - val_acc: 0.9206
 645 Epoch 321/1000
 646 69s 138ms/step - loss: 0.3551 - acc: 0.9619 - val_loss: 0.4883 - val_acc: 0.9233
 647 Epoch 322/1000
 648 69s 138ms/step - loss: 0.3467 - acc: 0.9635 - val_loss: 0.4820 - val_acc: 0.9247
 649 Epoch 323/1000
 650 69s 138ms/step - loss: 0.3468 - acc: 0.9621 - val_loss: 0.4795 - val_acc: 0.9225
 651 Epoch 324/1000
 652 68s 136ms/step - loss: 0.3386 - acc: 0.9651 - val_loss: 0.4927 - val_acc: 0.9205
 653 Epoch 325/1000
 654 68s 135ms/step - loss: 0.3368 - acc: 0.9644 - val_loss: 0.4823 - val_acc: 0.9205
 655 Epoch 326/1000
 656 68s 136ms/step - loss: 0.3284 - acc: 0.9667 - val_loss: 0.4691 - val_acc: 0.9236
 657 Epoch 327/1000
 658 69s 138ms/step - loss: 0.3255 - acc: 0.9658 - val_loss: 0.4734 - val_acc: 0.9252
 659 Epoch 328/1000
 660 68s 136ms/step - loss: 0.3255 - acc: 0.9648 - val_loss: 0.4795 - val_acc: 0.9230
 661 Epoch 329/1000
 662 68s 136ms/step - loss: 0.3257 - acc: 0.9638 - val_loss: 0.4681 - val_acc: 0.9223
 663 Epoch 330/1000
 664 68s 136ms/step - loss: 0.3181 - acc: 0.9648 - val_loss: 0.4670 - val_acc: 0.9215
 665 Epoch 331/1000
 666 68s 136ms/step - loss: 0.3138 - acc: 0.9660 - val_loss: 0.4821 - val_acc: 0.9185
 667 Epoch 332/1000
 668 68s 136ms/step - loss: 0.3140 - acc: 0.9648 - val_loss: 0.4727 - val_acc: 0.9202
 669 Epoch 333/1000
 670 69s 137ms/step - loss: 0.3102 - acc: 0.9663 - val_loss: 0.4632 - val_acc: 0.9231
 671 Epoch 334/1000
 672 68s 137ms/step - loss: 0.3085 - acc: 0.9663 - val_loss: 0.4611 - val_acc: 0.9240
 673 Epoch 335/1000
 674 68s 137ms/step - loss: 0.3019 - acc: 0.9679 - val_loss: 0.4614 - val_acc: 0.9238
 675 Epoch 336/1000
 676 69s 138ms/step - loss: 0.3046 - acc: 0.9654 - val_loss: 0.4635 - val_acc: 0.9202
 677 Epoch 337/1000
 678 68s 137ms/step - loss: 0.3015 - acc: 0.9660 - val_loss: 0.4599 - val_acc: 0.9228
 679 Epoch 338/1000
 680 69s 137ms/step - loss: 0.2992 - acc: 0.9662 - val_loss: 0.4577 - val_acc: 0.9207
 681 Epoch 339/1000
 682 69s 138ms/step - loss: 0.2942 - acc: 0.9669 - val_loss: 0.4702 - val_acc: 0.9172
 683 Epoch 340/1000
 684 69s 137ms/step - loss: 0.2924 - acc: 0.9675 - val_loss: 0.4545 - val_acc: 0.9211
 685 ...
 686 Epoch 597/1000
 687 68s 135ms/step - loss: 0.2366 - acc: 0.9703 - val_loss: 0.4557 - val_acc: 0.9103
 688 Epoch 598/1000
 689 68s 135ms/step - loss: 0.2399 - acc: 0.9697 - val_loss: 0.4449 - val_acc: 0.9117
 690 Epoch 599/1000
 691 67s 135ms/step - loss: 0.2397 - acc: 0.9689 - val_loss: 0.4359 - val_acc: 0.9147
 692 Epoch 600/1000
 693 68s 136ms/step - loss: 0.2341 - acc: 0.9717 - val_loss: 0.4224 - val_acc: 0.9169
 694 Epoch 601/1000
 695 lr changed to 0.0009999999776482583
 696 68s 136ms/step - loss: 0.2082 - acc: 0.9813 - val_loss: 0.3916 - val_acc: 0.9268
 697 Epoch 602/1000
 698 68s 136ms/step - loss: 0.1952 - acc: 0.9865 - val_loss: 0.3854 - val_acc: 0.9281
 699 Epoch 603/1000
 700 68s 136ms/step - loss: 0.1878 - acc: 0.9881 - val_loss: 0.3852 - val_acc: 0.9299
 701 Epoch 604/1000
 702 68s 136ms/step - loss: 0.1846 - acc: 0.9899 - val_loss: 0.3842 - val_acc: 0.9298
 703 Epoch 605/1000
 704 68s 135ms/step - loss: 0.1826 - acc: 0.9909 - val_loss: 0.3829 - val_acc: 0.9326
 705 Epoch 606/1000
 706 68s 136ms/step - loss: 0.1808 - acc: 0.9912 - val_loss: 0.3838 - val_acc: 0.9305
 707 Epoch 607/1000
 708 68s 136ms/step - loss: 0.1771 - acc: 0.9927 - val_loss: 0.3851 - val_acc: 0.9303
 709 Epoch 608/1000
 710 68s 136ms/step - loss: 0.1768 - acc: 0.9922 - val_loss: 0.3898 - val_acc: 0.9304
 711 Epoch 609/1000
 712 68s 135ms/step - loss: 0.1758 - acc: 0.9926 - val_loss: 0.3878 - val_acc: 0.9309
 713 Epoch 610/1000
 714 68s 136ms/step - loss: 0.1739 - acc: 0.9931 - val_loss: 0.3887 - val_acc: 0.9294
 715 Epoch 611/1000
 716 68s 136ms/step - loss: 0.1731 - acc: 0.9934 - val_loss: 0.3874 - val_acc: 0.9311
 717 Epoch 612/1000
 718 68s 136ms/step - loss: 0.1725 - acc: 0.9935 - val_loss: 0.3898 - val_acc: 0.9297
 719 Epoch 613/1000
 720 68s 135ms/step - loss: 0.1717 - acc: 0.9937 - val_loss: 0.3900 - val_acc: 0.9298
 721 Epoch 614/1000
 722 68s 136ms/step - loss: 0.1705 - acc: 0.9937 - val_loss: 0.3912 - val_acc: 0.9299
 723 Epoch 615/1000
 724 68s 136ms/step - loss: 0.1709 - acc: 0.9934 - val_loss: 0.3898 - val_acc: 0.9307
 725 Epoch 616/1000
 726 68s 136ms/step - loss: 0.1686 - acc: 0.9948 - val_loss: 0.3905 - val_acc: 0.9311
 727 Epoch 617/1000
 728 68s 136ms/step - loss: 0.1695 - acc: 0.9942 - val_loss: 0.3948 - val_acc: 0.9303
 729 Epoch 618/1000
 730 68s 136ms/step - loss: 0.1688 - acc: 0.9941 - val_loss: 0.3936 - val_acc: 0.9298
 731 Epoch 619/1000
 732 68s 136ms/step - loss: 0.1679 - acc: 0.9945 - val_loss: 0.3950 - val_acc: 0.9290
 733 Epoch 620/1000
 734 68s 136ms/step - loss: 0.1675 - acc: 0.9941 - val_loss: 0.3940 - val_acc: 0.9300
 735 Epoch 621/1000
 736 68s 136ms/step - loss: 0.1651 - acc: 0.9949 - val_loss: 0.3956 - val_acc: 0.9309
 737 Epoch 622/1000
 738 68s 136ms/step - loss: 0.1653 - acc: 0.9951 - val_loss: 0.3950 - val_acc: 0.9306
 739 Epoch 623/1000
 740 68s 136ms/step - loss: 0.1656 - acc: 0.9946 - val_loss: 0.3947 - val_acc: 0.9306
 741 Epoch 624/1000
 742 68s 136ms/step - loss: 0.1644 - acc: 0.9949 - val_loss: 0.3946 - val_acc: 0.9304
 743 Epoch 625/1000
 744 68s 136ms/step - loss: 0.1636 - acc: 0.9951 - val_loss: 0.3944 - val_acc: 0.9296
 745 Epoch 626/1000
 746 68s 136ms/step - loss: 0.1630 - acc: 0.9951 - val_loss: 0.3937 - val_acc: 0.9295
 747 Epoch 627/1000
 748 68s 136ms/step - loss: 0.1630 - acc: 0.9953 - val_loss: 0.3959 - val_acc: 0.9296
 749 Epoch 628/1000
 750 68s 136ms/step - loss: 0.1627 - acc: 0.9954 - val_loss: 0.3939 - val_acc: 0.9289
 751 Epoch 629/1000
 752 68s 136ms/step - loss: 0.1630 - acc: 0.9947 - val_loss: 0.3937 - val_acc: 0.9303
 753 Epoch 630/1000
 754 68s 135ms/step - loss: 0.1614 - acc: 0.9958 - val_loss: 0.3909 - val_acc: 0.9316
 755 Epoch 631/1000
 756 68s 137ms/step - loss: 0.1624 - acc: 0.9950 - val_loss: 0.3922 - val_acc: 0.9310
 757 Epoch 632/1000
 758 68s 135ms/step - loss: 0.1611 - acc: 0.9954 - val_loss: 0.3907 - val_acc: 0.9313
 759 Epoch 633/1000
 760 68s 136ms/step - loss: 0.1599 - acc: 0.9955 - val_loss: 0.3893 - val_acc: 0.9295
 761 Epoch 634/1000
 762 68s 136ms/step - loss: 0.1600 - acc: 0.9954 - val_loss: 0.3886 - val_acc: 0.9308
 763 Epoch 635/1000
 764 68s 136ms/step - loss: 0.1593 - acc: 0.9953 - val_loss: 0.3926 - val_acc: 0.9297
 765 Epoch 636/1000
 766 68s 136ms/step - loss: 0.1594 - acc: 0.9950 - val_loss: 0.3945 - val_acc: 0.9289
 767 Epoch 637/1000
 768 68s 136ms/step - loss: 0.1595 - acc: 0.9955 - val_loss: 0.3937 - val_acc: 0.9306
 769 Epoch 638/1000
 770 68s 136ms/step - loss: 0.1591 - acc: 0.9958 - val_loss: 0.3882 - val_acc: 0.9306
 771 Epoch 639/1000
 772 68s 135ms/step - loss: 0.1586 - acc: 0.9959 - val_loss: 0.3893 - val_acc: 0.9309
 773 Epoch 640/1000
 774 68s 136ms/step - loss: 0.1588 - acc: 0.9956 - val_loss: 0.3935 - val_acc: 0.9300
 775 Epoch 641/1000
 776 68s 135ms/step - loss: 0.1571 - acc: 0.9960 - val_loss: 0.3917 - val_acc: 0.9298
 777 Epoch 642/1000
 778 68s 136ms/step - loss: 0.1576 - acc: 0.9956 - val_loss: 0.3945 - val_acc: 0.9284
 779 Epoch 643/1000
 780 68s 136ms/step - loss: 0.1570 - acc: 0.9961 - val_loss: 0.3899 - val_acc: 0.9309
 781 Epoch 644/1000
 782 68s 136ms/step - loss: 0.1565 - acc: 0.9962 - val_loss: 0.3918 - val_acc: 0.9307
 783 Epoch 645/1000
 784 68s 136ms/step - loss: 0.1563 - acc: 0.9956 - val_loss: 0.3940 - val_acc: 0.9307
 785 Epoch 646/1000
 786 68s 136ms/step - loss: 0.1563 - acc: 0.9956 - val_loss: 0.3895 - val_acc: 0.9322
 787 Epoch 647/1000
 788 68s 136ms/step - loss: 0.1555 - acc: 0.9963 - val_loss: 0.3903 - val_acc: 0.9302
 789 Epoch 648/1000
 790 68s 135ms/step - loss: 0.1556 - acc: 0.9958 - val_loss: 0.3926 - val_acc: 0.9307
 791 Epoch 649/1000
 792 68s 135ms/step - loss: 0.1542 - acc: 0.9962 - val_loss: 0.3904 - val_acc: 0.9308
 793 Epoch 650/1000
 794 68s 136ms/step - loss: 0.1552 - acc: 0.9959 - val_loss: 0.3934 - val_acc: 0.9295
 795 Epoch 651/1000
 796 68s 136ms/step - loss: 0.1548 - acc: 0.9959 - val_loss: 0.3921 - val_acc: 0.9307
 797 Epoch 652/1000
 798 68s 136ms/step - loss: 0.1537 - acc: 0.9964 - val_loss: 0.3973 - val_acc: 0.9293
 799 Epoch 653/1000
 800 68s 136ms/step - loss: 0.1540 - acc: 0.9958 - val_loss: 0.3950 - val_acc: 0.9287
 801 Epoch 654/1000
 802 68s 136ms/step - loss: 0.1523 - acc: 0.9965 - val_loss: 0.3956 - val_acc: 0.9296
 803 Epoch 655/1000
 804 68s 137ms/step - loss: 0.1532 - acc: 0.9964 - val_loss: 0.3991 - val_acc: 0.9292
 805 Epoch 656/1000
 806 68s 136ms/step - loss: 0.1538 - acc: 0.9957 - val_loss: 0.3995 - val_acc: 0.9296
 807 Epoch 657/1000
 808 68s 136ms/step - loss: 0.1520 - acc: 0.9966 - val_loss: 0.3988 - val_acc: 0.9310
 809 Epoch 658/1000
 810 68s 136ms/step - loss: 0.1532 - acc: 0.9959 - val_loss: 0.3961 - val_acc: 0.9307
 811 Epoch 659/1000
 812 68s 136ms/step - loss: 0.1526 - acc: 0.9958 - val_loss: 0.3948 - val_acc: 0.9306
 813 Epoch 660/1000
 814 68s 136ms/step - loss: 0.1512 - acc: 0.9965 - val_loss: 0.3947 - val_acc: 0.9309
 815 Epoch 661/1000
 816 68s 136ms/step - loss: 0.1519 - acc: 0.9962 - val_loss: 0.3959 - val_acc: 0.9315
 817 Epoch 662/1000
 818 68s 136ms/step - loss: 0.1510 - acc: 0.9963 - val_loss: 0.3962 - val_acc: 0.9312
 819 Epoch 663/1000
 820 68s 136ms/step - loss: 0.1517 - acc: 0.9960 - val_loss: 0.3939 - val_acc: 0.9304
 821 Epoch 664/1000
 822 68s 135ms/step - loss: 0.1494 - acc: 0.9964 - val_loss: 0.3928 - val_acc: 0.9309
 823 Epoch 665/1000
 824 68s 135ms/step - loss: 0.1492 - acc: 0.9966 - val_loss: 0.3900 - val_acc: 0.9320
 825 Epoch 666/1000
 826 68s 136ms/step - loss: 0.1493 - acc: 0.9963 - val_loss: 0.3907 - val_acc: 0.9312
 827 Epoch 667/1000
 828 68s 136ms/step - loss: 0.1491 - acc: 0.9967 - val_loss: 0.3930 - val_acc: 0.9309
 829 Epoch 668/1000
 830 68s 136ms/step - loss: 0.1494 - acc: 0.9960 - val_loss: 0.3923 - val_acc: 0.9301
 831 Epoch 669/1000
 832 68s 136ms/step - loss: 0.1485 - acc: 0.9966 - val_loss: 0.3941 - val_acc: 0.9308
 833 Epoch 670/1000
 834 68s 135ms/step - loss: 0.1486 - acc: 0.9963 - val_loss: 0.3927 - val_acc: 0.9314
 835 Epoch 671/1000
 836 68s 135ms/step - loss: 0.1481 - acc: 0.9965 - val_loss: 0.3939 - val_acc: 0.9322
 837 Epoch 672/1000
 838 68s 136ms/step - loss: 0.1474 - acc: 0.9968 - val_loss: 0.3950 - val_acc: 0.9309
 839 Epoch 673/1000
 840 68s 136ms/step - loss: 0.1471 - acc: 0.9967 - val_loss: 0.3931 - val_acc: 0.9322
 841 Epoch 674/1000
 842 68s 136ms/step - loss: 0.1470 - acc: 0.9968 - val_loss: 0.3934 - val_acc: 0.9319
 843 Epoch 675/1000
 844 68s 136ms/step - loss: 0.1469 - acc: 0.9965 - val_loss: 0.3920 - val_acc: 0.9319
 845 Epoch 676/1000
 846 68s 136ms/step - loss: 0.1469 - acc: 0.9967 - val_loss: 0.3923 - val_acc: 0.9309
 847 Epoch 677/1000
 848 68s 136ms/step - loss: 0.1461 - acc: 0.9968 - val_loss: 0.3940 - val_acc: 0.9297
 849 Epoch 678/1000
 850 68s 135ms/step - loss: 0.1462 - acc: 0.9969 - val_loss: 0.3924 - val_acc: 0.9309
 851 Epoch 679/1000
 852 68s 136ms/step - loss: 0.1443 - acc: 0.9971 - val_loss: 0.3930 - val_acc: 0.9317
 853 Epoch 680/1000
 854 68s 135ms/step - loss: 0.1458 - acc: 0.9966 - val_loss: 0.3978 - val_acc: 0.9296
 855 Epoch 681/1000
 856 68s 135ms/step - loss: 0.1453 - acc: 0.9970 - val_loss: 0.3978 - val_acc: 0.9286
 857 Epoch 682/1000
 858 68s 136ms/step - loss: 0.1444 - acc: 0.9972 - val_loss: 0.3968 - val_acc: 0.9285
 859 Epoch 683/1000
 860 68s 136ms/step - loss: 0.1444 - acc: 0.9969 - val_loss: 0.3922 - val_acc: 0.9299
 861 Epoch 684/1000
 862 68s 136ms/step - loss: 0.1447 - acc: 0.9970 - val_loss: 0.3907 - val_acc: 0.9297
 863 Epoch 685/1000
 864 68s 136ms/step - loss: 0.1441 - acc: 0.9966 - val_loss: 0.3925 - val_acc: 0.9285
 865 Epoch 686/1000
 866 68s 135ms/step - loss: 0.1454 - acc: 0.9964 - val_loss: 0.3939 - val_acc: 0.9296
 867 Epoch 687/1000
 868 68s 135ms/step - loss: 0.1433 - acc: 0.9968 - val_loss: 0.3955 - val_acc: 0.9293
 869 Epoch 688/1000
 870 68s 136ms/step - loss: 0.1435 - acc: 0.9969 - val_loss: 0.3958 - val_acc: 0.9295
 871 Epoch 689/1000
 872 68s 136ms/step - loss: 0.1423 - acc: 0.9972 - val_loss: 0.3981 - val_acc: 0.9305
 873 Epoch 690/1000
 874 68s 136ms/step - loss: 0.1438 - acc: 0.9965 - val_loss: 0.3986 - val_acc: 0.9299
 875 Epoch 691/1000
 876 68s 136ms/step - loss: 0.1422 - acc: 0.9972 - val_loss: 0.3956 - val_acc: 0.9302
 877 Epoch 692/1000
 878 68s 136ms/step - loss: 0.1425 - acc: 0.9968 - val_loss: 0.3962 - val_acc: 0.9309
 879 Epoch 693/1000
 880 68s 136ms/step - loss: 0.1420 - acc: 0.9968 - val_loss: 0.3972 - val_acc: 0.9300
 881 Epoch 694/1000
 882 68s 136ms/step - loss: 0.1422 - acc: 0.9967 - val_loss: 0.3947 - val_acc: 0.9301
 883 Epoch 695/1000
 884 68s 136ms/step - loss: 0.1420 - acc: 0.9970 - val_loss: 0.3945 - val_acc: 0.9306
 885 Epoch 696/1000
 886 68s 136ms/step - loss: 0.1412 - acc: 0.9970 - val_loss: 0.3942 - val_acc: 0.9313
 887 Epoch 697/1000
 888 68s 136ms/step - loss: 0.1402 - acc: 0.9972 - val_loss: 0.3950 - val_acc: 0.9309
 889 Epoch 698/1000
 890 68s 136ms/step - loss: 0.1408 - acc: 0.9969 - val_loss: 0.3931 - val_acc: 0.9307
 891 Epoch 699/1000
 892 68s 136ms/step - loss: 0.1409 - acc: 0.9970 - val_loss: 0.3936 - val_acc: 0.9297
 893 Epoch 700/1000
 894 68s 136ms/step - loss: 0.1404 - acc: 0.9970 - val_loss: 0.3930 - val_acc: 0.9289
 895 Epoch 701/1000
 896 68s 136ms/step - loss: 0.1403 - acc: 0.9972 - val_loss: 0.3905 - val_acc: 0.9308
 897 Epoch 702/1000
 898 68s 136ms/step - loss: 0.1387 - acc: 0.9976 - val_loss: 0.3957 - val_acc: 0.9295
 899 Epoch 703/1000
 900 68s 135ms/step - loss: 0.1402 - acc: 0.9967 - val_loss: 0.3950 - val_acc: 0.9294
 901 Epoch 704/1000
 902 68s 136ms/step - loss: 0.1393 - acc: 0.9971 - val_loss: 0.3950 - val_acc: 0.9298
 903 Epoch 705/1000
 904 68s 136ms/step - loss: 0.1386 - acc: 0.9969 - val_loss: 0.3950 - val_acc: 0.9302
 905 Epoch 706/1000
 906 68s 136ms/step - loss: 0.1384 - acc: 0.9973 - val_loss: 0.3936 - val_acc: 0.9303
 907 Epoch 707/1000
 908 68s 135ms/step - loss: 0.1386 - acc: 0.9970 - val_loss: 0.3974 - val_acc: 0.9290
 909 Epoch 708/1000
 910 68s 136ms/step - loss: 0.1392 - acc: 0.9968 - val_loss: 0.3938 - val_acc: 0.9295
 911 Epoch 709/1000
 912 68s 136ms/step - loss: 0.1383 - acc: 0.9970 - val_loss: 0.3931 - val_acc: 0.9288
 913 Epoch 710/1000
 914 68s 135ms/step - loss: 0.1383 - acc: 0.9970 - val_loss: 0.3905 - val_acc: 0.9305
 915 Epoch 711/1000
 916 68s 136ms/step - loss: 0.1381 - acc: 0.9970 - val_loss: 0.3904 - val_acc: 0.9286
 917 Epoch 712/1000
 918 68s 136ms/step - loss: 0.1375 - acc: 0.9971 - val_loss: 0.3923 - val_acc: 0.9302
 919 Epoch 713/1000
 920 68s 136ms/step - loss: 0.1370 - acc: 0.9972 - val_loss: 0.3931 - val_acc: 0.9308
 921 Epoch 714/1000
 922 68s 136ms/step - loss: 0.1364 - acc: 0.9974 - val_loss: 0.3883 - val_acc: 0.9322
 923 Epoch 715/1000
 924 68s 136ms/step - loss: 0.1364 - acc: 0.9974 - val_loss: 0.3894 - val_acc: 0.9306
 925 Epoch 716/1000
 926 68s 135ms/step - loss: 0.1365 - acc: 0.9972 - val_loss: 0.3894 - val_acc: 0.9290
 927 Epoch 717/1000
 928 68s 135ms/step - loss: 0.1358 - acc: 0.9973 - val_loss: 0.3908 - val_acc: 0.9294
 929 Epoch 718/1000
 930 68s 136ms/step - loss: 0.1360 - acc: 0.9971 - val_loss: 0.3899 - val_acc: 0.9297
 931 Epoch 719/1000
 932 68s 135ms/step - loss: 0.1370 - acc: 0.9969 - val_loss: 0.3880 - val_acc: 0.9311
 933 Epoch 720/1000
 934 68s 135ms/step - loss: 0.1348 - acc: 0.9971 - val_loss: 0.3884 - val_acc: 0.9308
 935 Epoch 721/1000
 936 68s 135ms/step - loss: 0.1354 - acc: 0.9973 - val_loss: 0.3946 - val_acc: 0.9299
 937 Epoch 722/1000
 938 68s 136ms/step - loss: 0.1346 - acc: 0.9973 - val_loss: 0.3890 - val_acc: 0.9313
 939 Epoch 723/1000
 940 68s 136ms/step - loss: 0.1355 - acc: 0.9972 - val_loss: 0.3914 - val_acc: 0.9313
 941 Epoch 724/1000
 942 68s 136ms/step - loss: 0.1353 - acc: 0.9970 - val_loss: 0.3956 - val_acc: 0.9308
 943 Epoch 725/1000
 944 68s 136ms/step - loss: 0.1349 - acc: 0.9972 - val_loss: 0.3914 - val_acc: 0.9303
 945 Epoch 726/1000
 946 68s 136ms/step - loss: 0.1338 - acc: 0.9975 - val_loss: 0.3917 - val_acc: 0.9297
 947 Epoch 727/1000
 948 68s 136ms/step - loss: 0.1335 - acc: 0.9977 - val_loss: 0.3877 - val_acc: 0.9318
 949 Epoch 728/1000
 950 68s 135ms/step - loss: 0.1329 - acc: 0.9977 - val_loss: 0.3830 - val_acc: 0.9324
 951 Epoch 729/1000
 952 68s 136ms/step - loss: 0.1332 - acc: 0.9973 - val_loss: 0.3870 - val_acc: 0.9314
 953 Epoch 730/1000
 954 68s 136ms/step - loss: 0.1330 - acc: 0.9976 - val_loss: 0.3870 - val_acc: 0.9321
 955 Epoch 731/1000
 956 68s 136ms/step - loss: 0.1324 - acc: 0.9978 - val_loss: 0.3841 - val_acc: 0.9308
 957 Epoch 732/1000
 958 68s 136ms/step - loss: 0.1329 - acc: 0.9971 - val_loss: 0.3853 - val_acc: 0.9316
 959 Epoch 733/1000
 960 68s 137ms/step - loss: 0.1323 - acc: 0.9975 - val_loss: 0.3868 - val_acc: 0.9310
 961 Epoch 734/1000
 962 68s 136ms/step - loss: 0.1322 - acc: 0.9975 - val_loss: 0.3882 - val_acc: 0.9301
 963 Epoch 735/1000
 964 68s 135ms/step - loss: 0.1314 - acc: 0.9975 - val_loss: 0.3880 - val_acc: 0.9289
 965 Epoch 736/1000
 966 68s 136ms/step - loss: 0.1327 - acc: 0.9971 - val_loss: 0.3891 - val_acc: 0.9295
 967 Epoch 737/1000
 968 68s 135ms/step - loss: 0.1308 - acc: 0.9978 - val_loss: 0.3862 - val_acc: 0.9303
 969 Epoch 738/1000
 970 68s 136ms/step - loss: 0.1314 - acc: 0.9975 - val_loss: 0.3872 - val_acc: 0.9294
 971 Epoch 739/1000
 972 68s 136ms/step - loss: 0.1305 - acc: 0.9979 - val_loss: 0.3864 - val_acc: 0.9309
 973 Epoch 740/1000
 974 68s 136ms/step - loss: 0.1310 - acc: 0.9973 - val_loss: 0.3896 - val_acc: 0.9307
 975 Epoch 741/1000
 976 68s 136ms/step - loss: 0.1311 - acc: 0.9974 - val_loss: 0.3883 - val_acc: 0.9312
 977 Epoch 742/1000
 978 68s 136ms/step - loss: 0.1315 - acc: 0.9968 - val_loss: 0.3888 - val_acc: 0.9304
 979 Epoch 743/1000
 980 68s 136ms/step - loss: 0.1297 - acc: 0.9977 - val_loss: 0.3892 - val_acc: 0.9307
 981 Epoch 744/1000
 982 68s 136ms/step - loss: 0.1298 - acc: 0.9976 - val_loss: 0.3864 - val_acc: 0.9297
 983 Epoch 745/1000
 984 68s 136ms/step - loss: 0.1299 - acc: 0.9974 - val_loss: 0.3883 - val_acc: 0.9306
 985 Epoch 746/1000
 986 68s 135ms/step - loss: 0.1301 - acc: 0.9972 - val_loss: 0.3892 - val_acc: 0.9290
 987 Epoch 747/1000
 988 68s 136ms/step - loss: 0.1290 - acc: 0.9977 - val_loss: 0.3860 - val_acc: 0.9301
 989 Epoch 748/1000
 990 68s 136ms/step - loss: 0.1299 - acc: 0.9972 - val_loss: 0.3846 - val_acc: 0.9308
 991 Epoch 749/1000
 992 68s 136ms/step - loss: 0.1292 - acc: 0.9973 - val_loss: 0.3888 - val_acc: 0.9293
 993 Epoch 750/1000
 994 68s 135ms/step - loss: 0.1297 - acc: 0.9973 - val_loss: 0.3864 - val_acc: 0.9289
 995 Epoch 751/1000
 996 68s 135ms/step - loss: 0.1296 - acc: 0.9969 - val_loss: 0.3886 - val_acc: 0.9305
 997 Epoch 752/1000
 998 68s 136ms/step - loss: 0.1288 - acc: 0.9972 - val_loss: 0.3893 - val_acc: 0.9285
 999 Epoch 753/1000
1000 68s 136ms/step - loss: 0.1281 - acc: 0.9977 - val_loss: 0.3824 - val_acc: 0.9308
1001 Epoch 754/1000
1002 68s 136ms/step - loss: 0.1288 - acc: 0.9973 - val_loss: 0.3817 - val_acc: 0.9300
1003 Epoch 755/1000
1004 68s 136ms/step - loss: 0.1274 - acc: 0.9978 - val_loss: 0.3818 - val_acc: 0.9290
1005 Epoch 756/1000
1006 68s 136ms/step - loss: 0.1282 - acc: 0.9971 - val_loss: 0.3843 - val_acc: 0.9277
1007 Epoch 757/1000
1008 68s 136ms/step - loss: 0.1276 - acc: 0.9975 - val_loss: 0.3822 - val_acc: 0.9285
1009 Epoch 758/1000
1010 68s 135ms/step - loss: 0.1274 - acc: 0.9974 - val_loss: 0.3837 - val_acc: 0.9301
1011 Epoch 759/1000
1012 68s 136ms/step - loss: 0.1274 - acc: 0.9971 - val_loss: 0.3819 - val_acc: 0.9290
1013 Epoch 760/1000
1014 68s 136ms/step - loss: 0.1261 - acc: 0.9977 - val_loss: 0.3803 - val_acc: 0.9308
1015 Epoch 761/1000
1016 68s 136ms/step - loss: 0.1274 - acc: 0.9970 - val_loss: 0.3834 - val_acc: 0.9297
1017 Epoch 762/1000
1018 68s 136ms/step - loss: 0.1264 - acc: 0.9977 - val_loss: 0.3845 - val_acc: 0.9300
1019 Epoch 763/1000
1020 68s 136ms/step - loss: 0.1271 - acc: 0.9969 - val_loss: 0.3827 - val_acc: 0.9296
1021 Epoch 764/1000
1022 68s 136ms/step - loss: 0.1264 - acc: 0.9974 - val_loss: 0.3772 - val_acc: 0.9316
1023 Epoch 765/1000
1024 68s 136ms/step - loss: 0.1255 - acc: 0.9976 - val_loss: 0.3735 - val_acc: 0.9323
1025 Epoch 766/1000
1026 68s 135ms/step - loss: 0.1253 - acc: 0.9977 - val_loss: 0.3743 - val_acc: 0.9325
1027 Epoch 767/1000
1028 68s 136ms/step - loss: 0.1252 - acc: 0.9977 - val_loss: 0.3774 - val_acc: 0.9319
1029 Epoch 768/1000
1030 68s 136ms/step - loss: 0.1251 - acc: 0.9975 - val_loss: 0.3778 - val_acc: 0.9324
1031 Epoch 769/1000
1032 68s 136ms/step - loss: 0.1261 - acc: 0.9971 - val_loss: 0.3811 - val_acc: 0.9310
1033 Epoch 770/1000
1034 68s 136ms/step - loss: 0.1242 - acc: 0.9979 - val_loss: 0.3808 - val_acc: 0.9295
1035 Epoch 771/1000
1036 68s 136ms/step - loss: 0.1249 - acc: 0.9975 - val_loss: 0.3780 - val_acc: 0.9304
1037 Epoch 772/1000
1038 68s 136ms/step - loss: 0.1247 - acc: 0.9974 - val_loss: 0.3779 - val_acc: 0.9312
1039 Epoch 773/1000
1040 68s 136ms/step - loss: 0.1246 - acc: 0.9974 - val_loss: 0.3811 - val_acc: 0.9314
1041 Epoch 774/1000
1042 68s 135ms/step - loss: 0.1244 - acc: 0.9976 - val_loss: 0.3798 - val_acc: 0.9303
1043 Epoch 775/1000
1044 68s 136ms/step - loss: 0.1243 - acc: 0.9974 - val_loss: 0.3804 - val_acc: 0.9307
1045 Epoch 776/1000
1046 68s 135ms/step - loss: 0.1235 - acc: 0.9975 - val_loss: 0.3800 - val_acc: 0.9310
1047 Epoch 777/1000
1048 68s 136ms/step - loss: 0.1240 - acc: 0.9973 - val_loss: 0.3795 - val_acc: 0.9304
1049 Epoch 778/1000
1050 68s 136ms/step - loss: 0.1234 - acc: 0.9975 - val_loss: 0.3760 - val_acc: 0.9320
1051 Epoch 779/1000
1052 68s 136ms/step - loss: 0.1235 - acc: 0.9976 - val_loss: 0.3750 - val_acc: 0.9312
1053 Epoch 780/1000
1054 68s 136ms/step - loss: 0.1226 - acc: 0.9976 - val_loss: 0.3721 - val_acc: 0.9332
1055 Epoch 781/1000
1056 68s 136ms/step - loss: 0.1227 - acc: 0.9976 - val_loss: 0.3753 - val_acc: 0.9322
1057 Epoch 782/1000
1058 68s 135ms/step - loss: 0.1226 - acc: 0.9975 - val_loss: 0.3756 - val_acc: 0.9316
1059 Epoch 783/1000
1060 68s 135ms/step - loss: 0.1228 - acc: 0.9975 - val_loss: 0.3761 - val_acc: 0.9302
1061 Epoch 784/1000
1062 68s 136ms/step - loss: 0.1216 - acc: 0.9978 - val_loss: 0.3711 - val_acc: 0.9329
1063 Epoch 785/1000
1064 68s 136ms/step - loss: 0.1221 - acc: 0.9975 - val_loss: 0.3750 - val_acc: 0.9300
1065 Epoch 786/1000
1066 68s 136ms/step - loss: 0.1213 - acc: 0.9978 - val_loss: 0.3739 - val_acc: 0.9305
1067 Epoch 787/1000
1068 68s 136ms/step - loss: 0.1211 - acc: 0.9978 - val_loss: 0.3744 - val_acc: 0.9315
1069 Epoch 788/1000
1070 68s 136ms/step - loss: 0.1209 - acc: 0.9978 - val_loss: 0.3730 - val_acc: 0.9321
1071 Epoch 789/1000
1072 68s 136ms/step - loss: 0.1219 - acc: 0.9975 - val_loss: 0.3719 - val_acc: 0.9329
1073 Epoch 790/1000
1074 68s 135ms/step - loss: 0.1212 - acc: 0.9975 - val_loss: 0.3753 - val_acc: 0.9318
1075 Epoch 791/1000
1076 68s 136ms/step - loss: 0.1208 - acc: 0.9974 - val_loss: 0.3744 - val_acc: 0.9310
1077 Epoch 792/1000
1078 68s 135ms/step - loss: 0.1222 - acc: 0.9969 - val_loss: 0.3804 - val_acc: 0.9291
1079 Epoch 793/1000
1080 68s 136ms/step - loss: 0.1209 - acc: 0.9977 - val_loss: 0.3806 - val_acc: 0.9295
1081 Epoch 794/1000
1082 68s 136ms/step - loss: 0.1203 - acc: 0.9977 - val_loss: 0.3809 - val_acc: 0.9282
1083 Epoch 795/1000
1084 68s 136ms/step - loss: 0.1203 - acc: 0.9975 - val_loss: 0.3785 - val_acc: 0.9286
1085 Epoch 796/1000
1086 68s 136ms/step - loss: 0.1199 - acc: 0.9978 - val_loss: 0.3783 - val_acc: 0.9274
1087 Epoch 797/1000
1088 68s 136ms/step - loss: 0.1196 - acc: 0.9977 - val_loss: 0.3780 - val_acc: 0.9281
1089 Epoch 798/1000
1090 68s 135ms/step - loss: 0.1195 - acc: 0.9976 - val_loss: 0.3763 - val_acc: 0.9311
1091 Epoch 799/1000
1092 67s 135ms/step - loss: 0.1193 - acc: 0.9977 - val_loss: 0.3820 - val_acc: 0.9303
1093 Epoch 800/1000
1094 68s 136ms/step - loss: 0.1200 - acc: 0.9972 - val_loss: 0.3833 - val_acc: 0.9296
1095 Epoch 801/1000
1096 68s 135ms/step - loss: 0.1185 - acc: 0.9978 - val_loss: 0.3774 - val_acc: 0.9300
1097 Epoch 802/1000
1098 68s 136ms/step - loss: 0.1195 - acc: 0.9974 - val_loss: 0.3775 - val_acc: 0.9305
1099 Epoch 803/1000
1100 68s 136ms/step - loss: 0.1183 - acc: 0.9976 - val_loss: 0.3759 - val_acc: 0.9308
1101 Epoch 804/1000
1102 68s 136ms/step - loss: 0.1182 - acc: 0.9979 - val_loss: 0.3728 - val_acc: 0.9316
1103 Epoch 805/1000
1104 68s 136ms/step - loss: 0.1191 - acc: 0.9974 - val_loss: 0.3771 - val_acc: 0.9311
1105 Epoch 806/1000
1106 68s 136ms/step - loss: 0.1179 - acc: 0.9977 - val_loss: 0.3768 - val_acc: 0.9299
1107 Epoch 807/1000
1108 68s 136ms/step - loss: 0.1185 - acc: 0.9972 - val_loss: 0.3765 - val_acc: 0.9302
1109 Epoch 808/1000
1110 68s 136ms/step - loss: 0.1173 - acc: 0.9978 - val_loss: 0.3794 - val_acc: 0.9291
1111 Epoch 809/1000
1112 68s 136ms/step - loss: 0.1172 - acc: 0.9978 - val_loss: 0.3773 - val_acc: 0.9297
1113 Epoch 810/1000
1114 68s 136ms/step - loss: 0.1181 - acc: 0.9975 - val_loss: 0.3811 - val_acc: 0.9306
1115 Epoch 811/1000
1116 68s 136ms/step - loss: 0.1173 - acc: 0.9975 - val_loss: 0.3753 - val_acc: 0.9302
1117 Epoch 812/1000
1118 68s 136ms/step - loss: 0.1171 - acc: 0.9975 - val_loss: 0.3812 - val_acc: 0.9285
1119 Epoch 813/1000
1120 68s 136ms/step - loss: 0.1171 - acc: 0.9976 - val_loss: 0.3845 - val_acc: 0.9297
1121 Epoch 814/1000
1122 68s 136ms/step - loss: 0.1163 - acc: 0.9978 - val_loss: 0.3829 - val_acc: 0.9295
1123 Epoch 815/1000
1124 68s 136ms/step - loss: 0.1166 - acc: 0.9979 - val_loss: 0.3807 - val_acc: 0.9284
1125 Epoch 816/1000
1126 68s 136ms/step - loss: 0.1165 - acc: 0.9976 - val_loss: 0.3813 - val_acc: 0.9286
1127 Epoch 817/1000
1128 68s 135ms/step - loss: 0.1170 - acc: 0.9972 - val_loss: 0.3840 - val_acc: 0.9283
1129 Epoch 818/1000
1130 68s 136ms/step - loss: 0.1160 - acc: 0.9973 - val_loss: 0.3826 - val_acc: 0.9274
1131 Epoch 819/1000
1132 68s 136ms/step - loss: 0.1157 - acc: 0.9977 - val_loss: 0.3755 - val_acc: 0.9312
1133 Epoch 820/1000
1134 68s 136ms/step - loss: 0.1155 - acc: 0.9978 - val_loss: 0.3794 - val_acc: 0.9291
1135 Epoch 821/1000
1136 68s 136ms/step - loss: 0.1163 - acc: 0.9973 - val_loss: 0.3751 - val_acc: 0.9293
1137 Epoch 822/1000
1138 68s 137ms/step - loss: 0.1154 - acc: 0.9977 - val_loss: 0.3764 - val_acc: 0.9298
1139 Epoch 823/1000
1140 69s 137ms/step - loss: 0.1143 - acc: 0.9980 - val_loss: 0.3754 - val_acc: 0.9293
1141 Epoch 824/1000
1142 69s 138ms/step - loss: 0.1142 - acc: 0.9979 - val_loss: 0.3743 - val_acc: 0.9304
1143 Epoch 825/1000
1144 69s 138ms/step - loss: 0.1153 - acc: 0.9974 - val_loss: 0.3772 - val_acc: 0.9307
1145 Epoch 826/1000
1146 69s 138ms/step - loss: 0.1149 - acc: 0.9976 - val_loss: 0.3718 - val_acc: 0.9312
1147 Epoch 827/1000
1148 68s 137ms/step - loss: 0.1148 - acc: 0.9976 - val_loss: 0.3777 - val_acc: 0.9317
1149 Epoch 828/1000
1150 69s 138ms/step - loss: 0.1147 - acc: 0.9976 - val_loss: 0.3769 - val_acc: 0.9303
1151 Epoch 829/1000
1152 69s 138ms/step - loss: 0.1137 - acc: 0.9978 - val_loss: 0.3748 - val_acc: 0.9309
1153 Epoch 830/1000
1154 69s 138ms/step - loss: 0.1139 - acc: 0.9978 - val_loss: 0.3728 - val_acc: 0.9308
1155 Epoch 831/1000
1156 68s 137ms/step - loss: 0.1138 - acc: 0.9976 - val_loss: 0.3724 - val_acc: 0.9296
1157 Epoch 832/1000
1158 69s 138ms/step - loss: 0.1132 - acc: 0.9978 - val_loss: 0.3807 - val_acc: 0.9288
1159 Epoch 833/1000
1160 68s 137ms/step - loss: 0.1140 - acc: 0.9975 - val_loss: 0.3810 - val_acc: 0.9290
1161 Epoch 834/1000
1162 69s 138ms/step - loss: 0.1135 - acc: 0.9975 - val_loss: 0.3816 - val_acc: 0.9291
1163 Epoch 835/1000
1164 69s 138ms/step - loss: 0.1130 - acc: 0.9979 - val_loss: 0.3830 - val_acc: 0.9284
1165 Epoch 836/1000
1166 69s 138ms/step - loss: 0.1131 - acc: 0.9976 - val_loss: 0.3792 - val_acc: 0.9278
1167 Epoch 837/1000
1168 69s 137ms/step - loss: 0.1126 - acc: 0.9978 - val_loss: 0.3712 - val_acc: 0.9306
1169 Epoch 838/1000
1170 69s 137ms/step - loss: 0.1126 - acc: 0.9979 - val_loss: 0.3771 - val_acc: 0.9293
1171 Epoch 839/1000
1172 69s 138ms/step - loss: 0.1119 - acc: 0.9981 - val_loss: 0.3768 - val_acc: 0.9288
1173 Epoch 840/1000
1174 69s 138ms/step - loss: 0.1120 - acc: 0.9980 - val_loss: 0.3769 - val_acc: 0.9289
1175 Epoch 841/1000
1176 69s 137ms/step - loss: 0.1120 - acc: 0.9977 - val_loss: 0.3774 - val_acc: 0.9285
1177 Epoch 842/1000
1178 68s 136ms/step - loss: 0.1120 - acc: 0.9975 - val_loss: 0.3718 - val_acc: 0.9312
1179 Epoch 843/1000
1180 68s 136ms/step - loss: 0.1115 - acc: 0.9976 - val_loss: 0.3707 - val_acc: 0.9312
1181 Epoch 844/1000
1182 68s 136ms/step - loss: 0.1120 - acc: 0.9978 - val_loss: 0.3777 - val_acc: 0.9285
1183 Epoch 845/1000
1184 69s 137ms/step - loss: 0.1115 - acc: 0.9978 - val_loss: 0.3777 - val_acc: 0.9284
1185 Epoch 846/1000
1186 69s 138ms/step - loss: 0.1115 - acc: 0.9978 - val_loss: 0.3742 - val_acc: 0.9303
1187 Epoch 847/1000
1188 68s 137ms/step - loss: 0.1113 - acc: 0.9974 - val_loss: 0.3749 - val_acc: 0.9300
1189 Epoch 848/1000
1190 69s 138ms/step - loss: 0.1114 - acc: 0.9976 - val_loss: 0.3795 - val_acc: 0.9286
1191 Epoch 849/1000
1192 69s 138ms/step - loss: 0.1115 - acc: 0.9975 - val_loss: 0.3754 - val_acc: 0.9284
1193 Epoch 850/1000
1194 69s 138ms/step - loss: 0.1105 - acc: 0.9978 - val_loss: 0.3705 - val_acc: 0.9305
1195 Epoch 851/1000
1196 69s 138ms/step - loss: 0.1098 - acc: 0.9978 - val_loss: 0.3752 - val_acc: 0.9290
1197 Epoch 852/1000
1198 69s 138ms/step - loss: 0.1118 - acc: 0.9971 - val_loss: 0.3773 - val_acc: 0.9280
1199 Epoch 853/1000
1200 68s 137ms/step - loss: 0.1103 - acc: 0.9978 - val_loss: 0.3732 - val_acc: 0.9303
1201 Epoch 854/1000
1202 69s 137ms/step - loss: 0.1109 - acc: 0.9977 - val_loss: 0.3715 - val_acc: 0.9302
1203 Epoch 855/1000
1204 69s 137ms/step - loss: 0.1096 - acc: 0.9977 - val_loss: 0.3780 - val_acc: 0.9306
1205 Epoch 856/1000
1206 69s 137ms/step - loss: 0.1100 - acc: 0.9977 - val_loss: 0.3764 - val_acc: 0.9290
1207 Epoch 857/1000
1208 69s 137ms/step - loss: 0.1093 - acc: 0.9981 - val_loss: 0.3750 - val_acc: 0.9291
1209 Epoch 858/1000
1210 69s 138ms/step - loss: 0.1088 - acc: 0.9980 - val_loss: 0.3738 - val_acc: 0.9287
1211 Epoch 859/1000
1212 69s 138ms/step - loss: 0.1098 - acc: 0.9975 - val_loss: 0.3711 - val_acc: 0.9291
1213 Epoch 860/1000
1214 69s 138ms/step - loss: 0.1091 - acc: 0.9979 - val_loss: 0.3636 - val_acc: 0.9302
1215 Epoch 861/1000
1216 69s 138ms/step - loss: 0.1094 - acc: 0.9976 - val_loss: 0.3689 - val_acc: 0.9303
1217 Epoch 862/1000
1218 68s 137ms/step - loss: 0.1088 - acc: 0.9978 - val_loss: 0.3687 - val_acc: 0.9306
1219 Epoch 863/1000
1220 69s 137ms/step - loss: 0.1083 - acc: 0.9978 - val_loss: 0.3720 - val_acc: 0.9318
1221 Epoch 864/1000
1222 69s 138ms/step - loss: 0.1080 - acc: 0.9978 - val_loss: 0.3695 - val_acc: 0.9302
1223 Epoch 865/1000
1224 69s 138ms/step - loss: 0.1093 - acc: 0.9973 - val_loss: 0.3733 - val_acc: 0.9297
1225 Epoch 866/1000
1226 69s 138ms/step - loss: 0.1092 - acc: 0.9974 - val_loss: 0.3713 - val_acc: 0.9296
1227 Epoch 867/1000
1228 69s 137ms/step - loss: 0.1082 - acc: 0.9978 - val_loss: 0.3674 - val_acc: 0.9306
1229 Epoch 868/1000
1230 69s 138ms/step - loss: 0.1087 - acc: 0.9974 - val_loss: 0.3684 - val_acc: 0.9296
1231 Epoch 869/1000
1232 69s 138ms/step - loss: 0.1072 - acc: 0.9982 - val_loss: 0.3684 - val_acc: 0.9307
1233 Epoch 870/1000
1234 68s 137ms/step - loss: 0.1080 - acc: 0.9976 - val_loss: 0.3695 - val_acc: 0.9294
1235 Epoch 871/1000
1236 69s 138ms/step - loss: 0.1075 - acc: 0.9977 - val_loss: 0.3655 - val_acc: 0.9306
1237 Epoch 872/1000
1238 69s 138ms/step - loss: 0.1073 - acc: 0.9979 - val_loss: 0.3667 - val_acc: 0.9303
1239 Epoch 873/1000
1240 69s 138ms/step - loss: 0.1079 - acc: 0.9977 - val_loss: 0.3717 - val_acc: 0.9278
1241 Epoch 874/1000
1242 68s 137ms/step - loss: 0.1081 - acc: 0.9973 - val_loss: 0.3722 - val_acc: 0.9292
1243 Epoch 875/1000
1244 69s 138ms/step - loss: 0.1072 - acc: 0.9975 - val_loss: 0.3716 - val_acc: 0.9298
1245 Epoch 876/1000
1246 69s 137ms/step - loss: 0.1070 - acc: 0.9977 - val_loss: 0.3721 - val_acc: 0.9311
1247 Epoch 877/1000
1248 69s 137ms/step - loss: 0.1066 - acc: 0.9978 - val_loss: 0.3722 - val_acc: 0.9289
1249 Epoch 878/1000
1250 69s 137ms/step - loss: 0.1068 - acc: 0.9977 - val_loss: 0.3736 - val_acc: 0.9296
1251 Epoch 879/1000
1252 68s 137ms/step - loss: 0.1065 - acc: 0.9977 - val_loss: 0.3767 - val_acc: 0.9280
1253 Epoch 880/1000
1254 69s 138ms/step - loss: 0.1055 - acc: 0.9979 - val_loss: 0.3741 - val_acc: 0.9285
1255 Epoch 881/1000
1256 68s 137ms/step - loss: 0.1056 - acc: 0.9979 - val_loss: 0.3716 - val_acc: 0.9290
1257 Epoch 882/1000
1258 69s 138ms/step - loss: 0.1061 - acc: 0.9977 - val_loss: 0.3736 - val_acc: 0.9295
1259 Epoch 883/1000
1260 69s 138ms/step - loss: 0.1066 - acc: 0.9976 - val_loss: 0.3745 - val_acc: 0.9307
1261 Epoch 884/1000
1262 69s 137ms/step - loss: 0.1059 - acc: 0.9975 - val_loss: 0.3702 - val_acc: 0.9302
1263 Epoch 885/1000
1264 69s 138ms/step - loss: 0.1051 - acc: 0.9979 - val_loss: 0.3656 - val_acc: 0.9311
1265 Epoch 886/1000
1266 68s 137ms/step - loss: 0.1051 - acc: 0.9978 - val_loss: 0.3677 - val_acc: 0.9305
1267 Epoch 887/1000
1268 68s 137ms/step - loss: 0.1062 - acc: 0.9974 - val_loss: 0.3636 - val_acc: 0.9315
1269 Epoch 888/1000
1270 69s 137ms/step - loss: 0.1052 - acc: 0.9977 - val_loss: 0.3710 - val_acc: 0.9295
1271 Epoch 889/1000
1272 68s 137ms/step - loss: 0.1046 - acc: 0.9979 - val_loss: 0.3642 - val_acc: 0.9318
1273 Epoch 890/1000
1274 69s 138ms/step - loss: 0.1051 - acc: 0.9975 - val_loss: 0.3673 - val_acc: 0.9306
1275 Epoch 891/1000
1276 69s 138ms/step - loss: 0.1045 - acc: 0.9978 - val_loss: 0.3681 - val_acc: 0.9299
1277 Epoch 892/1000
1278 68s 137ms/step - loss: 0.1043 - acc: 0.9979 - val_loss: 0.3659 - val_acc: 0.9320
1279 Epoch 893/1000
1280 69s 137ms/step - loss: 0.1040 - acc: 0.9979 - val_loss: 0.3627 - val_acc: 0.9326
1281 Epoch 894/1000
1282 69s 138ms/step - loss: 0.1041 - acc: 0.9976 - val_loss: 0.3698 - val_acc: 0.9301
1283 Epoch 895/1000
1284 68s 137ms/step - loss: 0.1039 - acc: 0.9978 - val_loss: 0.3659 - val_acc: 0.9321
1285 Epoch 896/1000
1286 69s 137ms/step - loss: 0.1040 - acc: 0.9978 - val_loss: 0.3718 - val_acc: 0.9300
1287 Epoch 897/1000
1288 68s 137ms/step - loss: 0.1039 - acc: 0.9977 - val_loss: 0.3728 - val_acc: 0.9311
1289 Epoch 898/1000
1290 68s 137ms/step - loss: 0.1044 - acc: 0.9973 - val_loss: 0.3743 - val_acc: 0.9313
1291 Epoch 899/1000
1292 69s 137ms/step - loss: 0.1036 - acc: 0.9976 - val_loss: 0.3675 - val_acc: 0.9312
1293 Epoch 900/1000
1294 69s 138ms/step - loss: 0.1030 - acc: 0.9979 - val_loss: 0.3730 - val_acc: 0.9313
1295 Epoch 901/1000
1296 lr changed to 9.999999310821295e-05
1297 69s 138ms/step - loss: 0.1023 - acc: 0.9982 - val_loss: 0.3709 - val_acc: 0.9310
1298 Epoch 902/1000
1299 69s 137ms/step - loss: 0.1025 - acc: 0.9979 - val_loss: 0.3690 - val_acc: 0.9311
1300 Epoch 903/1000
1301 68s 137ms/step - loss: 0.1024 - acc: 0.9980 - val_loss: 0.3679 - val_acc: 0.9311
1302 Epoch 904/1000
1303 69s 137ms/step - loss: 0.1020 - acc: 0.9982 - val_loss: 0.3673 - val_acc: 0.9315
1304 Epoch 905/1000
1305 69s 138ms/step - loss: 0.1027 - acc: 0.9979 - val_loss: 0.3672 - val_acc: 0.9310
1306 Epoch 906/1000
1307 69s 138ms/step - loss: 0.1015 - acc: 0.9984 - val_loss: 0.3678 - val_acc: 0.9304
1308 Epoch 907/1000
1309 69s 138ms/step - loss: 0.1016 - acc: 0.9984 - val_loss: 0.3673 - val_acc: 0.9302
1310 Epoch 908/1000
1311 69s 138ms/step - loss: 0.1031 - acc: 0.9977 - val_loss: 0.3667 - val_acc: 0.9307
1312 Epoch 909/1000
1313 69s 139ms/step - loss: 0.1019 - acc: 0.9983 - val_loss: 0.3672 - val_acc: 0.9317
1314 Epoch 910/1000
1315 69s 137ms/step - loss: 0.1018 - acc: 0.9983 - val_loss: 0.3671 - val_acc: 0.9313
1316 Epoch 911/1000
1317 69s 137ms/step - loss: 0.1018 - acc: 0.9982 - val_loss: 0.3669 - val_acc: 0.9309
1318 Epoch 912/1000
1319 69s 137ms/step - loss: 0.1014 - acc: 0.9986 - val_loss: 0.3677 - val_acc: 0.9303
1320 Epoch 913/1000
1321 68s 137ms/step - loss: 0.1015 - acc: 0.9982 - val_loss: 0.3666 - val_acc: 0.9303
1322 Epoch 914/1000
1323 69s 138ms/step - loss: 0.1015 - acc: 0.9984 - val_loss: 0.3659 - val_acc: 0.9309
1324 Epoch 915/1000
1325 69s 138ms/step - loss: 0.1013 - acc: 0.9983 - val_loss: 0.3651 - val_acc: 0.9318
1326 Epoch 916/1000
1327 69s 138ms/step - loss: 0.1014 - acc: 0.9983 - val_loss: 0.3652 - val_acc: 0.9322
1328 Epoch 917/1000
1329 69s 137ms/step - loss: 0.1010 - acc: 0.9984 - val_loss: 0.3648 - val_acc: 0.9322
1330 Epoch 918/1000
1331 68s 137ms/step - loss: 0.1016 - acc: 0.9981 - val_loss: 0.3644 - val_acc: 0.9324
1332 Epoch 919/1000
1333 69s 138ms/step - loss: 0.1013 - acc: 0.9983 - val_loss: 0.3635 - val_acc: 0.9319
1334 Epoch 920/1000
1335 69s 138ms/step - loss: 0.1008 - acc: 0.9984 - val_loss: 0.3629 - val_acc: 0.9318
1336 Epoch 921/1000
1337 69s 138ms/step - loss: 0.1006 - acc: 0.9986 - val_loss: 0.3627 - val_acc: 0.9319
1338 Epoch 922/1000
1339 69s 137ms/step - loss: 0.1007 - acc: 0.9985 - val_loss: 0.3632 - val_acc: 0.9314
1340 Epoch 923/1000
1341 69s 137ms/step - loss: 0.1004 - acc: 0.9987 - val_loss: 0.3626 - val_acc: 0.9319
1342 Epoch 924/1000
1343 69s 138ms/step - loss: 0.1012 - acc: 0.9985 - val_loss: 0.3629 - val_acc: 0.9319
1344 Epoch 925/1000
1345 69s 138ms/step - loss: 0.1011 - acc: 0.9983 - val_loss: 0.3620 - val_acc: 0.9318
1346 Epoch 926/1000
1347 69s 138ms/step - loss: 0.1005 - acc: 0.9987 - val_loss: 0.3617 - val_acc: 0.9322
1348 Epoch 927/1000
1349 69s 138ms/step - loss: 0.1013 - acc: 0.9983 - val_loss: 0.3618 - val_acc: 0.9330
1350 Epoch 928/1000
1351 69s 138ms/step - loss: 0.1005 - acc: 0.9985 - val_loss: 0.3614 - val_acc: 0.9321
1352 Epoch 929/1000
1353 69s 137ms/step - loss: 0.1006 - acc: 0.9985 - val_loss: 0.3616 - val_acc: 0.9319
1354 Epoch 930/1000
1355 69s 138ms/step - loss: 0.1006 - acc: 0.9985 - val_loss: 0.3613 - val_acc: 0.9321
1356 Epoch 931/1000
1357 69s 138ms/step - loss: 0.1009 - acc: 0.9985 - val_loss: 0.3612 - val_acc: 0.9328
1358 Epoch 932/1000
1359 69s 138ms/step - loss: 0.1004 - acc: 0.9985 - val_loss: 0.3612 - val_acc: 0.9319
1360 Epoch 933/1000
1361 69s 138ms/step - loss: 0.1004 - acc: 0.9987 - val_loss: 0.3618 - val_acc: 0.9314
1362 Epoch 934/1000
1363 69s 137ms/step - loss: 0.1008 - acc: 0.9983 - val_loss: 0.3615 - val_acc: 0.9316
1364 Epoch 935/1000
1365 69s 138ms/step - loss: 0.1011 - acc: 0.9983 - val_loss: 0.3621 - val_acc: 0.9317
1366 Epoch 936/1000
1367 68s 137ms/step - loss: 0.1008 - acc: 0.9985 - val_loss: 0.3617 - val_acc: 0.9320
1368 Epoch 937/1000
1369 69s 138ms/step - loss: 0.1006 - acc: 0.9984 - val_loss: 0.3613 - val_acc: 0.9322
1370 Epoch 938/1000
1371 69s 137ms/step - loss: 0.1008 - acc: 0.9985 - val_loss: 0.3613 - val_acc: 0.9325
1372 Epoch 939/1000
1373 68s 137ms/step - loss: 0.1006 - acc: 0.9984 - val_loss: 0.3614 - val_acc: 0.9326
1374 Epoch 940/1000
1375 69s 137ms/step - loss: 0.1006 - acc: 0.9983 - val_loss: 0.3612 - val_acc: 0.9320
1376 Epoch 941/1000
1377 69s 137ms/step - loss: 0.1005 - acc: 0.9984 - val_loss: 0.3612 - val_acc: 0.9322
1378 Epoch 942/1000
1379 69s 138ms/step - loss: 0.1001 - acc: 0.9986 - val_loss: 0.3615 - val_acc: 0.9318
1380 Epoch 943/1000
1381 68s 137ms/step - loss: 0.0998 - acc: 0.9987 - val_loss: 0.3613 - val_acc: 0.9320
1382 Epoch 944/1000
1383 69s 137ms/step - loss: 0.1006 - acc: 0.9985 - val_loss: 0.3613 - val_acc: 0.9323
1384 Epoch 945/1000
1385 69s 138ms/step - loss: 0.1000 - acc: 0.9985 - val_loss: 0.3608 - val_acc: 0.9319
1386 Epoch 946/1000
1387 69s 138ms/step - loss: 0.1001 - acc: 0.9987 - val_loss: 0.3608 - val_acc: 0.9313
1388 Epoch 947/1000
1389 69s 138ms/step - loss: 0.0998 - acc: 0.9987 - val_loss: 0.3606 - val_acc: 0.9314
1390 Epoch 948/1000
1391 69s 138ms/step - loss: 0.1000 - acc: 0.9986 - val_loss: 0.3609 - val_acc: 0.9311
1392 Epoch 949/1000
1393 69s 138ms/step - loss: 0.0995 - acc: 0.9988 - val_loss: 0.3610 - val_acc: 0.9316
1394 Epoch 950/1000
1395 69s 137ms/step - loss: 0.0999 - acc: 0.9986 - val_loss: 0.3609 - val_acc: 0.9317
1396 Epoch 951/1000
1397 69s 137ms/step - loss: 0.1002 - acc: 0.9986 - val_loss: 0.3612 - val_acc: 0.9314
1398 Epoch 952/1000
1399 68s 137ms/step - loss: 0.0992 - acc: 0.9989 - val_loss: 0.3618 - val_acc: 0.9312
1400 Epoch 953/1000
1401 68s 137ms/step - loss: 0.0996 - acc: 0.9988 - val_loss: 0.3617 - val_acc: 0.9317
1402 Epoch 954/1000
1403 69s 138ms/step - loss: 0.0994 - acc: 0.9987 - val_loss: 0.3617 - val_acc: 0.9323
1404 Epoch 955/1000
1405 69s 138ms/step - loss: 0.1004 - acc: 0.9984 - val_loss: 0.3610 - val_acc: 0.9320
1406 Epoch 956/1000
1407 69s 138ms/step - loss: 0.1000 - acc: 0.9986 - val_loss: 0.3616 - val_acc: 0.9318
1408 Epoch 957/1000
1409 68s 137ms/step - loss: 0.1000 - acc: 0.9985 - val_loss: 0.3617 - val_acc: 0.9319
1410 Epoch 958/1000
1411 69s 138ms/step - loss: 0.0996 - acc: 0.9985 - val_loss: 0.3627 - val_acc: 0.9323
1412 Epoch 959/1000
1413 69s 137ms/step - loss: 0.0995 - acc: 0.9987 - val_loss: 0.3625 - val_acc: 0.9316
1414 Epoch 960/1000
1415 69s 137ms/step - loss: 0.0995 - acc: 0.9987 - val_loss: 0.3634 - val_acc: 0.9317
1416 Epoch 961/1000
1417 69s 138ms/step - loss: 0.0998 - acc: 0.9985 - val_loss: 0.3636 - val_acc: 0.9318
1418 Epoch 962/1000
1419 69s 138ms/step - loss: 0.0997 - acc: 0.9986 - val_loss: 0.3645 - val_acc: 0.9319
1420 Epoch 963/1000
1421 68s 137ms/step - loss: 0.1001 - acc: 0.9984 - val_loss: 0.3637 - val_acc: 0.9316
1422 Epoch 964/1000
1423 69s 138ms/step - loss: 0.0998 - acc: 0.9985 - val_loss: 0.3631 - val_acc: 0.9317
1424 Epoch 965/1000
1425 69s 137ms/step - loss: 0.0995 - acc: 0.9988 - val_loss: 0.3625 - val_acc: 0.9316
1426 Epoch 966/1000
1427 68s 137ms/step - loss: 0.0998 - acc: 0.9986 - val_loss: 0.3622 - val_acc: 0.9324
1428 Epoch 967/1000
1429 68s 137ms/step - loss: 0.1002 - acc: 0.9985 - val_loss: 0.3623 - val_acc: 0.9327
1430 Epoch 968/1000
1431 68s 137ms/step - loss: 0.0993 - acc: 0.9987 - val_loss: 0.3627 - val_acc: 0.9324
1432 Epoch 969/1000
1433 69s 137ms/step - loss: 0.0996 - acc: 0.9985 - val_loss: 0.3624 - val_acc: 0.9327
1434 Epoch 970/1000
1435 69s 138ms/step - loss: 0.0999 - acc: 0.9985 - val_loss: 0.3618 - val_acc: 0.9323
1436 Epoch 971/1000
1437 69s 137ms/step - loss: 0.1001 - acc: 0.9983 - val_loss: 0.3616 - val_acc: 0.9324
1438 Epoch 972/1000
1439 68s 136ms/step - loss: 0.0994 - acc: 0.9986 - val_loss: 0.3620 - val_acc: 0.9320
1440 Epoch 973/1000
1441 68s 136ms/step - loss: 0.0997 - acc: 0.9985 - val_loss: 0.3627 - val_acc: 0.9324
1442 Epoch 974/1000
1443 68s 136ms/step - loss: 0.1000 - acc: 0.9985 - val_loss: 0.3623 - val_acc: 0.9321
1444 Epoch 975/1000
1445 68s 135ms/step - loss: 0.0989 - acc: 0.9990 - val_loss: 0.3619 - val_acc: 0.9321
1446 Epoch 976/1000
1447 68s 136ms/step - loss: 0.0992 - acc: 0.9987 - val_loss: 0.3612 - val_acc: 0.9323
1448 Epoch 977/1000
1449 68s 136ms/step - loss: 0.0996 - acc: 0.9986 - val_loss: 0.3612 - val_acc: 0.9317
1450 Epoch 978/1000
1451 68s 136ms/step - loss: 0.0997 - acc: 0.9986 - val_loss: 0.3610 - val_acc: 0.9326
1452 Epoch 979/1000
1453 68s 136ms/step - loss: 0.0991 - acc: 0.9987 - val_loss: 0.3611 - val_acc: 0.9327
1454 Epoch 980/1000
1455 69s 137ms/step - loss: 0.0988 - acc: 0.9989 - val_loss: 0.3615 - val_acc: 0.9326
1456 Epoch 981/1000
1457 69s 137ms/step - loss: 0.0992 - acc: 0.9987 - val_loss: 0.3619 - val_acc: 0.9324
1458 Epoch 982/1000
1459 68s 137ms/step - loss: 0.0994 - acc: 0.9986 - val_loss: 0.3619 - val_acc: 0.9332
1460 Epoch 983/1000
1461 69s 137ms/step - loss: 0.0995 - acc: 0.9986 - val_loss: 0.3617 - val_acc: 0.9329
1462 Epoch 984/1000
1463 68s 137ms/step - loss: 0.0991 - acc: 0.9987 - val_loss: 0.3622 - val_acc: 0.9328
1464 Epoch 985/1000
1465 68s 137ms/step - loss: 0.0991 - acc: 0.9987 - val_loss: 0.3628 - val_acc: 0.9322
1466 Epoch 986/1000
1467 68s 137ms/step - loss: 0.0993 - acc: 0.9987 - val_loss: 0.3625 - val_acc: 0.9319
1468 Epoch 987/1000
1469 68s 137ms/step - loss: 0.0995 - acc: 0.9986 - val_loss: 0.3629 - val_acc: 0.9317
1470 Epoch 988/1000
1471 69s 137ms/step - loss: 0.0993 - acc: 0.9985 - val_loss: 0.3628 - val_acc: 0.9319
1472 Epoch 989/1000
1473 69s 137ms/step - loss: 0.0997 - acc: 0.9984 - val_loss: 0.3624 - val_acc: 0.9322
1474 Epoch 990/1000
1475 69s 138ms/step - loss: 0.0993 - acc: 0.9986 - val_loss: 0.3622 - val_acc: 0.9323
1476 Epoch 991/1000
1477 68s 137ms/step - loss: 0.0993 - acc: 0.9986 - val_loss: 0.3625 - val_acc: 0.9327
1478 Epoch 992/1000
1479 69s 137ms/step - loss: 0.0993 - acc: 0.9988 - val_loss: 0.3630 - val_acc: 0.9325
1480 Epoch 993/1000
1481 68s 137ms/step - loss: 0.0992 - acc: 0.9984 - val_loss: 0.3634 - val_acc: 0.9320
1482 Epoch 994/1000
1483 69s 138ms/step - loss: 0.0991 - acc: 0.9988 - val_loss: 0.3627 - val_acc: 0.9328
1484 Epoch 995/1000
1485 69s 138ms/step - loss: 0.0989 - acc: 0.9989 - val_loss: 0.3637 - val_acc: 0.9321
1486 Epoch 996/1000
1487 69s 138ms/step - loss: 0.0994 - acc: 0.9986 - val_loss: 0.3623 - val_acc: 0.9319
1488 Epoch 997/1000
1489 69s 138ms/step - loss: 0.0987 - acc: 0.9987 - val_loss: 0.3622 - val_acc: 0.9322
1490 Epoch 998/1000
1491 69s 138ms/step - loss: 0.0989 - acc: 0.9988 - val_loss: 0.3621 - val_acc: 0.9325
1492 Epoch 999/1000
1493 69s 138ms/step - loss: 0.0993 - acc: 0.9984 - val_loss: 0.3615 - val_acc: 0.9326
1494 Epoch 1000/1000
1495 69s 138ms/step - loss: 0.0986 - acc: 0.9988 - val_loss: 0.3614 - val_acc: 0.9323
1496 Train loss: 0.09943642792105675
1497 Train accuracy: 0.9982600016593933
1498 Test loss: 0.3614072059094906
1499 Test accuracy: 0.9322999995946885

在使用了shear_range = 30的数据增强以后，准确率降了呢。。

Minghang Zhao, Shisheng Zhong, Xuyun Fu, Baoping Tang, Shaojiang Dong, Michael Pecht, Deep Residual Networks with Adaptively Parametric Rectifier Linear Units for Fault Diagnosis, IEEE Transactions on Industrial Electronics, 2020, DOI: 10.1109/TIE.2020.2972458

https://ieeexplore.ieee.org/document/8998530

标签：acc,10,val,loss,调参,step,ReLU,Epoch,1000
来源： https://www.cnblogs.com/shisuzanian/p/12913541.html

本站声明： 1. iCode9 技术分享网（下文简称本站）提供的所有内容，仅供技术学习、探讨和分享；
2. 关于本站的所有留言、评论、转载及引用，纯属内容发起人的个人观点，与本站观点和立场无关；
3. 关于本站的所有言论和文字，纯属内容发起人的个人观点，与本站观点和立场无关；
4. 本站文章均是网友提供，不完全保证技术分享内容的完整性、准确性、时效性、风险性和版权归属；如您发现该文章侵犯了您的权益，可联系我们第一时间进行删除；
5. 本站为非盈利性的个人网站，所有内容不会用来进行牟利，也不会利用任何形式的广告来间接获益，纯粹是为了广大技术爱好者提供技术内容和技术思想的分享性交流网站。