首页 > 其他分享> 文章详细

TF2-HyperparameterSearch-1

2020-02-06 17:40:40 阅读：324 来源： 互联网

标签：loss 1s sample Epoch 11610 HyperparameterSearch 100 TF2

超参数搜索

为什么要超参数搜索

*神经网络有很多训练过程中不变的参数
*网络结构参数:几层, 每层宽度(神经元个数), 每层的激活函数
*训练参数: batch_size , 学习率 , 学习率衰减算法等

手工去试耗费人力

网格搜索

遗传算法

*对自然界的模拟
*A:初始化候选参数结合 -> 训练 -> 得到的模型指标作为生存概率
*B:选择 -> 交叉 -> 变异 -> 产生下一代集合
*C:重新到A

启发式搜索

*Auto-ML
*使用循环神经网络生成参数
*使用强化学习来进行反馈,使用模型来生成参数

import tensorflow as tf
import matplotlib as mlp
import matplotlib.pyplot as plt
%matplotlib inline
import numpy as np
import pandas as pd
import sklearn
import os
import sys
from tensorflow import keras
print("tf.__version__ = ",tf.__version__)
print(keras.__version__)

tf.__version__ =  2.0.0-alpha0
2.2.4-tf

#Get data
from sklearn.datasets import fetch_california_housing
housing = fetch_california_housing()
data = housing.data
target = housing.target

#split train test valid data
from sklearn.model_selection import train_test_split
x_train_all , x_test , y_train_all , y_test = train_test_split(
    data,target,random_state = 7 , test_size = 0.25
)
print(x_train_all.shape)
x_train , x_valid , y_train , y_valid = train_test_split(
    x_train_all,y_train_all,random_state = 11 , test_size = 0.25
)
print(x_train.shape)

(15480, 8)
(11610, 8)

#Normarlization
from sklearn.preprocessing import StandardScaler
scaler = StandardScaler()
x_train_scaled = scaler.fit_transform(x_train)
x_valid_scaled = scaler.transform(x_valid)
x_test_scaled = scaler.transform(x_test)

#bulit model


#learning rate [1e-4 , 3e-3, 1e-3 ,3e-3 , 3e-2 , 1e-2 , 3e-2]
#单参数for循环
# w = w + grad * learning_rate
learning_rates = [1e-4 , 3e-4, 1e-3 ,3e-3 , 1e-2 , 3e-2]
historys = []  
for lr in learning_rates:
    
    model = keras.models.Sequential([
        keras.layers.Dense(
            30,
            activation='relu',
            input_shape=x_train.shape[1:]
        ),
        keras.layers.Dense(1)
    ])
    model.summary()
    
    optimizer = keras.optimizers.SGD(lr)
    model.compile(
        loss="mean_squared_error",
        optimizer=optimizer
    )
    callbacks = [keras.callbacks.EarlyStopping(patience=5,min_delta=1e-2)]
    
    
    #Train
    history = model.fit(
        x_train_scaled,
        y_train,
        validation_data = (x_valid_scaled,y_valid),
        epochs = 100,
        callbacks = callbacks
    )
    historys.append(history)

Model: "sequential"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
dense (Dense)                (None, 30)                270       
_________________________________________________________________
dense_1 (Dense)              (None, 1)                 31        
=================================================================
Total params: 301
Trainable params: 301
Non-trainable params: 0
_________________________________________________________________
Train on 11610 samples, validate on 3870 samples
Epoch 1/100
11610/11610 [==============================] - 1s 80us/sample - loss: 4.2326 - val_loss: 3.6016
Epoch 2/100
11610/11610 [==============================] - 1s 64us/sample - loss: 3.0963 - val_loss: 2.6659
Epoch 3/100
11610/11610 [==============================] - 1s 64us/sample - loss: 2.3809 - val_loss: 2.0708
Epoch 4/100
11610/11610 [==============================] - 1s 63us/sample - loss: 1.9106 - val_loss: 1.6816
Epoch 5/100
11610/11610 [==============================] - 1s 64us/sample - loss: 1.5842 - val_loss: 1.4146
Epoch 6/100
11610/11610 [==============================] - 1s 63us/sample - loss: 1.3553 - val_loss: 1.2324
Epoch 7/100
11610/11610 [==============================] - 1s 65us/sample - loss: 1.1888 - val_loss: 1.1036
Epoch 8/100
11610/11610 [==============================] - 1s 64us/sample - loss: 1.0666 - val_loss: 1.0115
Epoch 9/100
11610/11610 [==============================] - 1s 65us/sample - loss: 0.9759 - val_loss: 0.9450
Epoch 10/100
11610/11610 [==============================] - 1s 64us/sample - loss: 0.9085 - val_loss: 0.8977
Epoch 11/100
11610/11610 [==============================] - 1s 66us/sample - loss: 0.8583 - val_loss: 0.8621
Epoch 12/100
11610/11610 [==============================] - 1s 61us/sample - loss: 0.8201 - val_loss: 0.8354
Epoch 13/100
11610/11610 [==============================] - 1s 62us/sample - loss: 0.7912 - val_loss: 0.8152
Epoch 14/100
11610/11610 [==============================] - 1s 62us/sample - loss: 0.7687 - val_loss: 0.7995
Epoch 15/100
11610/11610 [==============================] - 1s 62us/sample - loss: 0.7512 - val_loss: 0.7872
Epoch 16/100
11610/11610 [==============================] - 1s 62us/sample - loss: 0.7372 - val_loss: 0.7771
Epoch 17/100
11610/11610 [==============================] - 1s 63us/sample - loss: 0.7258 - val_loss: 0.7689
Epoch 18/100
11610/11610 [==============================] - 1s 62us/sample - loss: 0.7163 - val_loss: 0.7618
Epoch 19/100
11610/11610 [==============================] - 1s 62us/sample - loss: 0.7082 - val_loss: 0.7556
Epoch 20/100
11610/11610 [==============================] - 1s 62us/sample - loss: 0.7013 - val_loss: 0.7500
Epoch 21/100
11610/11610 [==============================] - 1s 62us/sample - loss: 0.6950 - val_loss: 0.7450
Epoch 22/100
11610/11610 [==============================] - 1s 63us/sample - loss: 0.6894 - val_loss: 0.7404
Epoch 23/100
11610/11610 [==============================] - 1s 63us/sample - loss: 0.6843 - val_loss: 0.7360
Epoch 24/100
11610/11610 [==============================] - 1s 62us/sample - loss: 0.6795 - val_loss: 0.7319
Epoch 25/100
11610/11610 [==============================] - 1s 61us/sample - loss: 0.6751 - val_loss: 0.7279
Epoch 26/100
11610/11610 [==============================] - 1s 63us/sample - loss: 0.6708 - val_loss: 0.7241
Epoch 27/100
11610/11610 [==============================] - 1s 63us/sample - loss: 0.6668 - val_loss: 0.7204
Epoch 28/100
11610/11610 [==============================] - 1s 62us/sample - loss: 0.6629 - val_loss: 0.7167
Epoch 29/100
11610/11610 [==============================] - 1s 61us/sample - loss: 0.6592 - val_loss: 0.7131
Epoch 30/100
11610/11610 [==============================] - 1s 62us/sample - loss: 0.6556 - val_loss: 0.7096
Epoch 31/100
11610/11610 [==============================] - 1s 62us/sample - loss: 0.6521 - val_loss: 0.7061
Epoch 32/100
11610/11610 [==============================] - 1s 63us/sample - loss: 0.6487 - val_loss: 0.7027
Epoch 33/100
11610/11610 [==============================] - 1s 65us/sample - loss: 0.6454 - val_loss: 0.6994
Epoch 34/100
11610/11610 [==============================] - 1s 61us/sample - loss: 0.6422 - val_loss: 0.6961
Epoch 35/100
11610/11610 [==============================] - 1s 62us/sample - loss: 0.6390 - val_loss: 0.6928
Epoch 36/100
11610/11610 [==============================] - 1s 63us/sample - loss: 0.6359 - val_loss: 0.6896
Epoch 37/100
11610/11610 [==============================] - 1s 62us/sample - loss: 0.6329 - val_loss: 0.6865
Epoch 38/100
11610/11610 [==============================] - 1s 62us/sample - loss: 0.6299 - val_loss: 0.6834
Epoch 39/100
11610/11610 [==============================] - 1s 61us/sample - loss: 0.6270 - val_loss: 0.6804
Epoch 40/100
11610/11610 [==============================] - 1s 62us/sample - loss: 0.6242 - val_loss: 0.6773
Epoch 41/100
11610/11610 [==============================] - 1s 62us/sample - loss: 0.6214 - val_loss: 0.6744
Epoch 42/100
11610/11610 [==============================] - 1s 63us/sample - loss: 0.6187 - val_loss: 0.6715
Epoch 43/100
11610/11610 [==============================] - 1s 63us/sample - loss: 0.6160 - val_loss: 0.6686
Epoch 44/100
11610/11610 [==============================] - 1s 64us/sample - loss: 0.6134 - val_loss: 0.6657
Epoch 45/100
11610/11610 [==============================] - 1s 64us/sample - loss: 0.6108 - val_loss: 0.6630
Epoch 46/100
11610/11610 [==============================] - 1s 64us/sample - loss: 0.6083 - val_loss: 0.6602
Epoch 47/100
11610/11610 [==============================] - 1s 65us/sample - loss: 0.6058 - val_loss: 0.6575
Epoch 48/100
11610/11610 [==============================] - 1s 64us/sample - loss: 0.6033 - val_loss: 0.6548
Epoch 49/100
11610/11610 [==============================] - 1s 63us/sample - loss: 0.6009 - val_loss: 0.6521
Epoch 50/100
11610/11610 [==============================] - 1s 62us/sample - loss: 0.5985 - val_loss: 0.6495
Epoch 51/100
11610/11610 [==============================] - 1s 63us/sample - loss: 0.5962 - val_loss: 0.6469
Epoch 52/100
11610/11610 [==============================] - 1s 62us/sample - loss: 0.5939 - val_loss: 0.6443
Epoch 53/100
11610/11610 [==============================] - 1s 62us/sample - loss: 0.5917 - val_loss: 0.6418
Epoch 54/100
11610/11610 [==============================] - 1s 63us/sample - loss: 0.5895 - val_loss: 0.6393
Epoch 55/100
11610/11610 [==============================] - 1s 63us/sample - loss: 0.5873 - val_loss: 0.6368
Epoch 56/100
11610/11610 [==============================] - 1s 63us/sample - loss: 0.5852 - val_loss: 0.6343
Epoch 57/100
11610/11610 [==============================] - 1s 62us/sample - loss: 0.5830 - val_loss: 0.6320
Epoch 58/100
11610/11610 [==============================] - 1s 62us/sample - loss: 0.5810 - val_loss: 0.6296
Epoch 59/100
11610/11610 [==============================] - 1s 62us/sample - loss: 0.5789 - val_loss: 0.6272
Epoch 60/100
11610/11610 [==============================] - 1s 63us/sample - loss: 0.5769 - val_loss: 0.6249
Epoch 61/100
11610/11610 [==============================] - 1s 63us/sample - loss: 0.5749 - val_loss: 0.6226
Epoch 62/100
11610/11610 [==============================] - 1s 62us/sample - loss: 0.5729 - val_loss: 0.6204
Epoch 63/100
11610/11610 [==============================] - 1s 62us/sample - loss: 0.5710 - val_loss: 0.6182
Epoch 64/100
11610/11610 [==============================] - 1s 63us/sample - loss: 0.5691 - val_loss: 0.6160
Epoch 65/100
11610/11610 [==============================] - 1s 63us/sample - loss: 0.5672 - val_loss: 0.6138
Epoch 66/100
11610/11610 [==============================] - 1s 62us/sample - loss: 0.5654 - val_loss: 0.6116
Epoch 67/100
11610/11610 [==============================] - 1s 62us/sample - loss: 0.5635 - val_loss: 0.6096
Epoch 68/100
11610/11610 [==============================] - 1s 62us/sample - loss: 0.5617 - val_loss: 0.6075
Epoch 69/100
11610/11610 [==============================] - 1s 61us/sample - loss: 0.5600 - val_loss: 0.6055
Epoch 70/100
11610/11610 [==============================] - 1s 62us/sample - loss: 0.5582 - val_loss: 0.6034
Epoch 71/100
11610/11610 [==============================] - 1s 62us/sample - loss: 0.5565 - val_loss: 0.6014
Epoch 72/100
11610/11610 [==============================] - 1s 63us/sample - loss: 0.5548 - val_loss: 0.5995
Epoch 73/100
11610/11610 [==============================] - 1s 62us/sample - loss: 0.5531 - val_loss: 0.5975
Epoch 74/100
11610/11610 [==============================] - 1s 62us/sample - loss: 0.5515 - val_loss: 0.5957
Epoch 75/100
11610/11610 [==============================] - 1s 60us/sample - loss: 0.5499 - val_loss: 0.5938
Epoch 76/100
11610/11610 [==============================] - 1s 61us/sample - loss: 0.5483 - val_loss: 0.5919
Epoch 77/100
11610/11610 [==============================] - 1s 61us/sample - loss: 0.5467 - val_loss: 0.5901
Model: "sequential_1"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
dense_2 (Dense)              (None, 30)                270       
_________________________________________________________________
dense_3 (Dense)              (None, 1)                 31        
=================================================================
Total params: 301
Trainable params: 301
Non-trainable params: 0
_________________________________________________________________
Train on 11610 samples, validate on 3870 samples
Epoch 1/100
11610/11610 [==============================] - 1s 72us/sample - loss: 3.2795 - val_loss: 2.0846
Epoch 2/100
11610/11610 [==============================] - 1s 62us/sample - loss: 1.5364 - val_loss: 1.2477
Epoch 3/100
11610/11610 [==============================] - 1s 62us/sample - loss: 1.0293 - val_loss: 0.9800
Epoch 4/100
11610/11610 [==============================] - 1s 63us/sample - loss: 0.8595 - val_loss: 0.8856
Epoch 5/100
11610/11610 [==============================] - 1s 62us/sample - loss: 0.7882 - val_loss: 0.8426
Epoch 6/100
11610/11610 [==============================] - 1s 63us/sample - loss: 0.7521 - val_loss: 0.8157
Epoch 7/100
11610/11610 [==============================] - 1s 62us/sample - loss: 0.7289 - val_loss: 0.7950
Epoch 8/100
11610/11610 [==============================] - 1s 61us/sample - loss: 0.7108 - val_loss: 0.7769
Epoch 9/100
11610/11610 [==============================] - 1s 62us/sample - loss: 0.6950 - val_loss: 0.7603
Epoch 10/100
11610/11610 [==============================] - 1s 65us/sample - loss: 0.6806 - val_loss: 0.7446
Epoch 11/100
11610/11610 [==============================] - 1s 65us/sample - loss: 0.6672 - val_loss: 0.7298
Epoch 12/100
11610/11610 [==============================] - 1s 64us/sample - loss: 0.6546 - val_loss: 0.7156
Epoch 13/100
11610/11610 [==============================] - 1s 65us/sample - loss: 0.6428 - val_loss: 0.7026
Epoch 14/100
11610/11610 [==============================] - 1s 63us/sample - loss: 0.6317 - val_loss: 0.6902
Epoch 15/100
11610/11610 [==============================] - 1s 61us/sample - loss: 0.6212 - val_loss: 0.6787
Epoch 16/100
11610/11610 [==============================] - 1s 64us/sample - loss: 0.6115 - val_loss: 0.6677
Epoch 17/100
11610/11610 [==============================] - 1s 64us/sample - loss: 0.6024 - val_loss: 0.6573
Epoch 18/100
11610/11610 [==============================] - 1s 62us/sample - loss: 0.5938 - val_loss: 0.6478
Epoch 19/100
11610/11610 [==============================] - 1s 65us/sample - loss: 0.5857 - val_loss: 0.6388
Epoch 20/100
11610/11610 [==============================] - 1s 62us/sample - loss: 0.5781 - val_loss: 0.6303
Epoch 21/100
11610/11610 [==============================] - 1s 63us/sample - loss: 0.5709 - val_loss: 0.6221
Epoch 22/100
11610/11610 [==============================] - 1s 63us/sample - loss: 0.5641 - val_loss: 0.6144
Epoch 23/100
11610/11610 [==============================] - 1s 62us/sample - loss: 0.5577 - val_loss: 0.6071
Epoch 24/100
11610/11610 [==============================] - 1s 63us/sample - loss: 0.5516 - val_loss: 0.5999
Epoch 25/100
11610/11610 [==============================] - 1s 64us/sample - loss: 0.5457 - val_loss: 0.5935
Epoch 26/100
11610/11610 [==============================] - 1s 63us/sample - loss: 0.5402 - val_loss: 0.5872
Epoch 27/100
11610/11610 [==============================] - 1s 63us/sample - loss: 0.5349 - val_loss: 0.5811
Epoch 28/100
11610/11610 [==============================] - 1s 64us/sample - loss: 0.5299 - val_loss: 0.5755
Epoch 29/100
11610/11610 [==============================] - 1s 64us/sample - loss: 0.5251 - val_loss: 0.5700
Epoch 30/100
11610/11610 [==============================] - 1s 63us/sample - loss: 0.5206 - val_loss: 0.5647
Epoch 31/100
11610/11610 [==============================] - 1s 64us/sample - loss: 0.5163 - val_loss: 0.5597
Epoch 32/100
11610/11610 [==============================] - 1s 64us/sample - loss: 0.5122 - val_loss: 0.5550
Epoch 33/100
11610/11610 [==============================] - 1s 64us/sample - loss: 0.5083 - val_loss: 0.5506
Epoch 34/100
11610/11610 [==============================] - 1s 63us/sample - loss: 0.5046 - val_loss: 0.5462
Epoch 35/100
11610/11610 [==============================] - 1s 64us/sample - loss: 0.5012 - val_loss: 0.5424
Epoch 36/100
11610/11610 [==============================] - 1s 66us/sample - loss: 0.4979 - val_loss: 0.5385
Epoch 37/100
11610/11610 [==============================] - 1s 64us/sample - loss: 0.4947 - val_loss: 0.5349
Epoch 38/100
11610/11610 [==============================] - 1s 63us/sample - loss: 0.4917 - val_loss: 0.5315
Epoch 39/100
11610/11610 [==============================] - 1s 64us/sample - loss: 0.4889 - val_loss: 0.5281
Epoch 40/100
11610/11610 [==============================] - 1s 64us/sample - loss: 0.4862 - val_loss: 0.5248
Epoch 41/100
11610/11610 [==============================] - 1s 63us/sample - loss: 0.4836 - val_loss: 0.5217
Epoch 42/100
11610/11610 [==============================] - 1s 63us/sample - loss: 0.4812 - val_loss: 0.5190
Epoch 43/100
11610/11610 [==============================] - 1s 63us/sample - loss: 0.4788 - val_loss: 0.5162
Epoch 44/100
11610/11610 [==============================] - 1s 62us/sample - loss: 0.4766 - val_loss: 0.5133
Epoch 45/100
11610/11610 [==============================] - 1s 62us/sample - loss: 0.4746 - val_loss: 0.5105
Epoch 46/100
11610/11610 [==============================] - 1s 63us/sample - loss: 0.4726 - val_loss: 0.5084
Epoch 47/100
11610/11610 [==============================] - 1s 62us/sample - loss: 0.4707 - val_loss: 0.5062
Epoch 48/100
11610/11610 [==============================] - 1s 63us/sample - loss: 0.4688 - val_loss: 0.5039
Epoch 49/100
11610/11610 [==============================] - 1s 63us/sample - loss: 0.4671 - val_loss: 0.5017
Epoch 50/100
11610/11610 [==============================] - 1s 61us/sample - loss: 0.4654 - val_loss: 0.4999
Epoch 51/100
11610/11610 [==============================] - 1s 62us/sample - loss: 0.4637 - val_loss: 0.4973
Epoch 52/100
11610/11610 [==============================] - 1s 62us/sample - loss: 0.4622 - val_loss: 0.4958
Epoch 53/100
11610/11610 [==============================] - 1s 62us/sample - loss: 0.4607 - val_loss: 0.4942
Epoch 54/100
11610/11610 [==============================] - 1s 62us/sample - loss: 0.4592 - val_loss: 0.4925
Model: "sequential_2"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
dense_4 (Dense)              (None, 30)                270       
_________________________________________________________________
dense_5 (Dense)              (None, 1)                 31        
=================================================================
Total params: 301
Trainable params: 301
Non-trainable params: 0
_________________________________________________________________
Train on 11610 samples, validate on 3870 samples
Epoch 1/100
11610/11610 [==============================] - 1s 71us/sample - loss: 1.4974 - val_loss: 0.9273
Epoch 2/100
11610/11610 [==============================] - 1s 62us/sample - loss: 0.8025 - val_loss: 0.8286
Epoch 3/100
11610/11610 [==============================] - 1s 62us/sample - loss: 0.7442 - val_loss: 0.7829
Epoch 4/100
11610/11610 [==============================] - 1s 63us/sample - loss: 0.7062 - val_loss: 0.7466
Epoch 5/100
11610/11610 [==============================] - 1s 61us/sample - loss: 0.6750 - val_loss: 0.7142
Epoch 6/100
11610/11610 [==============================] - 1s 62us/sample - loss: 0.6476 - val_loss: 0.6851
Epoch 7/100
11610/11610 [==============================] - 1s 62us/sample - loss: 0.6235 - val_loss: 0.6601
Epoch 8/100
11610/11610 [==============================] - 1s 62us/sample - loss: 0.6021 - val_loss: 0.6388
Epoch 9/100
11610/11610 [==============================] - 1s 61us/sample - loss: 0.5835 - val_loss: 0.6196
Epoch 10/100
11610/11610 [==============================] - 1s 61us/sample - loss: 0.5666 - val_loss: 0.6022
Epoch 11/100
11610/11610 [==============================] - 1s 62us/sample - loss: 0.5517 - val_loss: 0.5859
Epoch 12/100
11610/11610 [==============================] - 1s 62us/sample - loss: 0.5385 - val_loss: 0.5733
Epoch 13/100
11610/11610 [==============================] - 1s 62us/sample - loss: 0.5269 - val_loss: 0.5612
Epoch 14/100
11610/11610 [==============================] - 1s 62us/sample - loss: 0.5168 - val_loss: 0.5501
Epoch 15/100
11610/11610 [==============================] - 1s 61us/sample - loss: 0.5081 - val_loss: 0.5408
Epoch 16/100
11610/11610 [==============================] - 1s 62us/sample - loss: 0.5001 - val_loss: 0.5324
Epoch 17/100
11610/11610 [==============================] - 1s 61us/sample - loss: 0.4930 - val_loss: 0.5250
Epoch 18/100
11610/11610 [==============================] - 1s 61us/sample - loss: 0.4868 - val_loss: 0.5181
Epoch 19/100
11610/11610 [==============================] - 1s 61us/sample - loss: 0.4810 - val_loss: 0.5113
Epoch 20/100
11610/11610 [==============================] - 1s 62us/sample - loss: 0.4756 - val_loss: 0.5055
Epoch 21/100
11610/11610 [==============================] - 1s 61us/sample - loss: 0.4707 - val_loss: 0.4999
Epoch 22/100
11610/11610 [==============================] - 1s 62us/sample - loss: 0.4664 - val_loss: 0.4953
Epoch 23/100
11610/11610 [==============================] - 1s 62us/sample - loss: 0.4622 - val_loss: 0.4900
Epoch 24/100
11610/11610 [==============================] - 1s 62us/sample - loss: 0.4586 - val_loss: 0.4859
Epoch 25/100
11610/11610 [==============================] - 1s 63us/sample - loss: 0.4550 - val_loss: 0.4821
Epoch 26/100
11610/11610 [==============================] - 1s 61us/sample - loss: 0.4516 - val_loss: 0.4783
Epoch 27/100
11610/11610 [==============================] - 1s 62us/sample - loss: 0.4484 - val_loss: 0.4748
Epoch 28/100
11610/11610 [==============================] - 1s 62us/sample - loss: 0.4455 - val_loss: 0.4719
Epoch 29/100
11610/11610 [==============================] - 1s 61us/sample - loss: 0.4428 - val_loss: 0.4685
Epoch 30/100
11610/11610 [==============================] - 1s 62us/sample - loss: 0.4399 - val_loss: 0.4653
Epoch 31/100
11610/11610 [==============================] - 1s 62us/sample - loss: 0.4376 - val_loss: 0.4628
Epoch 32/100
11610/11610 [==============================] - 1s 62us/sample - loss: 0.4348 - val_loss: 0.4605
Epoch 33/100
11610/11610 [==============================] - 1s 61us/sample - loss: 0.4328 - val_loss: 0.4573
Epoch 34/100
11610/11610 [==============================] - 1s 62us/sample - loss: 0.4303 - val_loss: 0.4543
Epoch 35/100
11610/11610 [==============================] - 1s 61us/sample - loss: 0.4283 - val_loss: 0.4524
Epoch 36/100
11610/11610 [==============================] - 1s 61us/sample - loss: 0.4264 - val_loss: 0.4504
Epoch 37/100
11610/11610 [==============================] - 1s 61us/sample - loss: 0.4244 - val_loss: 0.4474
Epoch 38/100
11610/11610 [==============================] - 1s 62us/sample - loss: 0.4225 - val_loss: 0.4450
Epoch 39/100
11610/11610 [==============================] - 1s 62us/sample - loss: 0.4207 - val_loss: 0.4426
Epoch 40/100
11610/11610 [==============================] - 1s 62us/sample - loss: 0.4191 - val_loss: 0.4410
Epoch 41/100
11610/11610 [==============================] - 1s 61us/sample - loss: 0.4175 - val_loss: 0.4389
Epoch 42/100
11610/11610 [==============================] - 1s 61us/sample - loss: 0.4160 - val_loss: 0.4372
Epoch 43/100
11610/11610 [==============================] - 1s 61us/sample - loss: 0.4145 - val_loss: 0.4355
Epoch 44/100
11610/11610 [==============================] - 1s 61us/sample - loss: 0.4129 - val_loss: 0.4342
Epoch 45/100
11610/11610 [==============================] - 1s 62us/sample - loss: 0.4117 - val_loss: 0.4334
Epoch 46/100
11610/11610 [==============================] - 1s 61us/sample - loss: 0.4103 - val_loss: 0.4303
Model: "sequential_3"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
dense_6 (Dense)              (None, 30)                270       
_________________________________________________________________
dense_7 (Dense)              (None, 1)                 31        
=================================================================
Total params: 301
Trainable params: 301
Non-trainable params: 0
_________________________________________________________________
Train on 11610 samples, validate on 3870 samples
Epoch 1/100
11610/11610 [==============================] - 1s 71us/sample - loss: 1.1967 - val_loss: 0.6288
Epoch 2/100
11610/11610 [==============================] - 1s 62us/sample - loss: 0.5528 - val_loss: 0.5565
Epoch 3/100
11610/11610 [==============================] - 1s 61us/sample - loss: 0.5040 - val_loss: 0.5199
Epoch 4/100
11610/11610 [==============================] - 1s 65us/sample - loss: 0.4811 - val_loss: 0.5051
Epoch 5/100
11610/11610 [==============================] - 1s 66us/sample - loss: 0.4636 - val_loss: 0.4811
Epoch 6/100
11610/11610 [==============================] - 1s 65us/sample - loss: 0.4530 - val_loss: 0.4710
Epoch 7/100
11610/11610 [==============================] - 1s 66us/sample - loss: 0.4448 - val_loss: 0.4661
Epoch 8/100
11610/11610 [==============================] - 1s 64us/sample - loss: 0.4392 - val_loss: 0.4560
Epoch 9/100
11610/11610 [==============================] - 1s 63us/sample - loss: 0.4345 - val_loss: 0.4538
Epoch 10/100
11610/11610 [==============================] - 1s 63us/sample - loss: 0.4296 - val_loss: 0.4467
Epoch 11/100
11610/11610 [==============================] - 1s 64us/sample - loss: 0.4270 - val_loss: 0.4412
Epoch 12/100
11610/11610 [==============================] - 1s 64us/sample - loss: 0.4219 - val_loss: 0.4391
Epoch 13/100
11610/11610 [==============================] - 1s 64us/sample - loss: 0.4193 - val_loss: 0.4354
Epoch 14/100
11610/11610 [==============================] - 1s 63us/sample - loss: 0.4155 - val_loss: 0.4305
Epoch 15/100
11610/11610 [==============================] - 1s 63us/sample - loss: 0.4136 - val_loss: 0.4258
Epoch 16/100
11610/11610 [==============================] - 1s 63us/sample - loss: 0.4109 - val_loss: 0.4254
Epoch 17/100
11610/11610 [==============================] - 1s 62us/sample - loss: 0.4082 - val_loss: 0.4239
Epoch 18/100
11610/11610 [==============================] - 1s 62us/sample - loss: 0.4048 - val_loss: 0.4198
Epoch 19/100
11610/11610 [==============================] - 1s 63us/sample - loss: 0.4041 - val_loss: 0.4158
Epoch 20/100
11610/11610 [==============================] - 1s 63us/sample - loss: 0.4002 - val_loss: 0.4135
Epoch 21/100
11610/11610 [==============================] - 1s 62us/sample - loss: 0.3992 - val_loss: 0.4109
Epoch 22/100
11610/11610 [==============================] - 1s 61us/sample - loss: 0.3971 - val_loss: 0.4110
Epoch 23/100
11610/11610 [==============================] - 1s 64us/sample - loss: 0.3941 - val_loss: 0.4081
Epoch 24/100
11610/11610 [==============================] - 1s 62us/sample - loss: 0.3937 - val_loss: 0.4063
Epoch 25/100
11610/11610 [==============================] - 1s 62us/sample - loss: 0.3907 - val_loss: 0.4017
Epoch 26/100
11610/11610 [==============================] - 1s 63us/sample - loss: 0.3887 - val_loss: 0.4004
Epoch 27/100
11610/11610 [==============================] - 1s 62us/sample - loss: 0.3877 - val_loss: 0.3988
Epoch 28/100
11610/11610 [==============================] - 1s 62us/sample - loss: 0.3875 - val_loss: 0.3999
Model: "sequential_4"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
dense_8 (Dense)              (None, 30)                270       
_________________________________________________________________
dense_9 (Dense)              (None, 1)                 31        
=================================================================
Total params: 301
Trainable params: 301
Non-trainable params: 0
_________________________________________________________________
Train on 11610 samples, validate on 3870 samples
Epoch 1/100
11610/11610 [==============================] - 1s 72us/sample - loss: 0.9782 - val_loss: 0.7611
Epoch 2/100
11610/11610 [==============================] - 1s 63us/sample - loss: 0.6947 - val_loss: 0.4780
Epoch 3/100
11610/11610 [==============================] - 1s 63us/sample - loss: 0.4401 - val_loss: 0.4418
Epoch 4/100
11610/11610 [==============================] - 1s 63us/sample - loss: 0.4207 - val_loss: 0.4653
Epoch 5/100
11610/11610 [==============================] - 1s 64us/sample - loss: 0.4076 - val_loss: 0.4363
Epoch 6/100
11610/11610 [==============================] - 1s 64us/sample - loss: 0.3966 - val_loss: 0.4069
Epoch 7/100
11610/11610 [==============================] - 1s 63us/sample - loss: 0.3889 - val_loss: 0.4016
Epoch 8/100
11610/11610 [==============================] - 1s 63us/sample - loss: 0.3989 - val_loss: 0.3973
Epoch 9/100
11610/11610 [==============================] - 1s 63us/sample - loss: 0.3811 - val_loss: 0.3908
Epoch 10/100
11610/11610 [==============================] - 1s 64us/sample - loss: 0.3782 - val_loss: 0.3884
Epoch 11/100
11610/11610 [==============================] - 1s 64us/sample - loss: 0.3755 - val_loss: 0.3870
Epoch 12/100
11610/11610 [==============================] - 1s 63us/sample - loss: 0.3731 - val_loss: 0.3804
Epoch 13/100
11610/11610 [==============================] - 1s 64us/sample - loss: 0.3698 - val_loss: 0.3892
Epoch 14/100
11610/11610 [==============================] - 1s 64us/sample - loss: 0.3679 - val_loss: 0.3817
Epoch 15/100
11610/11610 [==============================] - 1s 64us/sample - loss: 0.3653 - val_loss: 0.3780
Epoch 16/100
11610/11610 [==============================] - 1s 63us/sample - loss: 0.3646 - val_loss: 0.3749
Epoch 17/100
11610/11610 [==============================] - 1s 62us/sample - loss: 0.4333 - val_loss: 0.3921
Model: "sequential_5"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
dense_10 (Dense)             (None, 30)                270       
_________________________________________________________________
dense_11 (Dense)             (None, 1)                 31        
=================================================================
Total params: 301
Trainable params: 301
Non-trainable params: 0
_________________________________________________________________
Train on 11610 samples, validate on 3870 samples
Epoch 1/100
11610/11610 [==============================] - 1s 72us/sample - loss: 5.8795 - val_loss: 0.8450
Epoch 2/100
11610/11610 [==============================] - 1s 64us/sample - loss: 0.5768 - val_loss: 0.4919
Epoch 3/100
11610/11610 [==============================] - 1s 64us/sample - loss: 0.4555 - val_loss: 0.4367
Epoch 4/100
11610/11610 [==============================] - 1s 64us/sample - loss: 0.4437 - val_loss: 0.4472
Epoch 5/100
11610/11610 [==============================] - 1s 66us/sample - loss: 0.4109 - val_loss: 0.4106
Epoch 6/100
11610/11610 [==============================] - 1s 64us/sample - loss: 0.3987 - val_loss: 0.4107
Epoch 7/100
11610/11610 [==============================] - 1s 61us/sample - loss: 0.4086 - val_loss: 0.4026
Epoch 8/100
11610/11610 [==============================] - 1s 62us/sample - loss: 0.3899 - val_loss: 0.4080
Epoch 9/100
11610/11610 [==============================] - 1s 64us/sample - loss: 0.3892 - val_loss: 0.3873
Epoch 10/100
11610/11610 [==============================] - 1s 62us/sample - loss: 0.3782 - val_loss: 0.3947
Epoch 11/100
11610/11610 [==============================] - 1s 62us/sample - loss: 0.3792 - val_loss: 0.4038
Epoch 12/100
11610/11610 [==============================] - 1s 63us/sample - loss: 0.3800 - val_loss: 0.3878
Epoch 13/100
11610/11610 [==============================] - 1s 64us/sample - loss: 0.3719 - val_loss: 0.3980
Epoch 14/100
11610/11610 [==============================] - 1s 62us/sample - loss: 0.3694 - val_loss: 0.3703
Epoch 15/100
11610/11610 [==============================] - 1s 63us/sample - loss: 0.3713 - val_loss: 0.3838
Epoch 16/100
11610/11610 [==============================] - 1s 62us/sample - loss: 0.3647 - val_loss: 0.3797
Epoch 17/100
11610/11610 [==============================] - 1s 62us/sample - loss: 0.3595 - val_loss: 0.3627
Epoch 18/100
11610/11610 [==============================] - 1s 63us/sample - loss: 0.3553 - val_loss: 0.3611
Epoch 19/100
11610/11610 [==============================] - 1s 63us/sample - loss: 0.3542 - val_loss: 0.3694

# history = model.fit(
#     x_train_scaled,
#     y_train,
#     validation_data = (x_valid_scaled,y_valid),
#     epochs = 100,
#     callbacks = callbacks
# )

def plot_learning_curves(history):
    pd.DataFrame(history.history).plot(figsize=(8,5))
    plt.grid(True)
    plt.gca().set_ylim(0,1)
    plt.show()
    
for lr , history in zip(learning_rates,historys):
    print("Learning_rate:",lr)
    plot_learning_curves(history)

Learning_rate: 0.0001

在这里插入图片描述

Learning_rate: 0.0003

在这里插入图片描述
[

Learning_rate: 0.001

在这里插入图片描述

Learning_rate: 0.003

在这里插入图片描述

Learning_rate: 0.01

在这里插入图片描述

Learning_rate: 0.03

在这里插入图片描述

wenqiang su 发布了50 篇原创文章 · 获赞 0 · 访问量 4380 私信关注

标签：loss,1s,sample,Epoch,11610,HyperparameterSearch,100,TF2
来源： https://blog.csdn.net/weixin_42681868/article/details/104199169

本站声明： 1. iCode9 技术分享网（下文简称本站）提供的所有内容，仅供技术学习、探讨和分享；
2. 关于本站的所有留言、评论、转载及引用，纯属内容发起人的个人观点，与本站观点和立场无关；
3. 关于本站的所有言论和文字，纯属内容发起人的个人观点，与本站观点和立场无关；
4. 本站文章均是网友提供，不完全保证技术分享内容的完整性、准确性、时效性、风险性和版权归属；如您发现该文章侵犯了您的权益，可联系我们第一时间进行删除；
5. 本站为非盈利性的个人网站，所有内容不会用来进行牟利，也不会利用任何形式的广告来间接获益，纯粹是为了广大技术爱好者提供技术内容和技术思想的分享性交流网站。

ICode9

TF2-HyperparameterSearch-1

超参数搜索

为什么要超参数搜索

网格搜索

遗传算法

启发式搜索