club.leiphone.com/page/ (二維碼自動識別)

本文為 AI 研習社編譯的技術博客，原標題 ：

Transfer Leaning: How a Pre-Trained CNN could be used as a Feature Extractor作者 | Rakesh Thoppaen翻譯 | Disillusion

校對 | 醬番梨 整理 | 菠蘿妹

原文鏈接：medium.com/@Rakesh.thop

遷移學習：如何將預訓練CNN當成特徵提取器

目標：學習如何使用預訓練網路對完全不同的數據集進行分類

遷移學習涉及到使用一個特定數據集上訓練的模型

然後將其應用到另一個數據集上

使用預訓練好的模型作為「捷徑」，從其沒有訓練過的數據中學習模式的能力。

深度學習的魅力在於預訓練好的模型能夠對完全不同的數據集進行分類。這種能力內用了這些深度神經網路結構（在ImageNet數據集上進行過訓練）的預訓練權重並把其應用在我們自己的數據集上。

在產業中能用到的預訓練模型如下：

  • Xception
  • VGG16
  • VGG19
  • ResNet50
  • InceptionV3
  • InceptionResNetV2
  • MobileNet

//這些預訓練模型是keras的一部分。

一些流行的現有深度學習框架包括：Tensorflow, Theano, Caffe, Pytorch, CNTK, MXNet, Torch, deeplearning4j, Caffe2 。

Keras

Keras 是一種高層API，Keras由Python編寫而成並能夠在TensorFlow、Theano以及CNTK上運行。Keras提供了一種簡單及模塊化的API去創建和訓練神經網路，省去了大部分複雜的細節。這讓你入門深度學習變得非常簡單。

Keras用到了一些以Theano、TensorFlow為後端的深度學習函數庫。同時，這些函數庫通過一些更低層次的函數庫與硬體進行溝通。例如，如果你在CPU上運行程序，Tensorflow或者Theano應用BLAS函數庫。另一方面，當你在GPU上運行程序時，它們則會應用CUDA和cuDNN函數庫。

如果你正在建立一個新系統，你可能會想看一下這篇文章。

Keras提供了一種讓訓練和評估模型變得極其簡單的工作流程。詳見下圖：

Keras Workflow

通過特徵提取進行遷移學習案例：花的分類

步驟0：排列數據——訓練/測試和配置文件

我們將使用來自牛津大學的FLOWERS17數據集，從這裡下載數據集。你可以選擇任何數據並使用以下代碼執行分類。將標註好的訓練數據和測試數據放在dataset文件夾中。

Folder Structure

保存下列json代碼並命名為conf.json在上圖的conf文件夾中。

對「model」, 「feature_path」, 「label_path」, 」results」 依據你將要選擇的網路進行修改。在下述案例種，我使用了mobilenet預訓練網路。

「num_classes」表示你數據集中類的數量。

{
"model" : "mobilenet",
"weights" : "imagenet",
"include_top" : false,
"train_path" : "dataset/train",
"test_path" : "dataset/test",
"features_path" : "output/flowers_17/mobilenet/features.h5",
"labels_path" : "output/flowers_17/mobilenet/labels.h5",
"results" : "output/flowers_17/mobilenet/results.txt",
"classifier_path" : "output/flowers_17/mobilenet/classifier.pickle",
"model_path" : "output/flowers_17/mobilenet/model",

"test_size" : 0.10,
"seed" : 9,
"num_classes" : 17
}

類似的，如果你想用只有4個類的inceptionV3而不是mobilenet，使用以下json代碼。

{
「model」 : 「inceptionv3」,
「weights」 : 「imagenet」,
「include_top」 : false,
「train_path」 : 「dataset/train」,
「test_path」 : 「dataset/test」,
「features_path」 : 「output/flowers_17/inceptionv3/features.h5」,
「labels_path」 : 「output/flowers_17/inceptionv3/labels.h5」,
「results」 : 「output/flowers_17/inceptionv3/results.txt」,
「classifier_path」 : 「output/flowers_17/inceptionv3/classifier.pickle」,
「model_path」 : 「output/flowers_17/inceptionv3/model」,

「test_size」 : 0.10,
「seed」 : 9,
「num_classes」 : 4
}

步驟1：特徵提取

# filter warnings
import warnings
warnings.simplefilter(action="ignore", category=FutureWarning)
# keras imports
from keras.applications.vgg16 import VGG16, preprocess_input
from keras.applications.vgg19 import VGG19, preprocess_input
from keras.applications.xception import Xception, preprocess_input
from keras.applications.resnet50 import ResNet50, preprocess_input
from keras.applications.inception_resnet_v2 import InceptionResNetV2, preprocess_input
from keras.applications.mobilenet import MobileNet, preprocess_input
from keras.applications.inception_v3 import InceptionV3, preprocess_input
from keras.preprocessing import image
from keras.models import Model
from keras.models import model_from_json
from keras.layers import Input

# other imports
from sklearn.preprocessing import LabelEncoder
import numpy as np
import glob
import cv2
import h5py
import os
import json
import datetime
import time
# load the user configs
with open(conf/conf.json) as f:
config = json.load(f)
# config variables
model_name = config["model"]
weights = config["weights"]
include_top = config["include_top"]
train_path = config["train_path"]
features_path = config["features_path"]
labels_path = config["labels_path"]
test_size = config["test_size"]
results = config["results"]
model_path = config["model_path"]
# start time
print ("[STATUS] start time - {}".format(datetime.datetime.now().strftime("%Y-%m-%d %H:%M")))
start = time.time()

# create the pretrained models
# check for pretrained weight usage or not
# check for top layers to be included or not
if model_name == "vgg16":
base_model = VGG16(weights=weights)
model = Model(input=base_model.input, output=base_model.get_layer(fc1).output)
image_size = (224, 224)
elif model_name == "vgg19":
base_model = VGG19(weights=weights)
model = Model(input=base_model.input, output=base_model.get_layer(fc1).output)
image_size = (224, 224)
elif model_name == "resnet50":
base_model = ResNet50(weights=weights)
model = Model(input=base_model.input, output=base_model.get_layer(flatten).output)
image_size = (224, 224)
elif model_name == "inceptionv3":
base_model = InceptionV3(include_top=include_top, weights=weights, input_tensor=Input(shape=(299,299,3)))
model = Model(input=base_model.input, output=base_model.get_layer(custom).output)
image_size = (299, 299)
elif model_name == "inceptionresnetv2":
base_model = InceptionResNetV2(include_top=include_top, weights=weights, input_tensor=Input(shape=(299,299,3)))
model = Model(input=base_model.input, output=base_model.get_layer(custom).output)
image_size = (299, 299)
elif model_name == "mobilenet":
base_model = MobileNet(include_top=include_top, weights=weights, input_tensor=Input(shape=(224,224,3)), input_shape=(224,224,3))
model = Model(input=base_model.input, output=base_model.get_layer(custom).output)
image_size = (224, 224)
elif model_name == "xception":
base_model = Xception(weights=weights)
model = Model(input=base_model.input, output=base_model.get_layer(avg_pool).output)
image_size = (299, 299)
else:
base_model = None

print ("[INFO] successfully loaded base model and model...")
# path to training dataset
train_labels = os.listdir(train_path)
# encode the labels
print ("[INFO] encoding labels...")
le = LabelEncoder()
le.fit([tl for tl in train_labels])
# variables to hold features and labels
features = []
labels = []
# loop over all the labels in the folder
count = 1
for i, label in enumerate(train_labels):
cur_path = train_path + "/" + label
count = 1
for image_path in glob.glob(cur_path + "/*.jpg"):
img = image.load_img(image_path, target_size=image_size)
x = image.img_to_array(img)
x = np.expand_dims(x, axis=0)
x = preprocess_input(x)
feature = model.predict(x)
flat = feature.flatten()
features.append(flat)
labels.append(label)
print ("[INFO] processed - " + str(count))
count += 1
print ("[INFO] completed label - " + label)
# encode the labels using LabelEncoder
le = LabelEncoder()
le_labels = le.fit_transform(labels)
# get the shape of training labels
print ("[STATUS] training labels: {}".format(le_labels))
print ("[STATUS] training labels shape: {}".format(le_labels.shape))
# save features and labels
h5f_data = h5py.File(features_path, w)
h5f_data.create_dataset(dataset_1, data=np.array(features))
h5f_label = h5py.File(labels_path, w)
h5f_label.create_dataset(dataset_1, data=np.array(le_labels))
h5f_data.close()
h5f_label.close()
# save model and weights
model_json = model.to_json()
with open(model_path + str(test_size) + ".json", "w") as json_file:
json_file.write(model_json)
# save weights
model.save_weights(model_path + str(test_size) + ".h5")
print("[STATUS] saved model and weights to disk..")
print ("[STATUS] features and labels saved..")
# end time
end = time.time()
print ("[STATUS] end time - {}".format(datetime.datetime.now().strftime("%Y-%m-%d %H:%M")))

步驟2：訓練

# organize imports
from __future__ import print_function
from sklearn.metrics import classification_report
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import confusion_matrix
import numpy as np
import h5py
import os
import json
import pickle
import seaborn as sns
import matplotlib.pyplot as plt
# load the user configs
with open(conf/conf.json) as f:
config = json.load(f)
# config variables
test_size = config["test_size"]
seed = config["seed"]
features_path = config["features_path"]
labels_path = config["labels_path"]
results = config["results"]
classifier_path = config["classifier_path"]
train_path = config["train_path"]
num_classes = config["num_classes"]
classifier_path = config["classifier_path"]
# import features and labels
h5f_data = h5py.File(features_path, r)
h5f_label = h5py.File(labels_path, r)
features_string = h5f_data[dataset_1]
labels_string = h5f_label[dataset_1]
features = np.array(features_string)
labels = np.array(labels_string)
h5f_data.close()
h5f_label.close()
# verify the shape of features and labels
print ("[INFO] features shape: {}".format(features.shape))
print ("[INFO] labels shape: {}".format(labels.shape))
print ("[INFO] training started...")
# split the training and testing data
(trainData, testData, trainLabels, testLabels) = train_test_split(np.array(features),
np.array(labels),
test_size=test_size,
random_state=seed)

print ("[INFO] splitted train and test data...")
print ("[INFO] train data : {}".format(trainData.shape))
print ("[INFO] test data : {}".format(testData.shape))
print ("[INFO] train labels: {}".format(trainLabels.shape))
print ("[INFO] test labels : {}".format(testLabels.shape))
# use logistic regression as the model
print ("[INFO] creating model...")
model = LogisticRegression(random_state=seed)
model.fit(trainData, trainLabels)
# use rank-1 and rank-5 predictions
print ("[INFO] evaluating model...")
f = open(results, "w")
rank_1 = 0
rank_5 = 0
# loop over test data
for (label, features) in zip(testLabels, testData):
# predict the probability of each class label and
# take the top-5 class labels
predictions = model.predict_proba(np.atleast_2d(features))[0]
predictions = np.argsort(predictions)[::-1][:5]
# rank-1 prediction increment
if label == predictions[0]:
rank_1 += 1
# rank-5 prediction increment
if label in predictions:
rank_5 += 1
# convert accuracies to percentages
rank_1 = (rank_1 / float(len(testLabels))) * 100
rank_5 = (rank_5 / float(len(testLabels))) * 100
# write the accuracies to file
f.write("Rank-1: {:.2f}%
".format(rank_1))
f.write("Rank-5: {:.2f}%

".format(rank_5))
# evaluate the model of test data
preds = model.predict(testData)
# write the classification report to file
f.write("{}
".format(classification_report(testLabels, preds)))
f.close()
# dump classifier to file
print ("[INFO] saving model...")
pickle.dump(model, open(classifier_path, wb))
# display the confusion matrix
print ("[INFO] confusion matrix")
# get the list of training lables
labels = sorted(list(os.listdir(train_path)))
# plot the confusion matrix
cm = confusion_matrix(testLabels, preds)
sns.heatmap(cm,
annot=True,
cmap="Set2")
plt.show()

步驟3：測試

# test script to preform prediction on test images inside
# dataset/test/
# -- image_1.jpg
# -- image_2.jpg
# ...
# organize imports
from __future__ import print_function
# keras imports
from keras.applications.vgg16 import VGG16, preprocess_input
from keras.applications.vgg19 import VGG19, preprocess_input
from keras.applications.xception import Xception, preprocess_input
from keras.applications.resnet50 import ResNet50, preprocess_input
from keras.applications.inception_resnet_v2 import InceptionResNetV2, preprocess_input
from keras.applications.mobilenet import MobileNet, preprocess_input
from keras.applications.inception_v3 import InceptionV3, preprocess_input
from keras.preprocessing import image
from keras.models import Model
from keras.models import model_from_json
from keras.layers import Input

# other imports
from sklearn.linear_model import LogisticRegression
import numpy as np
import os
import json
import pickle
import cv2
# load the user configs
with open(conf/conf.json) as f:
config = json.load(f)
# config variables
model_name = config["model"]
weights = config["weights"]
include_top = config["include_top"]
train_path = config["train_path"]
test_path = config["test_path"]
features_path = config["features_path"]
labels_path = config["labels_path"]
test_size = config["test_size"]
results = config["results"]
model_path = config["model_path"]
seed = config["seed"]
classifier_path = config["classifier_path"]
# load the trained logistic regression classifier
print ("[INFO] loading the classifier...")
classifier = pickle.load(open(classifier_path, rb))
# pretrained models needed to perform feature extraction on test data too!
if model_name == "vgg16":
base_model = VGG16(weights=weights)
model = Model(input=base_model.input, output=base_model.get_layer(fc1).output)
image_size = (224, 224)
elif model_name == "vgg19":
base_model = VGG19(weights=weights)
model = Model(input=base_model.input, output=base_model.get_layer(fc1).output)
image_size = (224, 224)
elif model_name == "resnet50":
base_model = ResNet50(weights=weights)
model = Model(input=base_model.input, output=base_model.get_layer(flatten).output)
image_size = (224, 224)
elif model_name == "inceptionv3":
base_model = InceptionV3(include_top=include_top, weights=weights, input_tensor=Input(shape=(299,299,3)))
model = Model(input=base_model.input, output=base_model.get_layer(custom).output)
image_size = (299, 299)
elif model_name == "inceptionresnetv2":
base_model = InceptionResNetV2(include_top=include_top, weights=weights, input_tensor=Input(shape=(299,299,3)))
model = Model(input=base_model.input, output=base_model.get_layer(custom).output)
image_size = (299, 299)
elif model_name == "mobilenet":
base_model = MobileNet(include_top=include_top, weights=weights, input_tensor=Input(shape=(224,224,3)), input_shape=(224,224,3))
model = Model(input=base_model.input, output=base_model.get_layer(custom).output)
image_size = (224, 224)
elif model_name == "xception":
base_model = Xception(weights=weights)
model = Model(input=base_model.input, output=base_model.get_layer(avg_pool).output)
image_size = (299, 299)
else:
base_model = None

# get all the train labels
train_labels = os.listdir(train_path)
# get all the test images paths
test_images = os.listdir(test_path)
# loop through each image in the test data
for image_path in test_images:
path = test_path + "/" + image_path
img = image.load_img(path, target_size=image_size)
x = image.img_to_array(img)
x = np.expand_dims(x, axis=0)
x = preprocess_input(x)
feature = model.predict(x)
flat = feature.flatten()
flat = np.expand_dims(flat, axis=0)
preds = classifier.predict(flat)
prediction = train_labels[preds[0]]

# perform prediction on test image
print ("I think it is a " + train_labels[preds[0]])
img_color = cv2.imread(path, 1)
cv2.putText(img_color, "I think it is a " + prediction, (140,445), cv2.FONT_HERSHEY_SIMPLEX, 1, (0,0,255), 2)
cv2.imshow("test", img_color)

# key tracker
key = cv2.waitKey(0) & 0xFF
if (key == ord(q)):
cv2.destroyAllWindows()

期望結果

如果在你的電腦上啟用了GPU，你可以加速特徵提取和訓練進程。

想要繼續查看該篇文章相關鏈接和參考文獻？

長按鏈接點擊打開或：

ai.yanxishe.com/page/Te

AI研習社每日更新精彩內容，觀看更多精彩內容：

機器學習 2019：AI 發展趨勢分析

使用 SKIL 和 YOLO 構建產品級目標檢測系統

谷歌開源BERT不費吹灰之力輕鬆訓練自然語言模型

使用 Tensorflow 完成簡單的強化學習 Part 1：好奇心驅動的學習

等你來譯：

如何用PyTorch訓練圖像分類器

掌握機器學習必須要了解的4個概念

給你的電腦做個簡單的「人臉識別認證」

取得自然語言處理SOA結果的分層多任務學習模型(HMTL)


推薦閱讀：
查看原文 >>
相關文章