인공 신경망 코드로 구현해서 다중 분류해보기 (1)

 

 

• 이전에 계속 배웠던 것들을 실제로 코드로 구현해보는데 (물론 프레임워크 쓰는거지만ㅋ) 기존에는 이해가 안되었던 것들이 왜 하는건지 어떤걸 하려는건지 보이니까 신기했다
• 역시 처음에는 이해가 안될지라도 꾸준히 계속 하는 것이 중요한 것 같다
• 그리고 내일 약속이 잡혀있는데 설렘반 기대반

 

 

 

 

 

 

---

 

 

 

 

 

 

1. TensorDataset과 DataLoader

• 입력 데이터를 쉽게 처리하고, 배치 단위로 잘러서 학습할 수 있게 도와주는 모듈
• Dataset : 학습시 사용하는 feature와 target의 pair로 이루어짐.
  
  • 아래에서 코드에서는 TensorDataset을 사용하여 Dataset 인스턴스를 생성했지만, 이미지의 사례와 같이 Dataset 클래스를 상속받아서 커스텀 인스턴스를 생성하는 형태로 많이 사용
• DataLoader: 학습 시 각 인스턴스에 쉽게 접근할 수 있도록 순회 가능한 객체(iterable)를 생성

• Sample code

from torch.utils.data import  TensorDataset, DataLoader

# X,y로 분할한 데이터를 tensor로 변환
X_train = torch.tensor(X_train, dtype=torch.float32)
X_test = torch.tensor(X_test, dtype=torch.float32)
y_train = torch.tensor(y_train, dtype=torch.int64)
y_test = torch.tensor(y_test, dtype=torch.int64)

# tensor를 TensorDataset으로 생성 - X와 y가 짝으로 이루어짐
train_dataset = TensorDataset(X_train, y_train)
test_dataset = TensorDataset(X_test, y_test)

# DataLoader 형태로 생성
train_dataloader = DataLoader(train_dataset, batch_size=10, shuffle=True)
test_dataloader = DataLoader(test_dataset, batch_size=10, shuffle=True)

• DataLoader가 하는 역할
  
  • shuffling
  • batch

2. Device 설정

• 일반적으로 인공신경망의 학습은 (가능하다면) GPU를 사용하는 것이 바람직함
• GPU를 사용하여 학습을 진행하도록 명시적으로 작성 필요
• 연산 유형에 따라 GPU에서 수행이 불가능한 경우도 존재하는데, 그럴 경우도 마찬가지로 명시적으로 어떤 프로세서
• 에서 연산을 수행해야하는지 코드로 작성해야함

device = 'cuda' if torch.cuda.is_available() else 'cpu'
model = NeuralNetwork().to(device)

• Cuda 설치 방법은 공식 문서 찾아보면서 하면 됨, 대신 cuda 랑 파이토치 버전을 잘 맞춰야 함. 초반에 이것 때문에 좀 고생;;

3. 신경망 생성

• torch.nn 패키지는 신경망 생성 및 학습 시 설정해야하는 다양한 기능을 제공

import torch.nn as nn

• 신경망을 nn.Module을 상속받아 정의
  
  • __ init __(): 신경망에서 사용할 layer를 초기화하는 부분
  • forward(): feed foward 연산 수행 시, 각 layer의 입출력이 어떻게 연결되는지를 지정

class NeuralNetwork(nn.Module):
    def __init__(self):
        super(NeuralNetwork, self).__init__()
        self.input_layer    = nn.Linear(4, 16)
        self.hidden_layer1  = nn.Linear(16, 32)
        self.output_layer   = nn.Linear(32, 3)
        self.relu = nn.ReLU()

    def forward(self, x):
        out =  self.relu(self.input_layer(x))
        out =  self.relu(self.hidden_layer1(out))
        out =  self.output_layer(out)
        return out

4. Model compile

• 학습 시 필요한 정보들(loss function, optimizer)을 선언
• 일반적으로 loss와 optimizer는 아래와 같이 변수로 선언하고, 변수를 train/test 시 참고할 수 있도록 매개변수로 지정해줌

learning_rate = 0.01
loss = nn.CrossEntropyLoss()
optimizer = torch.optim.Adam(model.parameters(),lr=learning_rate)

5. Train

• 신경망의 학습과정을 별도의 함수로 구성하는 것이 일반적
  
  • feed forward → loss → error back propagation → print(진행상황) 또는 로깅 → (반복)

def train_loop(train_loader, model, loss_fn, optimizer):
    for batch, (X, y) in enumerate(train_loader):
        X, y = X.to(device), y.to(device)
        pred = model(X)
        loss = loss_fn(pred, y)
        optimizer.zero_grad()
        loss.backward()
        optimizer.step()

6. Test

• 학습과정과 비슷하나 error back propagate하는 부분이 없음
  • feed forward → loss → print(진행상황) 또는 로깅 → (반복)

def test_loop(test_loader, model, loss_fn):
    size = len(test_loader.dataset)
    num_batches = len(test_loader)
    test_loss, correct = 0, 0

    with torch.no_grad():
        for X, y in test_loader:
            X, y = X.to(device), y.to(device)
            pred = model(X)
            test_loss += loss_fn(pred, y).item()
            correct += (pred.argmax(1) == y).type(torch.float).sum().item()

    test_loss /= num_batches
    correct /= size
    print(f"Test Error: \n Accuracy: {(100*correct):>0.1f}%, Avg loss: {test_loss:8f}\n")

7. Iteration

• 신경망 학습은 여러 epochs을 반복해서 수행하면서 모델을 구성하는 최적의 파라미터를 찾음
• 지정한 epochs 수만큼 학습과정과 평가과정을 반복하면서, 모델의 성능(loss, accuracy 등)을 체크함

epochs = 10
for i in range(epochs) :
    print(f"Epoch {i+1} \n------------------------")
    train_loop(train_dataloader, model, loss, optimizer)
    test_loop(test_dataloader, model, loss)
print("Done!")

8. 실습 : Iris 꽃 다중 분류

• iris 데이터셋을 사용하여 꽃의 품종을 구분하는 분류기를 신경망을 사용하여 구현
• iris 데이터셋은 4개의 feature와 3개의 label를 가지고 있음

[Step1] Load libraries & Datasets

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
import torch
import torch.nn as nn
from torch.utils.data import  TensorDataset, DataLoader

# 데이터 불러오기
iris = load_iris()
df = pd.DataFrame(data=iris.data, columns=iris.feature_names)
df['label'] = iris.target

# 데이터분할
y = df['label']
X = df.drop(['label'], axis=1)

X_train, X_test, y_train, y_test = train_test_split(X.values, y.values, random_state=42, stratify=y)

[Step2] Create DataLoader

X_train = torch.tensor(X_train, dtype=torch.float32)
X_test = torch.tensor(X_test, dtype=torch.float32)
y_train = torch.tensor(y_train, dtype=torch.int64)
y_test = torch.tensor(y_test, dtype=torch.int64)

train_dataset = TensorDataset(X_train, y_train)
test_dataset = TensorDataset(X_test, y_test)

train_dataloader = DataLoader(train_dataset, batch_size=10, shuffle=True)
test_dataloader = DataLoader(test_dataset, batch_size=10, shuffle=True)

[Step3] Set Network Structure

class NeuralNetwork(nn.Module):
    def __init__(self):
        super(NeuralNetwork, self).__init__()
        self.input_layer    = nn.Linear(4, 16)
        self.hidden_layer1  = nn.Linear(16, 32)
        self.output_layer   = nn.Linear(32, 3)
        self.relu = nn.ReLU()    
    
    def forward(self,x):
        out =  self.relu(self.input_layer(x))
        out =  self.relu(self.hidden_layer1(out))
        out =  self.output_layer(out)
        return out

[Step4] Create Model instance

device = 'cuda' if torch.cuda.is_available() else 'cpu'
print(f'device = {device}')

model = NeuralNetwork().to(device)

# device = cuda

[Step5] Model compile

# 모델 컴파일
learning_rate = 0.001
loss = nn.CrossEntropyLoss()
optimizer = torch.optim.Adam(model.parameters(),lr=learning_rate)

[Step6] Set train loop

def train_loop(train_loader, model, loss_fn, optimizer):
    size = len(train_loader.dataset)

    for batch, (X, y) in enumerate(train_loader):
        X, y = X.to(device), y.to(device)
        pred = model(X)

        # 손실 계산
        loss = loss_fn(pred, y)

        # 역전파
        optimizer.zero_grad()
        loss.backward()
        optimizer.step()
        
        loss, current = loss.item(), batch * len(X)
        print(f'loss: {loss:>7f}  [{current:>5d}]/{size:5d}')

[Step7] Set test loop

def test_loop(test_loader, model, loss_fn):
    size = len(test_loader.dataset)
    num_batches = len(test_loader)
    test_loss, correct = 0, 0

    with torch.no_grad():
        for X, y in test_loader:
            X, y = X.to(device), y.to(device)
            pred = model(X)
            test_loss += loss_fn(pred, y).item()
            correct += (pred.argmax(1) == y).type(torch.float).sum().item()

    test_loss /= num_batches
    correct /= size
    print(f"Test Error: \n Accuracy: {(100*correct):>0.1f}%, Avg loss: {test_loss:8f}\n")

[Step8] Run model

# 모델 실행
epochs = 20

for i in range(epochs) :
    print(f"Epoch {i+1} \n------------------------")
    train_loop(train_dataloader, model, loss, optimizer)
    test_loop(test_dataloader, model, loss)
print("Done!")

[8]
1초
# 모델 실행
epochs = 20

for i in range(epochs) :
    print(f"Epoch {i+1} \n------------------------")
    train_loop(train_dataloader, model, loss, optimizer)
    test_loop(test_dataloader, model, loss)
print("Done!")


#############실행결과#############
Epoch 1 
------------------------
loss: 1.229417  [    0]/  112
loss: 1.323886  [   10]/  112
loss: 1.411421  [   20]/  112
loss: 1.386905  [   30]/  112
loss: 0.733851  [   40]/  112
loss: 1.234458  [   50]/  112
loss: 1.305655  [   60]/  112
loss: 1.109646  [   70]/  112
loss: 1.298108  [   80]/  112
loss: 1.123639  [   90]/  112
loss: 1.252976  [  100]/  112
loss: 0.945725  [   22]/  112
Test Error: 
 Accuracy: 34.2%, Avg loss: 1.101148

Epoch 2 
------------------------
loss: 1.217736  [    0]/  112
loss: 1.141865  [   10]/  112
loss: 1.022278  [   20]/  112
loss: 1.132732  [   30]/  112
loss: 0.987784  [   40]/  112
loss: 1.072443  [   50]/  112
loss: 1.048232  [   60]/  112
loss: 1.058151  [   70]/  112
loss: 0.926058  [   80]/  112
loss: 1.073147  [   90]/  112
loss: 1.088172  [  100]/  112
loss: 1.048728  [   22]/  112
Test Error: 
 Accuracy: 34.2%, Avg loss: 1.009089

Epoch 3 
------------------------
loss: 0.949207  [    0]/  112
loss: 1.070655  [   10]/  112
loss: 0.997015  [   20]/  112
loss: 0.976548  [   30]/  112
loss: 0.954167  [   40]/  112
loss: 0.959842  [   50]/  112
loss: 0.941833  [   60]/  112
loss: 0.949394  [   70]/  112
loss: 1.013221  [   80]/  112
loss: 0.962253  [   90]/  112
loss: 0.961552  [  100]/  112
loss: 0.955065  [   22]/  112
Test Error: 
 Accuracy: 60.5%, Avg loss: 0.938423

...

Epoch 19 
------------------------
loss: 0.444971  [    0]/  112
loss: 0.436101  [   10]/  112
loss: 0.249345  [   20]/  112
loss: 0.421181  [   30]/  112
loss: 0.337932  [   40]/  112
loss: 0.201260  [   50]/  112
loss: 0.243774  [   60]/  112
loss: 0.387877  [   70]/  112
loss: 0.363123  [   80]/  112
loss: 0.357152  [   90]/  112
loss: 0.292318  [  100]/  112
loss: 0.610918  [   22]/  112
Test Error: 
 Accuracy: 97.4%, Avg loss: 0.348778

Epoch 20 
------------------------
loss: 0.302854  [    0]/  112
loss: 0.177556  [   10]/  112
loss: 0.458149  [   20]/  112
loss: 0.305042  [   30]/  112
loss: 0.256246  [   40]/  112
loss: 0.389602  [   50]/  112
loss: 0.365337  [   60]/  112
loss: 0.202098  [   70]/  112
loss: 0.348917  [   80]/  112
loss: 0.455435  [   90]/  112
loss: 0.360923  [  100]/  112
loss: 0.287352  [   22]/  112
Test Error: 
 Accuracy: 94.7%, Avg loss: 0.334741

Done!

 

 

 

 

 

 

---