PyTorch

Useful Tools for Training Neural Networks

Network structure visualization netron

2020-11-29

Running Jupyter Notebook/Lab on a remote server

SSH Tunneling Also known as port forwarding. I.e. setup a network tunnel (a connection for data to flow) from a local point to remote point. Example: ssh username@xx.xx.xx.xx -NL 1234:localhost:1234 1234:localhost:1234 means that any network request you send to port 1234 in your current system will be automatically forwarded to localhost:1234 from the remote system.

2020-11-29

Use tmux

What is tmux? Tmux is a terminal multiplexer You can start a Tmux session and then open multiple windows inside that session. Each window occupies the entire screen and can be split into rectangular panes.

2020-11-29

πŸ”₯🧾 General Training Steps Using PyTorch

Open in Google Colab General steps: Set device Set Dataset and DataLoader Define network model Build network model Define loss function and optimizer Define training process Train the model Store/Load weights

2020-11-26

πŸ”₯ Custom Datasets and Transforms

Custom Dataset In order to use our custom dataset, we need to inherit torch.utils.data.Dataset , an abstract class representing a dataset. override __len__ so that len(dataset) returns the size of the dataset.

2020-11-26

nn ModuleList vs. Sequential

import torch import torch.nn as nn import torch.nn.functional as F nn.Module Defines the base class for all neural network We MUST subclass it Example class Net(nn.Module): def __init__(self, in_c, n_classes): super().

2020-11-09

Saving and Loading Checkpoints

Motivation Saving and loading a general checkpoint model for inference or resuming training can be helpful for picking up where we last left off. When saving a general checkpoint, you must save more than just the model’s state_dict.

2020-11-06

πŸ”₯ Transfer Learning for Computer Vision

Handling settings for training and valiadtion phase flexibly πŸ’‘ Use Python dictionary Phase ('train' or 'val') as key For example: data_transforms = { # For training: data augmentation and normalization 'train': transforms.

2020-11-03

Using Convolution to Generalize

Convolutions Convolutions deliver locality and translation invariance If we want to recognize patterns corresponding to objects, we will likely need to look at how nearby pixels are arranged, and we will be less interested in how pixels that are far from each other appear in combination.

2020-10-27

Learning from Images

Dataset of images torchvision module: automatically download the dataset load it as a collection of PyTorch tensors For example, download CIFAR-10 dataset: from torchvision import datasets data_path = '../data-unversioned/p1ch7/' # root directory # Instantiates a dataset for the training data; # TorchVision downloads the data if it is not present.

2020-10-26