Extend Configs A common pattern is to extend an existing config, overriding and/or adding new config values to it. The extension is done by including the base configuration, and then
2023-02-23
Hydra is an open-source Python framework that simplifies the development of research and other complex applications. The name Hydra comes from its ability to run multiple similar jobs - much like a Hydra with multiple heads.
2023-02-23
2023-02-23
Before training the model, modules that need to be trained must be correctly registered. Otherwise, the unregistered modules would NOT be trained without errors or exceptions being thrown. Moreover, when we call model.
2022-03-09
Main Issues of Time Measurement GPU Execution Mechanism: Asynchronous Execution In multithreaded or multi-device programming, two blocks of code that are independent can be executed in parallel. This means that the second block may be executed before the first is finished.
2021-05-24
YACS stands for Yet Another Configuration System. It helps define and manage system configurations such as hyperparameters and architecture/module choices for training a model. A tool like this one is essential to reproducibility and is a fundamental component of the system.
2021-05-12
TorchScript A PyTorch model’s journey from Python to C++ is enabled by Torch Script, a representation of a PyTorch model that can be understood, compiled and serialized by the Torch Script compiler.
2021-04-21
Here I will document some problems and issues I encountered while training the neural network. Validation accuracy higher than training accuracy Possible reasons Regularization Regularization mechanisms, such as Dropout and L1/L2 weight regularization, are turned on at training time and turned off at testing time.
2021-02-23
What is data augmentation? To solve the problem that it’s hard to get enough data for training neural networks, image augmentation is a process of creating new training examples from the existing ones.
2021-01-17
Three core functions for saving and loading models: torch.save Saves a serialized object to disk. This function uses Python’s pickle utility for serialization. Models, tensors, and dictionaries of all kinds of objects can be saved using this function.
2021-01-17