PyTorch Lightning

The lightweight PyTorch wrapper for high-performance AI research. 

Lightning forces the following structure to your code which makes it reusable and shareable:

• Research code (the LightningModule).
• Engineering code (you delete, and is handled by the Trainer).
• Non-essential research code (logging, etc... this goes in Callbacks).
• Data (use PyTorch DataLoaders or organize them into a LightningDataModule).

Once you do this, you can train on multiple-GPUs, TPUs, CPUs and even in 16-bit precision without changing your code.

Lightning has over 40+ advanced features designed for professional AI research at scale.

Advantages over unstructured PyTorch

• Models become hardware agnostic
• Code is clear to read because engineering code is abstracted away
• Easier to reproduce
• Make fewer mistakes because lightning handles the tricky engineering
• Keeps all the flexibility (LightningModules are still PyTorch modules), but removes a ton of boilerplate
• Lightning has dozens of integrations with popular machine learning tools.
• Tested rigorously with every new PR. We test every combination of PyTorch and Python supported versions, every OS, multi GPUs and even TPUs.
• Minimal running speed overhead (about 300 ms per epoch compared with pure PyTorch).

https://github.com/PyTorchLightning/pytorch-lightning 

https://www.exxactcorp.com/blog/Deep-Learning/getting-started-with-pytorch-lightning

Lightning Flash

Lightning Flash offers a suite of functionality facilitating more efficient transfer learning and data handling, and a recipe book of state-of-the-art approaches to typical deep learning problems.

Like a set of Russian nesting dolls of deep learning abstraction libraries, Lightning Flash adds further abstractions and simplification on top of PyTorch Lightning. In fact we can train an image classification task in only 7 lines. We’ll use the CIFAR10 dataset and a classification model based on the ResNet18 backbone built into Lightning Flash. Then we’ll show how the model backbone can be repurposed for classifying a new dataset, CIFAR100,

While Lightning Flash is very much still under active development and has plenty of sharp edges, you can already put together certain workflows with very little code, and there’s even a “no-code” capability they call Flash Zero. For our purposes, we can put together a transfer learning workflow with less than 20 lines.

https://www.kdnuggets.com/2021/11/advanced-pytorch-lightning-torchmetrics-lightning-flash.html 

https://github.com/PyTorchLightning/lightning-flash 

Torchmetrics

TorchMetrics is a collection of 50+ PyTorch metrics implementations and an easy-to-use API to create custom metrics. It offers:

• A standardized interface to increase reproducibility
• Reduces boilerplate
• Automatic accumulation over batches
• Metrics optimized for distributed-training
• Automatic synchronization between multiple devices

You can use TorchMetrics with any PyTorch model or with PyTorch Lightning to enjoy additional features such as:

• Module metrics are automatically placed on the correct device.
• Native support for logging metrics in Lightning to reduce even more boilerplate.

https://github.com/PyTorchLightning/metrics 

https://torchmetrics.readthedocs.io/en/latest/