Commits on May 21, 2024

1. Update model.py …

   In this model the linear layers and conv layers are changed to the lazy version ,it reduces and provides flexibility in terms of memory consumption,explained below:
   Deferred Initialization:
   
   Lazy Layers: Unlike regular layers (nn.Linear, nn.Conv2d), lazy layers do not require the specification of input dimensions at the time of model construction. Instead, they infer the input shape the first time data passes through them.
   Memory Allocation: Memory for weights and biases is not allocated when the lazy layer is instantiated. It is only allocated when the layer processes the first batch of data during the forward pass.
   Flexibility:
   
   Dynamic Shape Inference: Lazy layers are useful when input dimensions might change or are not known in advance. This can be particularly beneficial in complex architectures or during rapid prototyping.
   Model Adaptability: They allow for building more adaptable models that can handle varying input sizes without requiring redefinition or adjustment of the layer parameters.
   Memory Efficiency:
   
   Deferred Memory Allocation: By deferring memory allocation until the forward pass, unnecessary memory usage during model instantiation is avoided. This can reduce the initial memory footprint, especially in large models or when using multiple layers with unknown dimensions.

   

   CHIRANJEET1729DAS authored May 21, 2024
   Configuration menu

   • View commit details
   • Copy full SHA for a87c70b
   • Browse repository at this point

   Copy the full SHA

   a87c70b View commit details

   Browse the repository at this point in the history