Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updated_model.py #1

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

CHIRANJEET1729DAS
Copy link

In this model the linear layers and conv layers are changed to the lazy version ,it reduces and provides flexibility in terms of memory consumption,explained below: Deferred Initialization:

Lazy Layers: Unlike regular layers (nn.Linear, nn.Conv2d), lazy layers do not require the specification of input dimensions at the time of model construction. Instead, they infer the input shape the first time data passes through them. Memory Allocation: Memory for weights and biases is not allocated when the lazy layer is instantiated. It is only allocated when the layer processes the first batch of data during the forward pass. Flexibility:

Dynamic Shape Inference: Lazy layers are useful when input dimensions might change or are not known in advance. This can be particularly beneficial in complex architectures or during rapid prototyping. Model Adaptability: They allow for building more adaptable models that can handle varying input sizes without requiring redefinition or adjustment of the layer parameters. Memory Efficiency:

Deferred Memory Allocation: By deferring memory allocation until the forward pass, unnecessary memory usage during model instantiation is avoided. This can reduce the initial memory footprint, especially in large models or when using multiple layers with unknown dimensions.

In this model the linear layers and conv layers are changed to the lazy version ,it reduces and provides flexibility in terms of memory consumption,explained below:
Deferred Initialization:

Lazy Layers: Unlike regular layers (nn.Linear, nn.Conv2d), lazy layers do not require the specification of input dimensions at the time of model construction. Instead, they infer the input shape the first time data passes through them.
Memory Allocation: Memory for weights and biases is not allocated when the lazy layer is instantiated. It is only allocated when the layer processes the first batch of data during the forward pass.
Flexibility:

Dynamic Shape Inference: Lazy layers are useful when input dimensions might change or are not known in advance. This can be particularly beneficial in complex architectures or during rapid prototyping.
Model Adaptability: They allow for building more adaptable models that can handle varying input sizes without requiring redefinition or adjustment of the layer parameters.
Memory Efficiency:

Deferred Memory Allocation: By deferring memory allocation until the forward pass, unnecessary memory usage during model instantiation is avoided. This can reduce the initial memory footprint, especially in large models or when using multiple layers with unknown dimensions.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant