[Feature Request] List of possible improvements

[Unco3892]

⚡ Feature Request

Motivation

Adding a list of general features to be implemented for the future versions.

Pitch

Major ideas

• Flexible Model Architectures: For the MDN model, it would be beneficial to make the neural network architecture modular, allowing for the incorporation of different layers and activation functions. For the SVGP model, adding the option to use different kernel functions could extend the model's flexibility.

• Improved Noise Modelling: Introducing the capability to use different types of noise in the MDN model could significantly improve the quality of uncertainty estimates.

• Classification Capabilities: Both the MDN and SVGP models could be extended to support binary and multi-class classification. This would involve modifying the likelihood function and the performance metric. This would, however, require extensive theoretical and empirical validation as MDNs and SVGPRs are not traditionally used for classification tasks.

• Hardware Utilization: The package could benefit from implementing multi-GPU support, which would allow for more efficient training of large models on large datasets. Optimizing the parallel data loading process for maximized CPU utilization could significantly improve overall performance.

• Benchmarking Performance: In the further iterations of this package, the performance, both in terms of speed and accuracy of prediction, can be measured against other models. For instance, one can compare our approach with a traditional GPR for larger and smaller datasets since, as already discussed, GPR has an algorithmic complexity of (O(N^3)) and does not scale well beyond a few hundred observations. Additionally, conformal prediction, and MAPIE can be compared with our techniques.

Minor ideas

• Remove any irrelevant dependencies that are already bundled via the main package + Make sure that the library is compatible with the current version or newer versions of the dependencies : Allow for better integrating with improved dependency management.

• Improved Code Documentation: The addition of type hints to the docstrings would offer better clarity and type checking, enhancing the readability and maintainability of the codebase. Additionally, the documentation can benefit from more examples.

• Add support (documentation) for R programming language through reticulate: Many users are interested also in having such techniques in R, which is particularly relevant since there are not many packages for uncertainty prediction.