Marc record tensornetwork entropy #94
Conversation
Merged branch 'Marc_record_fisher_trace' of https://github.com/zincware/ZnNL into Marc_record_fisher_trace
also moved the entropy calculations to the observables
saved the tensornetwork matrix in parsed data instead of a class variable
| loss_derivative = np.array([[5, 4, 3], [2, 1, 0]]) | ||
|
|
||
| trace = compute_fisher_trace(loss_derivative=loss_derivative, ntk=ntk) | ||
| assert trace == 638 / 2 |
There was a problem hiding this comment.
Where is the 1/2 coming from? Is this just a simple solution to the analytic trace or are you normalising?
There was a problem hiding this comment.
Are all of these notebooks cleared of their output cells?
| ------- | ||
| Magnitude entropy of the matrix.""" | ||
|
|
||
| magnitude_dist = compute_magnitude_density(gram_matrix=matrix) |
There was a problem hiding this comment.
Will this be a normalised matrix?
| if len(sorted_ntk.shape) == 4: | ||
| sorted_ntk = np.mean(sorted_ntk, axis=(2, 3)) | ||
|
|
||
| """ |
There was a problem hiding this comment.
This code block should be removed for the merge into main.
| from jax import vmap | ||
|
|
||
|
|
||
| def compute_tensornetwork_matrix(ntk: np.ndarray, targets: np.ndarray): |
There was a problem hiding this comment.
Please add an output type:
def my_func(...) -> np.ndarray:| return tensornetwork_matrix | ||
|
|
||
|
|
||
| def _get_start_and_end_value(i, target_counts): |
There was a problem hiding this comment.
Perhaps you can put this into the matrix utils module and add a test for it.
| @@ -244,7 +276,7 @@ def instantiate_recorder(self, data_set: dict = None, overwrite: bool = False): | |||
| if overwrite: | |||
| self._index_count = 0 | |||
|
|
|||
| # Check if we need an NTK computation and update the class accordingly | |||
| # Check if we need an NTK computation, update the class accordingly. | |||
There was a problem hiding this comment.
Why? Or was this the line length issue?
| @@ -342,7 +405,7 @@ def visualize_recorder(self): | |||
| ------- | |||
|
|
|||
| """ | |||
| raise NotImplementedError("Not yet available in ZnRND.") | |||
| raise NotImplementedError("Not yet available in ZnNL.") | |||
| @@ -593,6 +701,7 @@ def gather_recording(self, selected_properties: list = None) -> dataclass: | |||
| db_data = self._data_storage.fetch_data(selected_properties) | |||
| # Add db data to the selected data dict. | |||
| for item, data in selected_data.items(): | |||
| print(item) | |||
| @@ -618,3 +727,27 @@ def _export_in_memory_data(self) -> dataclass: | |||
| } | |||
|
|
|||
| return DataSet(**selected_data) | |||
|
|
|||
| def _adjust_ntk_shape(self, parsed_data): | |||
There was a problem hiding this comment.
Do you have tests for this? This can probably also be a matrix util but then it should be moved to the utils module and heavily tested.
No description provided.