RuntimeError: The size of tensor a (2048) must match the size of tensor b (1024) at non-singleton dimension 0

[ClancyCC]

Hello! I got RuntimeError as follow. I found the shapes of factor_best.reshape(-1, 256, 1) and absmax are respectively [2048,256,1] and [1024,256,1].

RuntimeError: The size of tensor a (2048) must match the size of tensor b (1024) at non-singleton dimension 0
  File "utils.py", line 82, in search
    tau = factor_best.reshape(-1, 256, 1) * absmax + tau0

And this is because of the cat operation, so could you explain the detail about entropy calculation? (By the way, I don‘t know why calculate entropy like these), Thanks！

# utils.py, line53, torch.cat operation, why?
def evaluate_entropy(weight_int8, blocksize):
    device = weight_int8.device
    _weight_int8 = weight_int8.reshape(-1, 1)
    weight_nf4 = torch.cat((_weight_int8//16, _weight_int8%16), 1).reshape(1, -1, blocksize)
    weight_nf4_repeat = weight_nf4.repeat(16, 1, 1).to(device)
    values = torch.tensor(range(16)).reshape(16, 1, 1).to(device)
    freqs = (weight_nf4_repeat==values).sum(dim=-1, keepdim=True) / blocksize
    entropy = -freqs * torch.log2(freqs)
    entropy = torch.where(torch.isnan(entropy), 0, entropy)
    entropy = entropy.sum(dim=0)
    return entropy