Setting probability_two_random_records_match with prior information

[judah-axelrod]

Hi there, I am using splink in a setting where I do have a prior about the probability of two random records matching. For simplicity, let's say I know 65% of the M records in df1 are found in the N records of df2. This would suggest that I could manually set probability_two_random_records_match = (0.65*M) / (N*M) = 0.65 / N.

But I'm unsure whether this probability is supposed to be among all M*N possible records or just among those that are actually compared after blocking, which is a fraction of the amount. For example, if I assume that blocking doesn't remove any true pairs from consideration, would it make more sense to set probability_two_random_records_match = (0.65*M)/cumulative_comparisons_after_blocking?

[RobinL]

Yes, it's among all M*N possible record comparisons and so is NOT sensitive to the blocking rules used for prediction

[judah-axelrod]

Thank you, very helpful!