[SciPy-User] Why do SciPy stats return very small but very high precision numbers?

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

[SciPy-User] Why do SciPy stats return very small but very high precision numbers?

Vicki Brown
Sometimes, I see results like this:

LeveneResult(statistic=1233.9270599281986, pvalue=6.212186164025708e-269) 

Is it even realistic to claim a value that is this small? (6.212 e-269)?

Other times, a function just returns 0.0.

Can someone point me to an explanation of how and why the functions decide o return 0 ("fer sure") vs "almost but not exactly 0, but close enough")

-- Vicki

Vicki Brown
cfcl.com/vlb




_______________________________________________
SciPy-User mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: Why do SciPy stats return very small but very high precision numbers?

Paul van Mulbregt
Hi Vicki,
Probabilities are often computed using a formula  that just returns a number and that number might be very small. E.g. the two-sided Kolmogorov-Smirnov test for large samples uses a theta function from scipy.special which can return probabilities as small as 1e-308. No attempt is made to decide that a value is so small that it should be replaced with 0. 
On the other hand if the probability is computed as p=1-q, where q is the output of the formula, then the smallest non-zero value returned will be about 1e-16.  
Some of the test/distributions compute the CDF, some the SF and some can do both.
I think returning a very small non-zero probability value is the correct thing to do and then let the user decide what to do with it. It may well be that a very small probability is sending a signal that the assumptions made in deciding to use this particular test are not valid. Or it could just be saying that the sample is very far removed from the target distribution. Either way it's up to the user who has the sample data to decide.

Paul

On Oct 15, 2019, at 3:11 PM, Vicki Brown <[hidden email]> wrote:

Sometimes, I see results like this:

LeveneResult(statistic=1233.9270599281986, pvalue=6.212186164025708e-269) 

Is it even realistic to claim a value that is this small? (6.212 e-269)?

Other times, a function just returns 0.0.

Can someone point me to an explanation of how and why the functions decide o return 0 ("fer sure") vs "almost but not exactly 0, but close enough")

-- Vicki

Vicki Brown
cfcl.com/vlb



_______________________________________________
SciPy-User mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/scipy-user

_______________________________________________
SciPy-User mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/scipy-user