Zener cards analysis

[Xenocrates]

I just carried out this experiment with my English class to practice writing experimental reports (not any freaky parapsychology stuff! ).

Zener cards are pictures of squares, circles, crosses, stars and waves - 5 types of picture. The experiment is for a 'sender' to pick the top card and imagine the picture and for the receivers to try to guess the picture using telepathy.

The results of one group, Lucia's, caught my eye. She had 5 receivers and 20 cards (i.e 5*20 guesses in total). 2 of the 5 receivers scored 0 correct answers and the other three scored only 1 each. i.e. only 3 correct guesses out of 100!

This must be extremely improbable. Can anyone give me the lowdown on analysing this result? What's the statistical significance of this experiment?

The other groups scored around 20% correct answers, as expected. So assuming Lucia followed instructions (and I think she did as cheats don't usually reduce their score ) what's the deal here?

 

[Xenocrates]

Also to clarify:

This experiment is the same as trying to guess the answer to 100 multiple choice questions where each has 5 choices and only getting three correct.

It's the same unless we bring telepathy into it, which I don't want to do ATM. It's only a report writing task with statistics.

 

[ParadigmShifter]

Probability of success = 0.2
Probability fail = 0.8

Probability (0 successes out of 20 trials) = 0.8^20 = 0.0115 ish i.e about 1 in 100.

Probability (1 success ot of 20 trials) = 20*0.8^19*0.2 = 0.0576 ish, i.e. about 1 in 20.

So not that eye-opening really.

http://mathworld.wolfram.com/BinomialDistribution.html

 

[Kennigit]

I'm guessing right now that the statistical significance of Lucia's group is the role of statistical anomaly.

Just one question- I assume that you wouldn't be told whether you got it right or wrong, otherwise it could narrow it down.

 

[ParadigmShifter]

I didn't notice that there were several groups guessing the cards. That makes it a bit more complicated.

 

[Xenocrates]

I first tried it with the binomial distribution, but that didn't really tell me what I wanted to know - the significance.

The only analysis that I could find 'off the shelf' is here:

http://paranormal.about.com/library/howto/htzener.htm

OK it's a paranormal website Anyway, if there's nought wrong with the statistics I'll go with that. I like it because it's in English rather than equations and suitable for my purposes in an English class. I just want to know if the maths is correct.

I followed through the formula and got an 'm' of 4.25. How significant is that really?

Also the binomial is used if the cards are replaced and reshuffled, but not in a closed system where getting a square diminishes the chance of getting another square.

@ I suspect that the sender told the receivers the answers as she went along. Against my instructions as usual.

 

[Xenocrates]

@ paradigm - yes it does complexify things!

It means the experiment is worthless to measure telepathy as you don't know which sender in the room you are receiving from! If telepathy exists that is....

Also there were four groups in the classroom, but each was independent.

 

[ParadigmShifter]

Right, thinking about it to analyse the statistical significance of multiple batches of trials you can consider each trial independent (since there is no knowledge shared between the guessers). So the statistical likelihood is still binomial, but with 100 trials. Of course, getting a specific number is rare for any trial involving 100 events (i.e. what is the probability of getting exactly 63 heads from 100 coin tosses, it's low).

So you will probably be more interested in the case "what is the probability of getting this amount of successes or less", which you can sum up using the formula from mathworld.

Since the events are independent it does not matter that one person guessed 0 correct, 3 guesed 1 correct, etc., it is just the total number of correct guesses in total that is important.

However, calculating the binomial coefficients (i.e. the entries in the (n+1)th row of Pascal's triangle) is pretty tedious.

Binomial coefficients: multipliers for the probabilities (for n trials and r successes)
1 1
1 2 1
1 3 3 1
1 4 6 4 1

etc.

The formula is nCr = n!/(n!*(n-r)!)

nCr is shorthand for the binomial coefficient.

where n is the total number of trials, and r is the number of successes.

The actual probability (n sucesses in r trials) is

nCr * pSuccess^r * pFail^(n-r)

where nCr is the binomial coefficient (see formula above).

EDIT: Changed N to r to save a bit of sanity.
EDIT2: Remembered to change both forumlae involving n and r.

 

[Kennigit]

If she told them the answers as they went along, or even a simple right or wrong, it would have increased the number of correct guesses. For example, if the 20 cards are split evenly the 5 ways, for 4 each, and it gets down to 5 cards left where there are 2 waves, 0 circles, 1 star, 1 cross, 1 suare, then you can narrow it down (guess wave- if right, then thats 1 right, wrong then 2/4 chance for next one).

 

[ParadigmShifter]

You can approximate binomials using the Normal distribution too, when the number of trials is large (20+).

That makes things alot easier. If you are familiar with hypothesis testing it's pretty easy to test the hypothesis that the number of correct guesses is exceptionally low. Mathworld again is your friend if you want to know about hypothesis testing.

 

[ParadigmShifter]

Ah... if you are using a finite deck too, and not shuffling after each trial, that makes it alot more complicated. I don't want to think about that really (goes for a lie down).

EDIT: Not shuffling basically gives the deck "memory" of previous events and you get into Bayesian probability... which, trust me, you don't want to get into on a random dicussion in a civ based forum.

 

[Kennigit]

I think right now this is the way to look at it-
1. somehow, experiment got screwed up/inconsistant between groups
2. statistical anomaly, meaning either that it was just lucky that way (bound to happen hypothetically, although unlikely, its still possible)
3. Telepathic abilities worked, but message got mixed up
4. Telepathic abilities worked, and as a joke she told them wrong cards.

Also, I think that when you crunch the numbers, if you guess something, get it wrong, then guess something else for the next one, its less likely than if you just randomly guessed again.

For example, lets say you guessed a circle for the first card and were told you got it wrong (against instructions). That means that it was a 1/4 chance that it would be the card you would guess next, bringing that guess to a 3/19 instead of 4/19. So maybe the people tried to figure it out, but instead kept decreasing their chances by guessing differently everytime. Not sure about the math, but it seems to make sense to me.

 

[Xenocrates]

I shuffled the whole deck (15 sets of zener cards - 75 cards) before dividing amongst the groups to try to get it to approximate the binomial. i.e. they wouldn't know how many of each card were in their part of the deck.

Thanks for help bros! I shall go into battle with more confidence now. Now the dodgy projector is my biggest fear!

Frankly it's been ~20 years since I last did this kind of stuff and I was pretty lost going through the calculations.

You'd love this school, they wake us up with Mozart at 6:30 and the lessons start at 7:30 and the school day finishes at 4:30 with compulsory homework class from 6:30 until 9:30. But fear not, the Chinese will never rule the World with teachers like me doing work on ghost picture analysis, 'Mike the headless chicken' and telepathy experiments!

 

[ParadigmShifter]

Right, looking at that website I think you messed up on step 11...

11. Complete the trial. The sender continues through the complete deck of cards, one card at a time, for as many trials as agreed upon. The experimenter should shuffle the cards between each trial.

Each guess is a "trial" here, not a run through of the whole deck. That would make the binomial analysis correct.

I'm not sure how they get the "m" statistic but it is probably an approximation of the Normal dist (the 1.96 figure sounds like it to me, which is the 95% significance level for Normal distribution of standard deviations from the mean). The Normal distribution approximates binomial extremely closely for 20+ trials where the success/fail probability is not extremely high or low.

 

[Xenocrates]

True. My experiment was pants. It wasn't easy to get them to understand the instructions anyway and there was only 40 minutes to do it. I didn't try to make a solid exp. only to get some numbers down, any numbers, and let them write a report on it. Hopefully they'll be able to identify the problems with my method in class today.

The boys were all around average, all a little under actually. Beth's group got 37 correct guesses out of 100 tries, but I suspect cheating. But Lucia's result seemed remarkably low and my eyebrows are raised!

 

[ParadigmShifter]

What was the total number of correct guesses out of 100?
Assuming the kids didn't know the number of symbols in each deck it is probably negligible the non-shuffling thing.

I could probably do a rough analysis of the likelihood given the number. If the total is a lot lower than 20/100 that would be significant but I can't give any figures at the moment.

EDIT: And I'd have to look out my stats tables too.

 

[Xenocrates]

The wonderful Lucia's group got 3 right out of 100. Two subjects = 0/20 and three had 1/20.

I did this testing myself when I was young (2 decades ago) and I got 70% + right on my two attempts (OK only two attempts). I've always been interested to see if telepathy gains credence, but it's not likely when few people experiment in this area. It's one of the modern taboos and I've been mauled twice for talking about it on OT already!

 

[ParadigmShifter]

3/100 sounds very low. I'll have a look and probably post the stats tomorrow. I suspect foul play

 

[Kennigit]

Nah, telepathy just doesn't make sense. Besides, if anyone tried to experiment, they would become distracted like Bill Murrey in Ghostbusters.

With the one group of 37/100, they probably were told if they were right or wrong. If you guess with intelligence, it can be narrowed down some, espescially if you get lucky and get some of the first ones right.

 

[ParadigmShifter]

I did some maths about the chance of getting 3 or less succesful guesses from 100 trials in a binomial trial with pSuccess = 0.2.

I summed up

0.8^100 + (100!/1!99!)0.8^99*0.2 + (100!/2!98!)0.8^98*0.2^2 + (100!/3!97!)0.8^97*0.2^3

I did some multilying out and rearranging and my final line of maths was

0.8^97*(0.512+12.8+158.4+1293.6) = 0.8^97 * 1465.312

Which is 5.8 * 10^-7 or thereabouts, approx 0.00000058 or 0.000058%

EDIT: 1 chance in 1715304 so nearly 2 million to 1 against.

So the results are statistically significant, I suspect foul play still.