1. Attention: We have put together a thread with tips and a tutorial video to help with using the new software. Please take a moment to check out the thread here: Trapshooters.com Tutorial & Help Video.
    Dismiss Notice

Recoil Test of Powders, part 2 (Winston)

Discussion in 'Shooting Related Threads' started by Neil Winston, Dec 26, 2009.

Thread Status:
Not open for further replies.
  1. Neil Winston

    Neil Winston Well-Known Member

    Joined:
    Jan 29, 1998
    Messages:
    7,840
    Last July I posted a modest test in which a single shooter tried to tried tell, by shooting, whether the powder in two consecutive shells was the same or different. The link above will give you a taste of the data, the discussion, and particularly the frantic aggressiveness of the devouts of the church of the “jab versus shove” dichotomy when their faith is questioned.

    Leaving out the crazies and the amazingly uninformed, most of the reasonable-unconvinced said that the subject pool (1) was too small and biased toward skepticism. They hinted that if there more testers, including some “believers,” that, as they say on Jeopardy, “the scores would really change.” That last requirement, that “belief” had some power to affect the result, is shared by promoters of ESP, who commonly dismiss unsuccessful attempts to verify any of it as the result of the negative vibration anyone who wants to know about the truth of anything brings to the test situation. It’s apparently one of these things where “you have to believe it to see it.”

    A smaller sub-group, beginning to discern the writing on the wall, changed the effect of different powders from “felt recoil” to “fatigue.” “You will be less tired after a day of shooting this compared with that,” they announced. I thought that was a completely unconvincing story, a detour to “somewhere else” that is stock-in-trade here on TS.com. I read English and know what is meant by the words “jab” and “shove” and know if either happens to me I don’t have to wait until tomorrow to find out which it was. However, if someone wants to do a blind “fatigue” test the field is open to them. I’m as open-minded as anyone about this stuff.

    I won’t go through the psychophysics again; it’s all in the first thread for anyone interested. This report is intended to provide the data which was requested in that earlier thread. I expect those who asked for this will fulfill their implied promise to accept the result now that the experiment they wanted has been done.

    Subjects: Eleven experienced shooters. Some are strong “believers” in jab versus shove, some equally strong disbelievers, still others were able to say they had no opinion but would be willing to help find out. A couple of early subjects had to be weeded out when it became clear that the presentation of stimuli (shell pairs) had not been well thought out.

    Equipment: as pictured on the earlier thread.

    Method: Each test consisted of ten trials. The subject fired a shell loaded with either Red Dot going (on the average) 1175 fps or PB with the same velocity. He or she did not know which powder was in that shell. Then the “test shell” was fired and the answer to the question “Were those two the same or different?” was recorded in the data book. Though not required, additional comments were welcomed and written down. About half the time this added insight was offered by the subjects, by some more often, by some less.


    Those comments were later analyzed based on the conventional wisdom of Red Dot and PB’s felt recoil. If the shooter said “the first one kicked more” and if it happened to be Red Dot, that comment was scored as “correct.” If it had been PB, then I called that “incorrect.” Words like “both were heavy” were scored using similar standards.

    Results:

    <a href="http://photobucket.com" target="_blank">[​IMG]</a>

    Discussion: In no case are we able to reject the “null hypothesis” that these powders cannot be distinguished by shooting them. No individual was able to do it at the standard scientific requirement of eight correct in ten, and as a group they were no better, with a general accuracy of only about 53%. The comments put an exclamation point on the “same or different” results since they were “wrong” at a far greater rate than they were “right,” the ratings being defined as described above.

    Conclusion: No one doing this could produce evidence which supports the idea that there’s any such thing is “jab versus shove” or that people who who think they are using those criteria are doing anything more than guessing. This was a surprise to some of the participants, no surprise at all to others.


    Neil

    ©2009 Neil Winston
     
  2. hmb

    hmb Well-Known Member

    Joined:
    Jan 29, 1998
    Messages:
    9,406
    Neil,

    Why have you set out on a mission to destroy all of the fundemental principles of trapshooting lore? HMB
     
  3. Unknown1

    Unknown1 Well-Known Member

    Joined:
    Nov 5, 2009
    Messages:
    3,395
    Neil,

    I hope those results didn't come as a surprise to you!

    MK
     
  4. racer

    racer TS Member

    Joined:
    Jan 29, 1998
    Messages:
    541
    What next? Are you going to prove I can't tell the difference between the pink and white AA wads? Dan :)
     
  5. Paladin

    Paladin Well-Known Member

    Joined:
    Jan 29, 1998
    Messages:
    4,383
    What were the comments from the two people that got 7 out of 10 correct? EM distinguished 7 out of 10. What did he feel? Were his comments eluding to random guesses?


    Comments were not required, but an interesting point is that only one person scored worse than 50% in "incorrect comments".

    What your test did prove is that 8 out of 10 could not tell a difference, and one person certainly did, unless he was guessing, witnessed by his comments, recorded or not recorded.
     
  6. Neil Winston

    Neil Winston Well-Known Member

    Joined:
    Jan 29, 1998
    Messages:
    7,840
    Sportshot, I'd like to address your question about comments before the other ones because the answer is quick.

    ED said, in the order he made the comment, which was half the time. I see that I gave him one more correct than he deserved:

    1. "Second Harder." And Red dot was second, so that's a "correct."

    2. "Second Harder." They were both Red Dot. so that's an "incorrect."

    3. "First Harder." PB was first, scored "incorrect"

    4. " "First Harder." PB was first, scored "incorrect"

    5. "Second Harder." Both Red Dot,"Incorrect."

    By now you see where that suspiciously high number of "incorrects" vs "corrects" comes from. You can only be correct one way, by picking the "right" powder when they are different. (If you think they are the same, you won't comment, generally.) But you can be wrong two ways, first by picking the wrong one, second by picking _either_ when in fact they are the same.

    I think this supports the contention that they are guessing. They comment about half the time so you would think 50/50 if they have equal chances to guess right or wrong. But given two ways to guess wrong they might be expected to be wrong twice as often, and they were.

    Neil
     
  7. fssberson

    fssberson Active Member

    Joined:
    Jan 29, 1998
    Messages:
    2,375
    Neil: You need to do a Chi Square comparison to determine the significant difference in your data. I love what you are doing and often stated that I could not tell the differenc in powders, primers, or wads. I can tell the difference in load weight = 1 oz. vs. 1 1/8 oz loads. What is your standard for determining the difference in felt recoil? 8 out of 10? Maybe, it needs to be 7 out of 10 or 15 out of 20?? This is great stuff. Fred
     
  8. brent375hh

    brent375hh TS Member

    Joined:
    Sep 15, 2009
    Messages:
    292
    Sportshot,

    Cut open a Winchester Universal and weight the 1 1/8 of shot. Then you will know why they kick less.
     
  9. Neil Winston

    Neil Winston Well-Known Member

    Joined:
    Jan 29, 1998
    Messages:
    7,840
    Could the 7-of-10-correct people "tell", to use Biged's words, or "distinguish" as Paladin puts it?

    This requires two answers, a long one, followed by a short. So I'm going to divide them into two separate posts, the first just a re-print of text I put on another thread in preparation for this. Readers who don't care to get into the messy details of decision making will miss little by skipping this one. It's covered in a few sentences in the next anyway.

    OK here's the long, non-mandatory one from Andrew's Quiz, my note to Wolfman. The null hypothesis in this thread's experiment is that a shooter can't tell if these two powders are the same or different, but that's just a detail. The argument is the same.

    "Wolfman, let me change that around a bit, will you? It's not critical here, but I'm soon going to be using the same line of argument Andrew has, and I don't want the herd to be spooked by a definitional problem once I have them safely moving down the arroyo.

    The null hypothesis here is that "target breaks cannot be read." Our task is to see if, using the established rules of experiment, we can reject that hypothesis. With the success rate of those who sent their tests back to Andrew, we are nowhere close to rejecting the null hypothesis, that is, nowhere near turning our backs on the idea that everyone is just guessing.

    We generally say we can reject that idea - that they are guessing - if the outcome of the experiment would only turn out this way about five times in a hundred if they were. Sure, that means we may give some credit where it's not due, but over time it's been a reasonable standard to balance the risks of two types of errors:

    1. That we will miss a "true" effect, that is, call something chance when it's really skill and

    2. That we will call some skill "real" when it's really just chance.

    It's important to remember how we are doing this, and what a "statistically significant" result tells us. Look at HMB's result, where he got 18 right by tracking wads. Let's assume that 18 of 23 is statistically significant and so we are able to reject the hypothesis that he's guessing. Does this mean he's "proved he can do it?"

    Far from it. In fact, we know that his result is one of those five (or fewer) in a hundred that are going to lead us to reject the null hypothesis in error. He got the score, but since he was basing his choices on random events (where the wad goes) he _can't_ be doing this by skill, no matter what the score. That's why we never conclude, when something is "statistically significant" it "proves" something is "real," it rather is a result that chips away at our proper scientific attitude of skepticism, the one that says that you have to do _something_ to convince me that there is anything here at all.

    In summary, we can maintain the null hypotheses if we don't see any reason to reject it.

    If we do see reason to reject it, and accept the alternative hypothesis, that this experimental subject has shown the skill, all we are doing is accepting a hypothesis which may or may not turn out to be right. We are not saying it's "proven;" it's just an idea which might be worth further investigation.

    Neil"

    © Neil Winston 2009
     
  10. Neil Winston

    Neil Winston Well-Known Member

    Joined:
    Jan 29, 1998
    Messages:
    7,840
    Biged and Paladin, you both feel that the 7-in-10-correct people have shown they can tell. Consider the following exchange at a bar:

    A guy says "I can control the fall of a coin flip with my mind." You are likely to say "Prove it."

    In ten flips he gets it right 7 times, is wrong 3. He says "Well that proves it, I can!" and you say . . . .?

    I doubt you are going to say you think he's proven anything, since you know by watching gambling on TV that such an outcome can occur by chance alone about 1 time in 9.

    So if you won't accept that he can control coins with his mind, why do you think these two subjects can "tell" powders when the cases are, at their heart, identical?

    Neil
     
  11. Paladin

    Paladin Well-Known Member

    Joined:
    Jan 29, 1998
    Messages:
    4,383
    Quoting Neil; "I see that I gave him one more correct than he deserved".

    Are you now saying EM scored 6 correct, 4 incorrect?

    What about MO, can you check his comments?
     
  12. K-80BUD

    K-80BUD TS Member

    Joined:
    Jan 31, 2008
    Messages:
    63
    Neil, An excellent test with the results I would expect. What amazes me is that with the results in black and white, people will not believe them. I would say to those who don't agree with the results that run their own tests and publish the results here. Bud Wood
     
  13. Paladin

    Paladin Well-Known Member

    Joined:
    Jan 29, 1998
    Messages:
    4,383
    "A guy says "I can control the fall of a coin flip with my mind." You are likely to say "Prove it."

    In ten flips gets it right 7 times, is wrong 3. He says "Well that proves it, I can!" and you say . . . .?"


    Come on Neil, you can't be serious with that type of comparison? If you are, I can certainly see where you are coming from.


    "why do you think these two subjects can "tell" powders when the cases are, at their heart, identical?"


    They aren't. They can't be.
     
  14. Neil Winston

    Neil Winston Well-Known Member

    Joined:
    Jan 29, 1998
    Messages:
    7,840
    No, Paladin. I though he got two comments right and three wrong. Rechecking showed he got one right, four wrong. But because of that bias making "incorrect" more likely, I'm not pushing the comments as being as "strong" as their ratios make them appear.

    Neil
     
  15. phirel

    phirel TS Member

    Joined:
    Jan 29, 1998
    Messages:
    9,556
    Neil- More factual data to support the conclusion that powder burning rates are so short in duration that nobody can really tell a difference between fast and slow powders. But, it will not change a believer. These individuals will believe something that is not based an critical measurements.

    Fred- Certainly we can tell the difference in recoil with different shot weights. But, in factory shells labeled 1 1/8 oz not every shell has exactly 1 1/8 oz of shot. Some have a little more, some have a little less. I have never hears the claim that anyone can tell the difference between the shells from one box. This difference is much greater than any difference in powder burning rates.

    Another complicating factor is gun mount. Gun mount can affect felt recoil, especially on the cheek. The way we mount and hold the gun is often a little different with different shots and this variation makes it difficult to test different powders.

    The measured difference between the push and jab is around 3-5/10,0000 of a second. Some believers claim they can distinguish between two events that differ by 3/10,000 of a second. I have difficulty believing them.

    Pat Ireland
     
  16. Neil Winston

    Neil Winston Well-Known Member

    Joined:
    Jan 29, 1998
    Messages:
    7,840
    Paladin, you wrote "Come on Neil, you can't be serious with that type of comparison?"

    I'm totally serious. It's the same test. Ten independent trials each with two possible outcomes, "right" or "wrong." The experimental design and logic are identical; the only difference is in the hardware.

    Neil
     
  17. Leo

    Leo Well-Known Member

    Joined:
    Jan 29, 1998
    Messages:
    3,044
    Electronic are a lot more repeatable than people. Why not tape a load cell to a shotgun butt and rest it in a post when you fire it. By recording pressure and time duration, you could conclusivley determine the outcome.

    I can tell the difference between a 1-1/8th ounce 3 dram load and a 1 ounce load. I can even tell the difference between a 1-1/16th ounce low recoil Fiocchi and a regular load, but I cannot tell the difference between a 1-1/8 ounce, 3 dram shell from brand to brand, including my own reloads.
     
  18. crusha

    crusha TS Member

    Joined:
    Jan 29, 1998
    Messages:
    5,762
    Hilarious.
     
  19. Paladin

    Paladin Well-Known Member

    Joined:
    Jan 29, 1998
    Messages:
    4,383
    The last piece of info that is missing is input from EM and MO, if they feel they could genuinely distinguish, or just guessing.


    If their response is that they could distinguish, you've got the task of telling them it doesn't matter, they are just random "hardware". LOL.


    It's time to blow snow.
     
  20. PerazziBigBore

    PerazziBigBore TS Member

    Joined:
    Jan 29, 1998
    Messages:
    3,859
    Looks to me like 20% of the people tested could tell.. 80% could not.. Sounds about right...
     
Thread Status:
Not open for further replies.
Search tags for this page

niel winston recoil test

,

niel winston recoil test trapshooters

,

winston recoil test trapshooters