Scientific Experiments in the Dojo?

Now and then I have had ideas that I have wanted to prove or disprove concerning certain practices in karate or the use of a certain supplement to enhance someone's training. More than once the idea has popped into my head that we all are armchair scientists in the dojo. We each have our own paradigms about the training world and we tinker with that paradigm to improve our own and our students' performance. However most karate instructors aren't properly trained as scientists, and almost nobody uses the scientific method to properly prove/disprove ideas or measure the differences between a standard and a new paradigm.

So to propose that we use our students as "lab rats" is not as crazy an idea as it first might seem. We all are doing it, but few do it the right way. When I say "the right way", I mean the following:

1) All efforts should start with a proper literature search; no experiments should be unnecessarily duplicated. All current knowledge should be brought to bear on the issue(s).
2) A proposal with a proper set of hypotheses and a sound experimentaldesign should be created. The principal investigator for a proposal should be a properly credentialed individual (MD, PhD, or Shihan).
3) The proposal should suggests hypotheses that are reasonably plausible. The subject matter should not be trivial.
4) The proposal should be reviewed by a human investigation committee. The committee should be made up of both scientists and lay people, karateka and nonkarateka.
5) Participants should be made aware of all known negative consequences of the experiment, and explicitly consent to being subjects. Informed consent documents should be signed and kept on file. 6) Results should be rigorously analyzed, reviewed, and presented to an appropriate audience. The results need not be made public (information could be kept "proprietary"), but they should definitely not go to waste.

I feel it's high time to start separating the myth from reality, the hype from the hope. This is not a proposal from Dr. Frankenstein's laboratory; rather it's an attempt to prod our group into being more disciplined about the knowledge we apply, teach, and create.

What do you think? This sure fits into the Shaolin/University model, doesn't it?

Bill Glasheen

Lets hear from you!

Dear Rich C.:

Yes. A cigar can be just a cigar. Thanks for the reminder.

Regarding "better" tips on stretching (and kicking), try Thomas Kurz's "Scientific Stretching", Stadion (?) Publishing. His book challenges some of the prevailing notions on stretching and offers tips on how to evaluate one's stretching potential and how to maximize it.

good training,

david moy

Bill:

This is gooooood very goood.......I would simply add one small thought. The martial arts systems are human constructs and as such they are artificial in form and "structure." Artifacts of our knowledge. I personally feel that the quest for the "traditional" way can be simply hiding behind the "Do" of the arts. An ignorant behavior that is well addressed in your reply. Tradition does not mean better. I think a man who had to train to save his life would embrace a lot of our modern findings...we have a duty to improve that which can be improved and modern "sport" science provides us with many useful and important tools.

ron


David

George is right - I am tickled somebody noticed and wrote such a thorough piece.

You have to understand something about me. Very few people (except for maybe
George) understand why I was so overjoyed to take my godan test with Manny
Neves (spelling). I don't know if you know this gentleman, but he is a
beast. He is also a lovable, respectful, and dedicated beast. I first worked
out with Manny a year before my test. The day before, Manny and I got a
chance to share a workout in George's class. I was very well aware of his
size, strength, and ability. At the end of our workout, Manny came up to me
and said "Bill, if we put them to sleep tomorrow, we both deserve to fail."
I gave him an ear-to-ear grin; he must have been reading my mind. The day of
the test was great fun. Great fun included Manny putting me on my butt twice
during the sparring match. Neither of us were perfect, but I think we showed
the lower ranks the joy of bringing the best out of ourselves. Art Rabesa
had plenty to tell us about our dan kumite, but WE GOT HIM TO CHUCKLE! It
wasn't about ego, or who is best, or whether we were going to pass. It was
all about discovering what we could create. We tapped into each other's
intensity and got more than we usually get.

Last night I was chatting with a smart GE engineer about the whole exchange.
Rich is a real concrete, straightforward guy. "Hey", he said, "I just want
to know if this technique or that exercise or this regimen is safe. I want
to know which set of stretches will keep me down in the lateral split. Now
you two are out in the stratosphere somewhere." Funny that I found myself
defending your position. You are right - far too much is made of waza and
far too little is understood about other critical pieces of our development.
Going into the nursery after my workout to pick up Chad I found myself
staring at the last scene of the Wizard of Oz that was playing on the TV.
'You've had the ability to go home all along, but you wouldn't have
appreciated it without the journey you took' said the good witch to Dorothy.
She - and you - make very good points.

You correctly point out many of the limits of science as applied by fallable
humans. You correctly point out how many inappropriately focus on what is
not important - including both martial artists and scientists. You spend
much of your working and hobby life seeing what is very frustrating in life.
You point out how screwed up many quests for knowledge, clarity, and
improvement are. That is indeed why I suggested things like a review
committee that would include martial artists as well as non-martial artists,
scientists as well as lay people. It's too easy to get wrapped up in one's
own narrow world and forget what really matters in life. It is so important
to tap into the breadth of humanity to occasionally reset our compass.

We need people like you. You care, you act, and you speak out. I am not
offended - I merely engage with those who test my thoughts and abilities
(that get's me in trouble with my wife a lot). You prodded dormant areas of
my pattern of thought and allowed me to get better persective.

Apology appreciated and not needed. Shoken accepted.

And by the way, I like the martial art area you are now dabbling in. I
learned aikido from a former green beret who also taught goju. The art is
only as good as the individuals who practice it and the limits they bring it
to. Plenty of good stuff in that whole arena to bring to sanchin ryu.

Hope to see you at camp, David.

Bill

Gentlemen: The cup is running over.

Freud is alleged to have once stated that "sometimes a cigar is just a cigar". To that point I believe that some people are quite interested in the quantifiable aspects of physical activites.I for one want to know which exercises can be injurious (ballistic stretching) and which are the most effective in improving perfomance. I do not want or need to hit the hardest, or be able to take the most punishment. What is important is that what I am doing produces measureable gain whether in mind, body or spirit.

In my experience, finding what works best for me in training is trial and error. Every so often a particular sequence of activities will give me a suddenand lasting improvement. Other exercises often just wear me down. Any approach that can focus on the former (PNF stretching) and eliminate any of the latter is going to be worthwhile.

To be effective, data must be real, not anecdotal. A scientific approach is a must. As an example, our martial arts pathfinders were not born with instinctive knowledge of pressure points and weapons techniques. They simply lined up prisoners of war, extra slaves, enemies etc. and began hacking and stabbing and bludgeoning away until they charted the most effective places to hack and stab the human body. It was all part of the "WAY".

Much can still be learned by studying the results of our training methods using a more modern and civilized approach.

Rich Castanet

[The following two letters from David Moy deal more with the process of this forum than the subject. I hope David will continue to contribute to this and other "Point of View!" topics.]

Subject: Apology

Dear Sensei,

Didn't mean to raise a controversy here. I am sending a reply to Mr. Glasheen and will do a graceful fade.

Thank you for your kind words about my writing by the way.

sincerely

david moy

David:
Please don't misinterprete Bill's words. That simply, is the way he discusses an issue. He is tickled pink that you are participating in the forum and certainly will be hurt if you believed to have insulted him.

Bill works out the way he talks. . . straight, honest and very much appreciative of other people's point of view. We may have "beaten" this subject into the ground, but please watch for other areas, or start a new one, where we as a group can expand our understanding. GEM


Dear Mr Glasheen,

I want to start off by apologizing. Clearly, I have offended you. My choice of words was not the best, especially with "lest you lead a bunch of students down the wrong path..." It wasn't my intention to say "you" literally, but I find it hard to say "we" since I am not an instructor/sensei/shihan. Merely, a student. It was my intention to sound a note of caution in response to your invitation, "Lets hear from you!" on this issue of experimentation in the dojo.

If you read my letter again, I think you will find that we are in agreement that the physical dimension of our practice can be measured and improved. And, with your example of "kenpo stretching" I agree wholeheartedly and have seen and personally experienced the detrimental effects (i.e. hamstring pull) due to this type of stretching. By all means, lets look at what is dangerous and what is more efficient. But, again, proceed with caution and clarity about what is being intended. My uneasiness I will come back to later on.

With respect to the other domains -- mind and spirit -- you are absolutely right in charging me with presumptiousness in assuming that these would not be included in your proposed research. I read your opening statement about research to "improve our own and our students'performance" to immediately mean within the physical realm. My error since I honestly cannot see how the other realms would be measured. Again, I am not a researcher and I could most definitely be wrong.

You mentioned and likened my "piece" to the book, "Confessions of a Bodybuilder. "Ouch!!!" A perfect shoken to the solar plex. I was hurt by that analogy because, like you, I am interested in the "process" and would have been disappointed if I were to read the book and find, in the end, the guy didn't talk about the "process" he went through. But, after some thought... You may be right about me. Who knows, I may yet stop practicing and say budo "sucks and we should all have butter on our vegetables." Perhaps, not tomorrow, but next week, next month... If I do and were to turn (or remain) dissatisfied enough with my life to seek the help of "an experienced psychologist" whom you suggest can make "MEASURABLE improvements" in my life, then the situation begs these questions. One, against whose standards would the improvement in my life be measured by -- the psychologist, society, or both? If not these, then the measurement must be against some unrecognized, undeveloped or unrefined standards relative to and pertaining to myself. How will these fit into an "experimental design"? Again, can we will ever agree on constructs for the "mind" and "spirit" and operationalized them; how do you test for "reliability" and "validity", especially given the (hopefully) diverse student body of the dojos in terms of age, socioeconomic status, racial and ethnic background, and any other number of variable factors.

Then we come to the "ethical" consideration of this experiment. You mentioned an ADD child, Asperger's Syndrome child, women students with sexual abuse histories (there are probably men students with that too), and an injured middle age man. Like you, I would like to help these folks. (BTW, I am one of those out-of-vogue, liberal "secular humanists" who work with "disadvantaged" children and adults.) But, in the proposed experimental design, how will you deal with the expectations of the ADD child and parents; how will you ask the sexually abused to participate in an "experiment" on the most effective approaches to make them feel safe, integrated and whole again? And, how long will you keep the experimental design going with an "experimental group" and a random "control" group to ensure the best possible longitudinal data and the success of the experiment?

These are my concerns and doubts, and why I immediately concluded the focus of your research to be strictly in the physical realm. My apologies and Godspeed if you and others can successfully address those issues in your research design.


Returning to the physical realm which has proven to be measurable, I agree it would be beneficial to examine what we do for sound physiological approaches. But, I will again sound my caution. In doing such experimentation, extreme care and consideration should be given to what's to be achieved and the possible ramifications, especially with students' expectations. Students come into the practice with different reasons. But, a preponderant (or ancillary) reason is the development of martial powress (based on my own unscientific poll). Without much explanation and caveats, could such an experiment not promote unrealistic and even unhealthy expectations -- a search for the "holy grail", the Ultimate Martial Art, as it were? Then, again, students don't go down that path unless they want to. I know. Like I said, I was one. I went from Hung Gar kung fu to Uechi Ryu (BTW, my kung fu sihings <sempais> admonished me for heading the wrong way), to boxing, to American Karate (kickboxing). But, outside of Uechi-Ryu, there were certainly instructors who implicitly or explicitly encouraged that route -- "my boxing style is more scientific than that other trainer's; boxing is good but why not add on kicks, do the heavy bag and sparr everyday; why not learn grappling so when you get them down, the fight always goes down, you can joint lock or choke them out..." Now. I do aikido. Ironically, aikido is probably more denigrated than most other arts for perceived "lack of effectiveness". As I heard one tell it -- "It's like two dancers doing underwater ballet." Of course, I still care about effectiveness, about safety especially to this old, aging body. But, this concern (obsession, perhaps) is tempered by a more primary and ascending motive to, as you so well put it, "strive for a better understanding of the process". It's taken awhile for me get here. But, along the way, the countless fellow students have who dropped out (which is fine and part of, perhaps, their own process) and, more tragically, died in an obsessed and dangerous search for that martial grail.

In your opening statement about research, you mentioned measuring the difference from the "standard and a new paradigm". Well, there is a historical parallel. Again, in mentioning it, I don't mean to equate you with that situation. But, it is a cautionary tale, one which you may well know, and one which could of course be judged to be totally irrelevant here.

Jigaro Kano, the founder of Judo was a practitioner of some of the older jujutsu (combat) styles and an educator. He felt that the older jujutsu styles should and could be transformed as a safe means for promoting the development of human potential and character. In the late 1800's, he set about developing his Kodokan school, incorporating and modifying techniques of various jujutsu schools to make them safe for general, public practice. To test and to promote his art, he started shiais (competitions) where he challenged practitioners of other arts to compete and see whose art was "more" effective. Most of the artists of course declined since their arts were for combat and victory was determined not by points but by maiming or killing the opponent. To top it off, he was able to recruit one Shiro Saigo to be his head instructor and exponent of judo. Saigo happened to be the adopted son of Tanomo Saigo, headmaster in the Daito Ryu aikijujutsu lineage, and was thus already well trained in a combat art. Shiro Saigo demolished all comers in competition, mostly using an (ironically) aikijujutsu technique called "Yama-arashi" (mountain storm). No one from Kodokan has reputedly used it since. Anyway, at the turn of the century, Saigo for unpublicized reasons left the Kodokan, never to practice judo nor Daito Ryu again as far as publicly known.

In addition to competition, the other "gift" to the marital arts world from Professor Kano was the belt ranking system. Because of his success in promoting everything judo, many if not all the martial arts (including now the Chinese arts), except the older koryu systems, have adopted some form of his belt ranking system. In judo, competition is highly valued and it is not unheard of for a newer but highly successful competitive student to quickly rise through the ranks over others who studied longer but have not been as successful in competition. By most accounts, Professor Kano was a man committed to the development of human potential as well as man with scholarly, "scientific" background and inclination. Yet, competitions and ranking are his questionable legacy to the extent these been have overly emphasized, leading to an often distorted perception of what and why we practice.

This letter and the previous were simply reflections of my opinion, not judgments against you or any other instructor or practitioner in this forum. By the very nature of our own individual practice, we each make the art our own. What I say or do can not change much of that for anybody else. Nevertheless, my apology for perhaps saying too much.

Respectfully,

david moy

Mr. Moy

Thank you so much for your provocative and thoughtful piece. You bring up some very valid, and interesting points. Your particular perception of what I have in mind makes me realize what preconceptions exist out there. I am glad you articulated your concerns.

I would first like to respond to the "why". Whether you realize it or not, we are all operating in a natural experiment. If uechiryu were the eight commandments and we all practiced the same way, well then there would be no point. However even the most basic elements of what we do are subject to question. There are many ways, many methods, and many approaches. Some are clearly more enlightened than others.

The "process" of martial training is a human construct. This process or way is not an aimless one. It is true that no one way is perfect and no one way is the best for all. However you cannot tell me that it isn't possible to assess "better" ways of approaching the journey. This reminds me of some of the most basic philosophical discussions about beauty dating back to the age of Aristotle. The adage "beauty is in the eye of the beholder" is a truism. However it is also true that a consensus of what is "beautiful" can be reached - hence beauty can be defined on a relative scale. Limiting beauty to one single dimension - the physical - is an argument commonly employed by naysayers to obfuscate discussion of the subject. There are several women I have met in my life who make my heart palpitate when they talk about their area of professional expertise. The reaction (in me) is just as powerful as the one many men get when they observe mammary glands at a provocative angle of suspension. Why do you assume that I only consider the physiology while you (and only you) have rejected that and found meaning in other dimensions of the journey? What is so special about what you do that has not been considered by others?

If you wish to peg the scientific method to a merely physiologic domain, well there is still plenty to be done. Have you been aware of the constant debate going on about our kotekitae? On the one extreme, we have people doing knuckle slammers on hard floors and sending young children home black & blue to their parents. On the other extreme, we have people out there stating that bone bruising causes cancer. Is the first set of practices safe? Is the second statement worthy of consideration? What about our Uechi warm up exercises? The junbi undo is riddled with a practice called "ballistic stretching", something now considered to be inferior training. In the seventies I was taught "kenpo stretching" by a well groomed Uechi instructor. This amounted to having a partner execute bouncing shoves to my shoulders as I tried to stretch forward. Today most people know that this particular type of stretching is counter productive - it triggers the stretch reflex which then actually causes the muscle to tighten. Additionally I have witnessed two individuals rupture discs in the lumbar area of their backs while engaging in this practice of pushing their "victim" on the shoulders. This, my friend, is a permanent injury - those two will be in pain for the rest of their lives. While the poor performance and occasional injuries were happening, complaints about the technique were blown off as "nonrepresentative anecdotes". We were all led to believe that 'sensei knows best'. Scholarly individuals have developed a discipline - the scientific method - for resolving such issues. Humans are irresponsible if they do not first question and then second resolve to answer significant questions. Certainly you have questioned some of your own previous ways and now think you have found a better way.

Reading your piece reminded me of a book I read several years ago "Confessions of a Bodybuilder". In it an Oxford graduate blows off grad school to Yale and decides to find himself through bodybuilding competition. He starts off small doing Nautilus in a gym. By the end of the book he has moved to muscle beach, CA and is working on free weights with the big boys. He ends up taking mega steroids and blowing every last dollar of an inheritance so he can spend 100% of his time lifting big, eating big, and sleeping big. Finally he enters a powerlifting and bodybuilding contest and his mediocre performance ends up being a message to him. His conclusion, in the vernacular, is that the sport sucks and we should all have butter on our vegetables. I was really pissed that I spent valuable time reading a decent book only to get to his stupid conclusions. How is it that this person can speak for me and what I get out of lifting weights in the gym? Where is the consideration of the process he went through? Did the guy ever consider that by going from super geek to killer pecs, he substituted one obsession for another? He will probably just throw himself the same way into something entirely different - with equal frustration. And who is to say that an experienced psychologist can't "assess" the issues here and make MEASURABLE improvements in this person's life after an appropriate intervention?

You write "To paraphrase the Tao Te Ching -- The way that can be named is not the *Way*. In other words, what we strive to measure ultimately cannot be measured." Physics has a phenomenon exactly like that - it's called the "Heisenburg uncertainty principle". Measurement's effect on what is measured is also a fundamental concept in engineering. Seriously the more we know about math and science, the more we realize that it mimics life - it IS life. The study of Chaos is all about nonlinear mathematics - a place where deterministic mathematical equations describe unpredictable behavior. Not surprisingly much of real life is best modeled with nonlinear constructs. The point is not to dwell on the outcome; it obviously cannot always be predicted and is often not the point at all. Instead we strive for a better understanding of the process, the path, or the "way" so that individuals can make THEIR OWN objective decisions about which way to go.

A big problem in life in general is that "experts" make decisions about appropriate methods that are not "evidenced based." At some point it is necessary to try a new idea because expanding our body of knowledge and options for living is generally considered a good thing. However untested ideas passed off as gospel are false knowledge. Recently I approached a private school about entering my son in kindergarten who will have his fifth birthday on August 31. The prevailing wisdom is that he should be held back because he is a "summer baby". The director of admissions showed me these detailed histograms with birthdays ranging over 21 months. Chad, if they accepted him, would have been the youngest. It didn't matter to them that he clearly tested above average on their own tests - they were concerned that he would be the last to get a driver's license in his class. But nowhere in all her charts, figures, and books was a well constructed (randomized trial) study that showed the outcome of starting kids early vs "normal" vs late. Who's to say that a "scrappy" Chad might not always consider life a challenge and continue to strive for exceptional performance beyond his school years - making discoveries that we all would benefit from. Instead I was subject to the whims of the "experts" who have predetermined that Chad must wait idle another year - one of the most formative years of a child's life. I need to find out what to do with my son who now "acts up" in daycare when they are working on lessons he did with me at age 2.

Please don't presume I dwell in the physiologic - I don't. Please don't presume that I "walked down the wrong path and take a bunch of students with you." My students choose their own paths and it is my job - my responsibility - to guide them. Furthermore you presume way too much by assuming that my or anyone else's path is a "wrong path" when you've never been in my dojo or lived a day in my life.

But please do take your questioning nature to the table with me. Please do take the attitudes which sent you in search for "better" ways to new, relevant questions. You can help me with the Richmond mother of an ADD child who wants to know how (what "way") I can positively impact his life. You can help me with another mother who has a child with Asperger's syndrome who wants to know if I can make her child look at others in the eyes when she talks to them. You can help me with the countless women students I have had who come to karate with a history of sexual abuse. You can help me with the 45 year old man who wants to know if he can still exercise in spite of severe back problems. Guess what - there isn't an Olympic champion in the bunch. And it really doesn't matter, does it?

I'm not like those who say "I don't like to study music because it ruins the experience for me; it's all metaphysical and cannot be measured." I enjoy music more now that I understand instrumentation, timing, chord progressions, timbre, meter, and style. And I recognize the difference between "not-so-good" and "better" music. And I appreciate excellence in all styles of music. Knowledge adds to our appreciation of all the fine things in life.

Bill Glasheen

Subject: Scientific Tests in the Dojo?

Dear Mr. Glasheen:

Yes and No to your question, but more importantly why?

Undoutably, the knowledge base of human physiology and "maximal performance" has increased tremendously in recent years. An outcome of Olympic and "world class" competitions. The athletes and trainers search ceaselessly for that new training technique/approach that will make the competitor not only "better" than what s/he was but the "best" in the world. I am sure scientific tests can be designed to capture the improvement of an athlete, be it aerobically, anaerobically, and/or musculoskeletal flexibility. I am likewise sure, one can devise tests to evaluate the biomechanical effectiveness of certain movements. All these can contribute to making an art more physically effecient. But, at what costs and to which purpose? My own feeling is that such testing and emphasis on it will lead only to more competitiveness, more "win/lose", more frustration, more disappointment...
How can it not be so when emphasis is placed on only the physicality of what we practice. And, what we practice is "budo" -- the martial way. A way connotates not only a physical practice but a practice of also the mind and spirit. Despite the rhetoric, so many in martial arts have already lost an appreciation of the development of mind and spirit. We see it students and instructors concerned with ranking, titles and the priveleges therein. We see athletic competitors. Rarely, do we see true humility, respect and human decency...

How would we proposed to measure scientifically the development of "mind" and the more transcendental "spirit". I work in the human services field.
Though not a MA nor a PHd, I have been involved with enough "scientific studies" to know how hard this goal might be. How would we agree on a "construct" and "operationalize" the "mind" and the "spirit". Could we ever truly measure what are intangible and transcendental. To paraphrase the Tao Te Ching -- The way that can be named is not the *Way*. In other words, what we strive to measure ultimately cannot be measured. What we end up measuring will surely not be what we wanted to measure. With respect the transcendental aspect of the martial way, one can only be... One does without regard to goals and objectives. Each moment/movement unfolds into the next. We stop and then start all over again... What use the memory (measure) of what I was, or the projection (desire/fantansy) of what I want to be. It is only this unfolding moment.

Sorry, if I am off on a tangent... <G> But, I been through that phrase of wanting "better" training methods, approaches, styles, etc. to be a "better" karateka, competitor, fighter. Finally, after reacting to every stress with the desire to knock someone's head off (with cleanest, quickest techniques possible, of course...), it dawned on me that "yes" I was a better fighter but I was not at peace." There was this constant desire to be "better" physically. But, ultimately it's a loser's path. How can it not be. We all age, we all die. The pertinent question should have been "am I a better person because of my martial arts practice?" --- and the simple answer would have been "I was not."

Now, I simply practice. Some practice go easier, some go harder. But, each practice is a a good one and a delight unto itself. Am I a better person? Perhaps... <g> Can you measure it? I doubt it.

I'm not saying "scientific testing" shouldn't be developed and implemented.
But, I would caution that those involved should be clear on the purpose of such testing, lest you walked down the wrong path and take a bunch of students with you...

good training,

david moy

 Yes, good idea. I would be very interested in actual research data based on
our body conditioning methods. Since adding martial arts to my fitness
regimen I have experienced definite, perhaps measurable, changes in my mind
and body.
For instance: muscle hardness, flexibility and range of motion, bone
density, pain threshold, mental toughness.....Also, I am actually a little
taller!
However, no needles or DRE's!
Rich Castanet

Return to home page

New counter. 3/15/97:

Click Here!