Teaching Scientific Method

A few days ago, PZ Myers of the Pharyngula fame (not the pharyngula stage, though - much more advanced in development) wrote a post (http://pharyngula.org/index/weblog/comments/an_interesting_teaching_example/) that links to this article (http://groups.google.co.uk/groups?selm=1107829738.090845.112680@g14g2000cwa.googlegroups.com) about creative ways to teach scientific method:

"I found that I had to teach the nature of science at both
the undergraduate and graduate levels for the honors class and molecular
genetics class that I taught. Even at the graduate level the understanding of
science cannot be taken for granted. It turned into my one lecture speel. I'd
hand out essays by Richard Feynman and Peter Medewar on the nature of science
for the students to read and then we'd work on a jigsaw puzzle. I'd use the
puzzle as an example of how science works. I'd use those cheap 100 piece kid
puzzles that you can buy at WalMart. I found that the two puzzles that I
purchased had an identical cut out pattern with different pictures. The first
thing that we'd do is turn over the pieces and I'd try and get the students to
think about the problem. Just looking at the pieces, can they come to some sort
of idea of what the picture was. Unless you have sometype of super genius that
can assemble the pieces in their mind the students can only come up with vague
ideas of what the picture might be. We do this in science all the time. Even the
assumption that it will make a picture that they can make sense of should be
pointed out to them. Try and get them to think about what they are doing. When
they start to assemble the puzzle ask them what they are doing. None of the
students I've had have tried the random assembly of just putting any two pieces
together. Get them to understand that they are hypothesis testing by grouping
the pieces by whatever character that they are using (color, pattern, shape).
Ask them why their hypotheses fail so often. Get them to understand the problem
that science deals with when you make assumptions based in incomplete data. If
they were able to take all the characteristics of each piece and make a perfect
analysis they would never be wrong in their choice of which pieces fit where,
but using the mark I eyeball and only a limited set of characters you often make
mistakes. You have to expect to be wrong quite often in science. You have to be
able to test your hypotheses. A few students always assemble the edge of the
puzzle first. I point out that this is just what scientist try and do when they
create a framework and build on it. We usually get the easiest pieces in place
first and the edges are the easiest pieces to fit because they only have three
interacting sides to consider. Science does what it can and builds on it. About
this time someone notices that I've taken away the corner pieces. When they ask
for the corners I ask them how they know that the puzzle has corners. It isn't a
trick question. We make assumptions like this all the time, and it is based on
our experience, but they can also see that some pieces are missing based on
their expected square side and only two interacting edges. They have an
hypothesis that something is missing and it is based on their experience and the
physical evidence. I throw out the corners and they have to scratch their heads
because I've given them the corners to another puzzle, but they still fit and
they still complete the outside of the puzzle. I tell them that science is full
of pieces that don't quite fit, but that are good enough to help us get a better
idea of what it is that we are working on. As the puzzle gets completed I make
them note how the qualitative as well as quantitative nature of the hypotheses
that they are testing improves as they acquire more knowledge of what the
picture looks like. The picture never gets perfect because the corners don't
match, but it is obviously good enough to get a pretty good idea of what the
picture is. I don't think that I've ever brought up creationism or ID in this
lecture, but if you want to you can just state the fact that ID as a "concept"
has never been able to place a piece in the puzzle of nature. They have tested
quite a few pieces to see if they fit, but there isn't a single one left in
place at the end of the day. Essentially, it is a concept with a 100% failure
rate upon testing. The only pieces left on the board are the ones that haven't
been tested yet. It has been found to be worse than just randomly picking any
two pieces and trying them to see if they fit. If any student doesn't believe
this, just ask them for a single piece that ID has placed in our scientific
knowledge. You won't find a list of these things at the Discovery Institute
because there are no ID scientific successes. The farce is that they have lists
of scientists that were or are religious and state their scientific successes
without telling anyone that usually these guys were responsible for kicking out
an ID piece from where it didn't belong. These guys are known for their
scientific contributions and not their ID contributions. This is why many
scientists define science in such a way that ID is excluded from consideration.
It simply has never worked, and it has been a monumental waste of time.
Definitions like those that exclude ID get put in place to protect the
incompetent from themselves. Most rational scientist can figure out for
themselves that they can think about ID, but they can't really expect to use it
for anything. Not a single success and a 100% failure rate upon testing is
pretty convincing to most scientists. "

The comments section on Pharyngula was also very interesting, with some valuable modifications suggested, e.g.:

"Here is how many disagreements among scientists arise: one thinks the piece is important and goes in the center; another it isn't important and is part of the edge (say the blue sky, not carrying much info); yet another thinks it's part of another puzzle altogether. And yet another that it isn't even a puzzle piece" [Revere]

"A virtue is that it shows science as a dynamic process. The closed nature of the assembly may be a drawback. Novices could easily conclude that the completed puzzle is what science is about. That the small puzzle is simply a step in the assembly of a much larger puzzle is implicit for us. Literal minds may need to assemble a series of increasingly larger puzzles to get the principle. It's disturbing to imagine what the most literal minds might require" [Les Lane]

"One variation I think I'd make is to at least remove one whole edge of the puzzle, to emphasize the fact that we don't "finish" science in an hour session" [PZ Myers]

"...wants to do this with her classes, with some other changes/refinements--such as splitting the puzzle into two ~40% chunks and having half the class work on each. "But how do you know that you're all working on the same puzzle?"" [JBWoodford]

Of course, I just came back from the store with two 100-piece puzzles that I will use later this week. I'll let you know how it worked. If you have any suggestions, write a comment.

I have, before, used a different approach - the light-switch example. Let me try to describe how I do it:

While the students are too busy doing something (e.g., a lab exercise) to notice where I am and what I am doing, I switch off the lights. I love my current teaching lab as it has no windows and just a single simple master switch, so the effect is immediate and the result is pitch darkness.

I ask what are they observing. They may say "darkness" or "light went out" or "you switched off the light". I can, of course, pick the answer I like, so I home in on "darkness" and frame it as "an observation that the room is now dark, while before it was lighted". Then I ask them to come up with a hypothesis to explain the observation ("you switched it off" comes in here), and an experiment to test that hypothesis ("flip the switch"). I flip the switch (so we can see each other again and nobody sneaks out) and the light comes on, but then, I continue: "What if the light did not come on?"

Over the next several minutes we go through a number of alternative hypotheses (and tests): burned lightbulb, bad socket, broken switch, flipped fuse, broken fuse, rotten (or mouse-chewed) wiring inside or outside the room, broken cable that brings power to the building, a failure at the power-relay, a failure at the power-plant, a terrorist attack, aliens taking over the Earth, etc....

Usually, the order of the ideas that come up is from simpler to more complicated. The later hypotheses depend on the negative results for the tests of previous hypotheses. The tests of later hypotheses require more and more time, money (for tools and materials), expertise (prior education how the system works), and usually more and more people involved. Yet at each step, the hypothesis is a simple yes/no question: it is either supported by evidence or it is not.

Then I go back to science and try to draw the parallels: how science is really a complex outgrowth of everyday use of basic common-sense, how in the early days of science it was possible for individuals without much prior knowledge to make observations, form hypotheses and test them without inordinate amounts of time, effort or money (e.g., "flipping the switch"), but how today, we have progressed to more complex and detailed studies that require many years of training, many people working co-operatively, quite a lot of time, effort and money (e.g., "fixing a blown power-plant"). Of course, some lines of research are at that stage, while others are still in the beginning and simple experiments by individual scientists can do the trick, but the federal funding agencies are unlikely to fund such research as it is not "sexy" and high-tech enough, not to mention that it asks questions about problems that some may have trouble understanding - as those were never mentioned in textbooks. What is mentioned in textbooks is stuff that has been researched to death. Many other questions have never been asked, thus they are not in the textbooks, but that does not mean they cannot be, or should not be asked.

I also make the point that many little children understand the connection between the flip of the switch and light coming on, but would have no idea that a lightbulb can burn out, or anything else about the way the system works. Similarly in science, we need to learn the basics first, teach these basics to the new generations of scientists, so they can go on and test more complex hypotheses. Generation by generation, the understanding gets better, but the sheer volume of knowledge gets bigger and requires more and more years of study to master.

Once the hypotheses get very complex they often go against the common sense, e.g., "if it looks like something is designed it must have been made by a designer", but the years of study of the way scientists have, over the centuries, gone through the endless series of simple tests, convince one that the counter-intuitive solutions are the correct ones. Thus, if you want to learn something about evolutionary biology, biochemists, physicists and engineers are not the people to trust - you have to ask people who have spent years studying evolutionary biology! This is just like you won't ask domestic law or immigration law questions to be explained by a lawyer whose expertise is constitutional law or tax law. I will never try to explain particle physics to you - I have read about it in popular literature and had a college-level course, but that does not make me an expert.

Finally, I make a point that not all science is hypothesis-testing. Much work concerns developing new ways/techniques of observing stuff (e.g., new staining techniques for cells, or new molecular techniques or new methods of sampling natural populations), or orrganizing knowledge (building taxonomy, or new mathematical models) and also much work involves just observing and cataloguing observations (e.g., discovering and listing all salamanders in North Carolina, or sequencing the complete human genome).


Related (somewhat): for several months now I have been looking for a website or a piece of software I can use to place a picture of my own choosing, break it into puzzle pieces, then put back the pieces one by one, finally copying some of the stages of the process into separate slides of a PowerPoint presentation I want to use for a talk. Does anyone know of such a website or software? Please let me know if you do. I am pretty sure it is possible as I have seen puzzles in kids' educational games (before they grew out of them and moved on to slash and burn bloody gory games).



Update: I did both the light-switch and the puzzle this morning. Both worked very well.
It took them about 10 minutes to figure out that corner-pieces were missing (upon which I gave them corners from the other puzzle). I did what PZ suggested and left out one whole edge. This made them much more uncomfortable than placing the corners from the wrong puzzle. In the end I gave them the edge-pieces with a big caveat that this is a part of the exercise that does not fit in the way science works, that perhaps that edge should be used to connect the puzzle to another puzzle and another and another, and all those little puzzles forming one huge puzzle, that is never finished.

A couple of neat things happened that made good teaching moments. First, one of the "wrong" corners turned out to be the key to linking two big parts that were, until then, sitting unconnected on the table. There was a side edge with a few more pieces added to it, and the bottom edge with many pieces added to it. The "wrong" corner piece linked those two portions and allowed them to see the outlines of the "big picture". I told them that although it makes them uncomfortable, the semi-correct piece was a key for subequent research and that in the end, they can always come back and reconsider the corner-piece and replace it with a correct one ("correct" meaning that it fits with color, pattern and shape, not just shape).

Second, the students had different "research styles": two started with edges, one homed in on one of the "characteristic" pieces (and eye of a cat) and built around it, another picked a random piece and worked on finding all its neighbors and so on. Each made progress, illustrating that different strategies and styles are all good, that there is no one "perfect" approach.

Finally, there was a point when one of the students found a piece that connected a lot of stuff together and, at that moment, ALL the students simulteneously oohed and aahed how that was a "key" piece. I stretched the metaphor further, pointing that this is like a study in Nature or Science, that everyone in the field ("consensus of the scientific community") agrees is a very important study.

I will definitely keep using both exercises in the future.