Does anyone have any evidence, anecdotal or otherwise, that gamification is good for teaching STEM ideas in the long term? I am wary of rewarding the brain with in-game loot for memorizing the rules of algebra rather than with the deep satisfaction that comes with understanding. Obviously, this latter type of reward cannot be as consistently provided and requires a certain maturity (maybe), but ultimately I think it's what drives most insightful people.
Here's a good example of what bothers me:
>As the game progresses, you’ll start seeing cards that are above and below each other, with a bar in the middle — and you’ll learn to cancel these out by dragging one onto the other, which then turns into a one-dot. And you’ll learn that a one-dot vanishes when you drag it onto a card it’s attached to (with a little grey dot between them). These, of course, are fractions — multiplication and division — but you don’t need to know that to play the game, either.
That last sentence is especially telling.
To me, gamification is suited for making necessary but painful tasks fun (e.g. cleaning your desk, tagging media, memorizing facts), but not for deep learning (e.g. algebra, quantum mechanics, object-oriented programming). But maybe, at 26, I'm just not with the times.
When most people speak of Math, what they have in mind is more its mechanism than its essence. This "Math" consists of assigning meaning to a set of symbols, blindly shuffling around these symbols according to arcane rules, and then interpreting a meaning from the shuffled result. The process is not unlike casting lots.
This mechanism of math evolved for a reason: it was the most efficient means of modeling quantitative systems given the constraints of pencil and paper. Unfortunately, most people are not comfortable with bundling up meaning into abstract symbols and making them dance. Thus, the power of math beyond arithmetic is generally reserved for a clergy of scientists and engineers (many of whom struggle with symbolic abstractions more than they'll actually admit).
I think gamification is a great way to teach symbol manipulation, and I think (contrary to Bret) that symbol manipulation is a prerequisite for deeper STEM ideas.
I do also believe that harder mathematical problems can also be gamified, but this process is much less well understood, and you'll probably want a few theorem prover experts around if you attempt a system like that.
I 100% agree that having symbol manipulation is required for deeper understanding and learning.
In my experience, being able to quickly and (fairly) reliably move symbols around in one's head makes futher learning much easier and faster, as one can follow proofs and explanations intuitively without feeling the need to work through each step laboriously to justify it.
As a simple example, being able to see the steps that were used to go between
(x - 1)^2 - 2 = 0
x = 1 +- sqrt(2)
at a glance gives one a higher "maths-learning" bandwidth, since one spends just the 3 seconds it takes to read those lines, instead of the extra 30 seconds physically doing the working.
(And, on a slightly different note, having some basic facts memorised like (for example) "d/dx sin(x) = cos(x)" is a little like the difference between data in L1 cache and that just in RAM. However, this doesn't mean that one should not understand why, or not be able to rederive it in a flash by drawing a diagram or whatever.)
> I think (contrary to Bret) that symbol manipulation is a prerequisite for deeper STEM ideas.
Can you elaborate on this further? I tend to disagree, but I don't know if I have all that much to say to back it up.
I mean, sure, there's just no good way to understand shear forces without being able to manipulate matrices (so I agree with dbaupp's sibling comment), but that doesn't mean you want to learn the rules of matrix algebra as if they were arbitrary rules enforced by the stick and carrots in a game.
In the case of mathematics, there is a fundamental sense in which the symbols and their manipulations are what you're studying. Think of it as if you were an archeologist: the symbols are the artifact of study. It's only half of the picture: you must(!) come up with a different way of intuitively understanding it--but at the end of the day if you really want to precisely say what it is you're talking about, it's symbols.
OK, I think I see what you're getting at. But we could devise a AltDragonBox game with a completely different set of arbitrary rules. (Possibly even rules that are inconsistent.) And as far as game play goes, AltDragonBox would work just as well. But there's a reason that the rules of algebra are what they are. And if you can't distinguish between DragonBox and AltDragonBox, I don't think you're learning what you need to learn.
Here's another way to look at it: we all know people who could ace their high school math tests because they had memorized the rules of manipulation but weren't good at math--as evidenced by their poor performance in college and failure to succeed in future STEM classes. If "the symbols and their manipulations are what you're studying" (which, I agree, there is some truth to), then what is it exactly that these people were lacking?
> there's a reason that the rules of algebra are what they are
But which algebra, and which rules ? There are infinitely many algebras ( eg. Algebra over the field of reals, Banach Algebra, relational algebra, boolean algebra, sigma algebra etc. ) The "rules" are really constructs you decide that apply to the elements of the space that conform to your algebra. So for example the reals are a field that have ordering, so you can talk about less than and greater than, but the complex numbers don't have an imposed order and you'd have to first define a norm to map them onto the reals. The AltDragonBox with its own inconsistent arbitrary rules will still have some algebraic encoding. Whether that's useful to you is debatable. Like in my algebra I could overload plus to mean multiply and square root to mean divide by 7 and add -3 and then try to figure out what exponentiation works out to. It would be interesting...maybe not useful, but its still an algebra. Maybe you won't have closure...the elements may not end up in a field or even in a semigroup...its a nice make-believe algebra.
Yes, that's a very impressive display of all the math you must surely know, but its completely misses the point. The rules of elementary algebra really are special, and it has to do with their correspondance to real things in real life. There's a reason mathematicians don't just enumerate all possible algebras and study them one by one.
I agree that it is important to recognize the limitations of this approach. As a high schooler (and even as an undergraduate in mathematics), you learn all sorts of formal systems. As a mathematics researcher, one of your charges is inventing interesting mathematical concepts to play around with. Obviously this is a bit much to ask out of DragonBox! And even more modestly, the ability to sit down with a problem and say, "Ah, but what really is going on here?" is a deep and difficult skill to impart. I don't know what these people were lacking, but if they didn't know how to move symbols around, I might have started there...
(BTW, an inconsistent set of rules would correspond to a version of DragonBox where there was a cheat code you could enter, and then easily solve every problem. So it would not "work just as well", and this would be pretty clear to a cheating gamer.)
OT: There's a great bit in Neal Stephenson's Anathem where these monk-like scientists are made to copy out subtly wrong mathematics and scientific proofs as a punishment.
AltDragonBox could be exactly that -- day and night cancel, except for symbols when its constellation is rising, in which case they divide, except for odd numbered Fridays in a leap year. Oh, and do it with numbers instead of day/night symbols.
Yes, there are loads of different forms of algebra and geometry. Sometimes people throw out what people though were fundamental rules and come up with new geometries (e.g. non-Euclidian geometry which turns out to be very helpful to describe relativity).
A lot of the time though, the other rules for alternative algebras produce very dull and boring algebras.
>but at the end of the day if you really want to precisely say what it is you're talking about, it's symbols
Hear hear! I'd go one step further and say its ALL symbols. Any associated real-life meanings that help a human intuitively understand the equation is purely coincidental and actually a distraction. I've repeated this argument ad-nauseam : http://news.ycombinator.com/item?id=4085558
Don't know who it was ( Martin Gardner ? ) who once said three dinosaur plus two dinosaur is still five dinosaur. The implication is that symbol pushing and symbol manipulation is way more fundamental than having humans around who can associate three and two with human artifacts and then add them to satisfy their intuition. The dinos will add up to 5 regardless of the human intuition.
If it's just all symbols then computers should be better at it than us. The reason we can prove deep theorems by choosing the right path through an impossibly large combinatorial space is because we perceive structure and meaning, and we use that to guide us. We gaim an intuition for something that's "going on underneath" and so don't just perform random searches.
Explaining simple proofs to students often leaves them feeling that they've followed the steps, but don't undertand. There is more than just symbols.
Humans have a lot more bandwidth and computational power than you might think. And the search space for most proofs is not all that large compared to say a go board once you consider how many different proofs also work.
PS: 1-10 petaflops by some estimates, just not that many significant digits per calculation.
When you say `most proofs' do you mean proofs that you would find in an intro level course, or do you something more. I strongly disagree that the search space for most proofs is as small as go. This may be true once you restrict to a suitably relevant field, but this is a nontrivial reduction which takes a great deal of insight!
Fermat's last theorem resisted the attempts of mathematicians for three hundred years because it required insights so complex they couldn't be formulated without a deep understanding of disparate subfields.
To tie this back to the go analogy, the search space of go is large because the branching factor is big (<400) and because the number of moves is quite large (<400 as well, for all but a very few bizarre situations). For real proofs, while the branching factor may be substantially smaller (given some axiomatic system), the length of the proof is much much longer. The exponent in proofs beat the branching factor of go.
I'm a big fan of intuitive learning and have mixed feelings too (downloading the game now to try it out). Some thoughts
1) Certain basics skills ("spelling and grammar") are needed to express/understand higher-level ideas (literature, poetry). Games help practice them. (Fear: gamification hurts internal motivation)
2) Fear: Assumption that "algebra-like" lessons automatically help algebra understanding. Does typing help piano playing? It's easy to assume both "use your fingers" and must correlate.
3) Fear: Reinforcement that math is about moving symbols around. We're trying to express ideas, symbols are their serialization. There's a "rule" that the same card must be added to both sides. We know it's to balance the equation. Does the kid know? What if the rule was to add the card twice to one side, and 0 times to the other? Why does one rule but not the other make sense?
I'm excited that this helps practice basic skills, but am afraid of ending up with a Chinese-room situation where we can manipulate symbols but intuit nothing. We already have hordes of calculus "graduates" who vaguely remember "x^n... drop the n, make the exponent n-1"... and what of it? Did it shift their perspective?
Update: After thinking more, I think the game is a good thing overall. For a young child (5, etc.) this game is giving them a new mental model of the world. Later on, when they learn arithmetic, and so on, it can be shown how this mental model corresponds to the rules. Giving children new analogies to work with is a good thing.
Yes, there is growing evidence that gamification, if done right, can be really good for teaching ideas. Zoran Popović [1], a faculty member here at the University of Washington, has been working on these issues for a few years now with his "Center for Game Science" [2]. And at a more advanced level, games like FoldIt [3] are starting to make small dents in difficult problems like protein folding.
i think the salient point about dragonbox is not the reward system, it's the fact that the rules of algebra have been mapped, transparently and isomorphically, to the game rules. playing the game doesn't just help you memorise the rules of algebra, it actually makes you think about how you use those rules to manipulate and transform expressions, which is the fundamental skill underlying "real" algebra.
I don't think deep learning is the starting point, for algebra; I think those who get it relatively quickly typically start from pattern recognition in applying the rules, and then the understanding grows beneath that. If that's the case, this kind of thing is fantastic. I could certainly be wrong, but either of us should start pointing at studies next...
To me, gamification is suited for making necessary but painful tasks fun (e.g. cleaning your desk, tagging media, memorizing facts), but not for deep learning (e.g. algebra, quantum mechanics, object-oriented programming). But maybe, at 26, I'm just not with the times.
The thing to be careful about here is the use of "gameification".
There's gameification of the Internet marketing bullshit variety, and there's making things more gameful. The suits ate up the first term and associated it with rewards, points and extrinsic motivation. The latter is a not-yet-bastardised term which simply means to try and find ways to make more tasks fun, to try to make systems explorable, malleable, and allow for failure. Games teach best when you're exploring systems and manipulating them, seeing what they do, manipulate them again, see the result of that. They're very refined incidental learning.
What happens if you use bullshit gamification is the Overjustification Effect, where the rewards begin to dominate and crowd out the intrinsic motivation, as people begin to focus on those instead. Intrinsic motivation is your "deep satisfaction", and you are right that rewards erode it. This has been written about a lot by Alfie Kohn (eg. "Punished by Rewards").
I don't know of any research that shows that gameful styles of work, as long as they're not attached to large amounts of rewards, are bad at all. Having played with DragonBox a little bit this morning, it seems to do a very good job of being gameful, and extracting the game out of algebra.
Some things I noticed:
1. The game really highlights the malleability of numbers. For me, this was the deep revelation about algebra, and it took me a long time to get there. The way the game is designed using touch, lets you fling numbers around and place them on top of each other and such, and doesn't let players get hung up on numbers or placement. We get to the understanding that algebra requires very quickly. You seem to see that as a loss, but I see that as a really big win.
2. As with all educational games, failure is easy and not punished. I haven't found a way for the game to show me solutions, but in a classroom setting, that wouldn't be a problem.
3. I don't find the rewards any more or less motivating than a checkmark next to my work, so I don't think we'll see the overjustification effect here. They just say whether you did it right or not. There's no achievement system that could encourage play that might be harmful to learning, just feedback on the specific problem you were solving.
Feedback is required for students. Feedback is also a form of reward. So threading the needle is not easy, but it looks like DragonBox did a really good job here.
All in all, I think it's a really good piece of work, and the developers deserve to make a fat chunk of change from it.
"1. The game really highlights the malleability of numbers. For me, this was the deep revelation about algebra, and it took me a long time to get there. The way the game is designed using touch, lets you fling numbers around and place them on top of each other and such, and doesn't let players get hung up on numbers or placement. We get to the understanding that algebra requires very quickly. You seem to see that as a loss, but I see that as a really big win."
This. One of the things my eldest daughter struggled with early on in Algebra was the variables. She kept insisting on having a 'value' for the variable up front because the abstraction bothered her. And math was about numbers right? (when you are 10 math is always 'numbers' it seems, even when that is arithmetic). One she got rid of the notion that math was 'numbers' rather it was a sequence of mutation rules against things which were infinitely mutable (within constraints), it went much better for her.
This program elegantly sidesteps that issue by starting of with boxes. Boxes are the real world equivalent of variables and they aren't numbers so they don't trigger that association per-maturely.
I think gamification may not be able to directly teach a deep understanding. However, I think it may be able to encourage the development of a work ethic that will help to build a deep understanding.
Anecdotally, my exposure to a constantly available stream of shallow stimulation (reddit, etc.) has decreased my ability to stay focused on initially unrewarding tasks. This seems like an effect that could be captured with a controlled experiment, so I'd love to know if it's been investigated.
To me, gamification is suited for making necessary but painful tasks fun (e.g. cleaning your desk, tagging media, memorizing facts), but not for deep learning (e.g. algebra, quantum mechanics, object-oriented programming).
We tend to think of understanding as "deep" and competence as "shallow." We tend to think of solving a problem for the first time as the valuable part of learning, and solving similar problems over and over as a waste of time. Maybe we even see it as stultifying, or as cheapening the experience of learning. Yet practice deepens understanding, and even if you don't believe that, you have to admit there's a long way to go between "understanding" algebra in the intellectual sense and mastering algebra in the mindless way that lets you use algebra when your mind is busy doing something else, such as learning chemistry or geometry. Anything that makes practice a little bit less boring will help kids develop fluency so they aren't distracted by understanding algebraic manipulations when they're supposed to be thinking about something else and just doing the algebra.
>rather than with the deep satisfaction that comes with understanding
This may not be a problem. For me, that satisfaction happens when understanding makes the world make more sense, less arbitrary; like you just got let in a joke that's been puzzling you for years. This is just one more thing that will make sense later. If anything, it'll enhance the effect.
One of the more popular TV channels in India, especially during the exams, is Topper ( http://www.yupptv.com/topper_tv_live.html )
The channel has its share of teaching, but mostly its crude gameification. Very effective channel. In terms of viewership during the exam season( totally unfair given India's huge student population ), it would easily surpass your CNN or Fox or whatever it is that Americans watch.
I watch a fair deal of Topper ( very addictive channel ). So they "teach" determinants by rapidly flashing square matrices on the screen and the competing student groups have to guess the value of the determinant. Not by computing adjoints and cofactors - that would be painfully slow. Mostly you use properties of determinants (http://en.wikipedia.org/wiki/Determinant#Properties_of_the_d... ). So if its a 3 by 3 and say a column is 3,1,4 and another column is 9,3,12, you know the value is zero ( because you could factor out the scalar multiple 3 and then two columns become identical, ergo value zero ).
Sometimes they'll flash a triangular matrix and all you have to do is multiply along the diagonals. Or you'd have row 1 = [4,1,2], row 2 = [5,3,5], row 3 = [1,2,3]. Some smartypants would correctly guess that row 2 was just the sum of the other two rows, so the determinant must be zero.
Is this sort of thing "useful" ? I don't know. But this is how I learnt much of my math in India...and they continue to use these games to this day. I can look at equations of lines & tell you if they slope up or down. I can tell you whether your parabola is convex or concave, where the focii are and what the lengths of the minor and major axis of your ellipse will be....tons and tons of repititive trivia, force-fed through pattern matching & gamification. Just by looking, no actual calculations! But this is one of the reasons Indian grad students tend to do well out here in STEM...we have no intuition but tons of gameified training. Once we are here, we'll get the intuition as well. To start with intuition would be a horrible idea, because the teacher quality back home is horible. Most of them honestly have no idea what a vector is or a complex number is...in most cases English is not our native tongue, so we can't even pronounce "surd" correctly, let alone know where it came from, but we all know that the root of 3 is a surd and its root isn't a surd and so on ( my math text: http://books.google.com/books?id=1C4iQNUWLBwC&lpg=PA25&#... )
imho, gamefication is unequivocally good in STEM, atleast upto college math level. Ultimately its all symbol pushing.
I think you are being a little narrow-minded in basically saying that your own 'deep satisfaction' in 'deep learning' is the single-goal of maths.
To me, there are many sides of maths, and different people know, utilise and enjoy these in different ways. If this tool provides a new 'in' I'm all for it - though of course its not going to teach everything, and its not going to be right for everyone (nothing is - even the best maths lessons).
Also - its aimed at non-maths time (ie. replacing 'angry birds') rather than competing for with other learning time (I wouldn't be so happy if they for instance started making things like this mandatory in school). So really I don't see where the loss is....
Here's a good example of what bothers me:
>As the game progresses, you’ll start seeing cards that are above and below each other, with a bar in the middle — and you’ll learn to cancel these out by dragging one onto the other, which then turns into a one-dot. And you’ll learn that a one-dot vanishes when you drag it onto a card it’s attached to (with a little grey dot between them). These, of course, are fractions — multiplication and division — but you don’t need to know that to play the game, either.
That last sentence is especially telling.
To me, gamification is suited for making necessary but painful tasks fun (e.g. cleaning your desk, tagging media, memorizing facts), but not for deep learning (e.g. algebra, quantum mechanics, object-oriented programming). But maybe, at 26, I'm just not with the times.
EDIT: I think ColinWright is getting at the same worry, and his comment is more fleshed out http://news.ycombinator.com/item?id=4106567