In our years of teaching and being around students (as I'm sure any fellow teacher that is reading this can attest to), we often get asked the question: "how should I study?" or "how do I get better at this or that?". In fields like math and physics, these questions are hard to answer because most definitions of "being good" at these things involves "being good" at solving problems. And the thing about solving problems is that it's a very personal, and intimate affair.
Sure, there are skills and bits of knowledge that you either will, or won't, have at any given time, but gaining those is the "easy" part. The hard part comes when you're facing a problem you haven't seen before and are at a complete loss as to what to do next. And that's where the personal, intimate part comes. You have to fight your inner demons, you have to find something to try, you have to know how your brain most effectively gets out of these ruts.
If we (society) had a well-tested and generally applicable formula for how humans can move from being stuck to un-stuck, then we'd all be Fields medalists, cancer would be cured, and we'd be driving cars that get us from point A to point B quickly, safely, without intervention, while powered by plastic bags and emitting only pure oxygen, H20, and the smell of freshly cut pine. But alas, we don't have such a formula, and therefore no one can tell you precisely and with certainty how you can get better at problem solving.
There is, however, one extremely powerful method that you can use to repeatedly put yourself in a position where you can learn about how your brain works, how it gets itself out of ruts, and how it makes progress.
Create Your Own Problems
Over our collective years teaching math and physics to undergrads and grads at various levels, there's been one pretty consistent pattern that has been noticed. Those who create their own problems, and then solve (or at least try to solve) them, get better, faster, at problem solving than those who don't. The benefits of doing so are indescribable and not limited to simply doing better on problem sets and/or exams. It's a great technique for eventually transitioning into research-level work, if that's your desired end-game. Before elaborating on the benefits, though, let's make clearer what exactly we mean.
How To Create Your Own Problems
Typically, "creating your own problems" involves taking problems that have been given to you and simply changing them (after you solve the original problem, if you can). Were you given an integral that has something like "1 + x^2" in it somewhere? If so, can you replace the 1 with a general constant "a" and solve this more general integral? Can you take your 1-dimensional kinematics question in physics and place it on an inclined plane instead? Or can you add friction to the problem? Can you do both?
This version of "creating your own problems" is probably my personal favorite, for many reasons. First, it usually allows you to "check your work" by taking the limit of your new problem that recovers your original problem. In the first example above, if we can solve the integral for a general "a", then we can check that our answer agrees with the original problem's answer when a=1. Similarly, if we take the angle of our inclined plane to be 0, or the coefficient of friction to be 0, we should recover the answer to the original 1-dimensional kinematics question.
Sometimes a really magical thing happens: you take this limit and your answer does not agree with what you expect. At this point there are a few options. One option is that you solved the first problem incorrectly. Another option is that you solved the new problem incorrectly. A third option is that you solved both incorrectly (hopefully you don't solve both incorrectly, but in such a way that the new answer still "flows" to the old answer in the limit!). And the fourth, truly magical option is that you solved both correctly, but your new, generalized problem is actually subtly not continuously related to your first problem.
One example of this fourth option is adding friction to physics problems — where the dynamics of a frictional system with arbitrarily small coefficients of friction can, sometimes, actually be fundamentally different than a frictionless system. Another example is adding a general parameter to an integral that might introduce a divergence. The integral might converge to different things in different regions of your new parameter, and if you're not careful this might be really confusing for a while.
Which brings us to the question of how to deal with the four options we just mentioned above. Our answer is: we don't know. You just kind of have to think really hard. Go back to the original version of the problem and make damn sure you solved it correctly. Once you're very confident you did that, do the same for your new version. If you still haven't found any mistakes, then start considering option 4 (the magical one). Can't find the subtle "regime change" that you may have introduced? Well, go back and check your work on parts 1 and 2 again.
Here's why this is so great: while you're doing all the stuff just mentioned in the previous paragraph, you're getting way better at solving stuff. You're getting to know how your brain works and how it struggles. You're mastering concepts, your way. And, since at least one of these problems is one that you came up with, it's probably going to be more fun going through this struggle than if it were just some textbook problem.
You might not know it, but another side-effect of going through all this is that it's preparing you for doing research as well.
Research Prep
Creating your own problems in this way is, in our opinion at least, the best possible research prep you can possibly do. In fact, it literally is research, just on problems that are likely not original.
Imagine you're a high-schooler learning integrals. If you take an integral and find a way to make it harder, then solve it, you probably have learned a lot but you probably haven't solved any truly original problem. And that's okay.
Now suppose that you're an undergrad and you take a problem on your topology problem set and make it a little harder (after you solve it, of course). Again, you probably learned a lot, which is great, but you probably haven't found anything publishable.
But now imagine you're a grad student, and your years of practicing "creating your own problems" has put you in a mindset where every problem and every proof you see, you're thinking about possible small little extensions. You're reading a paper and into your mind pops a little idea about some lemma you just read and how it could be extended. You let it jostle around in your brain for a week and you start to get more confident that you might be able to find some in-roads. Now, my friend, you are doing research, and extending that lemma very well could give you something publishable.
And nowhere along this journey did you consciously decide to "start doing research" — you were literally doing research the whole time, it's just that eventually you got to the point where your research hadn't been done before by anybody else. And that's a pretty neat feeling.
Creating The Right Kind Of Problems
Most research is not done by thinking about some brilliant new idea in a vacuum and then expounding upon it. 99.9995% of the time, research comes from reading someone else's work (or your own work from an earlier time), and finding some direction to generalize it and make it 2-10% harder than it was before.
Sometimes, though, when you take a solved problem and try to make it different and/or harder, you sometimes make it way too hard. Or worse, maybe you make it impossible (and impossible problems are very annoying). Being able to identify this situation, and more importantly, being able to get yourself back out of this situation, is a crucial skill to have. Why is it so hard now? What other info might you need in order to make it less hard? How can you backtrack a little without sacrificing too much of the interestingness of the problem?
This is the type of thinking that's involved in a lot of research, and fortunately, if you've been creating your own problems for years, you'll already be familiar with it.
It's also fascinating (and fun) to take a problem that you've solved, extend it in a way that you think is reasonable, and then find yourself utterly stuck. It updates your intuition about how problems of that type behave, since you initially thought it was going to be a tractable extension. I have no way of measuring just how valuable of a learning experience this is, but it certainly feels like a lot of learning (in the deepest sense of that word) gets done in this process.
In Short
The same way that a doctor might prescribe "exercise" as the single best and most generally applicable intervention that one can take to improve their health, we truly believe (but should not be trusted as much as you should trust your doctor!) that creating your own problems is the single best thing anyone can do to up their math and/or problem solving game. It puts you in a position to start wrestling the demons in your brain as often as possible, and learning how to tame them. It is also the single best preparation for research that there is — because it literally is research, just not on original problems (at first).
So, if you discover a problem that you particularly enjoyed working on, even if it's not original, let us know about it!