Tuesday, April 2, 2013

A portrait of an "unsolved problem"

I study high-temperature superconductivity (henceforth HTSC).  It's one of the largest fields of condensed matter physics, which itself is the largest field of physics.  HTSC is one of the outstanding unsolved problems in physics, and unsolved problems attract research.  The two big questions are:
  • What is the mechanism for HTSC?  That is, how does it work?
  • Can we find a superconductor that works at even higher temperatures, like room temperature?
My own work is purely on currently known superconductors, and is thus not directly related to the search for new superconductors.  My research is more directly related to the mechanism for HTSC.  But I'm an experimentalist, so I don't come up with mechanisms myself.  It would be more accurate to say that I test theories proposed by other scientists.

So my perspective is very limited.  Superconductivity is a vast field, and I occupy one tiny little corner.  I don't have a great idea of the big picture, because I'm too busy trying to understand the details of the stuff near my own corner.  And I don't even fully understand that.  Consider this a distorted portrait.

What "unsolved" means

In fact, superconductivity is already understood.  It was solved in 1957, when BCS theory was proposed.  BCS theory is named for its creators: Bardeen, Schrieffer, and Cooper.  The solution is that electrons pair up.  In order to pair up, there needs to be an attractive force between electrons.  There is an interaction between electrons and the ionic lattice that creates an effective attractive interaction between electron pairs.

But superconductivity reasserted itself as a mystery with the discovery of HTSC in 1986.  BCS theory does not work for HTSC materials.  It does not predict that superconductors could exist at such high temperatures (ie minus 140 degrees celsius).  We need a new theory of superconductivity for the newly discovered materials.  But it's not completely up for grabs.  We're still fairly sure that electrons must be pairing up due to some effective attractive interaction.  We're just unsure where the effective attractive interaction comes from.

Mind you, when I say low-temperature superconductivity is "understood" and high-temperature superconductivity is not understood, I'm not referring to my personal level of understanding.  I don't really understand BCS theory.  That is to say, I don't know how to calculate the electron-phonon interaction, and I don't know how to get from the microscopic theory to the Ginzberg-Landau theory.  But that previous sentence might have been gibberish to most of you.  Perhaps what I call "not understanding" is a much deeper understanding than the most educated lay person.

Surely when HTSC is solved, the solution will involve all these little technicalities.  I will not be able to understand the solution.  I will understand the cartoon picture that accompanies the solution, but I will not understand the calculations.

An excess, not a scarcity, of theories

People generally don't talk about mechanisms for HTSC.  The expression "elephant in the room" comes to mind.  My impression is that lots of mechanisms were proposed around 1986-1990, and then it became unfashionable.  The problem isn't that we don't have a theory, it's that we have too many theories.  We need evidence to knock down some of those theories.

New mechanisms for HTSC are occasionally proposed on ArXiV (which is where most physicists share their upcoming publications).  I often wonder if these are cranks.  There's nothing really to stop cranks from putting things up on ArXiV, since it's not peer-reviewed.  I'm told there's even an unwritten special section for cranks (the "general physics" section).  But perhaps many of these papers are completely legitimate and respectable.  The point is I wouldn't be able to the difference.  I've never gotten the impression that they have high impact anyway.

At March Meeting (a huge condensed matter physics conference with over 8000 talks) a few weeks ago, I saw a couple proposals for HTSC mechanisms.  One proposal was made during a 12 minute talk trying to explain the observations of some recent experiment.  The talk sounded exciting, but I didn't understand it at all.  That's not unusual; I don't understand most of the talks.

The other proposal occurred in a poster presentation.  The guy had a theory that did not involve electron pairing.  That makes it "wacky".  I had the impression that he was sort of a crank, since he said he was unable to get published or get funding.  But I respected him anyway.  I don't understand BCS theory, and I didn't understand his theory.  If I'm honest, I can't argue with him.  Let the knowledgeable theorists do the arguing.

I mentioned my impression that people don't really talk about mechanisms for superconductivity.  He said that's because everyone thinks superconductivity is already solved, and that the solution happens to be the idea they themselves proposed.  He alluded to (Nobel Laureate) Phil Anderson's theory.  I'm told that Anderson's mechanism involves electron pairing, but a repulsive force is sufficient to allow the pairing.  That sounds "wacky" too, but what do I know?

Approaching the problem indirectly

Earlier I said that as an experimentalist, I just test ideas proposed by theorists.  But we don't really talk about mechanisms for HTSC.  Instead, we test smaller ideas.

For example, one of the big debates is about a "kink" in the electronic structure.  Is it caused by an interaction between electrons and phonons, or an interaction between electrons and magnons?  And is it related to superconductivity or not?  I suppose there must be a class of theories involving phonons, and a class of theories involving magnons, but we don't talk about the theories directly.  We're just trying to establish the basic facts.

Another big debate is about the so-called "pseudogap" state, which is a strange state that has been observed above the superconducting temperature.  What is the nature of this state of matter?  Is it competing with superconductivity, or is it perhaps an incipient form of superconductivity?  Perhaps it has something to do with CDWs or stripes?  Not that any of this makes sense to you unless you're in the same field as me.  But once we figure out the answer, I'm sure I'll be able to draw a cartoon of it that you'll understand.

I think when we think of historical physics discoveries, we often think of the Eureka! moment.  Someone writes a great paper, and all the problems are solved as it clicks into place.  I'm suspicious of this narrative, because that's not how the field of HTSC looks.  It will be a slow and incremental progression.  Slowly working out incomprehensible technicalities.  But afterwards it will have looked simple.  We'll have a nice cartoon, and we'll tell stories about the scientists who, in a flash of brilliance, dreamed up those cartoons.


Sciatrix said...

I always find it very interesting when people in other disciplines to mine post observations like this, because it gives me a chance to see what is similar to the way people think in my field and what is different.

For example, in evolution/ecology/behavior, we also have a bit of a split between theorists who come up with mathematical models to explain natural phenomena and empiricists who go out and test those theories--but the split doesn't seem to be nearly as pronounced. There are a lot of people who do both some theoretical and some empirical work, and even people who do purely theoretical or purely empirical work are often exhorted to brush up on the part that isn't their specialty. While my research project is also geared to empirically test a piece of evolutionary theory, I'm expected to be familiar with the history of the theory, how the theory I am testing relates to other theories for the same evolutionary process, and have a passing ability to read competing models for that theory before I complete that project.

On the other hand, while there is a minority of EEB people trying to get biologists to start using ArXiV or related depositories, most biologists regard this with an air of deep suspicion and worry about being scooped. Work doesn't get publicly discussed until it's actually published, generally. That's one thing about physics that I rather wish biologists would learn from.

I'm just as suspicious of the narrative of "Eureka!" moments as you are. In my field, scientists tend to think about them a little less, possibly because Charles Darwin was the closest we come to an Archimedes or a Newton and he famously sat on his own "Eureka" moment for something like twenty years as he amassed tiny little bits of evidence to support it. But I suppose that isn't often very clear in pop science, when people are talking about Darwin and what he did to nonspecialists. Do actual physicists you know talk about physics in terms of Eureka! moments, or is it more something that you encounter among nonspecialists? I know that in the discussion sections I teach, one of the things I try very hard to get across to my non-scientist students is that science isn't a fast process of smart people having quick easy ideas; instead it's an incremental process as a lot of people evaluate ideas and test them to see if they hold up.

miller said...

ArXiV doesn't really get rid of the competitiveness. Perhaps EEB people are worried about posting in ArXiV, and giving ideas to rivals who can quickly publish in a journal. But physicists are worried about saying something in a conference, and giving ideas to rivals who can quickly publish on ArXiV. The first person to put it on ArXiV usually gets credit. (Mind you, my field is especially competitive.)

I think the "Eureka!" narrative occurs among both physicists and non-physicists. In physics popularizations it's common to ignore contemporaneous competing theories. In physics courses, it's common to refer to major discoveries by their year and discoverer, without going into detail. BCS theory was discovered in 1957, that's all we need to know.

And really, I would rather understand BCS better than understand its historical context better. But with that attitude, it should not be surprising when my current research experience does not resemble my simplistic historical knowledge.

miller said...

I'm told that the theoretical/experimental divide is more pronounced in physics than any other science. (It varies between fields, with the divide being especially pronounced in high energy physics, and much less pronounced in atomic molecular optics.)

From a first-person perspective, it's not hard to see why. Theoretical papers are literally 10 times harder to read than experimental ones. I can easily see spending years understanding that stuff, and then contributing to it.

Sciatrix said...

The odd thing about the competitiveness, at least in some branches of EEB (especially ecology) is that a lot of the work is system specific. That is, unless you're working on the same species or set of species, your work can't be completely identical to whatever other labs are doing, so you could still get the publication--just not all of the broader significance, but it's not like you would be unable to get anything out of your data set. And it's very common for a lab to be working on a system that only a few other labs work with, and usually they have different interests. Actually, I'm pretty sure that my lab is the only one in the world that is currently working with the species I study, except for a single postdoc who recently graduated from my lab.

(I'm more interested in ArXiV for the transparency than the competitiveness, though. It would be nice to know for sure what datasets are out there in the publication pipeline if I'm trying to answer a question and want to see if someone has found such-and-such process in a variety of species.)

The only other discipline I'm familiar with that even has that level of compartmentalized culture--where you wouldn't necessarily be able to follow work in a related subfield outside of yours--is actually mathematics, so that bit about theory and empiricism being especially distinct in physics isn't super surprising? I have always had the impression that theoretical physics is one of the closest disciplines to theoretical mathematics (as opposed to applied mathematics, under which falls both theoretical physics and theoretical biology). (I think, anyway--not a mathematician!)

I do have to actually understand the theory itself, but I admit that the theory I'm working with is really conceptually quite simple--and it's possible to explain it and its competing theories in a paragraph apiece. In fact, I once did so on tumblr. The math attempting to model the elaboration of traits over time is much more complex, and that I don't understand very much at all yet--I'm still just dipping my toes into modeling, and I suspect that I will end up mostly just familiar with the assumptions underlying the coefficients of the models and how to apply those to empirical work, not understanding too much deeper.

I do find the differences between physics courses and my biology courses really interesting, though--I am used to being told how such-and-such biologist discovered such-and-such theory, with a detailed explanation of the work's methodology and results or a quick story about why they started modeling that topic. That's fascinating that physics courses usually don't do that.

miller said...

I also have to understand some theory as well. But it's a fairly shallow understanding.

For instance, I've been making models of what our data should look like, especially the "kink" mentioned in the OP. To make the models, I have to understand the equations well enough that I can program numerical calculations of them. But this is all fairly well-established stuff, and I refer to papers in the 1970s.

I remember that bit you wrote about sexual selection. That was great.