I am in my second year of a graduate physics program. Students typically earn their PhD after five or more years. In a few years, I will be at a completely different stage of research, and have a completely different perspective. But right now, grad school is still somewhat "new".
My point of comparison is undergraduate university. I thought undergraduate physics was really easy. Obviously, I had many classmates who disagreed, which goes to show that my perspective is not necessarily representative. But that was me. I do very well in a class environment. I never had to study for tests because I already understood the material from the time it was mentioned in lecture.
Grad school is not a class environment. Or at least, not most of it. I've been taking a few classes every semester, but they are not very important, and the grades don't really matter. Soon I won't have any more classes to take. At that point, I'll divert all my attention to research, which is the real centerpiece of physics graduate school.
My impression is that research uses a different set of skills from those used in classes. It's hard to say exactly what that skill set is, but it includes self motivation, good communication skills, and good paper-reading skill. For me, this is somewhat of a disappointment, because I may be great in the classroom, but I am only decent at research skills. 'Twas to be expected, since life isn't a series of lectures, but still. I am most annoyed by all these papers. There is something to be said for the compact and efficient way that physics papers present information. But one thing I would not say for papers is that they are welcoming to people who are new to a topic. I would have a much easier time of it if they were in lecture format.
I came into grad school wanting to do theoretical physics, but now I am doing experimental physics. That's the way a lot of people do it, actually. For whatever reason, incoming students' interests skew towards theoretical, even though there is more room in experimental. An obvious possible cause is that theoretical physics is glamorous. String theory and cosmology are also glamorous, and thus also overrepresented among incoming students' interests. I've also heard it suggested that incoming students want to do theoretical physics because most undergraduate work is essentially theoretical. Students want to do more of the same, and think theoretical research will fit.
Myself, I just liked the idea of solving mathematical puzzles. I've been a puzzle enthusiast for a long time, as you know. But I was open to the idea of doing experimental physics. So I tried it. And now I see there are a lot of advantages to experimental work. And the thing is, I still get to solve puzzles! Last semester, I spent a lot of time trying to explain a feature in our data. I talked to a theorist about it, and he suggested a direction, but I still had to work out the rest. It was quite satisfying. This made me realize, experimentalists will always have an abundant supply of their own problems to solve, and theorists can't solve all of them.
I'm not sure what theoretical physics is like, but I suspect that it is not really much like undergraduate study after all. They probably have to read lots of theoretical papers, which are like ten times harder to read than experimental papers. And they probably do most calculations by computer modeling rather than pencil and paper like undergrads do. And I bet it's more stressful because it's more competitive too. Or so I imagine.
So yeah, I like where I am. My advisor fits the "perpetually absent" archetype, which suits me fine. I've met her several times, and she gives a great pep talk. Most of the time I just refer to the other grad students and postdocs for help, and they are very helpful. I have no complaints so far. Let's see if that changes in a few years!