There are actually four big issues here - memory usage, video card usage, cpu usage and stability - yep, it hits all three core components incredibly hard and the software side of it can get very shakey.
On the memory side it's cos the data structures needed to represent even a modest textured cuboid-box is quite large, multiply 'quite large' by 'many cuboid-boxes' and you get 'huge volumes of data'. Techniques can be applied to reduce how much of that data is required, but even then we're still talking 'large amounts of data'.
On the video side, well, all that data is vertex info, texture refrences, shader refrences, and so on. 'Large amounts of data' mean you're still presenting the video card with 'large numbers of fully textured, shader'd and lit triangles'. Chuck in high resolution and antialiasing and any card's going to be pushed.
The CPU takes a hit because of the physics processing. Basically if you have 'X' physical objects you're dealing with 'half X squared possible interactions'. If 'X' equals 'many' that's like 'half many squared possible interactions' - that's means '
A LOT of processor usage'. Again, this number can be reduced some through applying various techniques and technolgies like dual-core will definately help with future games, but even then, the physics calculations are the part that ask the most of the CPU in any game.
Physics is also where stability comes in - by which I mean the 'physical stability of the model' - all those interactions can amplify rounding and approximation errors in the physics code and make complex structures unstable.
Hardware physics processing cards will help a lot - by this I'm thinking of something like
this - give over all that physics to the physics processor and you can use the spare CPU power to do further preprocessing on the graphics data and to improve the game engine(s) - to give 'more dynamic' as well as 'more physical' game environments.
In the mean time, dual-channel and dual-core will at least help in these areas - two threads running on two cores accessing two parts of memory at the same time means you can run physics on one core, graphics and game engine on the other. Or you can mutli-thread further spliting the processing in almost all areas into multiple concurrently running threads - two available cores makes it much easier to interweave the theads.