I don't think desert is easier, I think they're just very different. 
As usual, you prove your ignorance and just have to chit on something if it doesn't fall exactly into YOUR definitions for fun, difficult, or a worthwhile hobby.  That said, you pretty much proved the point that it DOES take skill to rockcrawl by posting that doofus getting hung up in a section that ,dominictorreto>any rock crawler, any real rock crawler</dominictoretto> wouldn't even consider a challenge.  
Personally, I enjoy the challenge of just about every motorsport.  I've never done any sort of dirt-track racing, but honestly if I lived close to an active track I think that's the next thing I'd try.  I've done AutoX and Road Course at pretty competitive levels, and obviously I have done dune and desert but neither competitively (not that there's even a dune competition other than the one that shall not be named that only happens in secrecy by invite-only... cuz that's proven to be how you find the best: invite only competition you want, not an open event).   I've also done a ton of rock-crawling, but mostly as the spotter in the team... but I have a semi-competent rock rig now too.  
I can have just as much fun on a long trail day overcoming some gnarly obstacles and taking in nature (and mostly just getting away from people) as a dune day.  None of them have quite matched road course, but duning is the closest.  Rock crawling and trail driving are more of a slow, (sometimes) relaxing deal and more about the comradery than the "thrill" and adrenaline.  
View attachment 17017
-TJ