Pretty much, it's why I'm so picky in reading destiel. Normally I'd agree that both of them could switch. But the big problem for me is that bottomCas fans, just like bottomSam fans have a tendency to write Cas and Sam completely out of character, while turning Dean into an utter asshole.

It's hard to find a fic with those chars bottoming that actually feels right to me.