Is it sit on top or sit in? I fish from a sit in yak all the time, it can be a bit of a pain, but you can get into the skinny water really quietly. I once caught a six foot lemon shark from my yak. He towed me for miles before I wore him down. He wore me down pretty good too.
Another really good investment would be a sea anchor. Will keep you from getting blown around so much. I don't use rod holders, just set the pole in my lap. If I've got multiple poles, I lash one to the foredeck. I primarily flyfish, so I usually just take my trusty 2 weight bassin. Don't see the need for anything heavier.