This regretful trilogy is often misspoken as could of, should of, and would of. Misappropriating “of” for “have” is just lazy enunciation from the useful contractions of could’ve, should’ve, and would’ve.
Remember this, “of” speaks to a comparative relationship between one thing and another. Whereas “have” is a verb, an action word, that speaks to an activity done, doing, or could have been, should have been, or would have been under favorable circumstances.
How to fix it: just don’t say it. Okay?