If the glove doesn't fit ...

Words matter.

You remember the rest of it ... you must acquit. You remember it because it rhymes.

There is actually a cognitive bias for this phenomenon, the "rhyme as reason" effect. A cognitive bias is a pattern of deviation in judgment that occurs in particular situations, which may sometimes lead to perceptual distortion, inaccurate judgment, illogical interpretation, or what is broadly called irrationality.

Is O.J. running free today because of this bias?

Wikipedia explains the effect.

In experiments, subjects judged variations of sayings which did and did not rhyme, and tended to evaluate those that rhymed as more truthful (controlled for meaning). For example, the statement "What sobriety conceals, alcohol reveals" was judged to be more accurate than by different participants who saw "What sobriety conceals, alcohol unmasks". 

The effect could be caused by the Keats heuristic, according to which a statement's truth is evaluated according to aesthetic qualities; or the fluency heuristic, according to which things could be preferred due their ease of cognitive processing.

You would, of course, use rhyming rarely. You should liberally obey the fluency heuristic: A fluency heuristic in psychology is a mental heuristic where, if one out of two objects is processed more fluently, faster, or more smoothly, the mind infers that this object has the higher value with respect to the question being considered.