Confusion is not the only thing a trademark infringer can do. The trademark owner must also defend against trademark "dilution" which is the element being raised here.
Trademark "dilution" is the likelihood that the use of the mark will diminish the strength or value of the trademark by reducing the mark's distinctiveness or destroying the mark's image by connecting it to something negative or devaluing. No likelihood of confusion needs to be shown.
This can happen in one instance, or over a long period of time. For example, if "oyster" is used by O&P, and then by Tonka, and then by Mattel, and then by a pizza shop, and then by [add infringer A, B and C], then the word "oyster" becomes diminished as a brand. It's not so much confusion as the fact that it's been used so many times in so many different applications, perhaps some in a negative way, that it doesn't have any value anymore.
Or it could be one bad use, like using the word "oyster" for a drug helps that helps with constipation (with a clock symbol to boot)...just a joke example by me, but I hope you see the point.
So the word "oyster" becoming somewhat of a joke because of this issue, people buying the clock just because of the word "oyster" in conjunction with the negativity brought out by this issue, is evidence that the word "oyster" is actually being devalued.
So again, the trademark owner is under a constant duty to protect their trademark or run the risk of losing their trademark or the value of their trademark.