Do you know if they had bf/gf during the first seasons of Friends? Because I am like 90% sure they had a thing around that time.

Not entirely sure. Think Courteney had a pretty serious relationship with Michael Keaton, but that might have been over when Friends started? Matthew dated Julia Roberts for a bit, I think. Other than that, probably nothing too serious. And yes, they might have! They were super flirty back then.