While the native English speakers cannot make a definitive rule out of it, why should I worry?