Saying yes only to avoid rejection can leave you with an obligation you did not want. Respond according to your capacity and explain the boundary without attacking.