In reply to elidi.

Yes, that is possible, although it is rarely done in practice. I show a 2-D example in the article "Create scoring data when regressors are correlated."

If you use a full factorial experimental design, this problem doesn't occur, but a full factorial design is often too expensive in high dimensions.