You can make the isPrime method much more efficient by only checking against other prime numbers under p. Since you are building up the prime cache starting from 2 and working your way up there is no risk of missing primes.

I missed that the upper bound was Math.sqrt in my solution.

The reason this is very much not best practice is that it is very difficult for programmers to work together when type abstractions are violated. It is a great solution if the programmer is the only person that is ever going to rely on this code or need to modify it. If other programmers are involved, these sort of solutions can lead to unnecessary confusion and therefore bugs.

Also, if you are working on a personal project for long enough, you may look at a piece of code you did years ago and forget why your clever solution works or what it is meant to do.