You don't need a sieve to calculate prime factors. Its OP. Look at trial division.

I did that, but it's still taking too long because of how large the numbers are. I've been stuck with this all day.

You could try storing primes in a global/static variable. That way you only calculate them once during the first function call.

I know this feeling! I've been recently learning haskell and when i see other people's solution i'm like wow!, i'm still super bad at this. haha