You do a lookup in the whole set for each element, and still your code is very fast! I did kinda the same thing first but with a list and it becomes super slow when doing real work > 50k records.

I benchmarked 5 million records, and your code does it in: 10.307 seconds! (Didn't know set's where that fast.)
Mine don't check the whole list each time but could only manage: 3.9 seconds. (Yes it's faster but I needed way more code and tinkering. Your code has great roi.)

For 50 million records: your code does it in: 104.951 seconds.
For 50 million records: mine does it in: 52.141 seconds.