if letter == "x" or "X":

You need to put "letter ==" in your 2nd logical expression (in front of the "X").
If you don't then it will practically means:

if (letter == "x") or ("X" != 0)
...which will always be TRUE, since "X" ascii is not 0.

man I was working on a similar solution that failed:

for letter in s:
if letter == "x" or "X":
sum += 1

Could you explain why my answer fails ?

I am guessing its because I cannot compare 2 values at the same time to an iterator (letter) ?