Because the testing code wrongly entangles with the paren type to be defined. I managed to reduce the entanglement by using an independent definition in the testing code.
This may fix this problem.

Thank you, should be fixed by now.

Thank you, should be fixed now.

Thank you for the suggestion, it's inspiring.
It actually got me to ponder the design again and realized instance could not be imported before the data type is defined.
And test have been improved to prevent the situation from happening.

thank you for poniting out. you are right, But inorder to not invalidate already valid answers, I decide to update the test so this won't happen.

This function has been made into a next kata, try it out to see if there is any error
https://www.codewars.com/kata/5fff4fe1ffa782001ea74a3b/

Look like someone found a solution to use monoid. https://www.codewars.com/kata/reviews/5ff135ba11eac60001640111/groups/5ff5a370453bab0001982ef1

k must have type (Paren (S a) -> Paren (S a')).

notice k is in the middle of (PLeft . k . PRight),
which force k to accept the result of PRight and return the input of PLeft,
both should be of type Paren (S a).

(PLeft . k . PRight) as a whole though could be Paren a -> Paren a'

(Paren a -> Paren a) could unify with (Paren Z -> Paren Z)
but (Paren (S a) -> Paren (S a)) could not.

The go here has type:
go :: _int -> (Paren (S a) -> Paren (S a)) -> (Paren a -> Paren a)

Type safety is ensured by imposing restrictions
the more information you encode in types,
the less ways you can compose solutions(Which for often times, exclude some simpler solutions :(.. ).

Expecially for higher order functions, Haskell need its argument to have consistant typing throughout the proccess,
Maybe recursion scheme would help to do recursion, but that is another topic though.

:) You got a nice solution, much better than the my original.

And about the semigroup, If you means (:: Paren a),
consider (a::Paren (S Z)) and (b::Paren (S Z)),
simply stack them togather would have type (:: Paren (S (S Z))), does not work.

Thanks for pointing it out, tests improved

Thanks for pointing it out. Description added now.

I add a new function implementation requirement to better demonstrate the type restricted property.
Now, you can help out by trying the updated version :)