The cursor.equals method can return standard errors – it calls cursor.get_key on both cursors to retrieve the raw cursor key, so I suspect it could at least return ENOMEM if not other errors.
We don't really have an out-of-band error value (0 == the cursors are equal, >0 == POSIX, <0 == WT errors), so I'm not sure how to fix this.
This impacts the change you made to the Python code to handle cursor.equals returns, as well.