Edit the file /etc/X11/fs/config and at the end, comment out the line "no-listen = tcp". Then you must stop and start the service (restart won't do it) like this.
/etc/rc.d/init.d/xfs stop
/etc/rc.d/init.d/xfs start
After doing that, if X is running on the server, you are better to logout and restart (Application will become erratic).