I am wondering if I have 2 servers, one that is "main" server and the other one is just there(Both are using the same back-end code). How should I handle if main server crash and I want my clients to user the other server, making it so they don't have to restart their program or anything like that?

Here are some alternatives that might be much more robust:

• Implement a clustered web server and/or clustered database server:
  
  • http://blog.smartbear.com/devops/high-availability-basics-for-developers/
  • https://www.centos.org/docs/5/html/Cluster_Suite_Overview/s1-clstr-basics-CSO.html
• Implement a proxy:
  
  • http://www.haproxy.org/
  • http://wiki.alpinelinux.org/wiki/High_Availability_High_Performance_Web_Cache
• High Availability for your SQL database
  
  • http://blogs.msdn.com/b/sqlcat/archive/2013/11/20/sql-server-2012-alwayson-high-availability-and-disaster-recovery-design-patterns.aspx
  • https://docs.oracle.com/cd/E11882_01/server.112/e10803/toc.htm
  • http://dev.mysql.com/doc/mysql-ha-scalability/en/ha-overview.html
• Finally, to answer your original question about implementing failover at the application level, here are a few examples:
  
  • Failover Socket Client (.Net)
  • Multithreaded Forward Server Example

ADDENDUM:

• You might also want to look at memcached: http://www.javaworld.com/article/2078565/open-source-tools/use-memcached-for-java-enterprise-performance--part-1--architecture-and-setup.html