A closed-form symbol error rate (SER) formula for a generic OFDM system with M-ary phas shift keying (MPSK) modulation, and optimal phase detector for each subchannel, is derived. by employing the derived SER formula as the objective function, a novel OFDM system based on designed complex-valued unitary filter banks is presented to minimize the SER. A detection algorithm is presented for a generic OFDM system to jointly detect subchannel symbols received in one time frame. Monte-Carlo numerical simulations are performed to verify the theoretical SER analysis and compare the error performance of the proposed OFDM system with DFT-based OFDM and DWMT systems, and the new joint detection algorithm with the cinventional subchannel detection algorithm. The above efforts provide systematic approaches for the OFDM error performance analysis and the OFDM system design.