Fiber nonlinearity-induced penalty reduction in CO-OFDM by ANN-based nonlinear equalization.

We experimentally demonstrate ∼2  dB quality (Q)-factor enhancement in terms of fiber nonlinearity compensation of 40  Gb/s 16 quadrature amplitude modulation coherent optical orthogonal frequency-division multiplexing at 2000 km, using a nonlinear equalizer (NLE) based on artificial neural networks (ANN). Nonlinearity alleviation depends on escalation of the ANN training overhead and the signal bit rate, reporting ∼4  dBQ-factor enhancement at 70  Gb/s, whereas a reduction of the number of ANN neurons annihilates the NLE performance. An enhanced performance by up to ∼2  dB in Q-factor compared to the inverse Volterra-series transfer function NLE leads to a breakthrough in the efficiency of ANN.