466.4-Gbit/s PCS-64QAM wireless transmission enhanced by dual-polarized SISO link and vector-quantized variational autoencoders.

We successfully demonstrate photonics-assisted single-carrier 466.4 Gbit/s wireless transmission over 20 km SSMF and 6 m single-input single-output (SISO) wireless delivery at 92.5 GHz. This is achieved by employing a likelihood-aware vector-quantized variational autoencoder (VQ-VAE) equalizer for polarization demultiplexing and an advanced multiple-input multiple-output (MIMO) Volterra nonlinear equalizer (MIMO-VNE) further to compensate linear and nonlinear crosstalk between two polarizations. The capacity and robustness of the fiber-wireless system are extensively investigated, verifying that our proposed VQ-VAE with shared-layer autoencoder (SLAE) model outperforms state-of-the-art blind equalizers such as those based on constant modulus algorithm (CMA), VAE, and VQ-VAE algorithms. Additionally, the dual-polarized 40 Gbaud PCS-64QAM signal achieves a 2 dB power margin while meeting the 0.83 NGMI threshold, and a 0.05 gain in NGMI compared to traditional methods. This low-complexity training framework paves the way for enhancing wireless air interface capacities to approach those of fiber-optic transmission within fiber-wireless integration systems.