A cross-modal transfer-learning method (VisTabNet) that repurposes a Vision Transformer encoder pre-trained on images to solve small-sample tabular-data tasks. A lightweight adaptation network projects each tabular input into the patch-embedding format the frozen ViT encoder expects, so the image-pretrained self-attention stack can be applied directly to tables without designing a bespoke tabular architecture or training a transformer from scratch. It shows that pre-trained image models transfer usefully across modalities, outperforming ensemble methods and prior deep tabular models on datasets with fewer than ~1k samples.
Although deep learning models have had great success in natural language processing and computer vision, we do not observe comparable improvements in the case of tabular data, which is still the most…