Transductive Learning for Textual Few-Shot Classification: Proof of Proposition

cover
1 Mar 2024

This paper is available on arxiv under CC 4.0 license.

Authors:

(1) Pierre Colombo, Equall, Paris, France & MICS, CentraleSupelec, Universite Paris-Saclay, France;

(2) Victor Pellegrain, IRT SystemX Saclay, France & France & MICS, CentraleSupelec, Universite Paris-Saclay, France;

(3) Malik Boudiaf, ÉTS Montreal, LIVIA, ILLS, Canada;

(4) Victor Storchan, Mozilla.ai, Paris, France;

(5) Myriam Tami, MICS, CentraleSupelec, Universite Paris-Saclay, France;

(6) Ismail Ben Ayed, ÉTS Montreal, LIVIA, ILLS, Canada;

(7) Celine Hudelot, MICS, CentraleSupelec, Universite Paris-Saclay, France;

(8) Pablo Piantanida, ILLS, MILA, CNRS, CentraleSupélec, Canada.

A Proof of Proposition

Lemma 1. For any arbitrary r.v. X and countable r.v. Y , we have

where the first inequality follows by applying Jensen’s inequality to the function t 7→ − log(t).

Proof of Proposition 1: From Lemma 1, using Jensen’s inequality, we have

Table 5: Preliminary experiment results. Accuracy of the different backbone.

Using the identity given by (19) in expression (18), and setting β = 1/2, we obtain the following lower bound on I(X; Y ):

The inequality (6) immediately follows by replacing the distribution of the r.v. X with the empirical distribution on the query and P(y|x) with the soft-prediction corresponding to the feature x, which concludes the proof of the proposition.