Three stages of pruning
We studied how data pruning affects adversarial robustness and whether it can be used to train a more robust model. Pruning involves more than deciding which examples to discard: the data must first be prepared for computing statistics, those statistics must be turned into selection scores, and the model must then be trained on the chosen subset. We examined how choices in each of these three stages affect robustness.
Prepare the data and compute statistics
The preparation step determines the data on which the pruning statistics are computed. We investigated how this choice changes the information used to assess each training example, and what that means for robustness.
Score examples and select a subset
A scoring rule turns those statistics into a basis for choosing which examples to keep. We studied how the score and the resulting selection affect the robustness of a model trained on that subset.
Train on the pruned data
The selected data are then used to train the model. We examined how the training procedure affects the final robustness, alongside the choices made during preparation and selection.