How Far Can Better Data Substitute for Scale?

A controlled study of small vision-language models for calorie estimation, from three classes to continuous regression and structured ingredient mass.

4Bparameters in the task-specific model
$4reported cost of the main SFT run
85.2%frozen-test accuracy, n=822
+9.4 ppover the best frontier model tested

Bucket: frozen-test complete · fine / kcal / mass: validation-only · cost estimates: provisional

Abstract

A Qwen3.5-4B model trained on measured Nutrition5k targets scored 85.2% on a frozen three-class calorie test. The best frontier model tested scored 75.8%. Label source changed downstream model quality, and removing 111 contested examples caused a larger loss than changing model size. A second phase extends the same 4B SFT stack to ten-class prediction, continuous regression, and ingredient-mass extraction. Those extensions show strong validation gains, but are not yet frontier comparisons.

The question

The claim is not that a 4B model is generally smarter. The question is whether general-purpose scale is still necessary after the task, labels, and hard cases are fixed.

The experiment uses Nutrition5k. Its calorie and ingredient targets come from physical measurements, so every model and every label source can be checked against the same answer.

Before buying more compute, measure the labels and the hard cases.

What the inputs look like

Nutrition5k contains ordinary cafeteria plates rather than studio food photography. Each plate was captured from multiple side angles and from above, with measured dish mass, calories, macronutrients, and ingredient-level metadata.

Nutrition5k overhead sample showing a breakfast plate
Sample plate · 259 g · 357.81 kcal
Nutrition5k overhead sample showing chicken, vegetables, and greens
Sample plate · 604 g · 655.99 kcal
Nutrition5k overhead sample showing fruit and cottage cheese
Sample plate · 483 g · 351.89 kcal
Nutrition5k overhead sample showing a mixed green salad
Sample plate · 214 g · 184.99 kcal

Official Nutrition5k sample imagery · Google Research · CC BY 4.0

A 4B fine-tune beat every frontier model tested

The gold-label 4B fine-tune scored 85.2% on the frozen 822-case test. The strongest frontier model tested scored 75.8%. The same 4B model scored 50.6% before fine-tuning.

Task-specific 4B 85.2%

Frozen-test accuracy after supervised fine-tuning.

Best frontier model 75.8%

Same cases and the same output contract.

The model inherited the quality of its labels

One qualified human read was 78.7% accurate. Weighted consensus from about 18 reads reached 86.9%. The trained model followed the same order: 78.5% from single reads, 80.3% from consensus, and 85.2% from measured truth.

Label quality → model quality

Label accuracy on the left; frozen-test accuracy after training on the right

Single human read

Label
78.7
4B model
78.5

Weighted consensus

Label
86.9
4B model
80.3

Figure 2. Aggregation improved the label source by 8.2 points; the gain survived model training.

The model did not erase label noise. It compressed the quality ceiling of its supervision into a cheap inference model.

The 111 contested cases mattered more than scale

Removing about 5% of the training set reduced full-test accuracy from 85.2% to 78.5%. On 42 unseen hard cases, accuracy fell from 71.4% to 33.3%.

Hard-case ablation

Same 4B recipe and budget; only the 111 contested training cases changed

Full frozen test

All data
85.2
Removed
78.5
−6.7 points

42 unseen hard cases

All data
71.4
Removed
33.3
−38.1 points

Figure 3. Targeted hard examples produced the largest measured change in the study.

The operational implication is direct: route labeling effort toward examples that break current models. Adding easy examples can increase volume without adding much information.

The same 4B stack learned three harder outputs

The post-2 suite changes the target while keeping the same base model and measured image population. These are validation results on 247 cases.

fine

10-class bins

53.0%

Exact accuracy · 0.59 mean bin error · 9.7% errors ≥2 bins

kcal

Continuous calories

47.5

kcal MAE · 21.0 median error · 70.9% within ±50 kcal

mass

Ingredient mass

75.0%

Recall · 80.3% precision · 10.1 g mean absolute error

These new targets use measured gold labels. They extend the SFT method across output forms; they do not extend the human-consensus result or show a frontier-model win.

References

  1. Thames et al. Nutrition5k. CVPR, 2021.
  2. Hoffmann et al. Training Compute-Optimal Large Language Models. 2022.
  3. Swayamdipta et al. Dataset Cartography. EMNLP, 2020.
  4. Dawid and Skene. Maximum Likelihood Estimation of Observer Error-Rates Using the EM Algorithm. 1979.