shiwk24/MathCanvas-Imagen
Viewer • Updated • 10.1M • 1.51k • 5
How to use dleemiller/siglip2-math-base-patch16-256 with Transformers:
# Use a pipeline as a high-level helper
from transformers import pipeline
pipe = pipeline("zero-shot-image-classification", model="dleemiller/siglip2-math-base-patch16-256")
pipe(
"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/hub/parrots.png",
candidate_labels=["animals", "humans", "landscape"],
) # Load model directly
from transformers import AutoProcessor, AutoModelForZeroShotImageClassification
processor = AutoProcessor.from_pretrained("dleemiller/siglip2-math-base-patch16-256")
model = AutoModelForZeroShotImageClassification.from_pretrained("dleemiller/siglip2-math-base-patch16-256")This version of siglip2 is fine tuned on shiwk24/MathCanvas-Imagen using the code_derived_captions split.
I trained for 1 epoch on 4M math images, with a random selection between the tikz code or caption using a batch size of 640.
This is not a classification model, since the loss function was pairwise contrastive loss. Use for embedding or downstream classifier training is recommended.
Base model
google/siglip2-base-patch16-224