**Building an Image Classifier for Real Estate: To Fine‑Tune or Not to Fine‑Tune?**
In digital real estate services, hundreds of thousands of property listings are uploaded each year, many accompanied by dozens of photos that contain no textual description of what they depict. Automatically understanding these images—whether they show a kitchen, living room, garden, floor plan, or marketing shot—is essential for search, recommendations, and internal analytics. The goal is to assign multiple relevant tags from a set of 23 room‑ and content‑based classes, such as LIVING ROOM, KITCHEN, BEDROOM, GARDEN, and HALLWAY.
Because the same space can serve multiple functions (for example, a living room that includes a visible kitchen), classification is inherently multi‑label and context‑dependent. The threshold chosen for tagging can change based on the application: finding all kitchen photos may require identifying rooms that only partly show a kitchen, while a user requesting “kitchen photos” may expect only images where the kitchen is clearly visible. This makes the choice of modeling approach as much a business decision as a technical one.
At a high level, image classification can be performed in two common ways: using a third‑party vision‑language model (VLM) accessed via an API, or building a custom classifier by training a model on labeled data. Each path has important implications for cost, implementation effort, flexibility, and performance.
**Three Key Questions Before Training Anything**
1. **Prompt an API or train your own model?**
The decision depends heavily on your use case. Prompted APIs are simple to set up and require no training data, but they offer limited control over per‑class confidence and can struggle with vague or fine‑grained categories. Custom models, by contrast, require thousands of labeled examples and more upfront work, but they deliver per‑class scores, better handling of ambiguous cases, and greater flexibility when the task evolves. If your categories are fuzzy—such as distinguishing HALLWAY from LIVING ROOM—an API prompt may be imprecise, whereas a trained model can learn these distinctions from data.
2. **Which foundation model to use?**
When training your own classifier, the practical starting point is a pretrained open‑source vision model. Two common choices are SigLIP, which is trained with image‑caption objectives and excels at recognizing prominent objects, and DINO, a self‑supervised vision transformer that emphasizes layout and background due to its patch‑level training objective. The foundation model should align with your business priorities: object‑centric versus layout‑centric representations.
3. **To fine‑tune or keep the model frozen?**
A frozen feature extractor with a linear classifier is fast to train, inexpensive, and surprisingly effective, often reaching reasonable accuracy within minutes. However, it tends to produce low confidence scores and high under‑labeling rates. Fine‑tuning—especially with low‑rank adapters (LoRA)—updates only a small subset of parameters, preserves most of the foundation model’s strengths, and can significantly improve recall and calibration, at the cost of additional compute and training complexity.
**Performance in Practice**
In a production‑like evaluation on 40,000 manually annotated real estate photos, the differences between approaches were measurable but nuanced. A LoRA‑fine‑tuned SigLIP model achieved a micro averaged F1 score of 82.6%, improving recall by about 7 points over frozen SigLIP and reducing the share of unlabeled photos from around 9% to 3.4%. For specific classes like GARDEN and DINING AREA, fine‑tuning delivered gains of 26 and 15 F1 points respectively, largely because it better handled scenes where these areas appear only partially or in the background.
Choosing DINOv2 as a frozen alternative recovered more than half of the GARDEN gap without any fine‑tuning, highlighting how foundation‑model design alone can close substantial performance gaps. Yet, DINOv2 underperformed on more semantically driven classes such as KITCHEN and HALLWAY, suggesting that different models emphasize different signals.
**Cost, Effort, and Flexibility**
From a cost perspective, using an API can be expensive at scale—on the order of $1,500 per million images—while running a custom classifier on a GPU can reduce this to a few dollars. However, if you only process a few hundred images per day, the financial difference is negligible. Training a frozen linear model is nearly costless and can be done on a CPU in minutes; LoRA fine‑tuning requires GPU resources and more time but remains far cheaper than API inference at volume.
Operational flexibility is another advantage of custom models. With a trained classifier, you obtain well‑calibrated per‑class scores that can be thresholded consistently or adjusted as new categories emerge. API‑based approaches often rely on coarse confidence proxies that are harder to interpret and standardize across classes.
**When Is Fine‑Tuning Worthwhile?**
Fine‑tuning is most justified when under‑labeling is a serious problem for your workflow. If a small fraction of unlabeled photos is acceptable, a frozen model may be sufficient and far simpler to maintain. For classes where recall is critical—such as GARDEN or DINING AREA—fine‑tuning can deliver large improvements that are hard to obtain any other way. If you anticipate frequent changes to categories or rapid deployment of new classifiers, the faster, lighter approach of frozen features plus promptable APIs may be more practical, with the option to move to fine‑tuned models later as data and needs grow.
**Conclusion**
Whether you use an API, a frozen model, or a fine‑tuned classifier, automated image labeling clearly adds value across search, recommendations, and internal analytics in real estate platforms. Starting small with a prompted VLM is a low‑risk way to explore the problem, while a well‑planned fine‑tuning strategy offers the precision and stability required for production at scale. The best path depends on your data, your performance targets, and the trade‑offs you are willing to make between cost, simplicity, and accuracy.
Thank you for reading
