PRX Part 4: Our Data Strategy — How Photoroom Built a Million-Scale Dataset for Image Generation

If you’re building a generative AI model for images, you already know the dirty secret: the model is only as good as the data you feed it. And data at scale is a nightmare. I’ve been working with AI pipelines for years, and when I saw Photoroom’s latest update on their data strategy, I had to dig in. They published Part 4 of their PRX series on Hugging Face, and it’s a masterclass in pragmatic, high-quality dataset construction.

Photoroom, the company behind the popular background removal tool, is training their own image generation model. They’re not just grabbing random images from the web. They built a data pipeline that filters, deduplicates, and enriches millions of images. The result? A dataset that’s both vast and clean enough to train a production-grade model. Here’s what they did, why it matters, and how you can apply the same principles.

The Problem: Garbage In, Garbage Out

Every AI practitioner knows that raw data from the internet is chaotic. You get duplicates, low-resolution images, watermarks, and irrelevant content. Photoroom’s team faced exactly this. They needed a dataset large enough to train a diffusion model, but clean enough to avoid artifacts like warped text or meme screenshots. They didn’t just need quantity—they needed quality at scale.

Their approach was ruthless. They started with a massive crawl of public image datasets (like LAION-5B and COYO-700M), but then applied a series of filters that eliminated over 90% of the raw data. This is the kind of hard work most people skip, and it’s why their model performs well in production.

The Solution: A Multi-Stage Filtering Pipeline

Photoroom’s data strategy, as described in their blog post, is built on three core stages: deduplication, quality filtering, and semantic enrichment.

Deduplication at Scale

First, they removed near-duplicate images using perceptual hashing. This isn’t trivial when you’re dealing with hundreds of millions of images. They used a combination of dHash (difference hash) and pHash (perceptual hash) to cluster similar images and keep only the best version. This step alone cut their dataset by roughly 30%.

Quality Filtering

Next, they applied a battery of filters. These included:
- Resolution threshold: They discarded any image under 512x512 pixels.
- Aesthetic score: They used a CLIP-based aesthetic predictor to score images on visual appeal. Only images above a certain threshold were kept.
- Text and watermark detection: They used OCR and a watermark classifier to remove images with overlaid text or logos.
- NSFW detection: They ran a safety classifier to filter out inappropriate content.

Each filter was tuned to balance recall and precision. They didn’t want to throw away good data, but they were aggressive enough to keep the dataset clean.

Semantic Enrichment

Finally, they enriched the remaining images with captions. They used a combination of BLIP-2 and a custom captioning model fine-tuned on e-commerce and product photography. This is critical because the model needs to understand not just what’s in the image, but the context—like “a white ceramic coffee mug on a wooden table” vs. “a mug.”

Results: A Dataset That Works

After all this, Photoroom ended up with a dataset of about 100 million high-quality images. That’s still massive, but it’s a fraction of the original crawl. The key metric? Their model’s FID (Fréchet Inception Distance) score dropped significantly compared to training on unfiltered data. In production, this means fewer artifacts and more consistent outputs.

For example, when generating product images, the model no longer produces warped text or unrealistic shadows. This is a direct result of the data strategy. They also reported that training time decreased because the model converged faster on cleaner data.

Practical Takeaways for Your Own AI Projects

Here’s what I learned from Photoroom’s approach that you can apply today:

1. Don’t skip deduplication. It’s the most underrated step. Use perceptual hashing libraries like ImageHash in Python. It’s simple and saves you from data bloat.
2. Use multiple quality filters. A single metric (like resolution) isn’t enough. Combine aesthetic scores, NSFW classifiers, and watermark detection. Each filter removes a different type of noise.
3. Invest in captioning. If you’re training a text-to-image model, the quality of your captions matters as much as the images. Fine-tune a captioning model on your specific domain (e.g., fashion, furniture, food).
4. Measure, don’t guess. Track metrics like FID or CLIP score to validate your data pipeline. If the score improves, you’re on the right track.

If you’re building an AI product that relies on image generation, Photoroom’s data strategy is a blueprint. They didn’t invent new algorithms—they just executed the fundamentals with discipline.

Conclusion

Photoroom’s PRX Part 4 shows that data strategy is not glamorous, but it’s where the real work happens. In a world where everyone chases the next big model architecture, the winners are those who obsess over data hygiene. Whether you’re training a small classifier or a large diffusion model, the principles are the same: deduplicate, filter ruthlessly, and caption carefully.

For the full technical details, check out their original post on Hugging Face: Source. If you’re looking to integrate similar data pipelines into your own AI workflow, ASI Biont supports connecting to cloud storage and data processing tools via API — more at asibiont.com/courses.