🔄 In a Training Loop

Jiaqi Tang PRO

Jiaqi-hkust

7 12 20

https://jqt.me/

AI & ML interests

Multimodal Large Language Model

Recent Activity

reacted to theirpost with ➕ about 12 hours ago

IQA-T1: Evidence‑Based Image Quality Assessment with MLLMs Most MLLMs are blind to low‑level degradations—noise, blur, compression artifacts look the same as clean images in their internal representations. That leads to quality scores based on semantic “gut feeling” rather than real perceptual evidence. IQA-T1 changes that. We equip the model with a toolbox of 15 perceptual tools (noise residual maps, Fourier spectra, gradient maps, etc.) that generate structured visual evidence on demand. The model learns how to use tools via supervised fine‑tuning on our Q‑Tool dataset (11k evidence‑grounded reasoning chains), and when to call them via GRPO reinforcement learning that balances accuracy, tool count, and redundancy. The result: SOTA performance across 7 benchmarks (avg PLCC 0.795), using only 2.34 tools per image on average. Every predicted score is now interpretable and backed by hard visual evidence. All code, weights, dataset, and demo are open. Check them out and give it a spin! 📄 arxiv.org/abs/2607.12375v1 💻 github.com/zibuyu-02/IQA-T1 🤗 model/data: huggingface.co/zibuyu-02/IQA-T1 🎮 demo: huggingface.co/spaces/Jiaqi-hkust/IQA-T1

reacted to theirpost with 🤝 about 14 hours ago

reacted to theirpost with 😎 2 days ago

View all activity

Organizations

Posts 6

Post

2739

IQA-T1: Evidence‑Based Image Quality Assessment with MLLMs

Most MLLMs are blind to low‑level degradations—noise, blur, compression artifacts look the same as clean images in their internal representations. That leads to quality scores based on semantic “gut feeling” rather than real perceptual evidence.

IQA-T1 changes that. We equip the model with a toolbox of 15 perceptual tools (noise residual maps, Fourier spectra, gradient maps, etc.) that generate structured visual evidence on demand. The model learns how to use tools via supervised fine‑tuning on our Q‑Tool dataset (11k evidence‑grounded reasoning chains), and when to call them via GRPO reinforcement learning that balances accuracy, tool count, and redundancy.

The result: SOTA performance across 7 benchmarks (avg PLCC 0.795), using only 2.34 tools per image on average. Every predicted score is now interpretable and backed by hard visual evidence.

All code, weights, dataset, and demo are open. Check them out and give it a spin!

📄 arxiv.org/abs/2607.12375v1
💻 github.com/zibuyu-02/IQA-T1
🤗 model/data: huggingface.co/zibuyu-02/IQA-T1
🎮 demo: huggingface.co/spaces/Jiaqi-hkust/IQA-T1

Post

3989

🚀 Introducing Robust-U1: Teaching MLLMs to Self-Recover Corrupted Visual Content

Multimodal Large Language Models (MLLMs) have achieved impressive visual understanding, yet they remain highly brittle under real-world corruptions—noise, blur, compression artifacts, adverse weather.

Standard MLLMs suffer dramatic performance drops, and existing robustness solutions come with fundamental limits: black‑box feature alignment lacks interpretability, while white‑box text reasoning cannot restore the lost pixel‑level visual details. This raises a crucial question:

🧐 Can MLLMs recover corrupted visual content by themselves?

If the answer is yes, we can move beyond merely “compensating” for corruption and instead build a more intrinsic, generalizable form of resilience. Robust-U1 is our answer to that question.

💡 Paper: https://arxiv.org/abs/2606.08063
🔗 Code: github.com/jqtangust/Robust-U1
🌍 Demo: Jiaqi-hkust/Robust-U1