| ---
|
| license: apache-2.0
|
| datasets:
|
| - leduckhai/S-Chain
|
| language:
|
| - en
|
| ---
|
| |
| <p align="center"> |
| <img src="./SChain_icon.png" alt="S-Chain logo" width="70"> |
| </p> |
|
|
| <h1 align="center">S-Chain: Structured Visual Chain-of-Thought for Medicine</h1> |
|
|
|
|
|
|
| [](https://arxiv.org/abs/2510.22728) |
| [](https://huggingface.co/leduckhai/S-Chain) |
| [](https://huggingface.co/datasets/leduckhai/S-Chain) |
| [](https://github.com/leduckhai/S-Chain/blob/main/DATASET_LICENSE.md) |
| [](https://s-chain.github.io/) |
| [](https://github.com/leduckhai/S-Chain) |
|
|
| --- |
|
|
| ⭐ **If you find this project helpful, please consider giving it a [star on GitHub](https://github.com/leduckhai/S-Chain)!** |
|
|
| --- |
|
|
| <p align="center"> |
| <a href="https://github.com/leduckhai" target="_blank"><strong>Khai Le-Duc</strong></a><sup>* 1,2✉</sup>, |
| <a href="https://scholar.google.com/citations?user=_NIyeykAAAAJ&hl=en" target="_blank"><strong>Duy M. H. Nguyen</strong></a><sup>* 3,4,24✉</sup>, |
| <a href="https://scholar.google.com/citations?user=5CbQH_kAAAAJ&hl=en" target="_blank"><strong>Phuong T. H. Trinh</strong></a><sup>* 5</sup>, |
| <strong>Tien-Phat Nguyen</strong><sup>* 6</sup>, |
| Nghiem T. Diep<sup>** 3</sup>, |
| An Ngo<sup>** 7</sup>, |
| Tung Vu<sup>** 8</sup>, |
| <a href="https://scholar.google.com/citations?user=trFdwLkAAAAJ&hl=en" target="_blank"><strong>Trinh Vuong</strong></a><sup>9</sup>, |
| Anh-Tien Nguyen<sup>10,11</sup>, |
| Mau Nguyen<sup>12</sup>, |
| Van Trung Hoang<sup>13</sup>, |
| <a href="https://scholar.google.com/citations?user=IMryD1YAAAAJ&hl=en" target="_blank"><strong>Khai-Nguyen Nguyen</strong></a><sup>14</sup>, |
| <a href="https://scholar.google.com/citations?user=ZAuQIqwAAAAJ&hl=en" target="_blank"><strong>Hy Nguyen</strong></a><sup>15</sup>, |
| Chris Ngo<sup>2</sup>, |
| <a href="https://scholar.google.com/citations?user=k_4zYecAAAAJ&hl=en" target="_blank"><strong>Anji Liu</strong></a><sup>16</sup>, |
| <a href="https://scholar.google.com/citations?user=Xs7cKMwAAAAJ&hl=en" target="_blank"><strong>Nhat Ho</strong></a><sup>17</sup>, |
| <a href="https://scholar.google.com/citations?user=Khifj_MAAAAJ&hl=en" target="_blank"><strong>Anne-Christin Hauschild</strong></a><sup>11</sup>, |
| <a href="https://scholar.google.com/citations?user=SmqouhIAAAAJ&hl=en" target="_blank"><strong>Khanh Xuan Nguyen</strong></a><sup>18</sup>, |
| <a href="https://scholar.google.com/citations?user=UrTlMiwAAAAJ&hl=en" target="_blank"><strong>Thanh Nguyen-Tang</strong></a><sup>19</sup>, |
| <a href="https://scholar.google.com/citations?user=cnncomYAAAAJ&hl=en" target="_blank"><strong>Pengtao Xie</strong></a><sup>20,21</sup>, |
| <a href="https://scholar.google.com/citations?user=v7i6Uz4AAAAJ&hl=en" target="_blank"><strong>Daniel Sonntag</strong></a><sup>3,22</sup>, |
| <a href="https://scholar.google.com/citations?user=23ZXZvEAAAAJ&hl=en" target="_blank"><strong>James Zou</strong></a><sup>23</sup>, |
| <a href="https://scholar.google.com/citations?user=p5vLzq0AAAAJ&hl=en" target="_blank"><strong>Mathias Niepert</strong></a><sup>4,24</sup>, |
| <a href="https://scholar.google.com/citations?user=EQw8d9AAAAAJ&hl=en" target="_blank"><strong>Anh Totti Nguyen</strong></a><sup>25✉</sup> |
| </p> |
|
|
|
|
| <p align="center"> |
| <em>*Co-first authors; order randomized | **Co-second authors</em><br> |
| <em>✉ Corresponding Authors</em> |
| </p> |
| <details> |
| <summary><strong>🎓 Affiliations</strong> (click to expand)</summary> |
| 1. University of Toronto, Canada |
| 2. Knovel Engineering Lab, Singapore |
| 3. German Research Centre for Artificial Intelligence |
| 4. University of Stuttgart, Germany |
| 5. Chonnam National University, South Korea |
| 6. Singapore University of Technology and Design |
| 7. Bucknell University, USA |
| 8. Concordia University, Canada |
| 9. Korea University |
| 10. Justus Liebig University Giessen, Germany |
| 11. University Medical Center Göttingen, Germany |
| 12. Japan Advanced Institute of Science and Technology |
| 13. Hue University, Vietnam |
| 14. College of William & Mary, USA |
| 15. Deakin University, Australia |
| 16. National University of Singapore |
| 17. University of Texas at Austin, USA |
| 18. University of California, Berkeley, USA |
| 19. New Jersey Institute of Technology, USA |
| 20. University of California San Diego, USA |
| 21. MBZUAI, UAE |
| 22. Oldenburg University, Germany |
| 23. Stanford University, USA |
| 24. Max Planck Research School for Intelligent Systems (IMPRS-IS), Germany |
| 25. Auburn University, USA |
| </details> |
| --- |
| <p align="center"> |
| ✨ In honor of |
| <a href="https://en.wikipedia.org/wiki/H%E1%BA%A3i_Th%C6%B0%E1%BB%A3ng_L%C3%A3n_%C3%94ng" target="_blank"><strong>Hải Thượng Lãn Ông (海上懶翁) – Lê Hữu Trác (黎友晫)</strong></a>, |
| the father of Vietnamese traditional medicine ✨ |
| </p> |
| ## 🔍 What is S-Chain? |
| S-Chain is the first large-scale dataset of **Structured Visual Chain-of-Thought (SV-CoT)**: |
| each reasoning step is explicitly linked to visual evidence via bounding boxes. |
| This enables training and evaluating *grounded* medical VLM reasoning instead of |
| hallucinated justifications. |
| - **12,000 medical images** with expert bounding boxes. |
| - **700k+ VQA / rationale pairs** across **16 languages**. |
| - Each sample: image, question, answer, stepwise SV-CoT, and per-step visual regions. |
|
|
| We show that supervising VLMs with SV-CoT: |
| - Improves interpretability |
| - Improves grounding fidelity (reasoning actually points to the right region) |
| - Improves robustness across models and languages |
|
|
| <p align="center"> |
| <img src="main_pipeline.png" alt="Alt text" width="1400"/> |
| </p> |
|
|
|
|
| ## 📣 News |
|
|
| - **[Oct 2025]** Updated experiment scripts and checkpoints for ExGra-Med and LLaVA-Med. See the [readme](architectures/Exgra-Med-CoT/README.md) for detailed instructions. |
| - **[Oct 2025]** Dataset and project site released. |
|
|
| ## Citation |
| If you find this work useful, please cite our paper: [https://arxiv.org/abs/2510.22728](https://arxiv.org/abs/2510.22728) |
|
|
| ``` |
| @article{leduc2025schain, |
| title={S-Chain: Structured Visual Chain-of-Thought For Medicine}, |
| author={Le-Duc, Khai and Trinh, Phuong T. H. and Nguyen, Duy M. H. and Nguyen, Tien-Phat and Diep, Nghiem T. and Ngo, An and Vu, Tung and Vuong, Trinh and Nguyen, Anh-Tien and Nguyen, Mau and Hoang, Van Trung and Nguyen, Khai-Nguyen and Nguyen, Hy and Ngo, Chris and Liu, Anji and Ho, Nhat and Hauschild, Anne-Christin and Nguyen, Khanh Xuan and Nguyen-Tang, Thanh and Xie, Pengtao and Sonntag, Daniel and Zou, James and Niepert, Mathias and Nguyen, Anh Totti}, |
| journal={arXiv preprint}, |
| eprint={2510.22728}, |
| url={https://arxiv.org/abs/2510.22728}, |
| year={2025} |
| } |
| ``` |
|
|
| ## ⚖️ Important Notice on Dataset Usage |
|
|
| The S-Chain dataset is provided solely for research and educational purposes. |
| It may contain human or machine annotation errors, as well as potential biases or inconsistencies inherent to medical data. |
| Users are expected to exercise appropriate caution in interpretation and ensure ethical and non-commercial use. |
