Jinyu Yang Senior Applied Scientist Amazon Stores Foundational AI (SFAI) - Rufus Palo Alto, CA Email: viyjy (at) amazon (dot) com Google Scholar     Github

Biography

I obtained my Ph.D. degree from the computer science department at University of Texas at Arlington (Fall 2018-May 2022; John S. Schuchman Outstanding Doctoral Student). My PhD advisor is Prof. Junzhou Huang. I received my M.S degree (2017) and B.S degree (2015) from SDSU and Jilin University, respectively.

Research

Multimodal, Multimodal LLM, Vision-Language Pretraining, Self-Supervised Learning, Text-to-Image Generation

Selected Publications

* Indicate Equal Contribution; The underline authors are PhD interns mentored by me

1. Chuong Huynh, Jinyu Yang, Ashish Tawari, Mubarak Shah, Son Tran, Raffay Hamid, Trishul Chilimbi, Abhinav Shrivastava. “CoLLM: A Large Language Model for Composed Image Retrieval”. In Proc. of IEEE Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA, June 2025. (CVPR'25) (acceptance rate=22.1%) [Project Website]


2. Sirnam Swetha, Jinyu Yang, Tal Neiman, Mamshad Nayeem Rizve, Son Tran, Benjamin Yao, Trishul Chilimbi, Mubarak Shah. “X-Former: Unifying Contrastive and Reconstruction Learning for MLLMs”. In Proc. of The 18th European Conference on Computer Vision, Milan, Italy, September 2024. (ECCV'24) (acceptance rate=27%)


3. Xinliang Zhu, Michael Huang, Han Ding, Jinyu Yang, Kelvin Chen, Tao Zhou, Tal Neiman, Ouye Xie, Son Tran, Benjamin Yao, Doug Gray, Anuj Bindal, Arnab Dhua. “Bringing multimodality to Amazon visual search system”. ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Barcelona, Spain, August 2024. (KDD'24)


4. Kushal Kumar, Tarik Arici, Tal Neiman, Jinyu Yang, Shioulin Sam, Yi Xu, Hakan Ferhatosmanoglu, Ismail Tutar. “Unsupervised multi-modal representation learning for high quality retrieval of similar products at e-commerce scale”. Proceedings of the 32nd ACM International Conference on Information and Knowledge Management. (CIKM 2023)


5. Jinyu Yang, Jingjing Liu, Ning Xu, Junzhou Huang. “TVT: Transferable Vision Transformer for Unsupervised Domain Adaptation”. IEEE Winter Conference on Applications of Computer Vision, Waikoloa, Hawaii, USA, January 2023. (WACV'23)


6. Jinyu Yang, Jiali Duan, Son Tran, Yi Xu, Sampath Chanda, Liqun Chen, Belinda Zeng, Trishul Chilimbi, Junzhou Huang. “Vision-Language Pre-Training with Triple Contrastive Learning”. In Proc. of IEEE Conference on Computer Vision and Pattern Recognition, New Orleans, USA, June 2022. (CVPR'22) (acceptance rate=25%)


7. Jiali Duan, Liqun Chen, Son Tran, Jinyu Yang, Yi Xu, Belinda Zeng, Trishul Chilimbi. “Multi-modal Alignment using Representation Codebook”. In Proc. of IEEE Conference on Computer Vision and Pattern Recognition, New Orleans, USA, June 2022. (CVPR'22) (acceptance rate=25%)


8. Jinyu Yang, Chunyuan Li, Weizhi An, Hehuan Ma, Yuzhi Guo, Yu Rong, Peilin Zhao, Junzhou Huang. “Exploring Robustness of Unsupervised Domain Adaptation in Semantic Segmentation”. In Proc. of the 18th International Conference on Computer Vision, Virtual, October 2021. (ICCV'21 Oral) (acceptance rate=3%)


9. Jinyu Yang, Peilin Zhao, Yu Rong, Chaochao Yan, Chunyuan Li, Hehuan Ma, Junzhou Huang. “Hierarchical Graph Capsule Network”. In Proc. of the 35th AAAI Conference on Artificial Intelligence, February 2021. (AAAI'21) (acceptance rate=21%)


10. Jinyu Yang, Weizhi An, Chaochao Yan, Peilin Zhao, Junzhou Huang. “Context-Aware Domain Adaptation in Semantic Segmentation”. IEEE Winter Conference on Applications of Computer Vision, January 2021. (WACV'21)


11. Jinyu Yang, Weizhi An, Sheng Wang, Xinliang Zhu, Chaochao Yan, Junzhou Huang. “Label-Driven Reconstruction for Domain Adaptation in Semantic Segmentation”. In Proc. of The 16th European Conference on Computer Vision, Glasgow, United Kingdom, August 2020. (ECCV'20) (acceptance rate=26%)


12. Chaochao Yan*, Qianggang Ding*, Peilin Zhao, Shuangjia Zheng, Jinyu Yang, Yang Yu, Junzhou Huang. “RetroXpert: Decompose Retrosynthesis Prediction like A Chemist”. In Proc. of the 34th Annual Conference on Neural Information Processing Systems, Vancouver, Canada, December 2020. (NeurIPS'20 Spotlight) (acceptance rate=3%)


13. Jinyu Yang, Anjun Ma, Adam D. Hoppe, Cankun Wang, Yang Li, Chi Zhang, Yan Wang, Bingqiang Liu, and Qin Ma. "Prediction of regulatory motifs from human Chip-sequencing data using a deep learning framework". Nucleic Acids Research (IF=19.160) (2019) [Project Website]


14. Jinyu Yang, Xin Chen, Adam McDermaid, Qin Ma. "DMINDA 2.0: integrated and systematic views of regulatory DNA motif identification and analyses". Bioinformatics (IF=6.937) (2017) [Project Website]

Working Experience

Dec 2024 - Now	Senior Applied Scientist Amazon Stores Foundational AI (SFAI) - Rufus Palo Alto, CA, USA
May 2022 - Dec 2024	Applied Scientist II Amazon Search Science & AI Palo Alto, CA, USA
Sep 2021 - Dec 2021	Applied Scientist Intern Visual Search & AR Team, Amazon Search Palo Alto, CA, USA
May 2021 - Aug 2021	Research Intern Image & Video Group, Kuaishou US R&D Center Palo Alto, CA, USA

Intern Mentorship

• Zhiruo Zhou, PhD@University of Southern California, 2022 Summer (Now at Apple)
• Swetha Sirnam, PhD@University of Central Florida, 2023 Summer and Fall
• Ryan Huynh (Chuong), PhD@University of Maryland, College Park, 2024 Summer and Fall

Review Service