Jinyu Yang Applied Scientist II Amazon Stores Foundational AI (SFAI) Palo Alto, CA Email: viyjy (at) amazon (dot) com Google Scholar     Github

Biography

I obtained my Ph.D. degree from the computer science department at University of Texas at Arlington (Fall 2018-May 2022; John S. Schuchman Outstanding Doctoral Student). My PhD advisor is Prof. Junzhou Huang. I received my M.S degree (2017) and B.S degree (2015) from SDSU and Jilin University, respectively.

Research

Multimodal, Multimodal LLM, Vision-Language Pretraining, Self-Supervised Learning, Text-to-Image Generation

Publications

* Indicate Equal Contribution; The underline authors are interns mentored by me

1. Sirnam Swetha, Jinyu Yang, Tal Neiman, Mamshad Nayeem Rizve, Son Tran, Benjamin Yao, Trishul Chilimbi, Mubarak Shah. “X-Former: Unifying Contrastive and Reconstruction Learning for MLLMs”. In Proc. of The 18th European Conference on Computer Vision, Milan, Italy, September 2024. (ECCV'24) (acceptance rate=27%)


2. Xinliang Zhu, Michael Huang, Han Ding, Jinyu Yang, Kelvin Chen, Tao Zhou, Tal Neiman, Ouye Xie, Son Tran, Benjamin Yao, Doug Gray, Anuj Bindal, Arnab Dhua. “Bringing multimodality to Amazon visual search system”. ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Barcelona, Spain, August 2024. (KDD'24)


3. Kushal Kumar, Tarik Arici, Tal Neiman, Jinyu Yang, Shioulin Sam, Yi Xu, Hakan Ferhatosmanoglu, Ismail Tutar. “Unsupervised multi-modal representation learning for high quality retrieval of similar products at e-commerce scale”. Proceedings of the 32nd ACM International Conference on Information and Knowledge Management. (CIKM 2023)


4. Jinyu Yang, Jingjing Liu, Ning Xu, Junzhou Huang. “TVT: Transferable Vision Transformer for Unsupervised Domain Adaptation”. IEEE Winter Conference on Applications of Computer Vision, Waikoloa, Hawaii, USA, January 2023. (WACV'23)


5. Jinyu Yang, Jiali Duan, Son Tran, Yi Xu, Sampath Chanda, Liqun Chen, Belinda Zeng, Trishul Chilimbi, Junzhou Huang. “Vision-Language Pre-Training with Triple Contrastive Learning”. In Proc. of IEEE Conference on Computer Vision and Pattern Recognition, New Orleans, USA, June 2022. (CVPR'22) (acceptance rate=25%)


6. Jiali Duan, Liqun Chen, Son Tran, Jinyu Yang, Yi Xu, Belinda Zeng, Trishul Chilimbi. “Multi-modal Alignment using Representation Codebook”. In Proc. of IEEE Conference on Computer Vision and Pattern Recognition, New Orleans, USA, June 2022. (CVPR'22) (acceptance rate=25%)


7. Jinyu Yang, Chunyuan Li, Weizhi An, Hehuan Ma, Yuzhi Guo, Yu Rong, Peilin Zhao, Junzhou Huang. “Exploring Robustness of Unsupervised Domain Adaptation in Semantic Segmentation”. In Proc. of the 18th International Conference on Computer Vision, Virtual, October 2021. (ICCV'21 Oral) (acceptance rate=3%)


8. Jinyu Yang, Peilin Zhao, Yu Rong, Chaochao Yan, Chunyuan Li, Hehuan Ma, Junzhou Huang. “Hierarchical Graph Capsule Network”. In Proc. of the 35th AAAI Conference on Artificial Intelligence, February 2021. (AAAI'21) (acceptance rate=21%)


9. Jinyu Yang, Weizhi An, Chaochao Yan, Peilin Zhao, Junzhou Huang. “Context-Aware Domain Adaptation in Semantic Segmentation”. IEEE Winter Conference on Applications of Computer Vision, January 2021. (WACV'21)


10. Jinyu Yang, Weizhi An, Sheng Wang, Xinliang Zhu, Chaochao Yan, Junzhou Huang. “Label-Driven Reconstruction for Domain Adaptation in Semantic Segmentation”. In Proc. of The 16th European Conference on Computer Vision, Glasgow, United Kingdom, August 2020. (ECCV'20) (acceptance rate=26%)


11. Chaochao Yan*, Qianggang Ding*, Peilin Zhao, Shuangjia Zheng, Jinyu Yang, Yang Yu, Junzhou Huang. “RetroXpert: Decompose Retrosynthesis Prediction like A Chemist”. In Proc. of the 34th Annual Conference on Neural Information Processing Systems, Vancouver, Canada, December 2020. (NeurIPS'20 Spotlight) (acceptance rate=3%)


12. Chaochao Yan, Sheng Wang, Jinyu Yang, Tingyang Xu, Junzhou Huang. “Re-balancing Variational Autoencoder Loss for Molecule Sequence Generation”. ACM Conference on Bioinformatics, Computational Biology, and Health Informatics, September 2020. (ACM-BCB'20)


13. Yuzhi Guo, Jiaxiang Wu, Hehuan Ma, Jinyu Yang, Xinliang Zhu, and Junzhou Huang. "WeightAln: Weighted Homologous Alignment for Protein Structure Property Prediction", IEEE International Conference on Bioinformatics and Biomedicine, Seoul, Korea, December 2020. (BIBM'20)


14. Jinyu Yang, Anjun Ma, Adam D. Hoppe, Cankun Wang, Yang Li, Chi Zhang, Yan Wang, Bingqiang Liu, and Qin Ma. "Prediction of regulatory motifs from human Chip-sequencing data using a deep learning framework". Nucleic Acids Research (IF=19.160) (2019) [Project Website]


15. Jinyu Yang, Xin Chen, Adam McDermaid, Qin Ma. "DMINDA 2.0: integrated and systematic views of regulatory DNA motif identification and analyses". Bioinformatics (IF=6.937) (2017) [Project Website]


16. Sheng-Yong Niu*, Jinyu Yang*, Adam McDermaid, Jing Zhao, Yu Kang, Qin Ma. "Bioinformatics tools for quantitative and functional metagenome and metatranscriptome data analysis in microbes". Briefings in Bioinformatics (IF=13.994) (2017)


17. Bingqiang Liu, Jinyu Yang, Yang Li, Adam McDermaid, Qin Ma. "An algorithmic perspective of de novo cis-regulatory motif finding based on ChIP-seq data". Briefings in Bioinformatics (IF=13.994) (2017)


18. Yu Zhang*, Juan Xie*, Jinyu Yang, Anne Fennell, Chi Zhang, Qin Ma. "QUBIC: a bioconductor package for qualitative biclustering analysis of gene co-expression data". Bioinformatics (IF=6.937) (2017)

Working Experience

May 2022 - Now	Applied Scientist II Amazon Search Science & AI Palo Alto, CA, USA
Sep 2021 - Dec 2021	Applied Scientist Intern Visual Search & AR Team, Amazon Search Palo Alto, CA, USA
May 2021 - Aug 2021	Research Intern Image & Video Group, Kuaishou US R&D Center Palo Alto, CA, USA

Intern Mentorship

• Zhiruo Zhou, PhD@University of Southern California, 2022 Summer (Now at Apple)
• Swetha Sirnam, PhD@University of Central Florida, 2023 Summer and Fall
• Ryan Huynh (Chuong), PhD@University of Maryland, College Park, 2024 Summer

Review Service