Report · Page 209

On the Opportunities and Risks of Foundation Models
209
Alex Wang, Amapreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R Bowman. 2019b. GLUE: A Multi-
Task Benchmark and Analysis Platform for Natural Language Understanding. In International Conference on Learning
Representations (ICLR).
Ben Wang. 2021. Mesh-Transformer-JAX: Model-Parallel Implementation of Transformer Language Model with JAX.
https://github.com/kingoflolz/mesh-transformer-jax.
Ben Wang and Aran Komatsuzaki. 2021. GPT-J-6B: A 6 Billion Parameter Autoregressive Language Model. https://github.
com/kingoflolz/mesh-transformer-jax.
Haojie Wang, Jidong Zhai, Mingyu Gao, Zixuan Ma, Shizhi Tang, Liyan Zheng, Yuanzhi Li, Kaiyuan Rong, Yuanyong Chen,
and Zhihao Jia. 2021c. PET: Optimizing Tensor Programs with Partially Equivalent Transformations and Automated
Corrections. In 15th USENIX Symposium on Operating Systems Design and Implementation (OSDI 21). 37–54.
Lijun Wang, Wanli Ouyang, Xiaogang Wang, and Huchuan Lu. 2015b. Visual tracking with fully convolutional networks. In
Proceedings of the IEEE international conference on computer vision. 3119–3127.
Mingzhe Wang and Jia Deng. 2020. Learning to Prove Theorems by Learning to Generate Theorems. In Advances in
Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020,
December 6-12, 2020, virtual, Hugo Larochelle, Marc’Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan-Tien
Lin (Eds.). https://proceedings.neurips.cc/paper/2020/hash/d2a27e83d429f0dcae6b937cf440aeb1-Abstract.html
Sinong Wang, Belinda Z Li, Madian Khabsa, Han Fang, and Hao Ma. 2020c. Linformer: Self-Attention with Linear Complexity.
arXiv preprint arXiv:2006.04768 (2020).
Tongzhou Wang and Phillip Isola. 2020. Understanding contrastive representation learning through alignment and uniformity
on the hypersphere. In International Conference on Machine Learning. PMLR, 9929–9939.
Tianlu Wang, Jieyu Zhao, Mark Yatskar, Kai-Wei Chang, and Vicente Ordonez. 2019d. Balanced datasets are not enough:
Estimating and mitigating gender bias in deep image representations. In Proceedings of the IEEE/CVF International
Conference on Computer Vision. 5310–5319.
Wenhui Wang, Sen Yang, Xiang Zhang, and Jing Li. 2014. Drug repositioning by integrating target information through a
heterogeneous network model. Bioinformatics 30, 20 (2014), 2923–2930.
Xiaolong Wang, David Fouhey, and Abhinav Gupta. 2015a. Designing deep networks for surface normal estimation. In
Proceedings of the IEEE conference on computer vision and pattern recognition. 539–547.
Xiaozhi Wang, Tianyu Gao, Zhaocheng Zhu, Zhengyan Zhang, Zhiyuan Liu, Juanzi Li, and Jian Tang. 2021a. KEPLER: A
unified model for knowledge embedding and pre-trained language representation. Transactions of the Association for
Computational Linguistics 9 (2021), 176–194.
Xuefeng Wang, Eric P Xing, and Daniel J Schaid. 2015c. Kernel methods for large-scale genomic data analysis. Briefings in
bioinformatics 16, 2 (2015), 183–192.
Yu Wang, Jinchao Li, Tristan Naumann, Chenyan Xiong, Hao Cheng, Robert Tinn, Cliff Wong, Naoto Usuyama, Richard
Rogahn, Zhihong Shen, et al. 2021b. Domain-Specific Pretraining for Vertical Search: Case Study on Biomedical Literature.
In ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD).
Zihan Wang, Karthikeyan K, Stephen Mayhew, and Dan Roth. 2020a. Extending Multilingual BERT to Low-Resource
Languages. arXiv:2004.136