AI Sweden, together with RISE and WASP WARA Media & Language, are developing a large-scale generative language model for the Nordic languages, and primarily Swedish.
GPT-SW3 is the first truly large-scale generative language model for the Swedish language. Based on the same technical principles as the much-discussed GPT-3, GPT-SW3 will help Swedish organizations build language applications never before possible.
Do you want access to GPT-SW3?
We are now inviting developers and practitioners to a controlled pre-release of the models to try them out. Please note that we provide the model weights for download on Hugging Face and that the models are not available through an API or user-friendly web interface.
The pre-release is an important step in the process of knowledge building and validating the model (with 126M, 356M, 1.3B, 6.7B, 20B, 40B) and collecting feedback on both what works well and what does not.
The models are accessible in a private repository under a modified RAIL license on Hugging Face, where we also provide both a model card and a datasheet - please note that you will need significant computation power. In order to access the repository and use the model you need to apply using this form. All applicants will have to approve the license and go through manual approval before the model is provided. The pre-release is intended for organizations and individuals in the Nordic NLP ecosystem.
Join the conversation on Discord and reach out to Francisca Hoyer to share your learnings! And keep an eye out for workshops and seminars for deep dives into use cases and collective problem solving.
Want to read more? Check out our page on Medium.
Facts
Project partners: AI Sweden, RISE and WASP WARA Media & Language.
The current GPT-SW3 models are trained on Linköpings University’s supercomputer, Berzelius, using the Nemo Megatron framework from NVIDIA.