Image by Author | Created on imgflip

Transformers have truly transformed the natural language processing realm, underpinning all state-of-the-art NLP applications. Google Bard, OpenAI’s ChatGPT, and beyond: they're all powered by large (transformer) language models–trained on a massively large corpus–with reinforcement.

The most recent success–OpenAI's ChatGPT is built on top of the GPT-3 family of large language models. This article presents a list of free courses on large language models that’ll help you get a deeper technical understanding.

Let’s dive right in!

CS324: Large Language Models by Stanford University

The CS324: Large Language Models course by Stanford University covers everything you need to know about large language models:

• Capabilities of large language models 
• Harms associated with large language models such as toxicity, misinformation, privacy risk, social biases, and more 
• Modeling and training of large language models such as encoder-only, decoder-only, and encoder-decoder architectures 
• Parallelism 
• Scaling and adaptation of large language models 

The course materials and suggested reading are available on the course website.

Understanding Large Language Models by Princeton University

COS 597G: Understanding Large Language Models offered by Princeton University is another free course that takes you from the basics to advanced concepts in large language models. The course materials and suggested reading are available on the course website, with the syllabus covering the following:

• Basics of large language models.
• In-depth review of BERT, T5, and GPT 3.
• Prompting language models.
• Scaling and risks in large language models.
• Retrieval-based language models.
• Multimodal language models.

Large Language Models by ETH Zürich

Large language models offered by Rycolab at ETH, Zürich is a brand new course that’s currently running (Spring 2023). The course officially started on February 21st, 2023, and the lecture slides and suggested reading will be gradually updated in the course website. This course will help you learn the following:

• Probabilistic foundations probability foundations 
• Modeling foundations
• Neural network modeling and inference 
• Training, fine-tuning, and inference 
• Parallelism and scaling up 
• Security and misuse

CS224n: Deep Learning for NLP by Stanford University

Taught by Prof. Chris Manning at Stanford, CS224n: Deep learning for NLP is a must-take course for anyone interested in natural language processing. From traditional NLP and linguistics concepts all the way up to large language models and ethical challenges, this course provides a comprehensive and solid foundation in the field of natural language processing.

The lectures from the Winter 2021 and Spring 2022 offerings are available on YouTube.

HuggingFace Transformers Course

If you’re looking to learn all about transformers and start building your own NLP applications for natural language inference, summarization, question answering, and more, look no further than the free HuggingFace Transformers course. 

It's organized into three sections that’ll help you become familiar with the HuggingFace ecosystem:

• Using HuggingFace transformers
• The Datasets and Tokenizers libraries
• Building production-ready NLP applications

Other Useful Resources for Large Language Models

So far we covered free courses on large language models. Next, we’ll go over other useful resources to get your feet wet.

Jay Alammar’s Article Series on Large Language Models

From The Illustrated Transformer to Applying Massive Language Models in the Real World with Cohere, Jay Alammar’s technical blog is one of the best resources to understand the ins and outs of natural language processing.

Understanding Large Language Models - A Transformative Reading List

Sebastian Raschka has put together Understanding Large Language Models - A Transformative Reading List of research work around large language models. The reading list will help you understand the breakthroughs in the NLP space over the years: from RNNs in the pre-transformer era to Google BERT to today's ChatGPT.

LangChain

LangChain is a Python library that helps you build useful applications on top of large language models. Some examples include question-answering over a domain-specific corpus, training agents to solve specific problems, and more.

You can check out the documentation for info on setting up the development environment, getting started, and API reference.

Here's a LangChain demo by Harrison Chase, the creator of LangChain.

Wrapping Up

I hope you found this round-up of resources on large language models helpful. We’ve provided a mix of courses, reading lists and other helpful resources, and frameworks that can help you build your own powerful LLM-based applications. 

If you’re looking to learn more about how ChatGPT works, check out this list of free resources to learn ChatGPT.
 
 
Bala Priya C is a technical writer who enjoys creating long-form content. Her areas of interest include math, programming, and data science. She shares her learning with the developer community by authoring tutorials, how-to guides, and more.