Review of Chapter 1 of Building LLM from Scratch

The more lectures I listen to, the more I feel that learning by exploring on your own leaves a longer-lasting impression and leads to learning more than just listening to lectures.

That's why I immediately signed up when I heard that there was a 'Building LLM from Scratch' reading challenge on Inflearn.

Just like last time when I realized the importance of algorithms at Haenghae 99, I hope to learn a lot this time as well.

1. Summary

To summarize the essence of Chapter 1, the beginning of LLM starts with the transformer architecture.

In the transformer architecture, the language is encoded, and the target language is decoded to produce the translated language.

The encoder is the process of tokenizing and vectorizing the language, and decoding is the process of outputting the language.

By isolating the decoding part into a program, you can create services like GPT. GPT repeatedly executes the decoder until the sentence is complete.

To build such a language model, you need learning data.

By using open-source learning data, you can build a model and create various LLM services through fine-tuning.

2. Impressions

Reading the book, I was able to understand the structure of LLM accurately.

Techniques such as BERT, which emerged during vectorization, and One-Hot Encoding used for encoding, were all methods of word vectorization.

Just learning this one aspect made the whole challenge worthwhile.

I plan to thoroughly read Chapter 2 as well.