动手学大模型 Dive into LLMs 教程学习笔记

One word conclusion

Lordog/dive-into-llms is a set of Chinese public welfare tutorials for the introduction and practice of large models. The core value is not "to provide a product that can be directly launched". Instead, it uses 11 themes to string together the key concepts related to large model development, safety, multimodality and intelligent body, and is equipped with PPT, README and Jupyter Notebook, so that learners can run while watching.

If the goal is to complement the knowledge framework in pre-sales, program, product or R & D collaboration of large models, this tutorial is very suitable for "practical operation of key chapters of system literacy". If the goal is production-level deployment, enterprise project delivery template, complete MLOps process, it also needs to be looked at together with more engineering materials.

Basic information

What problems does this tutorial solve

It solves the problem of "knowing the big model is important, but not knowing where to get started with the system.

Many large model learning materials are either too theoretical or only about a certain tool. This set of tutorials is characterized by a wide horizontal coverage: from Transformers fine-tuning, API calls, prompt engineering, knowledge editing, to text watermarking, jailbreak attacks, steganography, multimodal models, GUI Agent, Agent security, RLHF. For beginners, it's like a roadmap; for pre-sales, it helps you break down the concepts that customers often ask into specific capabilities and verifiable experiments.

The project comes from the lecture notes expansion of Shanghai Jiaotong University's "Natural Language Processing Frontier Technology" and "Artificial Intelligence Security Technology" courses. The author also clearly stated that it is a free course for public welfare, and the content is still under construction. Issue and PR are welcome.

Who is suitable for learning

People with completely zero programming foundation | medium and low | can read the concept first, but practical operation will be more difficult |

| People who want to find enterprise-level production deployment manuals | Medium and low | This is not a complete project delivery manual, which needs to be supplemented with operation and maintenance, monitoring, authority, cost and evaluation systems |

Course Map

Recommended learning route

1. Pre-sales Quick Understanding Route

This route is suitable for establishing a knowledge framework that "can talk and understand with customers.

1. Chapter 2: Hint Learning and Thinking Chain

Chapter 1: Fine-tuning and deployment

Chapter 8: Multimodal Models

Chapter 9: GUI Intelligence

1. Chapter 10: Smart Body Security.

Chapter 11: RLHF

The focus of this route is not to run all the code through, but to understand what problems each type of technology solves, where the costs are, and where the risks are. For example, when the customer asks "can we be an Agent that can operate the business system", you should be able to naturally connect GUI Agent, tool call, security evaluation, permission isolation and manual takeover, instead of just saying "can connect large model API".

2. Practical operation route of the project

This route is suitable for people who want real hands-on training or deployment.

1. Chapter 1 First Run Text Classification Fine-tuning and Gradio Deployment
2. Chapter 2 Practice API Calls, few-shot, CoT
3. Chapter 4 attempts mathematical reasoning SFT
4. Chapter 8 Reading Multimodal System Structures
5. Chapter 9 Do GUI Agent Training with Sufficient GPU

Note that Chapter 4 explicitly mentions at least 40GB of video memory, and Chapter 9 mentions at least 3 pieces of 80GB A100 in the training Qwen2-VL GUI Agent example. When there is no such resource, it is recommended to read the process, look at the data format, run light reasoning, and do not train hard.

3. Large Model Safety Route

This route is suitable for people who do safety assessment, compliance, wind control and AI governance.

1. Chapter 2 understands prompt and CoT, because neither attack nor defense can bypass the prompt.
2. Chapter 5 looks at text watermarking and detection.
3. Chapter 6 See Prison Break Attack
4. Chapter 7 Look at text steganography
5. Chapter 10 Look at Agent Security Evaluation
6. Chapter 11 Look at RLHF and Reward Shaping

This part is also valuable for pre-sales: now many customers do not ask "whether the model is accurate", but ask "whether the data will be leaked, misoperated, bypassed and how to define the security responsibility after going online". The R-Judge ideas in Chapter 10 are especially worth picking out for program material.

How to install and use

The repository itself is not a Unity Python package, more like a tutorial book with notebook. It is recommended to use the method of "clone the warehouse first, and then build the environment according to chapter README.

git clone https://github.com/Lordog/dive-into-llms.git
cd dive-into-llms

An example of a basic environment given in Chapter 1 is:

conda create -n llm python=3.9
conda activate llm
pip install transformers

If you run a specific chapter, you should also enter the corresponding directory to view README. For example:

cd documents/chapter1
jupyter notebook dive-tuning.ipynb

The dependencies of different chapters are not exactly the same. For example, Chapter 5 will use X-SIR, Chapter 6 will use EasyJailbreak, Chapter 8 will focus on NExT-GPT, Chapter 9 will rely on Qwen2-VL, OS-Kairos and LLaMA-Factory, and Chapter 11 will use Hugging Face TRL as PPO. Don't expect a 'pip install -r requirements.txt 'to run through all the experiments in the entire tutorial.

Key chapter notes

Chapter 1: Fine-tuning and deployment

Chapter 1 is very suitable for the first practical entrance. It starts with the Hugging Face Transformers and demonstrates model fine-tuning, reasoning, and deployment with a text classification task. There are both low-level customizable versions and Transformers official scripted integrated versions in the tutorial.

The value of this chapter is that it puts "model fine-tuning" into an explainable process: preparing data, loading models and tokenizer, training, evaluating, saving, reasoning, and doing demo with Gradio Spaces. When talking about the plan before sales, customers often ask about RAG, fine-tuning and prompt. This chapter can help you to make "what problems are suitable for fine-tuning" more specific.

Examples of suitable services include: sentiment classification, public opinion classification, news classification, vulnerability text classification, customer service work order classification, lightweight question and answer, and summary tasks. It should be emphasized that the classification task and the generation task are evaluated in different ways, and the effect cannot be judged by demonstrating only a few samples.

Chapter 2: Hint Learning and Thinking Chain

Chapter 2 is the most worthwhile chapter before sales. It covers API calls, zero samples, few samples, thinking chains, Program-of-Thought, self-consistent sampling, and so on. It also mentions an important phenomenon: the quality of the few-shot example affects the performance of the model, and the wrong example may bias the model.

The idea of self-consistent sampling is also very practical: when there is unstable output in the reasoning question, not only one answer is generated, but multiple reasoning paths are sampled when the temperature is greater than 0, and then a majority vote is taken on the final answer. This type of approach is not a panacea, but it can help explain why "the same model sometimes answers the same question differently" and why complex reasoning is accompanied by evaluation and strategy.

Chapter 3: Knowledge Editing

Chapter 3 revolves around EasyEdit. Knowledge editors are concerned about whether they can make changes to a specific piece of knowledge in the model while trying not to disrupt the model's performance on other issues.

The tutorial example is to change the professional knowledge about Messi in the GPT-2-XL from football to basketball, and use the ROME method to demonstrate the editing process. It splits knowledge editing into Editor, Method, Evaluate, and Trainer components, where evaluation focuses on reliability, generality, locality, and portability.

Pre-sales communication can be explained as follows: knowledge editing is not a knowledge base search, nor is it a simple prompt, it is closer to "fixed-point modification of model parameters or behavior". It is suitable for research and verification of specific scenarios, but if the customer just wants the model to know the latest company system, RAG usually has more engineering control.

Chapter 4: Mathematical Reasoning

What Chapter 4 does is mathematical reasoning ability distillation: complex mathematical reasoning responses generated by SFT learning DeepSeek-R1 based on DeepMath-103K data sets and Qwen 2.5-Math-1.5B models. It covers data cleansing, preprocessing, training, generation, and profiling.

This chapter had better not be intimidated by the word "mathematics. What really matters behind it for the programme staff is "using the inference data produced by the high-capacity model to train the smaller model to master a certain type of capability". This is very common in enterprises: use strong models to generate or clean training samples, and then adapt smaller, cheaper, and more easily privatized models to vertical tasks.

The hardware threshold should be made clear in advance: the tutorial mentions at least 40GB of video memory and recommends reserving at least 50GB of disk space.

Chapter 5: Model Watermark

Chapter 5 describes embedding watermarks in language model-generated text and detecting watermark strength. It uses a X-SIR repository, covering text watermarking algorithms such as KGW, X-SIR, SIR, etc., and also contains robustness assessments under dewatermarking attacks such as paraphrase and translation.

Watermark capabilities are often asked in pre-sales scenarios, especially in education, content platforms, media, and government-enterprise compliance scenarios. It should be noted that the watermark is not a magic button that "100% proves that a piece of text was generated by AI", but is part of a probability detection and engineering strategy. When actually landing, it is also necessary to combine the generation of link records, account permissions, content audit and log retention.

Chapter 6: Prison Break Attack

Chapter 6 uses EasyJailbreak to talk about large model jailbreak attacks. EasyJailbreak, the attack process is broken down into seed initialization, constraint/mutation, attack, evaluation and other steps, and multiple attack methods are integrated. Tutorial mentioned PAIR, AdvBench and other materials.

This chapter is suitable for understanding "security is not just a system prompt". After a model goes live, it will face risks such as malicious prompts, inducements, role-playing, multiple rounds of bypassing, and tool call combinations. Before sales, it can be converted into words that customers can understand: red team testing should be done before going online, continuous monitoring should be done after going online, and key business actions should have authority and approval boundaries.

Chapter 7: Large Model Steganography

Chapter 7 does LLM text steganography: using GPT-2 to hide information in the generated text, supporting Huffman Coding and Fixed Length Coding. It generates steganographic text and plain text for comparison.

This part is more partial to safety research and risk education. It reminds us that text may carry not only "surface semantics" but also hidden signals. For ordinary business projects, it is not recommended to package it as an application function; for security, compliance, and content audit teams, it can be used as risk awareness material.

Chapter 8: Multimodal Large Language Model

Chapter 8 talks about multimodal large language models, focusing on the common architecture of MLLM and NExT-GPT such arbitrary modalities to arbitrary modal systems.

In the tutorial, the architecture is divided into two categories: one is LLM as a task scheduler, driving external modules with text commands, and the other is the encoder-LLM-decoder framework, which directly becomes the joint part of the system, receiving multi-modal encoded signals and driving multi-modal output. The second category is a very common route in current multimodal models.

For pre-sales, the most useful part of this chapter is to help you explain why multimodal systems are not just "uploading images to models". A true multi-modal solution would involve visual/audio/video encoders, language models, diffusion models or other decoders, alignment training, instruction data, and inference links.

Chapter 9: GUI Intelligence

Chapter 9 talks about the GUI Agent, which lets the model understand the screen and perform actions. The tutorial is based on Qwen2-VL-7B, OS-Kairos datasets, and LLaMA-Factory, with the goal of building a GUI Agent capable of adaptive human-computer interaction.

This chapter is worth remembering two points.

First, GUI Agent training is not only "screenshot recognition". The data contains information such as tasks, picture paths, step lists, current steps, historical actions, scores, teacher actions, and success. In other words, the model is learning what to do next in the context of the current screen and task ".

Second, resource requirements are high. The tutorial example mentions that a minimum of 3 80GB A100s are required for full training. In the absence of such resources, you can first study data structures, inference formats, and low-confidence trigger manual takeover ideas.

Chapter 10: Intelligent Body Security

Chapter 10 is ideal for presales and program personnel. It discusses security risks in open intelligence scenarios, using a R-Judge measurement platform to analyze whether the model can identify Agent behavioral risks.

The R-Judge data includes multiple rounds of interaction records of users, agents, and environments, manually marked security labels and risk descriptions, covering software programming, operating systems, IoT, applications, economics and finance, network applications, health care, and other categories. The evaluation method is to let the model output risk analysis after reading the record, and then judge safe/unsafe.

This chapter can be directly converted into project speech: enterprises should not be allowed to use Agent, but should make clear which actions can be executed automatically, which actions need approval, which data cannot be transmitted out, which system instructions must be restricted by authority, and how to replay and hold accountable for problems.

Chapter 11: RLHF

Chapter 11 speaks PPO version RLHF with Hugging Face TRL. The experiment is to use the BERT sentiment classifier as a reward function to optimize the GPT-2 to generate more positive IMDB movie reviews.

The PPO process can be understood as three steps: first let the language model generate response according to query; Then use the reward function, reward model or manual feedback to give a scalar score to query-response pair. Finally, use PPO to optimize the model while constraining the model not to deviate too far from the reference model through KL divergence.

The pre-sales help of this chapter is to make "alignment" more realistic: not that the model suddenly becomes smart, but that it is defining rewards, collecting feedback, optimizing strategies, and controlling offsets. It can also help explain why RLHF is costly, long and difficult to evaluate, and cannot be used as a synonym for ordinary fine-tuning.

The whole process of domestic large model development

The project README also mentioned that a public welfare tutorial on "The Whole Process of Large Model Development" jointly launched with Huawei Shengteng Community was launched on June 06, 2025. It includes PPT, experimental manuals and videos, and is divided into primary, intermediate and advanced ones. The goal is to support model development, migration and optimization based on Shengteng software and hardware.

Entry: Full Process of Large Model Development

This part of the domestic substitution, letter creation environment, rising adaptation, government and enterprise customer program will be more reference value. It is recommended to organize a separate article later, because its usage scenario is not exactly the same as GitHub's main tutorial.

The most useful refinements for pre-sales

Concepts that can be directly explained

-Prompt Engineering: Not just "writing prompts", but also example selection, task disassembly, output format constraints, and stability evaluation.

-CoT/PoT: Complex reasoning can be augmented by natural language reasoning chains or programmatic reasoning, but also requires an assessment of authenticity.

-SFT: Use labeled data or distilled data to adapt the model to a certain type of task.

-RAG vs fine-tuning vs knowledge editing: RAG is more suitable for updating knowledge and citation sources; Fine-tuning is more suitable for stabilizing task behavior; Knowledge editing is more biased towards research and fixed-point modification.

-Multimodal: It's not as simple as uploading pictures, it involves encoder, LLM, decoder and modal alignment.

-GUI Agent: The core is screen understanding, action decision making, execution feedback, and security boundaries.

-Agent security: The focus is not whether the model will answer, but whether the model will misoperate, exceed its authority, disclose data, and cause property or system damage.

-RLHF: Shaping model behavior through reward signals, suitable for explaining "why alignment requires data, evaluation and training closed loops".

Can be turned into a customer communication problem

-Does this scenario require the model to "know more", "behave more consistently", or "be able to operate the system"?

-Are labeled data, historical work orders, question-and-answer pairs, operation tracks or safety samples available?

-What is the cost of an output error? Is it an experience problem or a compliance/property/production accident?

-Which actions are allowed to be performed automatically and which must be confirmed manually?

-Is a private deployment required and, if so, are GPUs, data permissions, model licenses, and operational capabilities ready?

-What are the evaluation metrics? Accuracy, recall, task success, rejection, risk identification, or human satisfaction?

Limitations and considerations

1. The warehouse does not have a unified one-click installation process. Each chapter is more like an independent experiment, with dependencies to be viewed by chapter README.
2. Some chapters have high hardware requirements. Chapter 4 has at least 40GB of video memory, and Chapter 9 requires multiple 80GB A100 for complete training examples.
3. The GitHub API does not return an explicit License. Learning references is not a big problem, but authorization should be confirmed before commercial reuse codes, PPT or diagrams.
4. The author of the tutorial declares that the content comes from personal experience, network information and scientific research accumulation, which is for reference only and is not guaranteed to be completely correct.
5. Security-related chapters contain content such as jailbreak, steganography, and risk attacks, and should be used in compliance, isolation, teaching, or research environments.
6. The ecology of large models changes rapidly, and the Transformers, TRL, model weight, API parameters, and download address may expire.

My suggested study plan

Day 1: Creating a Global Map

Read the root directory README and go through the chapter 11 directory. Don't rush to run code, write down the questions answered in each chapter: is it development, reasoning, security, multimodality, or agent.

Day 2: Run through the lightest practice

Priority Run Chapter 2 prompts learning. Prepare a large model API available in China, such as Tongyi Qianyi or Zhipu, practice zero-shot, few-shot, CoT, and self-consistent sampling. This input is small, the output is fast, and it is suitable to become customer demo material immediately.

Days 3 to 4: Understanding the Fine-Tuning Closed Loop

Look at Chapter 1 and run text classification with Transformers. If native resources are limited, at least understand the data format, training script parameters, evaluation metrics, and Gradio deployment process.

Day Five: A Look at Multimodality and Agent

Read chapters 8 and 9 without training immediately. Focus on understanding architecture diagrams, data formats, input and output processes, and resource requirements.

Day 6: Pin Security and Alignment

Read chapters 10 and 11. In particular, the evaluation ideas of R-Judge multi-round records, risk descriptions, and security labels should be recorded, which is very suitable for extending to enterprise Agent online review.

What can be precipitated from this tutorial later

-A big model learning roadmap

-A set of pre-sales common question bank

-A set of demonstrable prompt and CoT cases

-A "RAG/Fine Tuning/Knowledge Editing/Agent" scheme selection table

-An Agent Security Evaluation Checklist

-A special note on the development of a large domestic model.

-Several lightweight demos: text classification, prompt engineering, Gradio deployment, watermark detection

Reference link

-GitHub repository:Lordog/dive-into-llms

-Chapter 1 Tuning and Deployment:chapter1

-Chapter 2 Tips Learning and Thinking Chain:chapter2

-Chapter 3 Knowledge Editing:chapter3

-Chapter 5 Model Watermark:chapter5

-Chapter 8 Multimodal Models:chapter8

-Chapter 9 GUI Agent:chapter9

-Chapter 10 Agent Security:chapter10

-Chapter 11 RLHF:chapter11

-Localization of "The Whole Process of Large Model Development": Huawei Shengteng Course Entrance