SandLogicTechnologies commited on
Commit
1640bcd
·
verified ·
1 Parent(s): 673f158

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +184 -0
README.md ADDED
@@ -0,0 +1,184 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+
3
+ license: apache-2.0
4
+ language:
5
+ - en
6
+ base_model:
7
+ - openbmb/MiniCPM5-2B
8
+ tags:
9
+ - text-generation
10
+ - causal-language-model
11
+ - long-context
12
+ - coding
13
+ - tool-calling
14
+ - agentic
15
+ - reasoning
16
+ - conversational
17
+ ---
18
+
19
+ # MiniCPM5-2B
20
+
21
+ MiniCPM5-2B is a compact dense language model from the MiniCPM5 series, developed by OpenBMB for local assistants, coding agents, tool-use workflows, reasoning tasks, and resource-constrained deployment. The model is designed to provide strong capabilities within a relatively small parameter and memory footprint.
22
+
23
+ The model contains approximately 2.52 billion parameters and uses a standard LlamaForCausalLM architecture with Grouped Query Attention. It supports a native context length of up to 131,072 tokens, making it suitable for applications that require long-context processing while maintaining a compact model size.
24
+
25
+ MiniCPM5-2B is the final post-trained release in the MiniCPM5-2B model line. Its training pipeline includes supervised fine-tuning, reinforcement learning, and On-Policy Distillation (OPD), with dedicated training aimed at reasoning, coding, tool use, and agentic capabilities.
26
+
27
+ ---
28
+
29
+ ## Model Overview
30
+
31
+ - **Model Name**: MiniCPM5-2B
32
+ - **Base Model**: openbmb/MiniCPM5-2B
33
+ - **Architecture**: LlamaForCausalLM
34
+ - **Model Type**: Dense Causal Language Model
35
+ - **Parameter Count**: 2,516,756,480
36
+ - **Context Window**: 131,072 tokens
37
+ - **Modalities**: Text
38
+ - **Languages**: English, Chinese
39
+ - **Developer**: OpenBMB
40
+ - **License**: Apache 2.0
41
+
42
+ ---
43
+
44
+ ## Quantization Formats
45
+
46
+ This repository provides GGUF-converted versions of `MiniCPM5-2B` for efficient local inference. The supplied conversion includes an F16 reference file together with IQ3_M, IQ4_NL, IQ4_XS, and Q6_K variants.
47
+
48
+ The original F16 conversion file was deleted after conversion, while the four quantized variants were retained.
49
+
50
+ ### IQ3_M
51
+
52
+ - Size reduction of approx 75.69% (1.14 GB) compared to 16-bit (4.69 GB)
53
+ - Aggressive 3-bit quantization intended to substantially reduce the memory and storage requirements of the 2B model
54
+ - Suitable for compact local deployments, edge-oriented applications, and systems where available memory is limited
55
+ - Provides a practical low-footprint option for conversational, reasoning, coding, and tool-use workloads
56
+ - The lower numerical precision can introduce greater changes in generation behavior on demanding reasoning and long-context tasks compared with higher-precision variants
57
+
58
+ ### IQ4_NL
59
+
60
+ - Size reduction of approx 70.15% (1.40 GB) compared to 16-bit (4.69 GB)
61
+ - 4-bit non-linear quantization designed to retain more parameter information than the 3-bit representation
62
+ - Provides a balanced option for applications requiring both reduced memory consumption and dependable generation quality
63
+ - Suitable for coding, reasoning, conversational generation, tool-use workflows, and long-context inference
64
+ - Requires a larger model footprint than IQ3_M while providing a higher-precision representation
65
+
66
+ ### IQ4_XS
67
+
68
+ - Size reduction of approx 71.64% (1.33 GB) compared to 16-bit (4.69 GB)
69
+ - Compact 4-bit quantization designed for efficient local deployment
70
+ - Offers a practical middle ground between aggressive compression and higher-precision representations
71
+ - Suitable for local assistants, coding workflows, tool-calling applications, and long-context tasks
72
+ - Provides a slightly smaller footprint than IQ4_NL while remaining within the same 4-bit quantization class
73
+
74
+ ### Q6_K
75
+
76
+ - Size reduction of approx 58.85% (1.93 GB) compared to 16-bit (4.69 GB)
77
+ - Higher-precision 6-bit K-Quant representation designed to retain more parameter information than the lower-bit variants
78
+ - Suitable for workloads where generation fidelity is prioritized while still benefiting from a substantially smaller model footprint than F16
79
+ - Provides a larger representation than IQ3_M, IQ4_NL, and IQ4_XS while remaining considerably smaller than the F16 conversion
80
+ - The higher bit depth is expected to reduce quantization-related information loss compared with the lower-bit variants, although comparative benchmarking is required to establish the actual quality difference
81
+
82
+ ---
83
+
84
+ ## Training Overview
85
+
86
+ MiniCPM5-2B follows a multi-stage training process covering base training, mid-training, and post-training. The training approach is part of OpenBMB's UltraData-based training pipeline.
87
+
88
+ ### Base and Mid-Training
89
+
90
+ The early stages focus on developing core language capabilities, training stability, and adaptation to target data distributions.
91
+
92
+ Training data released alongside the model includes:
93
+
94
+ - Ultra-FineWeb
95
+ - Ultra-FineWeb-L3
96
+ - UltraX
97
+ - UltraData-Code
98
+ - UltraData-Math
99
+
100
+ These datasets support general language learning as well as specialized improvements in coding and mathematical capabilities.
101
+
102
+ ### Post-Training
103
+
104
+ The final MiniCPM5-2B release uses a three-stage post-training process:
105
+
106
+ - Supervised Fine-Tuning (SFT)
107
+ - Reinforcement Learning (RL)
108
+ - On-Policy Distillation (OPD)
109
+
110
+ The SFT stage uses deep-thinking data to develop reasoning and general conversational capabilities. Subsequent RL stages train specialized teachers for mathematics, coding, agentic tasks, writing, and related domains. OPD then combines capabilities from multiple RL-trained expert models into the final release model.
111
+
112
+ The model card reports that the RL + OPD process improves reasoning and general capabilities as well as agentic capabilities across the evaluated benchmarks.
113
+
114
+ ---
115
+
116
+ ## Core Capabilities
117
+
118
+ - **Reasoning**
119
+ Designed for analytical and multi-step reasoning tasks, including mathematical and technical problem solving.
120
+
121
+ - **Coding**
122
+ Provides capabilities for code generation, code reasoning, debugging, and software-development workflows.
123
+
124
+ - **Long-Context Processing**
125
+ Supports a 131,072-token context window for processing long documents, codebases, conversations, and extended task histories.
126
+
127
+ - **Tool Use**
128
+ Designed for workflows where the model interacts with external tools and structured tool interfaces.
129
+
130
+ - **Agentic Workflows**
131
+ Supports coding-agent and general agent tasks requiring multiple steps, tool interaction, and task execution.
132
+
133
+ - **Instruction Following**
134
+ Post-training improves the model's ability to follow user instructions across conversational and structured tasks.
135
+
136
+ - **Mathematical Reasoning**
137
+ Includes specialized post-training for mathematical reasoning and reports strong results across mathematical evaluation tasks.
138
+
139
+ - **Conversational Generation**
140
+ Supports general conversational interactions while maintaining a compact deployment footprint.
141
+
142
+ ---
143
+
144
+ ## Example Usage
145
+
146
+ ### llama.cpp
147
+
148
+
149
+ ```bash
150
+ ./llama-cli \
151
+ -m SandLogicTechnologies/MiniCPM5-2B_IQ4_NL.gguf \
152
+ -p "Explain the architectural differences between dense and Mixture-of-Experts language models."
153
+ ```
154
+
155
+ ## Recommended Use Cases
156
+
157
+ - Local AI assistants
158
+ - Coding-agent workflows
159
+ - Software-development applications
160
+ - Tool-calling systems
161
+ - Agentic AI applications
162
+ - Mathematical reasoning systems
163
+ - Long-context document processing
164
+ - Conversational applications
165
+ - Structured generation workflows
166
+ - Research involving compact language models and agentic systems
167
+
168
+ ---
169
+
170
+ ## Acknowledgments
171
+
172
+ These quantized models are based on the original work by the **OpenBMB** development team.
173
+
174
+ Special thanks to:
175
+
176
+ - The **OpenBMB** team for developing and releasing [MiniCPM5-2B](https://huggingface.co/openbmb/MiniCPM5-2B).
177
+
178
+ - **Georgi Gerganov** and the `llama.cpp` open-source community for enabling efficient GGUF-based local inference.
179
+
180
+ ---
181
+
182
+ ## Contact
183
+
184
+ For any inquiries or support, please contact us at [support@sandlogic.com](mailto:support@sandlogic.com) or visit our [Website](https://www.sandlogic.com/).