BroAlanTaps
/

Stage1-PCC-Lite-4x

Feature Extraction

text-generation

Model card Files Files and versions

Model Card for Model ID

PCC-Lite-4x Model:

Compressor: Continue pre-training based on GPT2-Large

Downloads last month: 33

Safetensors

Model size

0.8B params

Tensor type

BF16

·

Model tree for BroAlanTaps/Stage1-PCC-Lite-4x

Base model

openai-community/gpt2-large

Finetuned

(129)

this model

Dataset used to train BroAlanTaps/Stage1-PCC-Lite-4x

Collection including BroAlanTaps/Stage1-PCC-Lite-4x

PCC-Pretrained

Pretraining Context Compressor for Large Language Models with Embedding-Based Memory • 13 items • Updated Mar 2 • 1