Add `library_name` and usage example to model card

#1
by nielsr HF Staff - opened
Files changed (1) hide show
  1. README.md +34 -4
README.md CHANGED
@@ -1,19 +1,49 @@
1
  ---
2
- license: apache-2.0
3
  language:
4
  - en
5
  - zh
 
6
  pipeline_tag: text-generation
 
7
  ---
8
 
9
  # BlockFFN-Medium
10
 
11
  This is the original 0.5B BlockFFN checkpoint used in the paper *BlockFFN: Towards End-Side Acceleration-Friendly Mixture-of-Experts with Chunk-Level Activation Sparsity* for acceleration tests.
12
- You can load and use this model simply by using `AutoTokenizer` and `AutoModelForCausalLM`.
13
 
14
  Links: [[Paper](https://arxiv.org/pdf/2507.08771)] [[Codes](https://github.com/thunlp/BlockFFN)]
15
 
16
- ### Citation
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
17
 
18
  If you find our work useful for your research, please kindly cite our paper as follows:
19
 
@@ -25,4 +55,4 @@ If you find our work useful for your research, please kindly cite our paper as f
25
  year={2025},
26
  url={https://arxiv.org/pdf/2507.08771},
27
  }
28
- ```
 
1
  ---
 
2
  language:
3
  - en
4
  - zh
5
+ license: apache-2.0
6
  pipeline_tag: text-generation
7
+ library_name: transformers
8
  ---
9
 
10
  # BlockFFN-Medium
11
 
12
  This is the original 0.5B BlockFFN checkpoint used in the paper *BlockFFN: Towards End-Side Acceleration-Friendly Mixture-of-Experts with Chunk-Level Activation Sparsity* for acceleration tests.
 
13
 
14
  Links: [[Paper](https://arxiv.org/pdf/2507.08771)] [[Codes](https://github.com/thunlp/BlockFFN)]
15
 
16
+ ## Usage
17
+
18
+ You can load and use this model simply by using `AutoTokenizer` and `AutoModelForCausalLM` from the `transformers` library.
19
+
20
+ ```python
21
+ from transformers import pipeline, AutoTokenizer, AutoModelForCausalLM
22
+ import torch
23
+
24
+ # Assuming the model ID is "SparseLLM/BlockFFN-Medium"
25
+ model_id = "SparseLLM/BlockFFN-Medium"
26
+
27
+ # Load tokenizer and model
28
+ tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
29
+ model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.bfloat16, trust_remote_code=True)
30
+
31
+ # Create a text generation pipeline
32
+ pipe = pipeline(
33
+ "text-generation",
34
+ model=model,
35
+ tokenizer=tokenizer,
36
+ torch_dtype=torch.bfloat16,
37
+ device_map="auto",
38
+ )
39
+
40
+ # Example usage
41
+ prompt = "The quick brown fox jumps over the lazy"
42
+ result = pipe(prompt, max_new_tokens=50, do_sample=True, top_p=0.9, temperature=0.7)
43
+ print(result[0]["generated_text"])
44
+ ```
45
+
46
+ ## Citation
47
 
48
  If you find our work useful for your research, please kindly cite our paper as follows:
49
 
 
55
  year={2025},
56
  url={https://arxiv.org/pdf/2507.08771},
57
  }
58
+ ```