prompt eval time = 3282.51 ms / 436 tokens ( 7.53 ms per token, 132.83 tokens per second) eval time = 11093.61 ms / 362 tokens ( 30.65 ms per token, 32.63 tokens per second) total time = 14376.13 ms / 798 tokens
prompt eval time = 10964.14 ms / 1409 tokens ( 7.78 ms per token, 128.51 tokens per second) eval time = 36610.31 ms / 1105 tokens ( 33.13 ms per token, 30.18 tokens per second) total time = 47574.45 ms / 2514 tokens
prompt eval time = 11592.94 ms / 1345 tokens ( 8.62 ms per token, 116.02 tokens per second) eval time = 29251.55 ms / 839 tokens ( 34.86 ms per token, 28.68 tokens per second) total time = 40844.49 ms / 2184 tokens
prompt eval time = 21192.54 ms / 2218 tokens ( 9.55 ms per token, 104.66 tokens per second) eval time = 62331.64 ms / 1625 tokens ( 38.36 ms per token, 26.07 tokens per second) total time = 83524.18 ms / 3843 tokens
Chemical classification remains useful but is rendered somewhat redundant by the broad range of chemical entities now available and by the absence of any clear structure–activity relationships for newer drugs. The chemistry of some older drugs does relate to their propensity to cause movement disorders.
Hans Asperger used to love telling the story of his life; thus all I have to do is retell it. He was born in Vienna in 1906. His grandfather’s family had been farmers east of the capital of the Austro-Hungarian Monarchy for many generations. As a high school student, he became acquainted with the “German Youth Movement.” It was in this movement that this achievement-oriented and intellectual young man was to find all those things he valued most throughout his lifetime. There he discovered friendship, mountaineering, nature, art as a source of strength and repose, and literature—the medium in which he moved and lived.
这里暂且使用默认的元指令“You are a helpful assistant”，但于其后加入 /think 或 /no_think 来控制是否启用思考链；其余条件（量化级别、llama.cpp 提交等）和后记一相同。
数学问题测试
设a,b为正实数，满足 a*b=a+b+3；求a,b的取值范围。
启用思考链时的输出
思考链出现了重复思考和无穷重复输出的现象，故中止了推理。此处不再粘贴整个思考链。
关闭思考链时的输出
我们已知：
$$ ab = a + b + 3 \quad \text{（1）} $$
其中 $ a, b > 0 $，要求 $ a, b $ 的取值范围。
第一步：整理方程
从（1）式出发：
$$ ab - a - b = 3 $$
我们尝试将左边配方：
$$ ab - a - b = (a - 1)(b - 1) - 1 $$
所以：
$$ (a - 1)(b - 1) - 1 = 3 \Rightarrow (a - 1)(b - 1) = 4 $$
第二步：设变量替换
令：
$$ x = a - 1, \quad y = b - 1 $$
则 $ x > -1, y > -1 $（因为 $ a, b > 0 $），且：
$$ xy = 4 $$
所以 $ x $ 和 $ y $ 是满足 $ xy = 4 $ 的实数，且 $ x > -1, y > -1 $。
第三步：求 $ a, b $ 的取值范围
由 $ x = a - 1 $，$ y = b - 1 $，得：
$$ a = x + 1, \quad b = y + 1 $$
所以：
$$ a = x + 1, \quad b = \frac{4}{x} + 1 $$
因为 $ x > -1 $，且 $ xy = 4 $，所以：
$ x > 0 $，否则 $ y = \frac{4}{x} $ 会变成负数，导致 $ b = y + 1 < 0 $，不满足 $ b > 0 $。
所以 $ x > 0 $，从而 $ y > 0 $
因此：
$$ a > 1, \quad b > 1 $$
第四步：总结取值范围
我们得到：
$ a > 1 $
$ b > 1 $
且满足 $ (a - 1)(b - 1) = 4 $
也就是说，$ a $ 和 $ b $ 是满足这个关系的正实数，且都大于 1。
最终答案：
$$ \boxed{a > 1, \quad b > 1} $$
同时满足：
$$ (a - 1)(b - 1) = 4 $$
这是 $ a, b $ 的取值范围。
在终端里一窥量化情况
1 2 3 4 5 6
llama_model_loader: - type f32: 241 tensors llama_model_loader: - type q2_K: 96 tensors llama_model_loader: - type q3_K: 37 tensors llama_model_loader: - type q4_K: 180 tensors llama_model_loader: - type q5_K: 14 tensors llama_model_loader: - type q6_K: 11 tensors
翻译测试
同样关闭思考链的情况下，给予前述第二条英语片段来翻译，可以见到译文质量再上一个台阶。
另一医学著作的普通英语片段
Hans Asperger used to love telling the story of his life; thus all I have to do is retell it. He was born in Vienna in 1906. His grandfather’s family had been farmers east of the capital of the Austro-Hungarian Monarchy for many generations. As a high school student, he became acquainted with the “German Youth Movement.” It was in this movement that this achievement-oriented and intellectual young man was to find all those things he valued most throughout his lifetime. There he discovered friendship, mountaineering, nature, art as a source of strength and repose, and literature—the medium in which he moved and lived.
近期因工作需要，着手在笔记本上部署 AI 助手来帮忙翻译。开源翻译助手 Pot 最近遇到些问题：先是 DeepL 接口的默认 token 废弛，再是在整整一周时间内（两次滚动更新之间）都打不开。现在不少大语言模型都具备多语言能力，不过公共接口都有试用限制，我便考虑本地部署大模型，作为翻译小助手；要求既表意准确，推理也要足够快。 早前曾尝试在资料服务器上托管，但未果。
由于缺少独立 GPU，处理器算力也有限，这里要求推理框架尽可能地轻量化。基于 Python 的各种框架：Pytorch、Transformers 等等可能太重了，不仅因为执行效率比不上 C++ 程序，也因为要连带一大堆 NVIDIA 专有框架一起安装。基于 C++ 的 llama.cpp 貌似可以接受，不仅自身开销小，也自带一个 Web app 和 OpenAI-like API 服务用来试用、托管模型。
“I want you to act as a translator. I will speak to you in English and you will translate it and answer in Chinese. Keep the meaning same, but make them more literary. I want you to only reply the translations and nothing else, do not write explanations.” ——面向外国企业训练的模型，如 LLaMa、Gemma
Chemical classification remains useful but is rendered somewhat redundant by the broad range of chemical entities now available and by the absence of any clear structure–activity relationships for newer drugs. The chemistry of some older drugs does relate to their propensity to cause movement disorders.
摘自： Taylor D M, Barnes T R E, Young A H. The Maudsley® Prescribing Guidelines in Psychiatry [M]. 14th ed. Hoboken, US: John Wiley & Sons, 2021: 3.
另一医学著作的普通英语片段
Hans Asperger used to love telling the story of his life; thus all I have to do is retell it. He was born in Vienna in 1906. His grandfather’s family had been farmers east of the capital of the Austro-Hungarian Monarchy for many generations. As a high school student, he became acquainted with the “German Youth Movement.” It was in this movement that this achievement-oriented and intellectual young man was to find all those things he valued most throughout his lifetime. There he discovered friendship, mountaineering, nature, art as a source of strength and repose, and literature—the medium in which he moved and lived.
摘自： McPartland J C, Klin A, Volkmar F R. Asperger syndrome: assessing and treating high-functioning autism spectrum disorders [M]. 2nd ed. New York, US: The Guilford Press, 2014: Foreword, x.
