Update README.md
Browse files
README.md
CHANGED
|
@@ -13,13 +13,14 @@ language:
|
|
| 13 |
|
| 14 |
## I miss model XXX
|
| 15 |
|
| 16 |
-
I am not the only one to make quants. For example, Lewdiculous makes high-quality imatrix quants of many
|
| 17 |
small models *and has a great presentation*. I either don't bother with imatrix quants for small models (< 30B), or avoid them
|
| 18 |
because I saw others already did them, avoiding double work.
|
| 19 |
|
| 20 |
-
Other notable people which do quants are Nexesenex, bartowski, dranger003 and Artefact2. I'm not saying
|
| 21 |
-
anything about the quality, because I probably forgot some really good folks in this list, and I wouldn't
|
| 22 |
-
even know, anyways
|
|
|
|
| 23 |
even if the creator might provide far fewer quants than me.
|
| 24 |
|
| 25 |
As always, feel free to request a quant, even if somebody else already did one, or request an imatrix version
|
|
@@ -69,7 +70,7 @@ not generate imatrix quants for models primarily meant for non-english usage. Th
|
|
| 69 |
english over other languages. But from (sparse) testing data it looks as if this doesn't actually make a big
|
| 70 |
difference. More data are always welcome.
|
| 71 |
|
| 72 |
-
Unfortunately, I do not have the
|
| 73 |
equivalent set in the future and publish set.
|
| 74 |
|
| 75 |
## Why are you doing this?
|
|
@@ -78,7 +79,7 @@ Because at some point, I found that some new interesting models weren't availabl
|
|
| 78 |
source, TheBloke, had vanished. So I quantized a few models for myself. At the time, it was trivial - no imatrix,
|
| 79 |
only a few quant types, all them very fast to generate.
|
| 80 |
|
| 81 |
-
I then looked into huggingface more closely than just as
|
| 82 |
good thing, so others don't have to redo the work on their own. I'm used to sharing most of the things I make
|
| 83 |
(mostly in free software), so it felt naturally to contribute, even at a minor scale.
|
| 84 |
|
|
|
|
| 13 |
|
| 14 |
## I miss model XXX
|
| 15 |
|
| 16 |
+
I am not the only one to make quants. For example, **Lewdiculous** makes high-quality imatrix quants of many
|
| 17 |
small models *and has a great presentation*. I either don't bother with imatrix quants for small models (< 30B), or avoid them
|
| 18 |
because I saw others already did them, avoiding double work.
|
| 19 |
|
| 20 |
+
Other notable people which do quants are **Nexesenex**, **bartowski**, **dranger003** and **Artefact2**. I'm not saying
|
| 21 |
+
anything about the quality of their quants, because I probably forgot some really good folks in this list, and I wouldn't
|
| 22 |
+
even know, anyways (bartowski tends to not generate imatrix quants, but is generally very quick).
|
| 23 |
+
Model creators also often provide their own quants. I sometimes skip models because of that,
|
| 24 |
even if the creator might provide far fewer quants than me.
|
| 25 |
|
| 26 |
As always, feel free to request a quant, even if somebody else already did one, or request an imatrix version
|
|
|
|
| 70 |
english over other languages. But from (sparse) testing data it looks as if this doesn't actually make a big
|
| 71 |
difference. More data are always welcome.
|
| 72 |
|
| 73 |
+
Unfortunately, I do not have the rights to publish the testing data, but I might be able to replicate an
|
| 74 |
equivalent set in the future and publish set.
|
| 75 |
|
| 76 |
## Why are you doing this?
|
|
|
|
| 79 |
source, TheBloke, had vanished. So I quantized a few models for myself. At the time, it was trivial - no imatrix,
|
| 80 |
only a few quant types, all them very fast to generate.
|
| 81 |
|
| 82 |
+
I then looked into huggingface more closely than just as a download source, and decided uploading would be a
|
| 83 |
good thing, so others don't have to redo the work on their own. I'm used to sharing most of the things I make
|
| 84 |
(mostly in free software), so it felt naturally to contribute, even at a minor scale.
|
| 85 |
|