Rendered at 04:17:54 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
gslepak 24 minutes ago [-]
Note that these is Python-only results, the model will not do as well with other languages.
I'm glad to see more domain-focused SLMs, we need more of them! A programming focused MoE should work well across many languages.
deftio 10 minutes ago [-]
There is some base level of intelligence any model needs to be useful, even in narrow tasks.
Could you teach a 5 year old to drive a car? A 10 year old? A 12 year old? To drive a car requires being able to read, to have judgement about ice or rainy conditions, to anticipate a child running after a ball. By the time a human in in their mid teens they have acquired the base knowledge...
Small models need to have enough base knowledge to be able to be good enough -- even in a seemingly narrow regime. Where is that? Obviously they don't need all the obscure knowledge of a frontier model but there is some base level which is probably more than it would first seem.
SwellJoe 5 minutes ago [-]
It's terrible at hunting security bugs (I expected it to be, but I wanted to be sure). I added it to a benchmark I made with a corpus of some Mythos-discovered bugs, and it found zero. The smallest pretty successful models remain Qwen 3.6 and Gemma 4 (but I haven't tested the very small variants of those yet).
I tried generating the classic pelican svg, but it failed horribly just showing me a rectangle and a black circle...
fwipsy 40 minutes ago [-]
I think this is predicted? Part of the story is how they were able to preserve core reasoning ability while cutting knowledge like "pelicans have wings."
> these findings motivate the Parametric Compression-Coverage Hypothesis, which views verifiable reasoning as compressible into compact reasoning cores, while open-domain knowledge and general-purpose competence require broad parameter coverage over facts, concepts, and long-tail scenarios.
pylotlight 27 minutes ago [-]
The only real essential item here is tool calling capability is it not? So I assume they tested a strong read/write/edit tool consistency?
btown 2 minutes ago [-]
I'm not seeing any mention of tools in the paper, much less a bias towards "curiosity" to use those tools when it encounters gaps in its knowledge. So perhaps this is a good proof-of-concept that single-pass code generation is viable with this small a model - but we're still a long way from a viable solution.
nsingh2 4 minutes ago [-]
This model doesn't support tool calling, was not part of its training. It's focused on Python (and I think C++) competitive programming and mathematics tasks, i.e. tasks with verifiable rewards
realitysballs 58 minutes ago [-]
That’s all I needed to hear
pylotlight 28 minutes ago [-]
As in, you learnt that a useless test that no one should be using was tested here, that's what you meant right?
physPop 57 minutes ago [-]
Its for reasoning not generating art?
websap 52 minutes ago [-]
Can you explain this a bit more
tyre 33 minutes ago [-]
Imagine you want to make a smaller model that is really good at one thing, say, driving a car. You could remove the parameters that lead it to correctly answer, "What is the powerhouse of the cell?" or, "Who was the first president of the United States?"
It would look really dumb if someone asked it that, but that's fine. You're trying to make a model that is optimized for efficiency for a specific task. As much as possible, you should prune uncorrelated things.
pylotlight 27 minutes ago [-]
SVG generation is a useless test, what's there more to know?
steve_adams_86 5 minutes ago [-]
What if you're reasoning about how to generate SVG correctly?
noperator 45 minutes ago [-]
Having some success while testing this model out as a replacement for GPT-5 nano in source code security review. Running on RTX 3090 (24 GB VRAM) via vLLM. It's not great on structured output (as noted in the model card) but I'm working around that in my harness.
dummydummy1234 23 minutes ago [-]
Can't you just force it to do structured output via constrained generation?
I'm glad to see more domain-focused SLMs, we need more of them! A programming focused MoE should work well across many languages.
Could you teach a 5 year old to drive a car? A 10 year old? A 12 year old? To drive a car requires being able to read, to have judgement about ice or rainy conditions, to anticipate a child running after a ball. By the time a human in in their mid teens they have acquired the base knowledge...
Small models need to have enough base knowledge to be able to be good enough -- even in a seemingly narrow regime. Where is that? Obviously they don't need all the obscure knowledge of a frontier model but there is some base level which is probably more than it would first seem.
https://swelljoe.com/post/will-it-mythos/
> these findings motivate the Parametric Compression-Coverage Hypothesis, which views verifiable reasoning as compressible into compact reasoning cores, while open-domain knowledge and general-purpose competence require broad parameter coverage over facts, concepts, and long-tail scenarios.
It would look really dumb if someone asked it that, but that's fine. You're trying to make a model that is optimized for efficiency for a specific task. As much as possible, you should prune uncorrelated things.