I think the point they’re making though is that the numbers seem too good to be true.
ie. Does anyone know the payback time for a B100 used just for inference? I assume it’s more than a couple of months? Or is it just training that costs so much?
Like the point of being more intelligent than someone or something is to an extent being able to simulate their brain and thinking with your own brain.
We’re cleverer than animals because we can simulate all their actions before they do them.
You can’t simulate something more advanced than yourself.
The opposite conclusion would also be obvious. We're a social species that might have deep primitives for evaluating the intelligence of another without needing to simulate the whole shebang.
I don't see how that would follow. If we are talking about "intelligence" in the formal sense of induction/prediction then it is a profoundly memory-hard problem as a matter of theory. This is to "learning" what the speed of light is to physics.
You can't replace the larger simulation required (i.e. more state/RAM) with a faster processor.
Sticking with the computation analogy, it could be a long-term memory look up. If memories were passed down the generations, people could simply memorize actions of individuals deemed smarter. Over a large sample size, a heuristic would emerge. Kind of like knowing there is always a sunset following a sunrise without understanding the solar system.
It is a zero sum game because you have a finite state budget for representing heuristics. Increasing the "smartness" (and therefore state required) of one heuristic necessarily requires reducing the smartness of other heuristics. The state is never not fully allocated, the best you can do is reallocate it.
This places an upper bound on the complexity of the patterns you can learn. At the limit you could spend 100% of resources building a maximally accurate model of a single thing but there are limits to ROI. Pre-digested learning makes it more efficient to acquire heuristics but it doesn't change the cost of representing it.
Some simple state machines are resistant to induction by design e.g. encryption algorithms.
I think that's kind of how all the religions were started. Smart people being tired of reasoning with dumb ones and instead going with "do this, because that's the will of God".
lm studio offers an Anthropic compatible local endpoint, so you can point Claude code at it and it'll use your local model for it's requests, however, I've had a lot of problems with LM Studio and Claude code losing it's place. It'll think for awhile, come up with a plan, start to do it and then just halt in the middle. I'll ask it to continue and it'll do a small change and get stuck again.
Using ollama's api doesn't have the same issue, so I've stuck to using ollama for local development work.
Claude Code is fairly notoriously token inefficient as far as coding agent/harnesses go (i come from aider pre-CC). It's only viable because the Max subscriptions give you approximately unlimited token budget, which resets in a few hours even if you hit the limit. But this also only works because cloud models have massive token windows (1M tokens on opus right now) which is a bit difficult to make happen locally with the VRAM needed.
And if you somehow managed to open up a big enough VRAM playground, the open weights models are not quite as good at wrangling such large context windows (even opus is hardly capable) without basically getting confused about what they were doing before they finish parsing it.
If you want to experiment with same-harness-different-models Opencode is classically the one to use. After their recent kerfluffle with Anthropic you'll have to use API pricing for opus/sonnet/haiku which makes it kind of a non-starter, but it lets you swap out any number of cloud or local models using e.g. ollama or z.ai or whatever backend provider you like.
I'd rate their coding agent harness as slightly to significantly less capable than claude code, but it also plays better with alternate models.
I don't get why I would use Claude Code when OpenCode, Cursor, Zed, etc. all exist, are "free" and work with virtually any llm. Seems like a weird use case unless I'm missing something.
previously I have found claude code to be just better than the alternatives, using large models or local. It is, however, closer now and not much excuse for the competition after the claude code leak. Personally, I will be giving this a go with OpenCode.
> I don't get why I would use Claude Code when OpenCode, Cursor, Zed, etc. all exist, are "free" and work with virtually any llm. Seems like a weird use case unless I'm missing something.
I'm with you on this. I've tried Gemma and Claude code and it's not good. Forgets it can use bash!
However, Gemma running locally with Pi as the harness is a beast.
They read the Three Body Problem but forgot that light exists. For aliens with interferometers looking at Earth there's little question there's some sort of interesting active chemistry (life) here.
Theres no hiding that fact. If they're within about 100 light years they'll be watching the effects of the Industrial Revolution on the atmosphere. Even if they're don't know the exact cause the spectra of pollutants and rates of change will give hints the changes are unlikely to be from random natural processes.
Outside of 100 light years but pretty much anywhere in the galaxy (assuming interferometers capable of getting spectra of Earth) will know there's some sort of life here. Even if you want to assume some aliens don't recognize life as we understand it they'll at least see extremely interesting and varied chemistry.
The idea you're going to hide Earth's biosignatures is silly. Trying to hide our technology signatures is pointless. At about 70 light years any interested aliens will start seeing isotopes resulting from above ground nuclear testing.
Telescopes aren't magic, and space is big. There are 100 billion+ stars in the galaxy. Within a 100 light-year radius, there are 27 ( https://en.wikipedia.org/wiki/List_of_star_systems_within_95... ). Nobody's looking at Earth. If any hypothetical civilization were to find our system, it would be by blanketing the entire galaxy in 100 billion drones and checking every single star, in which case the dark forest doesn't matter anyway.
First that's just star systems within 100 lightyears of Earth, systems with one of more gravitationally bound stars in them. There are thousands of stars within 100 light years of Earth. Most are red dwarfs but there's about a thousand F, G, and K class stars.[0]
While telescopes indeed are not magic, an alien species at least as advanced as us could have telescopes capable of not only finding Earth but gathering spectra from it. It's certainly no guarantee Earth would be found but there's no hiding from anyone looking. There's no masking the chemistry of life on Earth and likewise no masting techno-signatures in the atmosphere.
If they are at our current tech level, to "see" Earth, then Earth would need to pass in front of the Sun from their point of view. That means they would need to be somewhere in the same pane as the Earth's orbit.
That's a transiting detection, there's other detection methods for exoplanets. Even a coarse grained survey with a ground based traditional telescope can find our solar system thanks to Jupiter's gravitational influence on the Sun. Doppler shift's in the Sun's spectra come from Jupiter tugging at it gravitationally. With interferometry and coronagraphy spectra of planets in our system can be gathered without needing to see our system edge-on. Then of course for aliens on the ecliptic there's transiting spectra of Earth.
The number of techniques for detecting exoplanets makes the Dark Forest concept silly. There's no hiding our solar system from alien observation. For dedicated observers (at the right distances) there's no hiding the existence of life, the Industrial Revolution, or above ground nuclear testing.
ie. Does anyone know the payback time for a B100 used just for inference? I assume it’s more than a couple of months? Or is it just training that costs so much?
reply