It simply means the tokenizer's training corpus may have included a massive amount of German literature or accidentally oversampled a web page where that word was frequently repeated. Look up "glitch tokens" to learn more.
Yes, I agree that the value of Apple's chips is hard to beat, but there's still a massive bottleneck with hardware accessibility. The 128 GB MacBook described costs over $5,000 on their website, and in the consumer space, even the most often recommended GPU which is a 3090 with 24 GB VRAM, you can find used going for $700 at minimum. This effectively prices out all non-professional users who don't have money to spend to simply have parity with the $20 subscription on their phones. (and even for those who do, they have to cope with the fact that the model will always be dumber than ChatGPT, and that their hardware will grow outdated very quickly)
Also a noticeable disconnect between the hardware we have and the primary focus of open-source labs (scale up and cater to their enterprise customers, just look at GLM-5's increase to 744B parameters, double from GLM-4.5's 355B). We really just need some kind of Cambrian explosion in cheap hardware for local models to be feasible.
Curious if you agree that local is where Apple's bet is long term - it's out of reach now, but I found the jump in capability for the top line laptop interesting. Presumably chip development hasn't focused all that much on running LLM's for all that long, I'm wondering what kind of jump we'll see two or three releases down the line.
That's fine, it's pretty good slop and from the comments history even entertaining at times.
> my grandmother had a cookie jar collection and I always thought it was weird until I realized she was basically running a primitive NFT gallery except the tokens were actually useful because they contained cookies
I would recommend using the ropey crate for easy performance gains. A string buffer is quick to implement but you will hit a wall as soon as you need to edit large files.
It's not that bad. You need really large files to notice.
The largest realistic file I'll ever touch - sqlite3 amalgamation with 270k lines and 9.1 kB - still takes only 6 ms to memmove it on my poor laptop.
Any regular up-to 10k lines file is memmoved in order of microseconds.
Yes, absolutely. I've since switched to rope-backed buffers, but I don't think the rope itself is actually adding much from a performance standpoint, even for really very large files.
We talk about big-O complexity a lot when talking about things like this, but modern machines are scarily good at copying around enormous linear buffers of data. Shifting even hundreds of megabytes of text might not even be visible in your benchmark profiling, if done right.
When benchmarking, I discovered that the `to_pos`/`to_coord` functions, which translate between buffer byte positions and screen coordinates, were by far the heaviest operation. I could have solved that problem entirely simply by maintaining a list of line offsets and binary-searching through it.
That's true for code editing, but it's nice to not have to reach for a different solution when editing huge files. Sometimes I like to open up big log files, JSON test data, etc.
Unmentioned in the post, but I have since switched to a third-party rope library (crop, not ropey). At some point I'd like to implement one myself, but for now this does the job.
reply