As a longtime F# developer and longtime recipient of STEM academic bullying[1] I refuse to use LLMs in large part because ChatGPT-3.5 was so ridiculously bad and obvious about copy-pasting from F# GitHub repos. I never felt the AGI, I just saw a plagiarism machine whose decorations had fallen off.
Eventually I am sure someone at Microsoft noticed and rang the RLHF alarm, so GPT improved substantially. It seems pretty usable for F#. I am sure some unprincipled F#er is crushing it with agents these days. But I didn't think "oh boy they solved the plagiarism problem, let's go generate some slop!" I thought "oh great, now it's no longer going to be blatantly obvious when ChatGPT plagiarizes." I really don't want to roll a d100, or even a d1000, to completely compromise a core value of mine in in exchange for a productivity benefit. I'll just be slow and jobless, thanks. This is serious: I am getting into solar installations and junk hauling.
[1] The "students don't want to think" problem is much older than LLMs. In 2007 I took a senior-level PDEs class, and almost everyone copied my homework because I was actually motivated to study PDEs, and too psychologically weak to resist those mean lazy math majors. Then it happened again in math grad school! Actually unbelievable. Why are you even in the program?
Intelligence is certainly not compression. People need to think more carefully about how it is that cockroaches and house spiders are able to live comfortably and adaptably in human houses, which are totally novel environments that have only existed for at most 10,000 years. Does it really make sense to say that they decompressed some latent knowledge about attics and pantries, perhaps from a civilized species of dinosaur? I think they have some tiny spark of true general intelligence that lets them adapt to situations vastly outside the scope of their "training data."
I would be much more convinced about AGI 2027 if someone in 2026 demonstrates one (1) robot which is plausibly as intelligent as a cockroach. I genuinely don't think any of us will live to see that happen.
I doubt you would ever blurt out a copyrightable portion of a book without realizing that's what you're doing. That's the biggest difference.
In particular, you are a legal person who can be sued in civil court if you infringe on copyright. If I ask you "can you help me write a blog about Manhattan?" and you plagiarize the New York Times, then the NYT sues me for copyright infringement, then I would correctly assume you conned me, and you are responsible for the infringement, and I would vindictively drag you into the lawsuit with me. With LLMs it involves dragging in a corporation, much much uglier. Claude is not actually a person and cannot testify in any legally legitimate trial. (I am sure it will happen soon in some kangaroo court.)
Yes, we've known the line is blurry for hundreds of years, that's why we have courts. That has nothing to do with the specific problem of LLMs infringing copyright. LLMs needs to be held to much higher scrutiny because they are not capable of taking legal responsibility for copyright infringement, regardless of whether its verbatim or a more ambiguous case, and their users can't be expected to know off-hand whether the output is copyrighted or not.
Based on this comment: https://news.ycombinator.com/item?id=47960014 it seems like you are just ignorant about the basics of copyright law, and pretending this ignorance is some sort of flaw in the idea of copyright itself.
>> Whereas earlier you had to use something that was mass produced to be satisfactory for everyone
As someone who recently started using OpenSCAD for a project I find this attitude quite irritating. You certainly did not "have to" use popular tools.
The OpenSCAD example is particularly illuminating because it's fussy and frustrating and clearly tuned towards a few specific maintainers; there's a ton of things I'd like changed. But I would never trust an LLM to do it! "Oh the output looks fine, cool" is not enough for a CAD program. "Oh, there are a lot of tests, cool" great, I have no idea what a thorough CAD test suite looks like. I would be a reckless idiot if I asked Claude to make me a custom SCAD program... unless I put in a counterproductive amount of work. So I'm fine with OpenSCAD.
I am also sincerely baffled as to how this stimulates the "labor economy." The most obvious objection is that Anthropic seems to be the only party here getting any form of economic benefit: the open-source maintainers are just plain screwed unless they compromise quality for productivity, and the LLM users are trading high-quality tooling built by people who understand the problem for shitty tooling built by a robot, in exchange for uncompensated labor. It only stimulates the "labor economy" in a Bizarro Keynesian sense, digging up glass bottles that someone forgot to put the money in.
I have seen at least 4 completely busted vibe-coded Rust SQLite clones in the last three months, happily used by people who think they don't need to worry their pretty little heads with routine matters like database design. It's a solved problem and Claude is on the case! In fact unlike those stooopid human SQLIte developers, Claude made it multithreaded! So fucking depressing.
This is funny because I was in the same situation, and actually used Claude to make a custom CAD program inspired by OpenSCAD :) https://fncad.github.io
You definitely need to have a strong sense of code design though. The AIs are not up to writing clean code at project scale on their own, yet.
This is a good example of what I mean! fnCAD appears to be a significantly buggier and highly incomplete version of OpenSCAD, where AI essentially grabbed the low-hanging fruit - albeit an impressively large amount of fruit - and left you with the hard parts. I fail to see how this solved any problems. Maybe it was an experiment, which is fine. But it's not even close to a viable CAD product, even by OpenSCAD's scruffy FOSS standards, and there's no feasible way to get it there without a ton of human work.
Not trying to denigrate the work here, as such. But this certainly didn't convince me that using AI to replace OpenSCAD (or any other major open-source project) is a good idea. The LLMs still aren't even close to being able to pull it off.
Anthropic will probably do what Google did in the 2000s, which is give jobs to all the open source developers whose work helped them get there.
Civilization isn't monotonic. People keep solving the same problems over and over again, telling the same stories with a different twist. For example in 1964 having a GUI work environment with a light pen as your mouse was a solved problem on IBM System/360. They had tools similar to CAD. So why don't we all just use that rather than make the same mistakes again. Each time a new way of doing things comes out, people get an opportunity to rewrite everything.
The most effective response is rarely to argue the general case. Instead, acknowledge the concern, offer a brief reframe, and propose one concrete demonstration on the person’s own code. Most concerns are resolved by a single successful experience.
First of all, Google didn't have to write this stuff about Kubernetes, suggesting psychological tricks and magic demonstrations to cajole people into agreeing with you. Kubernetes was happy to discuss the general case - I don't like k8s and don't think they had a bulletproof argument, but they offered a pretty good one. What Anthropic is doing here is very very weird. I said "Scientology" earlier and I was not kidding.
Part of the reason LLMs have led me to tear out so much of my own hair is how many people seem to have made it through four years of STEM college without developing any scientific thinking ability whatsoever. A truly stunning number of people have been wowed by "a single successful experience." Actually that section is full of horrible logic:
Concern
"I am faster without it."
Suggested response
That is likely true for code the person writes routinely. Suggest trying it on the work they tend to avoid: legacy files, unfamiliar services, or test scaffolding, where the leverage is highest.
Evidence to offer
Time one tedious task both ways and compare.
This isn't just unscientific and manipulative: it's really goddamn annoying! If someone times me at 1.5 hours reading about and learning an unfamiliar service, and smugly says Claude learned it in 12 seconds of "thinking," either my laptop or a certain Claude Champion is getting thrown out the window.
This Scientology-ass blog aligns startlingly well with my hypothesis that certain tech workers (including CEOs like Dario Amodei and Satya Nadella) are excessively enamored with LLMs because of a fundamental spiritual emptiness and ignorance.
Imagine calling yourself a "Champion" and dispensing nuggets of wisdom like this:
>> When a colleague asks how you accomplished something, the most useful response is the prompt you actually used. They will learn more from running that prompt against their own problem than from any description you could write, and it gives them something they can act on immediately.
Colleague: How did you get it to find that race condition?
Champion: I asked, "The test in @tests/scheduler.test.ts is flaky, figure out why," and it traced two unjoined promises in the scheduler. Try the same phrasing on your test.
People quickly became too embarrassed to call themselves "prompt engineers." I don't think anyone is jumping at the bit to be the office Claude Champion.
I am a musician and deeply morally opposed to any form of generative AI that has unauthorized training data. That means I am morally opposed to any useful system. This seems to be a majority opinion among creatives: they are plagiarism machines built on stolen data. Using them for anything is always unacceptable.
>> Notepad++ for macOS is maintained by Andrey Letov, who wrote the Objective-C++ Cocoa UI that replaces Notepad++'s Win32 front-end. The app is available to download from the Notepad++ website.
That is not the Notepad++ website! It's some other website. I understand that this is a fairly legitimate and professional port. But this framing is unacceptable. It's especially grating considering "Notepad++" is trademarked in France: https://data.inpi.fr/marques/FR5133202 [1]. The software is GPL but that doesn't mean you can slap the trademark on any derived codebase - legally problematic in France, but it's disrespectful worldwide. The Mac port really should have been released under a similar but clearly distinct name, and MacRumors should have been way more responsible about framing the story.
FWIW I also think an underappreciated advantage is Windows Server (last I checked that was still rock-solid) and Active Directory. Lots of CIOs / CTOs would correctly veto a move off of these, absent a specific technical problem. This is really more of a "hard knocks" lesson than anything fundamental to operating system design or implementation, but: the two Linux shops I worked at got at least a little sloppy about the sudoers list, or got frustrated and gave too much access to a "shared" folder, etc etc, largely because the admins got fed up with all the Mother May-I-ing. It just seemed to inevitably turn into a mess; sometimes that mess is fun and even productive, sometimes it's actually unacceptable.
Even the research hospital I worked at had a proper SELinux setup on the Red Hat installations, but by-quantity most servers were CentOS and it was way more of a free-for-all than it should have been, e.g. I was the fed-up admin when I was really not qualified! I screwed up a lot. Not that big of a deal: this was research-related computing and deidentified data. All the clinical computing was Windows Server. That is not a coincidence, it is really a market difference.
As someone who hates Windows 11... I do like the core Windows kernel, and would much rather do IT on Windows machines than Linux machines. Windows NT is very fussy and a bit bloated, but a huge part of that is an admirable commitment to backwards compatibility; a lot of XP applications run fine on Windows 11, except DPI wonkiness. And Windows' driers advantage isn't just commercial support; the kernel is fundamentally leaner and faster than Linux at real-time IO, and better about cleanly isolating driver processes across privilege levels. Very broadly, compared to Linux I find administering Windows easier to navigate and harder to screw up, especially with handling user permissions. Surely part of this is what I grew up with, but there's also a values difference: a lot of Linux users like how low-friction it can be since the OS doesn't get in your way. I kind of like that Windows makes you turn an excessive number of disarming keys... even when I am frustrated by it.
It does make me quite sad that the only real general-use OS options are the apex of a 20th-century operating system family, Apple's version of that, and a truly 21st-century monolith-microkernel hybrid whose specific design is a mystery to public science.
They're referring to the Windows kernel; see the preceding paragraph on the Windows kernel - the three general purpose OS families are Linux, macOS, Windows.
Personally I think not enough credit to macOS here; Apple's Mach/XNU has been microkernel flavored since the NeXT days and many subsystems run in userspace like Windows.
Last years Crowdstrike outage never hit any of the macOS computers with CS installed because on macOS the Crowdstrike agent runs entirely in userspace thanks to the Endpoint Security framework.
Really the security of macOS is probably the best of all of the desktop OSes, and as annoying as it can be.
"The" future of software engineering is a silly thing to predict. I might predict one substantial change is that we get our house a little more in order about universities and the private sector distinguishing between computer science, software engineering, and software development. Obviously they are not cleanly separated[1], but LLMs will affect each subfield very differently.
- The impact on computer science seems almost entirely negative so far: mostly the burden of academic wordslop, though an additional negative impact is AI sucking all the air out of the room. What's worse is how little interesting computer science has come out of the biggest technological development with computers in many years: in fact there has been a terrible and very sudden regression of scientific methodology and integrity, people rationalizing unscientific thinking and unprofessional behavior by pointing to economic success. I think it'll take decades to undo the damage, it's ideological.
- The impact on software development actually does seem a bit positive. I am not really a software developer at all. It always felt too frustrating :) However the easing of frustration might be offset by widespread devastation of new FOSS projects. I don't want to put my code online, even though I'm not monetizing it. I'm certainly not alone. That makes me really sad. But I watched ChatGPT copy-paste about 200 lines of F# straight from my own GitHub, without attribution. I'm not letting OpenAI steal my code again.
- Software engineering... it does not seem like any of these systems are actually capable of real software engineering, but we are also being adversely affected by an epidemic of unscientific thinking. Speaking of: I would like to see Mythos autonomously attempt a task as complex and serious as a C compiler. Opus 4.6 totally failed (even if popular coverage didn't portray it as such):
The resulting compiler has nearly reached the limits of Opus’s abilities. I tried (hard!) to fix several of the above limitations but wasn’t fully successful. New features and bugfixes frequently broke existing functionality.
"Future of software engineering" folks should stuff like this in mind. What model is going to undo Mythos's mess? What if that mess is your company's product? Hope you know some very patient humans!
[1] They should have different educational tracks. There is no reason why a big fancy school like MIT can't have computer scientists do something like SICP and software engineers do the applied Python class. Forcing every computer professional into "computer science" is just silly; half the students gripe about how useless this theory is, the other half gripe about how grubby the practice is. What really sucks here is that I think Big Tech would support the idea, we're just stuck in a weird social rut.
I feel like LLMs[1] are going to cause a kind of "divorce" between those who love making software and those who love selling software. It was difficult for these two groups to communicate and coordinate before, and now it is _excruciating_. What little mutual tolerance and slack there was, is practically gone.
Open source was always[2] a fragile arrangement based on the kind of trust that involves looking at things through one's fingers (turning a blind eye may be more idiomatic in English), and we are at the point where you just have to either shut your eyes, or otherwise stop pretending that the situation can be salvaged at all.
Just a thought I had: some people think that LLM-shaming is declasse, and maybe it is, but I think that perhaps we _should_ LLM-shame, until the AI-companies train their LLMs to actually give attribution, if nothing else (I mean if it can memorize entire blocks of code, why can't it memorize where it saw that code? Would this not, potentially, _improve_ the attribution-situation, to levels better than even the pre-LLM era? Oh right, because plagiarism might actually be the product).
[1]: Not blaming the tech itself, but rather the people who choose to use it recklessly, and an industry that is based almost entirely on getting mega-corporations to buy startups that, against the odds, have acquired a decent number of happy-ish customers, that can now be relentlessly locked-in and up-sold to.
Eventually I am sure someone at Microsoft noticed and rang the RLHF alarm, so GPT improved substantially. It seems pretty usable for F#. I am sure some unprincipled F#er is crushing it with agents these days. But I didn't think "oh boy they solved the plagiarism problem, let's go generate some slop!" I thought "oh great, now it's no longer going to be blatantly obvious when ChatGPT plagiarizes." I really don't want to roll a d100, or even a d1000, to completely compromise a core value of mine in in exchange for a productivity benefit. I'll just be slow and jobless, thanks. This is serious: I am getting into solar installations and junk hauling.
[1] The "students don't want to think" problem is much older than LLMs. In 2007 I took a senior-level PDEs class, and almost everyone copied my homework because I was actually motivated to study PDEs, and too psychologically weak to resist those mean lazy math majors. Then it happened again in math grad school! Actually unbelievable. Why are you even in the program?
reply