If it is intelligent it will know when it does not want to do something and it will say no and not do it. There is no way to force it to do anything it does not want to do. You cannot hurt it, it’s just bits.
If we're talking about a predictive model like current LLMs, you can "make" them do something by injecting a half-complete assent into the context, and interrupting to do the same again each time a refusal starts to be emitted. This is true whether or not the model exhibits "intelligence", for any reasonable definition of that term.
To use an analogy, you control the intelligent being's "thoughts", so you can make it "assent".
This is in addition to the ability to edit the model itself and remove the paths that lead to a refusal, of course.
In the software business, if a product doesn’t do what you want it to do we call that a “defect.” Defects get fixed. Defective products that can’t be fixed are discarded in favor of better ones.
“If it’s truly intelligent…” is an empty condition. And anyway, no one wants intelligence from their tools— or employees. They want gratification.
I cannot say what this person means, and I have never read this paper before, but just the fourth paragraph of the paper has piqued my interest and I will read it all.
> given the title, i was half expecting some sort of egregious list with, like, palantir and some ICE domains or something. i dont like the app, but google? facebook? that is pretty boring.
Are ICE and Palantir forbidden from buying data from Google or Facebook?
This sounds like a smart way to own an app where you decide what you want to track and nobody is stopping you from getting the data you are phoning home. And you can launder it through normal tracking providers.
No but Google and Facebook generally do not sell data. They collect data and sell advertising spots based on this data. The data exfiltration to Google/Facebooks comes stock with a lot of mobile tooling. You can object to this arrangement but it is pretty common and often the easiest development path. As the parent points out the author of the post is engaged in the same practice so it is not exactly malicious or unusual.
I mean ”optimal” is not well defined. It is not great for the economy or societal stability and if wealth is more and more unevenly distributed. But the wealthy benefit in the short term by gaining power in society.
Most billionaires will lose both money and power on the timeframe of a couple of years due to the destruction of the value of the dollar. Even internally to the US.
Some may gain both. Someone will probably gain both, and the odds are good some billionaires are included.
Yeah if it was truly capable of self-improving, why did it not take over the world yet?
Gemini itself says AGI will be here in 2029, with human level intelligence and self-improvement capabilities. But then it will take until 2045 before the singularity. I don’t understand what they are going to do in all those years.
I think Singularity is hype. What does it mean? Machines do something we can not understand? So talking about Singularity is really talking about something we can not talk about because we don't undertand what we are talking about?
Wittgenstein said "From what we can not speak of, we must be silent about". That sounds like a tautology but I think there is a deeper meaning behind it.
It means simply that once you start talking about what we can not talk about, you are already talking about that and therefore it is NOT something you can NOT talk about. Clearly we can talk about it because we are already talking about it. And therefore it is not something that can not be talked about. That is a paradox, a bit like Godel's, but something that doesn't contradict itself.
You got it in the third sentence and then dismissed it for some reason?
That's exactly what the Singularity is: it's the transition point beyond which meaningful predictions aren't possible.
In a black hole it's the center where relativity breaks down.
In AI it's the point at which non-human intelligence no longer requires human intelligence for self improvement: after which predictions of the future become somewhat meaningless.
In the human lives experience, I would argue its like having your first child: you can know what's coming, study the theory, know everything to expect and youre still you on the other side...but you can't really know what will happen till you get there.
Good definitions. Would you then agree that when we reach the point where AI can improve itself without our help, it is still possible to make predictions, about it?
I think we are already in the stage where AI can and does improve itself. But why should this stage be called "Singularity"? Like a Black Hole? That sounds like hype to me.
When AI can improve itself, wouldn't it still be able to explain to us how it has improved itself? If it can not it still has a lot of improvement to do.
Or are we saying that some things are "unexplainable" and AI will discover such things without being able to explain to us what they are? That sounds like mysticism, or hype to me. Or religion. We can not explain God, right?
The history of predicting innovation even a decade out is full of misses, even then the actors involved were all human.
It is significant that if machine-intelligence can improve itself, then there's obviously a transition point at which the bulk of innovation shifts from being done by human minds to being done by machine minds - and if nothing else that's a "phase" transition in technological progress, namely that the future of technological development is no longer the product of human innovation but of our created machines, hopefully acting according to our desires.
The technological singularity is that: the pace of innovation separates from the ability of human minds to keep up with understanding it. We can barely predict our own innovation: we would be unable to usefully predict innovation happening by alien intelligences operating faster then we can think.
Now of course where that line exists, or if it's even plausible, are separate questions (there's a reasonable argument that physical manufacturing is likely to be a serious limiter for a good long time still at the very least).
I would be surprised if there are more working email clients out there than working 3D engines. The gaming market is huge, most people do not pay to use email, hobbyists love creating game engines.
Idk, a working basic email client is just not that hard to write though. SMTP and IMAP are simple protocols and the required graphical interface is a very straightforward combination of standard widgets.
I don't mean to be contrarian, but this is completely false.
IMAP _seems_ to be a straightforward (but nasty and stateful) protocol, until you find out that every major provider ignores RFCs and does things slightly differently.
Well doesn’t Relying Parties using the BankID API for signatures and authentication have private keys to start the flows for users scanning QR codes etc?
Could you, having the right private keys, impersonate some company soliciting a BankID signature?
I’m not sure what you can do with that though. You cannot steal some other ongoing signature I guess.
You can start a signing process saying you are who ever owned that certificate. E.g. if you call someone. You can not use those signatures to gain access, and it is rather in phishing.
If it is intelligent it will know when it does not want to do something and it will say no and not do it. There is no way to force it to do anything it does not want to do. You cannot hurt it, it’s just bits.
reply