Does this level of detail seem strange to anybody else? Shining such a strong light on OpenAI's moderation/manual review efforts seems like it would draw unwanted attention to the fact that ChatGPT conversations are anything but private, and seems somewhat at odds with their recent outrage about the subpoena for user chats in the NYT case.
Manual reviews of sensitive data are ok as long as their own employees are the reviewers, I suppose?
I thought that was pretty open? Even their more privacy-oriented Zero Data Retention agreement (which isn’t so easy to get on your business account) includes an exception “where needed to comply with law or combat misuse”
that creepy feeling of "being watched" has mostly kept me from taking advantage of any SOTA models, i only dabble in a few local ones.
The level of detail does not seem surprising. they're both charged with maintaining a facade of privacy while eliminating any and all miss-use. Certainly they heavily analyze basically everything given to them.
And generally as a society we've been ok with basically zero privacy as long as the data we send stays inside the company we sent it too. Google reads all your emails? Sure thing, read away, just don't send them to the popo. Apple knows when you're ovulating? no problem, just don't tell Amazon. etc
Get list of your inputs mixed with generated ones and ask some model to tell you which ones are yours.
Other than that the approach in general is weak, most people likely generate lots of noise themselves. It's just about that one time you asked about X.
I was in Shanghai recently and while casually testing one of their AI chat bots I typed "What do you think of the situation in Taiwan?".
It started discussing like a Western bot would - "it's complicated, etc. etc." and around 5s it abruptly stopped and regurgitated the same line the CCP uses "... it's an unalienable part of China etc. etc.".
After printing the line, a popup opened and my camera was activated. The app wanted me to submit my information, presumably to decide what to do with me next time I enter China.
1) All the lights and modern buildings cannot hide that China is a creepy authoritarian state underneath.
2) Given the bot started printing the Western consensus first, I bet $10 it was trained by distilling ChatGPT or Gemini.
> After printing the line, a popup opened and my camera was activated. The app wanted me to submit my information, presumably to decide what to do with me next time I enter China.
Was this on your personal device? I'm just wondering how it activated your camera. I would love more details!
An increasing use of AI is to gather user feedback. The Chatbot UI detected an error state, and then loaded a feedback vendor, who then popped the camera open for their interactive feedback session
I've run into this a few times, now.
So what OP is saying is plausible, I just don't appreciate their added and probably incorrect conclusion that it's because the government of China wants to do something to them
What are you talking about? Why are you using imprecise language like "popped the camera open?"
You've run into a site you view on chrome/firefox/safari accessing your camera without granting access a few times now?
Can you give us an example of a site that does this so we can reproduce? Or could you retract your statement and clarify that you did grant camera permissions for that site previously?
Otherwise, you're saying very casually there's a huge bug and security issue that no one else has detected but you personally have seem multiple times.
I've run into people on the internet misremembering things or not understanding how the browser works more times than I've run into browsers allowing access to system devices like the camera without a permission prompt.
> Or could you retract your statement and clarify that you did grant camera permissions for that site previously?
I never said anything about granting permissions. I can respond to your other points, in turn, but first I would like you to confirm that I am who you think you are responding to :) I am not OP.
In case you don't think I'm OP, then, well I was being imprecise. Yes, it requires browser/app/manifest permissions. Your paranoid and aggressive tone implies you're not giving me any benefit of the doubt, as I speak informally in a casual web forum discussion about understanding what happened.
I'd suspect rather than interactive feedback, it might have been trying to let him log in with a QR code. "A popup opened and wanted me to submit my information" sounds like a login/registration form.
If this were true, why didn’t the chatbot immediately recognize that the word “Taiwan” should trigger the response? Detecting the word “Taiwan” has been possible since before most of us were born.
China has more restrictions on what you can say than the U.S. but what you are describing is not reality. Some westerner asking Deepseek about Taiwan is completely uninteresting. Just as the government do not chase people over VPN usage.
China doesn’t try to hide that they are an authoritarian state. They don’t need to. Most people in China are no less happy with their government than westerners are with their governments. Governments reflect culture. And as for foreigners, our view of China is far worse than it actually is, China doesn’t need to hide anything, people who visit China will come away with a more positive view of the country than those who do not visit.
> If this were true, why didn’t the chatbot immediately recognize that the word “Taiwan” should trigger the response?
Not recognizing they were outputting wrongthink until after it was being streamed to the user is a known behavior with some Chinese chatbot apps. A quick search found an example of DeepSeek doing it: https://www.reddit.com/r/OpenAI/comments/1ic3kl6/deepseek_ce...
I don't think his story is genuine, but it showing the "wrong" answer before correcting itself is known behavior.
I've seen it from the non-Chinese ChatGPT before. Something was deemed to be violating the sensitivity filters or something, and it refused to answer. But only after I saw part of the real answer streamed to the output, and then redacted and replaced.
My wife grew up in Shanghai, and you'll have to go quite some distance to find someone more critical of the PRC and CCP than she is. And it's with good reason.
She grew up during the cultural revolution, and was largely raised by her grandmother because literally every other person in her extended family was in prison or work camp, not because of anything they had actually done wrong, but for political reasons because the whole family was blacklisted.
And that's not just the old days. Her father died as a direct result of Chinese Covid policy. During the pandemic her cousins still in the country would ask her (on Skype) "is X true?", and largely their perception of what was going on was false. She would exfiltrate encrypted news reports to them - until those started getting blocked. Her dad's estate still has affairs that need to be resolved, but we've decided not to return to China until Xi is gone, as it's just not safe. It doesn't get much airplay, but there are currently a couple of hundred Americans who are being illegally detained in China right now. It's not worth the risk.
My first trip to China was about 30 years ago, shortly after we got married. And back then, I would have said that you were right. Honestly, it felt like for the average person in their day-to-day-lives, the Chinese were less under the governmental thumb than we are. People from the countryside would bring their produce into the city to sell, or cook dumplings and buns to sell on the side of the street - stuff that in America we'd have to get permits for. It seemed that the oligarchy had an understanding with the people: let us control the big picture, and we'll look the other way for the little things. But Chinese politics is a pendulum swinging very widely. From Tienanmen Square and Tank Man, it had swung quite a bit the other way. But today, it's come back 180-degrees. Xi is really trying for a Cultural Revolution 2.0.
These impressions largely match what I hear from other Chinese immigrants - except for Party members, who tend not to want to talk about it at all. I'm afraid that you've been listening to too much propaganda.
i don't doubt your experience, but just know it might be skewed and not representative of everyone's opinions
the sense i get from my chinese friends are that the CCP is an annoying parent but they understand the challenges both domestic and international and largely agree with the compromises
How do they feel about and respond when asked about the Taiwan question?
Do they either clam up or act like it's a mortal insult to suggest that an independent democratic nation should not live in fear of impending violent conquest?
Because that's the kind of reaction that makes the reports of "happy life, all's good" a little harder to digest.
Not saying that's a unanimous opinion / response, of course. But it certainly seems to be the default.
The majority of US support for Taiwan and it's current situation is owed entirely to supporting a military junta from the mainland that massacred the local Taiwainese who objected to it and suppressed civil society.
Are you saying you would've been neutral on an invasion of Taiwan before 1985 or so, since it wasn't a democracy?
“there are currently a couple of hundred Americans who are being illegally detained in China right now”
Compared to the U.S. which currently has no foreign nationals detained illegally?
Pick any country and you will find political dissidents. The existence of angry emigrants is not evidence that a country is worse than we could ever imagine.
The fact that the USA and others are also trending authoritarian isn't really relevant. The point I was trying to make is that people have legit fears of the PRC government, enough so that legitimate business like settling a deceased parent's affairs isn't sufficient to convince people to enter the country.
You haven't addressed at all the parts about blacklisting whole families for political reasons, or horrible return-to-normal policies for covid-19 three years ago, or the general pendulum-swing-back-to-evil trend.
I don't doubt you, but what if someone's else's wife felt differently. Would that counteract your wife? Or is your wife special in an objective sense and her intuitions about hypotheticals are more valid than anyone else's?
Your wife feels a certain way and wanted to avoid a certain hypothetical. But since it didn't happen, we have no way of knowing how relevant these feelings are.
How can we address blacklisting and covid response if you are insisting that any comparison isn't relevant and that we should evaluate it with no baseline?
Sheesh, an actual Whataboutism. The fact that "the US does it too!" won't help Grandparent poster/his wife if they get detained in China. GP says "there are currently a couple of hundred Americans who are being illegally detained in China right now", most likely they are dual citizens, or were born in China, and from China's point of view, one can't lose the Chinese citizenship, and they're detaining their own citizens.
I would also like to know if these are dual citizens or not. I think it would be newsworthy if hundreds of US passport holders who do not have chinese passports also were being held in China and not charged with any crime and unable to access consular services.
Sensationalizing claims then qualifying them later is inherently dishonest.
>And as for foreigners, our view of China is far worse than it actually is, China doesn’t need to hide anything, people who visit China will come away with a more positive view of the country than those who do not visit.
To the extent that's true, it's because they won't let you see the uyghur reeducation camps.
We can get videos from remote hellholes of Africa like Dafur and Mali but apparently,that's too much to ask in Xinjiang.We can't even get satellite images to show us evidence of this so called wigur genocide
The Xinjiang Police Files: A 2022 leak of over 5,000 police photos, internal documents, and spreadsheets revealing the scale of detention, with images showing prisoners shackled, hooded, and under guard in 2018.
The China Cables (2019): Leaked, classified instructions on how to run the camps, including directives to ensure "no escapes" promote "repentance" and use full video surveillance.
Satellite Imagery Analysis: Researchers from the Australian Strategic Policy Institute (ASPI) identified over 380 suspected detention sites, including new construction and expansion, often featuring guard towers and razor wire.
Testimonies and Research: Former detainees have reported torture, rape, forced sterilization, and intense indoctrination to abandon their religious and cultural practices.
Government Documentation: The Karakax list, a leaked document, provided detailed, case-by-case justifications for detention, such as having too many children or wearing a veil.
Are you this incredulous when someone reports that the US locks up more Black people capita than White?
Someone defending the US could make the same claims you are that everyone is out to make the US look bad. That multiple independent groups are fabricating evidence etc…
1. give links or one link to the collection of above "evidence" to let others to get conclusion by their own. BTW, I've seen some ("Leaked, classified instructions...) but easily get different interpretation.
2. Also using "I" is better than "We". That means you get your conclusion, not representing others.
On the other hand you can travel to Xinjiang, visit mosques, Uighur museums, experience Uighur culture, observe Uighurs just minding their own business in their daily life.
“Subjected to arbitrary arrests and forced labor, sterilizations to torture, more than one million Uyghurs, Kazakhs, Kyrgyz, and other minorities are estimated to have been locked up in so-called “re-education” camps and prisons in the region over the last decade, according to the UN Office of the High Commissioner for Human Rights.”
UN High Commissioner on Human Rights Michelle Bachelet actually visited Xinjiang and made no such assertions. Whoever did release the report you're referencing, they waited until immediately after her term ended to release it (within hours). Pretty conspicuous.
No it was actually released hours before her term ended not after. And the reason she held off releasing until the last minute is because of pressure from China to refrain from releasing it.
In addition to releasing the report she released a 131 page Chinese rebuttal simultaneously. Not the actions one would expect of a shadowy group at the UN out to get China.
“Bachelet’s damning report was published with only 11 minutes to go before her term came to an end at midnight Geneva time. Publication was delayed by the eleventh-hour delivery of an official Chinese response that contained names and pictures of individuals that had to be blacked out by the UN commissioner’s office for privacy and safety reasons.”
The organization’s human rights office delivered its much-delayed report minutes before Michelle Bachelet, the U.N. high commissioner for human rights, was to leave office.
I agree it was reported this way, but do we have evidence it was actually prepared and published by her at that time? The report conspicuously does not mention the viewpoint and concerns of the High Commissioner as other OHCHR reports do nor does it reference Bachelet's findings from her May 2021 visit to Xinjiang.
DeepSeek would print all it's mental gymnastics to censor itself in the reasoning phase directly to the user, before shutting down the conversation. Apparantly such an odd move is a thing in China.
I think one of the reasons why AI companies are valued this high is you can actually inspect what user inputs & outputs are.
It's basically an OSINT siphon.
In this Chinese case, the tokens are leaked at least twice since ChatGPT offer no direct access, you have to use some kind of Openrouter-like service, and they are also have to be in clear-text during transmission.
I'm pretty sure they can just prompt any convo in the background and ask "is this conversation sensitive ?" and the model can answer without this being added to the context of the convo.
One hopes the CIA/Secret service would be willing to provide the human to do the reviewing but sadly I've worked for European telco's and I know better.
If I were doing this sort of thing, I would make certain to ban accounts that were too obvious while leaving ones that are subtle enough, so that the other side has less reason to suspect I am tracking their inputs and feeding them disinformation.
> While the US certainly isn't pure, its scale of surveillance intrusion is light compared to China
I assume that for someone to believe this, they either have to believe the U.S. has poorer surveillance capability than China, or, more likely, they consider U.S. surveillance unintrusive and Chinese surveillance intrusive.
> ... or, more likely, they consider U.S. surveillance unintrusive and Chinese surveillance intrusive.
Of course. What's the point of surveillance if you're not going to use it to enforce dogma? I think you can reasonably evaluate a country's surveillance by looking at the pettiness of the arrests & censorship they make.
See this chinese tech reviewer[1] being bullied by the government for putting a spotlight on chinese phone makers cheating about benchmarks. I'm not sure the US is at this point yet...
The amount of information about everything that people are giving OpenAI is astronomical, information that was previously kept closely guarded is now just freely flowing through foreign servers.
Truly a paradise for american intelligence. Would have expected that the chinese officials be briefed on not using us tech companies, but opsec is hard to teach, and even harder to always follow.
> “This is what Chinese modern transnational repression looks like,” Ben Nimmo, principal investigator at OpenAI, told reporters ahead of the report’s release. “It’s not just digital. It’s not just about trolling. It’s industrialized. [...]
There's something poetic about OpenAI being asked to comment on mis-use of their slop generator, and their answer is composed entirely of AI slop.
i kinda get the impression this was from 2023 and also it is not clear what this dissident did, hard to evaluate whether i should care without knowing that
> “It’s not just digital. It’s not just about trolling. It’s industrialized. It’s about trying to hit critics of the CCP [Chinese Communist Party] with everything, everywhere, all at once.”
Does this level of detail seem strange to anybody else? Shining such a strong light on OpenAI's moderation/manual review efforts seems like it would draw unwanted attention to the fact that ChatGPT conversations are anything but private, and seems somewhat at odds with their recent outrage about the subpoena for user chats in the NYT case.
Manual reviews of sensitive data are ok as long as their own employees are the reviewers, I suppose?
reply