>an example of a judge taking a hyper-skeptic position on technology - namely, distrusting the zoom feature because it might add in extra information not present in the original image.
Some mocked the judge but here's an example where an AI upscaling algorithm added a human face that was never there:
EDIT: based on some replies I see, my cite isn't to claim that Apple iOS is using AI enhancement during zoom to add fake pixels. The point is to consider why the judge (who is not a computer programmer) is skeptical. Therefore, his request for expert testimony to rule out that the zoom function doesn't add false information is not that unreasonable.
In other words, what's obvious to a HN tech audience doesn't always apply to judges who don't have the same technical knowledge of how zoom algorithms actually work. So considering the judge's state of mind, the "fact" that iOS zoom doesn't add false incriminating pixels isn't a fact to his level of satisfaction unless the prosecutors bring in an expert to confirm it.
The issue is the expert who was brought it in was not an expert. They failed to answer even the simplest of questions related to upscaling and did nothing to inform the court of the nature of the upscaling used.
Anyone would be supportive of judges being cautious over upscaling, which we know has the potential to introduce false information because it's doing inference and interpolation.
As Twisol said, zooming doesn't inherently require interpolation. But when it does require interpolation, it's usually an extremely simple and well-understood form like nearest neighbour interpolation that's not going to result in things like faces being inserted.
It also only uses the available data in the image itself on which to work.
It's fundamentally different from neural network-driven inference and interpolation that attempts to fill in pixels using evidence from other images that the network has been trained on.
This is just pivoting. In this case zooming uses interpolation which "has the potential to introduce false information" exactly as you mentioned. It using only the available data doesn't mean that interpolation can't go wrong. So what if it's not AI?
And yes, it's so well understood that the prosecution had no idea of what's going on, claiming that it's "common sense" that it works just like a magnifying glass, which is simply wrong. Using bilinear (or bicubic) interpolation makes the edges of an image blurrier, especially when you're zooming in as much as they did on low-quality footage.
Come now, you're reaching. The defence attorney specifically claimed that "iPads, which are made by Apple, have artificial intelligence in them that allow things to be viewed through three dimensions and logarithms." and "[The iPad] uses artificial intelligence or their logarithms to create what they believe is happening. So this isn't actually enhanced video; this is Apple's iPad programming creating what it thinks is there, not what necessarily is there."
Aside from the confusion about logarithms, where I presume he meant algorithms, and the false claim about 3D, the defence was making a claim about artificial intelligence being used to upscale the footage. Which is inaccurate, and a specific claim being made that's unsupported by any evidence.
Funny enough, the same sort of question arose with regard to a crime scene image that was enlarged by the state's forensic lab, but a simple explanation of the mechanism used was deemed sufficient.
> Aside from the confusion about logarithms, where I presume he meant algorithms, and the false claim about 3D, the defence was making a claim about artificial intelligence being used to upscale the footage. Which is inaccurate, and a specific claim being made that's unsupported by any evidence.
With AI being used in ever more parts of technology, it is not surprising that a layperson wants to clarify if such technology is being applied here.
Apple has been advertising the usage of AI in dealing with photos ("Deep Fusion" of 2019 IIRC), so the suspicion is warranted.
Also, things like Deep Fusion are capture-time technologies, they don't alter the photo or video each time it's viewed. There's also already precedent in US courts for handling automatic enhancements applied to digital camera evidence.
I'm reaching? Like, why would the defense's wrong argument be relevant? You're not arguing with the defense here.
Your claim was that the judge's call is wrong because zooming doesn't actually do interpolation and has thus no potential to introduce wrong information. I've told you that zooming in this case does interpolate and that interpolation has the potential to go wrong.
When interpolation goes wrong it’s predictably wrong, which is not true of more complex algorithms using AI. If we can predict how it goes wrong we can extract the meaningful information from it.
It was a claim made by the defence, which the judge insisted that the prosecution prove the absence of, rather than requiring the defence to substantiate.
That's not how technical claims of that nature are normally handled.
The prosecution presented modified evidence, claiming that the modification doesn't change it, which the defense objected to. Why would the burden of proof not fall on the prosecution?
What does "built-in" have to do with anything? The entire point of the issue was that it wasn't just zoom in, it was enlargement (think upscaling). The software has to actively interpolate and add pixels where it thinks they're necessary in order to do so, and that changes the image. In most cases, it's probably not a big deal. But when it's from a nighttime camera of a person several hundred feet away that's only a few pixels in the original frame, there's a lot to go wrong there.
So if in 5 years the iPad really uses AI when zooming, it's perfectly fine because it's a built-in feature? What an absurd argument.
And both the prosecutions and defense's claims were fantastical, unsubstantiated (beyond some incorrect 'common sense' handwaving) and wrong - so since it's the prosecution that wants to enter new evidence, it's perfectly fine to ask them to back to it up. This isn't 'proving a negative', it's just asking them to prove that what they said (that zooming doesn't change the evidence).
This wasn't 'new evidence', it was an iPad being used to play a video for Rittenhouse during cross-examination, in which he was being asked to describe the events being shown.
Both prosecutors and defence attorneys have used the same mechanism countless times in previous cases, without issue, because it is actually common sense that pinch to zoom is not going to substantively change what's shown in a video like that.
There was not a question regarding visual facts previously. Much of the video usage was used to either establish timing or things someone said.
In this case the prosecution wanted to significantly zoom into very grainy footage to ask about an object a few pixels wide, on footage so grainy the prosecution couldn't identify the face of their own lead witness.
The defense raised the question regarding zooning altering or adding pixels and the judge said flatly he didn't know. All the prosecution had to do was bring in an expert on video to explain how zoom works - or have the expert do a non-fractional proportional zoom.
I wonder how reliable even the source video is at pixel-scale? If it's encoded with something like MPEG then not every macroblock is updated every frame, and even if it was then there are still compression artifacts created by the discrete cosine tranform.
Its common sence that you can't take over control of a car over the internet and kill everyone in it, that acess to a website that shows realtime location of half of US polulation would at least be properly protected, that a car's accelerator pedal doesn't cause random and unpredictable effect thanks to spagghetti code, that when Fujistsu charges a postman for theft, they are not just covering up bug in their tracking and accounting software.
Most states use a form of common law, which is completely based on what is common practice and what is common sense. Or at least was in England hundreds of years ago...
That does not mean we ask Joe Blogs what his common sense tells him about highly technical subjects, whether that's effects of sniffing glue, bank interest accrual or image upscaling algorythms
True. And the case of what upscaling does is technically truly interesting and nuanced... but that doesn't mean that people don't have a sense of what zooming in does to an image, CSI "zoom and enhance" be willfully ignored.
They may have a sense of it, and it's not at all clear whether the sense they have of what will happen matches reality.
Which is why both sides can ask the other side to substantiate their claims or provide expert testimony to justify their claims. If there is no issue, then the other side will be able to relatively easily find someone to testify to that or provide other evidence.
It's how the system is meant to work. If this is being used abusively, the judge will catch on soon enough.
>That's not how technical claims of that nature are normally handled.
That's because criminal law is purposefully biased in favor of the defense. If the prosecution wants to argue that they defendant appears to be pointing his gun at somebody when they zoom in on a photo with an ipad then they need to prove that the ipad's software can be trusted to do so reliably. If there are programs that can upscale images by creating new data then they need to prove that the photo app on their ipad is not one of those programs.
This should have been done with a program that is open to scrutiny (and ideally one that's open-source) so that the algorithm used to upscale the image is known. It's not enough to just assume that the ipad uses bilinear interpolation when there's a reasonable doubt that it could be using another algorithm.
How is a non-expert, and somebody whose job specifically requires them to be sceptical and to question unstated assumptions, supposed to be sure of that?
It's a simple enough thing to verify, or require the defence (in this case) to substantiate the claim without throwing out the method of showing evidence entirely.
Courts have to deal with questions like this all the time and handle them well enough. Technology isn't magic, nor do we have to go down a rabbit hole on each point about what might be there rather than what is there.
This inverts the burden. The prosecution proposed the exhibit, and so the burden is on them to show that everything about it - that the chain of custody was valid, that the processing done on it was meaningful - is correct. It's not the defense's place to prove them wrong.
All of the other evidence was processed through forensics software, and displayed on Windows laptops. Then, at the last minute, the prosecutor asks to show this late-discovered evidence on an iPad. Watching the trial, that felt suspicious - why should this one piece of evidence be shown in a novel way?
It’s not a new exhibit being introduced, and the zoomed in form is not evidence. Therefore there’s no chain of custody question.
It was being used to show the footage to the witness on the stand, in order for them to explain what’s shown on it. As has been done in American courts hundreds of times before.
Now sure, the witness and prosecution or defence could argue that a witness doesn’t need to make guesses about what they’re seeing on zoomed in low-quality footage when under cross-examination. That happens all the time, too.
What doesn’t happen every time is for iPads to be rejected for this purpose because of magical AI ‘logarithms’ inventing imagery.
That means the jury will be able to see this as its own exhibit. (It may have been assigned two exhibit numbers.)
And I'd say it makes sense. Each additional processing step can change the image. For example, if one used the waifu2x algorithm to scale the image up, that would certainly be wrong.
It is not magic, but, as other posters alluded, sometimes it can be merely an educated guess. It is hard to accept a life changing verdict based on a guess.. even if it is an educated one.
So no, each time a given technique is used, the way it works has to be explained so that a decision based on facts of the case is made.
Nothing wrong with that, but a common sense explanation by an expert witness ought to be enough. Does Apple need to be dragged into each and every case to testify as to how pinch to zoom works, as was demanded here?
Courts have established mechanisms for handling technical questions around evidence. It's not like any of this is new.
The judge did not give the prosecution time to find an expert witness, did not ask his clerks to search for similar usage of iPad zooming in previous cases, and did not require the defence to provide expert testimony of their own to substantiate their claim.
In the end the iPad was not used. Instead a 4K TV was used, which ironically probably upscaled the image too.
This wasn’t a new evidence exhibit being introduced. Did you actually read the transcript, or are you reacting to this story based on what you’ve read elsewhere?
I would argue they DO NOT handle them well enough at all, and I have a feeling that if the person on trial had a different political affiliation you would also be claiming they do not.
Courts routinely get technology wrong, and use this the send innocent people to prison all the time. I would say the courts get technology related evidence wrong more than they do correct.
> I have a feeling that if the person on trial had a different political affiliation you would also be claiming they do not.
That's an ad hominem argument. In truth I don't care who's on trial, as the outcome of this specific trial is at this point an American culture war & politics issue that I'm uninterested in.
I am interested in how well courts handle technology and whether either prosecutors or defence attorneys can throw a wrench in proceedings with fanciful woo like "iPads, which are made by Apple, have artificial intelligence in them that allow things to be viewed through three dimensions and logarithms", and have that tolerated by judges who themselves are often technologically illiterate.
If the pre-zoomed image was already downscaled (e.g. to fit a small display), then up to a point, zooming should simply downscale less, i.e. remove fewer pixels. Beyond that point, yes, a zoom would have to perform some kind of interpolation.
In the CSI "zoom and enhance", anything beyond the simplest forms of interpolation would be part of "enhance", which is to say, not part of "zoom".
Even downscaling uses interpolation, but sure. The zoomed in part definitely wasn't shown at its native resolution though.
And bilinear interpolation is as simple as you can get besides nearest neighbor, yet it still has the potential to introduce wrong information/cause artifacting. So even in the "CSI" sense, zooming can cause issues (unless you want to argue that only nearest neighbor is CSI 'zooming'? But then not even browsers do only CSI 'zooming').
While as far as I know no pinch-to-zoom software does this today, I can totally imagine that in just a few years it might be common for pinch-zoom in browsers and photo gallery apps to do AI upscaling.
Remember, for most users, a photo only has to be as accurate as their own memories. If extra detail is added to make the photo look better, it doesn't matter as long as it isn't detail the user remembers differently.
Apple already uses AI to enhance photos with their Deep Fusion and Google makes even more use of AI in their Pixel 6. Added detail where it doesn't exist seems like a very reasonable concern.
And in a trial you only need to slightly modify a facial expression to affect the outcome. Imagine if some defendant appears to smile while a victim describes its ordeal.
You can't zoom without upscaling in some form. If you zoom in on an image in your browser, the image gets upscaled. It's going to be using a simple form of upscaling (default is bilinear), but it's still upscaling.
Of course you can, by having the image originally be displayed downscaled for instance. The back camera of your average phone has a much higher resolution than its display, or most computer displays for that matter.
AI-driven upscaling which fills in pixels using data from other images is quite different to algorithmic interpolation that only uses the available information in the image being worked on.
Then this is a problem inherent with digital photography. Any image you see is interpolated from Bayer data. There are multiple algorithms with different results, the colour needs to be adjusted, and the camera distortion needs to be reversed. All of this happens before you see anything on your screen.
A zooming or scaling algorithm that only uses available image data is deterministic, it should get the same result every time.
One that relies on a neural network trained on other images is not, it might produce a subtly different image each time it's run and introduce artefacts from other images.
What? Not only is this a bizarre argument (why would deterministically creating wrong information be okay?), it's not even correct. There's nothing that fundamentally prevents an image scaling NN from being deterministic and I doubt you think that deterministically adding someone's face would be fine, right?
‘Wrong’ information, if it’s expected, understood, and predictable, is still usable. Especially if we know it’s not going to introduce completely alien artefacts into an image like Ryan Reynold’s face.
All of the standard zoom algorithms fall into that category. Whether bilinear, nearest neighbour, or similar, they’re all going to produce similar outcomes with which, importantly, a teenaged witness on the stand will be familiar. Same with the way TVs upscale content.
It’s not like this is new evidence being introduced in its zoomed-in form. It’s an iPad and footage or images being used in the same way prosecutors and defence attorneys have used them for years when cross-examining witnesses.
The reason it’s okay to be deterministic and wrong in engineering is because it’s a lot easier to explain the conditions under which the wrong result will be produced and how you can identify those situations in the specific photo. The reason I think a simple resize that deterministically adds misinformation is better than a neural network is because I can calculate the bounds under which the algorithm produces wrong information and label the image. And I can do that without running the algorithm.
And how do you resolve conflicting technological claims from the defense and prosecution? By expert witness. Which is what the judge did. So what’s the problem?
That's not what happened. The judge placed the burden of proving that zooming didn't do that on the prosecution, and only afforded them a few minutes to find an expert witness.
Some mocked the judge but here's an example where an AI upscaling algorithm added a human face that was never there:
https://petapixel.com/2020/08/17/gigapixel-ai-accidentally-a...
For terminology trivia, that algorithm seems to be doing a human version of pareidolia :https://en.wikipedia.org/wiki/Pareidolia
EDIT: based on some replies I see, my cite isn't to claim that Apple iOS is using AI enhancement during zoom to add fake pixels. The point is to consider why the judge (who is not a computer programmer) is skeptical. Therefore, his request for expert testimony to rule out that the zoom function doesn't add false information is not that unreasonable.
In other words, what's obvious to a HN tech audience doesn't always apply to judges who don't have the same technical knowledge of how zoom algorithms actually work. So considering the judge's state of mind, the "fact" that iOS zoom doesn't add false incriminating pixels isn't a fact to his level of satisfaction unless the prosecutors bring in an expert to confirm it.