Two important features I insist on for products I develop:
1. Staged rollout of firmware updates. It’s common practice for apps and software but for some reason it’s less common with firmware. Rolling out to 1% (or less, depending on scale) of devices and waiting a day is cheap insurance. Side note: Build a good relationship with customer service people so you hear about these things immediately.
2. A failsafe firmware reset back to factory state. Some sequence that resets the device completely back to the way it was when it came out of the box, firmware included, as a last resort. In conjunction, your automated tests need to confirm that every factory firmware you’ve ever released can update to the latest firmware.
> A failsafe firmware reset back to factory state.
This doesn't work if your threat model includes denying rollbacks to prevent exploiting bugs in old firmware. I'd love to be able to roll-back firmware on some of my devices to allow me to "jailbreak" them using old firmware.
In some cases your newer firmware may be blowing e-fuses that prevent old firmware from functioning. See the Nintendo Switch, for an example.
To be clear: I think this is anti-consumer and wrong, but manufacturers absolutely do it.
Edit: I also think it should be illegal, by way of consumer regulation. I don't think consumers should have option to waive their right to manufacturers not damaging hardware they own.
This doesn't get enough attention, waaaay too many of these issues are traced back to the vendor trying to "prevent" someone from using their product in a way that they don't like.
Why else would a soundbar need updates anyway? It either performs its well defined functions when you bought it or they sold you a device that doesn’t input/output sound.
Updates for these types of things always fall into three categories. Either they’re gimping some unanticipated usage, they’re trying to insert ads, or they’re trying to gather more usage data.
Sibling mentioned CEC fixes— this one is huge. CEC is lovely in concept but I ended up having to disable it completely across my setup as there was just way too many bits of weird behaviour with devices turning themselves on and then switching the TV or AVR to their input apropos of nothing.
I feel like CEC tried way too hard to be magical instead of exposing enough control for the user to be able to block certain commands from problematic devices, or even just designate that device X will always be the boss in a particular setup.
Yup, game consoles are ground zero for this. I hit the button on the PS5 controller only to have the receiver and TV power on, then the PS4 wakes up for some reason and then switches the AVR to its input.
My Sony UHD player also seems to want to grab the input sometimes too, so maybe it's Sony that's the source of the problems haha.
And again, it's all just so maddening because it feels like it would go away if I could be like "Hey, AVR should never send power-on messages to its input devices." Because then I would just power on the device I actually want to use, it would turn on the AVR and TV, and we'd be golden.
> And again, it's all just so maddening because it feels like it would go away if I could be like "Hey, AVR should never send power-on messages to its input devices."
Yeah, that sounds a weird "feature" in the first place.
If I manually turn on the UHD player/Chromecast/PS5/whatever, it makes sense that the TV also turns on and switches to the respective input.
I could also sort of imagine that if I switched the TV to some input source, it might be convenient if the device connected to that input turns on. (Not by a lot, though. You need the device's remote/gamepad/whatever anyway to tell it what to do, so the one button press saved doesn't really buy you much.)
But what makes no sense for me is the TV turning on all input devices when it's being turned on itself. When would you ever want to have the PS4, the PS5 and the HD player running, let alone as the default behavior?
That sounds like a genuine bug in the TV.
(Also, you sound as if you have some sort of "2 <-> n" setup with n input and 2 output devices. I have no idea how CEC would even be supposed to behave in such a setup. Would an input device turn on both output devices?
I suspect the issue is largely with the receiver (a VSX-935), as that's seemingly the component sending a turn-on signal to its inputs.
If I could, I would have probably run everything to the TV and just done all the audio over eARC, but the TV is on the other end of a 50' HDMI cable, so I definitely need the receiver as an in-rack multiplexer.
I have a laptop, steamdeck, Nintendo Switch and chromecast all connected to an LG TV and all the ouput switching and remote pass-through works as expected. Maybe just a lucky combination ?
> Why else would a soundbar need updates anyway? It either performs its well defined functions when you bought it or they sold you a device that doesn’t input/output sound.
Unfortunately there are soooo f..ing many devices out there that don't follow the specs, no wonder given how long and complex alone the Bluetooth specifications are, and HDMI/HDCP (which a soundbar with ARC support needs...) is even worse, and don't even try to get me started on CEC because that is an even bigger pile of dung, or stuff like GPUs that run HDMI over DVI, MHL or USB-C in DP mode and god knows what else people expect to "magically work" with a 5 dollar adapter they got off of Alibaba. And no, "audit products to follow the specs" isn't a foolproof solution either. That means that everyone has to deal with everyone else's quirks and at least the most popular devices and their manufacturers have to supply firmware updates to react upon reports of quirks.
I thought HDMI and DVI use the same signalling (at least the 'digital part' of DVI, was it DVI-D?), just over a different connector?
In my memory only the connectors competed for adoption, and Home Entertainment industry opted for HDMI and the PC-industry opted for DVI, while the signalling was not contested (besides DVI also being able to carry analog signalling with full spin-out, and HDMI carrying audio instead).
My memory might not serve me well here though.
I never thought HDMI would win :( but it makes sense I guess - Computers/their use changed :(
And the obvious solution is to isolate the device from the world. Most of my stereo is isolated from “the world”, and some parts are close to 30 years old. Why does a soundbar need contact with the internet?
That kinda defeats the point of having a device. Sure it works in some cases but we're talking about a soundbar here and that has to interact with other devices. It's whole purpose is to interact with other devices.
Even if it doesn't need to contact the internet you're still going to want it to connect through cables. There's good reason to connect through bluetooth.
But why should it contact over the internet? Well it sure is nice to be able to stream music from my NAS. There's utility in that. There's also utility in the parent company updating firmware to support new audio codecs. Or to support new algorithms. If my device is gaining more utility, that's a great thing! And of course, if it is connected wirelessly in any way (including bluetooth) I sure as hell would like updates with respect to security.
Without this, the thing becomes e-waste. The environment moves. Time marches on. No thing can exist in isolation, no matter how hard you try. Again, software rots, not because the software changes, but because the world does.
But that's not the problem here. The problem is abuse of that power. It isn't for the benefit of the customer. The problem is managers pushing to release before things are ready. The need for speed with no direction. To not even consider in the calculus of decision making the tremendous costs of when things go wrong. And how this lesson is never learned despite facing the problem time and time again. Issues like this now cost tons of engineering hours, tons of lawyer hours, and ultimately will cost tons in rebates and refunds. How many weeks of work is that equivalent to? Sure, it doesn't always result in catastrophic failure like this, sometimes it results in smaller failures, sometimes small enough they can be brushed off. But those are still costs that no one considers. That's the problem here.
In my case, my stereo is connected to an inexpensive Airplay adapter.
So I do get all the advantages of a connected device, but if the adapter is bricked, I can easily replace just that small device. And more likely, when there’s a new standard, most of my equipment is unaffected.
No, you are missing my point. In the same way as we do (or at least should do) when we develop software, we isolate the volatile parts from the stable ones. The loudspeakers have looked the same for decades. No revolutionary changes in amplifiers in a long time. The same with DACs. That means that when a software update bricks my adapter, or a new much better standard comes along, or I decide to leave the Apple ecosystem, I only need to replace one small part of my stereo system, not all of it.
This should be done internally to the device. I do agree that nothing you do should affect how speaker sure input is processed. But if you want those other features it's much more convenient to integrate them on device or rather place them within the housing as there's lots of empty space.
With electronics you can still isolate functionality like in software how we wrap things into functions. But like software sometimes we need to break that for optimization. Think like Apple M chips. They do it in the most annoying way, but integration is helpful. Ideally in a speaker though you should be able to fuck everything up and still allow for raw input.
As for the Apple thing, well that's a bigger issue because we really should be using open protocols and fuck walled gardens. Walled gardens are part of the problem we're talking about
At least in theory these Samsung sound bars are supposed to adapt to the listening environment to more accurately render the intended surround sound. They also have various non-trivial inputs (including wireless ones) as well as support for additional real speakers and subwoofers which again might need changes for compatibility.
Of course they could be designed to be simpler and have whatever input device is used (e.g. the TV) handle fancy features like mobile phone support.
Sure, you could do everything through a static circuit and require things being fed with speaker wire. But if you add a microcontroller you're going to be able to do much more, get better sound quality, and protect your equipment. Do your speakers have batteries? Do they plug into wall? Either way you can better control power levels. Do you want to boost bass? Fix corrupted signals? Do you want to process signals from anything other than a bare wire?
Sure, you don't need a microcontroller in a speaker. But we also don't need them in our cars. You don't need them in your fucking kettle. But personally, I find them useful and considering how cheap they are it's worth the basically $0 increased price.
See my other argument. The issue isn't that there's a microcontroller in the speaker. The issue is bricking the device. Don't confuse the means in which a bad actor operates with the bad actor themselves. You'll never stop the bad actor by just banning everything tool they abuse. You'll end up with nothing.
Imagine your signal comes in degraded. Some extra noise on the wire because it is passing next to a faulty wire in your walls or something. You can then do a FFT (example) and pull out the noise and rebalance the signal. Maybe an easy way to think of this is with radio since you're very used to dealing with static in that domain but fundamentally there's nothing different than signal coming through a wire other than the technicalities of the medium through which it's transmitted.
There's much more signal processing you can do besides FFT btw and many can improve signal quality and thus sound quality. Even something like a built in equalizer. Sure, you can do this all with hardware by creating all the right filters but you can do more in a smaller package with a computer
Innocuous product features like streaming music, integration with Alexa/Google, connecting to TV and other speakers via wifi. Oh and collecting analytics data and selling to ad networks...
Modern soundbar are bugged Bluetooth enabled, also with ship with interfacing protocols, while legacy bluetooth/wifi drivers are ok, protocols just break
Also, time-to-market pressures can result in initial shipments having (minor but not showstopping) firmware bugs. Post-sale firmware upgrades can be beneficial for the customer.
It’s the norm because people rather buy one single product that does it all.
The alternative to an all-in-one sound bar is having regular 5.1 speakers, a nice receiver, a nice streaming box, and maybe a dumber TV and you will have absolutely the best setup but it’s a lot of putting pieces together, more space usage, and either money (if you want it right away) or a lot of waiting (if you want to get it used).
I had a Yamaha that had a dtsx firmware addition upgrade after it shipped. Not sure if it wasnt ready at product ship, or some way to avoid licensing fees, but I dont know how they would track who upgraded as it wasnt network enabled.
Sennheiser Max has a full computer and os running inside, they can upgrade it quite a bit. Biggest limitation on the device is HDMI 2.0 preventing 20gbps video passthrough of hdmi 2.1, however they should be able to add new audio codecs.
I actually picked up a Samsung soundbar for my mom this past Christmas and there were quite a few negative reviews. Usually around the soundbar dropping its connection. However diving deeper on them seems to revealed that the issue was resolved with an update. It's not super smart though and needs a USB drive or phone app to update. So it has prevented this situation from happening.
Considering the soundbar connects to a TV, console, phone, etc that are constantly releasing new versions and upgrades it makes sense to build in the function to something as simple as a soundbar to fix bugs and compatibility issues.
Samsung doesn't have the greatest track record with updates though so obviously you don't want to jump the gun on these. Hopefully not a Galaxy Watch 4 situation where they need to be mailed to Samsung to be reset because they didn't think about this during the design phase.
More hardware is sold at cost or at a loss, compensated with ads. I don't like the model either, but that's how it is.
If price isn't the only factor for some, it is for many who would otherwise not buy these things. Sellers picked up on that long ago.
Other comments wish to see regulations, they can't outwit those marketing tricksters. For profit enterprise can, and will offer more alternatives with bigger stamps about privacy, ad-less certified and whatnot.
While I agree with your broad statement, I have a TCL (with built-in Roku) TV that has a bug in the sound processing. Either it becomes very quiet, drops out completely, or comes in and out with a lot of stuttering. Happens irregularly, typically though not always weeks apart (though on no schedule I've identified), solved with a reboot of the TV (which of course can't just be done by turning it off and back on - you have to select "restart system" from the menus).
I owned it for at least six months before this occurred the first time.
In theory, I could do a USB update of the firmware and hope that fixes it. In practice, they want my serial number to let me download it. No thanks, I'll pass, even though it's never been connected to WiFi or Ethernet and never will be. I'll just reset it every once in a while.
> they want my serial number to let me download it.
Out of curiosity, why is that a problem to you? Granted, it is strange; I went through the process for my TCL Roku who's wifi stopped working (still not fixed, and now a second, 3yo TCL Roku has bricked itself. nice!)
I don't care in principle, but it's not just that. You have to give your serial, you have to boot the TV to the update, which then sends a challenge-response to their servers that must be correctly answered (you use your computer for this, so the TV isn't actually on the internet) for the upgrade to proceed.
I don't know what's in that data. And if I don't know what's in it, I'm not inclined to proceed; you might need my serial number to know if you're giving me the right software, but you don't need challenge/response for that. They sold me a cheap TV in hopes of collecting info on everything I watch, whether via Roku or just screen analysis. No thanks, and I have no interest in making it easier for them to break into my WiFi. I'm sure it would connect itself automatically to an open WiFi.
It's a little paranoid, but they really are out to get us (or at least our data).
A lot of consumer products ship with half-baked software and/or firmware. I wish Polk would fix the bug(s) that cause my soundbar to freeze and need a reboot several times per week. But it's an old product that's not longer sold, so I'm probably SOL.
The problem usually aren't vendors. The problem usually are rightsholders - the movie/TV series industry still didn't get the Spotify memo, and the console game industry... well it's hard to say they don't have a point insisting on serious DRM given how rampant piracy becomes once there's an easy-enough root method available.
IoT integrations like Alexa come with numerous security requirements that are often good ideas in theory but lead to hacky workarounds to meet certification requirements
Spotify made 1 billion $ of profit in 2024. Hard to call that unprofitable.
My point is, it (and Youtube) killed piracy for the most part when it comes to music. Trading CDs full of mp3s used to be a sport in school a decade or two ago, these days why would anyone even want to invest the time when Spotify has everything anyway at a price point school kids can afford it?
Netflix used to become the same thing for movies, but the greed of studios killed it and now it's more expensive to have the large stream services than cable TV.
> the movie/TV series industry still didn't get the Spotify memo
I'm not sure that's really a memo I'd like them to get. We don't need more subscription services where you don't get to own you content and everything can be taken away at any time.
In what way? Console makers wouldn't gain anything by weakening DRM and making devices rootable. It's not like they are making that much money from device sales.
Of course then you have MS which basically just turned XBox into a cheap but totally locked down gaming PC (since there are very few Xbox exclusives these days).
Not always. There's a time and a place for including end users in your threat model. These would include scholastic and carceral settings, where in both cases the end user may, as an example, desire access to resources that have been deemed inappropriate.
I disagree that a software in a school setting should see students as adversaries. Cheating is a much higher level problem that is better dealt with education and negative reinforcement. After all, those students will need to become participants in a society where we definitely don't want this level of mutual distrust around every corner.
But in any case, students are usually NOT the customer here even if they are the end user.
Yup! Depends on what's a higher priority: Preventing catastrophic destruction of the device, OR, "protecting" some IP from ultra-small-scale piracy, even though ultimately anyone bent on piracy will be able to pirate anyway.
Clearly the latter is heavily preferred by most companies.
even with that "requirement" add special minimal recovery that can be booted with special buttons sequence by bootloader and allows some form of flashing signed firmware.
this should be especially trivial when your device have some usb ports.
you can keep all requirements of only newer or the same version of firmware to flash, with all refuse checks.
if you mess up, you can allow consumers to flash fix using regular pendrive
Sometimes they do it because it’s contractually required if they want to get access to proprietary standards, for example to allow them to play copy-protected content.
Copyright and patent have morphed into evils that drive anti-consumer and anti-competitive behavior, and have driven a “subscription” model that allows rent seekers to achieve their wildest dreams.
Big part of the UBNT vs Cambium dispute. IIRC UBNT won in court, but just to prevent the Cambium firmware being installed on their hardware the next few firmware versions fixed it so that it cant be easily reverted.
Whats worse is that a lot of the affected hardware was near or EOL anyway, so Cambium was simply helping rescue devices headed for the scrap heap.
I think the correct way to do this is to allow a rollback to the immediately previous working version. Before updating, write current firmware to failsafe data storage, then do the update. Then a firmware reset sends you back to the last good version. I'm pretty sure this is already done by many hardware and software manufacturers, such as me.
Is that applicable here? We're talking about speakers. For most/low security devices, a firmware rollback, or a firmware-download mode, are fine. In this case, it would probably have prevented millions in losses, with the risk being a...jailbroken speaker?
This practice should simply be illegal or at least make the manufacturer liable for a full refund plus interest. We shouldn't let manufacturers brick devices that we own.
Android systems can do this today. After an orderly shutdown of new software, then it can mark the new stuff as good and not allow older software to boot.
The funny part is the Samsung update that bricked a10 phones was a update to smart things, so it couldn't use the Android A/B capability to roll back lol
Most companies don't do this because it's not one of their organizational priorities to have reliable updates. The infrastructure is usually custom built and maintained by a couple of folks who have a dozen other responsibilities they're told are more important. Testing is usually limited by hardware availability and release velocity. "One of every board revision we've ever produced" simply isn't available and waiting two days to run through every firmware version before you release updates is a conversational non-starter with the PMs.
There are commercial offerings (like mender.io, never used) that basically specialize in providing rock solid update infrastructure, but that again takes investment and organizational priority that doesn't exist for non-feature code.
I'm working on embedded systems and I've seen and heard some horror stories just on the device's side. Piles and piles of pre- and post-reboot shell scripts filled with race conditions against the system's services and themselves. When these break, if you're lucky a factory reset is enough to fix the system, if you're unlucky they become field bricks.
I'm trying to buck the trend though and on the new embedded system I'm working on, I've specifically designed the upgrade system to be as reliable as I can make it. It goes something like this:
- The new firmware is downloaded to the secondary application slot.
- Just prior to rebooting, the entire state data of the system is serialized as a document and stored on a flash partition.
- The upgrade flag is set, the system reboots and MCUboot does its thing.
- The new firmware finds out a upgrade happened, clears out all the data partitions, restores from the document and then clears out its partition.
The system is basically sanitized and restored after each upgrade. It's also the same codepath that handles saving and restoring the system's configuration by the end-user as well as settings management. If the document schema is for an older version, run the N-to-N+1 schema upgraders on it prior to applying instead of trying to patch the system in-place. If something goes horribly wrong, flip a jumper to trigger the heavy-duty sanitization that nukes the entire external flash (internal flash only contains the bootloader, primary application slot and factory parameters so it's essentially read-only once the application boots).
It might be hubris, but I hope it's good enough that I'll never see a bricked card that can't be resurrected by a factory reset with this project (assuming no hardware damage, no internal flash corruption and no bricking firmware getting signed with production keys seeping through the cracks despite all the checks in place).
That's a strong start, but be careful if your system ever evolves beyond a single logical processor. You'll need additional orchestration to have reliable updates in a distributed system with semi-independent processors. The update on one might succeed, while another fails. Depending on when the old images were produced, the new images might not be able to talk to each other. Depending on their relative roles in the system (e.g. one sets up the power supply or network for the other, or acts as the time master to do certificate validation) this may or may not be an easily fixable issue even if each system locally thinks it's okay.
This sort of functional interdependency has become increasingly common in embedded these days with heterogenous SoCs.
One thing I've seen before is to separate downloading from rebooting, broadcast the manifest for the updates between all the independent processors (all updates need a declarative manifest for so, so many reasons) to check locally, and only proceed when they all agree. Rollbacks are initiated if they can't see everyone with their expected versions afterwards.
Fortunately, it's a single no-frills MCU running the Zephyr RTOS. It does communicate with another system, but they are so very loosely coupled to the point that we really don't care whatever is running on the other side.
I won't get into details, but in some of the horrors stories I've heard the distributed system happened to be entirely software in nature. There are plenty of creative ways to mess up an upgrade on a uniprocessor system.
We already have a watchdog timer. We could automatically trigger a factory reset after N bootloops following an upgrade, but it's up to the end-user to decide to flip the switch so we won't go there.
I kept the summary short and simple, partly because that product isn't out yet and also because I don't want to bury the lead with a lot of extraneous details that we do take into consideration, but are irrelevant to the big picture idea of an upgrade method that factory resets the card and restores its state with a codepath shared with the end-user save/reset and configuration mechanisms.
Different industry, but I (a long time ago) worked in a place that built scientific instruments.
> "One of every board revision we've ever produced"
The, ah, "special" people we had running engineering didn't even put in the work to be capable of the software querying the board rev. We had to play games like running certain motors past a position limit and seeing if there were limit switches there (or not) to guesstimate board revs.
I completely agree with both points and would add a third: design for offline use first (maybe treat every OTA update as - this might be the final version this device ever receives).
Products should work perfectly fine without an internet connection, heck that's how they worked until 5-7 years ago. Core features should never depend on cloud services, and updates should be opt-in, not forced.
Offline first approach respects user autonomy and creates a natural safety net against bad updates. Plus, it means your product keeps working even when servers change or get shut down years later or a nuclear war happens.
Sure, connectivity has benefits, but a speaker's main job is playing sound, not phoning home. Building offline-first also forces better engineering decisions about longevity and graceful degradation.
It's so hard to find any offline-first apps/devices nowawdays, which is sad to see in a world of algorithms and AI.
But you see, the problem with offline use is the manufacturer can't claw back value in the future. How will you keep shareholders happy if you can't arbitrarily push ads, hobble existing functionality, or impose a new subscription service?
Exactly - that's the flaw in trying to extract infinite growth from finite products. We've turned durable goods into rental services without consent, all to please quarterly earnings reports.
The tragedy is that "respecting customer ownership" is now seen as leaving money on the table rather than building lasting brand loyalty through quality.
I get the sense that #2 is viewed as a risk for DRM, given all the work that goes into preventing firmware downgrades to potentially insecure firmware. Specifically thinking of the Nintendo Switch[1] that goes so far as to blow fuses on each firmware upgrade!
eFuses were already on the Xbox 360/PS3 generation. Smartphones also use them to lock out proprietary photography algorithms if you unlock the bootloader.
Great points! As an addendum to this, if #2 becomes untenable for whatever reason (such as a vulnerability in the factory firmware image), then this #3 would be good to strive for as well:
3. have a set of conditions to mark the running firmware image as "safe" and have it become the new fallback firmware image for this scenario. That way you can have a recently up-to-date firmware version constantly trailing the new ones
IMO this is a terrible idea for many reasons but the most important of which is: As a consumer I should have the right to have my device revert any b.s. update and get my setup to how it was the day I bought it.
So many companies have begun rolling out updates that makes the device I purchased call home before allowing any user functions and if/when that server goes down my device becomes a brick. This behavior essentially invalidates my ownership of the product and renders it to a service, provided at will by the manufacturer.
Your idea ensures my device will one day become a brick as soon as the manufacturer decides to mark their update requiring internet check-ins “safe”.
If you think I’m exaggerating check out Louis Rossmann‘s YouTube channel.
FWIW, my background is in B2B hardware and that's the perspective I am coming here with. Out of curiosity though, how do you weigh your value of control vs. security vulnerabilities? Modern speaker systems allow some form of wireless connectivity, so there is bound to be something and not all consumers will be savvy enough to keep up with security updates on their own.
My thoughts on security vulnerabilities is that they exist on any out of date firmware and that should be expected. I’ve never rolled back to factory settings and assumed that this device is now exposable on a DMZ.
Specifically I’m talking about consumer devices, which are almost always behind a NAT config + firewall. If your soundbar has a vulnerability it’s pretty much irrelevant if someone has already breached your network.
If we’re talking about enterprise networking equipment, I still stand by my concerns that the the owner should be able to revert back to stock but the burden of responsibility is on the technician configuring this device, not the manufacturer.
It seems to me the mentality has become that since end users tend to be bad at system administration, they shouldn't be allowed to do it, for their own good.
I reject this mentality. I don't think it's necessary or desirable to make it impossible for people to do things that have negative consequences for themselves. Put a "here there be dragons" warning on the firmware rollback, bootloader unlock, or similar dangerous operation and let people take responsibility for the outcome.
In the case of consumer devices, most people won't even try those things; those who do risk further problems for the chance of a better outcome. In the case of enterprise networking equipment, there's an IT department that, in theory has the skills and resources necessary to make good decisions about technology.
There will always be security issues, so "but security" is not a reason to prevent a consumer from doing whatever they want with a thing that they purchased from you (I'm of course just speaking morally/ethically here since there's no legal provisions preventing that in most places).
If I pay you for a product, you have no moral right to tell me what I can and cannot do with that product, up to and including messing with the firmware, installing known-bad firmwares, wiping it and building my own firmware, whatever I want. It's mine, I paid for it, stop violating my private property rights.
I think I agree with you generalle but just from a logics perspective, this is a bad argument:
> There will always be security issues, so "but security" is not a reason to prevent a consumer from doing whatever they want with a thing that they purchased from you
Just because there will always be security issues doesn't mean you shouldn't try to take care of the low hanging fruit.
Not the person you replied to, but I'm literally pulling wire again to avoid dealing with that dichotomy. And hardware developers that think OTW firmware updates are a neat idea >:(
Unfortunate you'd need to weave that all the way through the whole product stack in order not to end up in a state that looks like it's working at first glance but actually isn't doing what it is supposed to - like everything running but not showing an image, or everything running except networking is dead (-> also no further updates possible), or (remote) input devices, etc etc
From the manufacturer's point of view, a sufficient "safe" state is "can receive and apply a firmware update" -- worst case scenario you can always push out a new re-signed and renumbered version of the older working version.
Network connectivity would need to be in the set of checks to determine if an update was successful. Also, there should hopefully be QA. If you only have one smoke-test for a firmware image it should be whether or not it can upgrade/downgrade a new image from that one.
You need to have the firmware equivalent of a platform team.
It's common now for medium and large companies to have some variant of a cloud platform team: People responsible for shared practices, infrastructure, and processes in the cloud.
Smart hardware companies have done the same for decades. You have a firmware platform team that handles things like update protocols, recovery protocols, testing checklists, on-device OTA update architecture, and other critical functions.
When you're a company like Samsung that continuously releases and develops products this actually increases your time to market rather than decreasing it. You let each product team focus on the parts of the firmware that make their product valuable and free them from having to roll their own update systems
Samsung has multiple such teams.
In my experience with the broader industry, platform teams are usually less than a dozen people who own millions of lines of mostly-external code. You don't usually get the luxury of careful deliberation and comprehensive testing because you're doing too busy putting out fires and chasing down manufacturer errata.
Samsung might be one of the good ones, but sadly most hardware manufacturers treat firmware and software like just another line item on the BOM. Like a screw or a silicon gasket: Source it from some "supplier," spoon it into the product somewhere on the assembly line, and then never touch it again. I've seen a hardware manufacturer that doesn't even use source control or branching. When they have a new hardware product, they take the software that is closest in functionality, hack it until it works with the new hardware, and then set the software back on the shelf until next time.
It's almost exact same thing as purchasing an insurance.
If the management folks have personal health insurance, surely they must understand the concept and the need. And this is a much better deal because unlike actual insurance this is more like "invest once, enjoy forever" type of thing. And multi-stage boot chain, recovery partition and staged rollouts are not some rocket science that needs some serious expertise.
Yet, here we go. Humans are not really rational actors after all, and collective humans are even less so.
I suppose the closest equivalent would be motherboards with dual BIOS.
There if something goes wrong during an update, you always have a backup BIOS with the previous version (not necessarily factory settings). If the system fails to boot, it automatically switches to the backup BIOS and restores the main BIOS to the last working version.
For this $1500 street price soundbar, I'm wondering whether they consciously decided not to invest in BOM cost or software effort that would help avoid bricking.
I'm not sure I understand various industries' conventions...
While interviewing for a principal engineer job, I was meeting individually with a bunch of team leads and managers, and one engineer asked how would I design firmware updating for the company's product (which was more critical, complex, and expensive than a soundbar).
I assumed they were probably trying to see whether I would throw in some robustness/resilience (not oversimplify it). So I sketched it out, while hitting notes like diffs, downloading and assembling in staging space, imperfect networking, having at least two firmware "slots", backing out upon boot loop or failure soon after boot, gradual deployment to installed base, contrasting with some less-critical consumer product firmware update practices, etc.
(Either that was a bad answer, or they got distracted thinking about something I'd said, because I was getting odd subconscious backchannel cues, and they were unresponsive when I tried elicit more requirements or guidance about what they were looking for. Maybe there was some standard embedded systems programmer canned answer that I was supposed to recite (analogous to the Web brogrammer 'system design' interview), and they couldn't think of how to nudge me towards the shibboleth without saying it?)
#2 has been a godsend in the custom/HEDT PC market. Many expensive motherboards now come with a "dual BIOS" system that gives you an older known working image to boot from, in case flashing a new version broke something that can't be easily undone.
Another amazing feature is the ability to flash a BIOS from an unbootable system. You insert a flash drive with the firmware file into a USB port, press a hardware button and the BIOS gets updated, even without a CPU socketed.
This is a requirement for any motherboard I purchase now. I have enjoyed the ability to use AMD CPUs that are slightly outside of the generational support or enable features I am not promised.
Without the ability to flash from USB without a CPU doing this requires keeping spare CPUs that will work just to flash.
As a user/customer, if I'm part of that 1% with an issue and get the same sort of "canned" response you see on the mentioned thread, I feel like me as a user doesn't matter. I guess the next step is calling customer support and then having the person on the phone making me go through their checklist of things I've already tried and again, feeling like this is of no use.
I think it usually takes a big rollout for these big companies to actually "hear" their users.
The second point is the really important one here. Mistakes happen, having a factory reset that actually works is crucial to avoiding extremely expensive recalls.
I'm reminded of the time a random NPR station accidentally bricked the infotainment systems on thousands of Mazdas and because there was no factory reset feature they had to spend millions replacing head units. That's just bad design.
I wonder if that opens a threat vector from a security point of view? If an attacker knows that the golden firmware has some critical vulnerability which they can exploit easily, they can activate it at will by bricking the device and waiting for it to restart.
They could, and that's been a way for attackers to "jailbreak" devices and load custom firmware in the past. Though for the sake of reducing eWaste and enabling device repurposing and reuse, I do think this is the best path for firmware-updatable devices.
Attackers aren't usually in a position to reset firmware, and if they are they might as well do a whole host of other things like replace the device with a compromised one. I don't think there is much of a point to trying to protect from that.
The golden firmware should reset to the old/first firmware of the device and nothing else. Keep it as simple as possible and restore the customer device back to an operational state.
The reset would be done physically. If there was some danger of the device being exploited after being reset, advice could be included for those performing the reset to prevent this.
For example, to not connect it to a network and to manually perform an update to the latest version with some physical media.
I prefer to keep the factory firmware reset to a manual process that requires user intervention.
For example, holding down the reset button for 10 seconds after plugging the device in.
In my experience, it's not a good idea to have a device automatically roll back firmware and erase user data after failed boots. These mechanisms get triggered too easily during certain power outages (power comes on then goes off just long enough to cause multiple failed boots) or when users are doing simple things like rearranging their power cables.
Ability to reset to original out of the box firmware is not only about failsafe. It's also a protection from "bug fixes" taking away features you had out of the box.
I'm still pissed off about LG removing record to disk option from our TV after an upgrade. I've only connected it to internet & upgraded assuming some of those bug fixes resolved few dlna issues otherwise it's always on internet block list.
The important feature here I would insist on is to let the user decide when to do a firmware update. Not the other way round. That's the way to build a good consumer relationship.
Why on earth a sound bar needs to update its firmware? Why firmware needs to be in a couple of tweeters and a woofer? It should basically output audio from an input source.
Another good one is; please always split any security updates from feature changes (and backport the updates per whatever versioning policy you have for those lagging the latest).
After many years of being burned I always delay system level non-security -related updates at least several days after launch to mitigate the risk.
> 2. A failsafe firmware reset back to factory state.
Do you mean like a physical button? That could work, though I'm not sure I've ever seen it. Holding down power for 10 seconds (or whatever) usually just erases user data, but doesn't reset firmware. Are you aware of any device that does this? But does it require some meta-firmware to roll back the firmware? What if that meta-firmware has a security flaw and needs to be updated? And that update is faulty?
If you're talking about a code sent from your servers to devices to reset, that seems like asking for the impossible. If a firmware update bricks the device, that may very well brick its ability to receive codes at all.
In both situations, it starts to feel like a problem of infinite regress...
Reverting to factory state seems riskier than last known good state. You could run into things like TLS root authorities not being recognised, deprecated cipher suites, etc. Just because that version worked a decade ago, it doesn’t mean it’s compatible with the world today.
> > Just because that version worked a decade ago, it doesn’t mean it’s compatible with the world today.
> That's why I said you have to include this in your test procedures.
You can’t test the world. Even if your servers can correctly respond to requests from old software, it doesn’t mean that the network between you will too.
Networking surely does introduce complications especially when TLS is now basically considered required and cert lifetimes are being limited for 'security' reasons. However most consumer devices have functionality, often their primary/most important function, to which network connectivity isn't even needed. For instance, a speaker producing sounds.
In the factory reset state, things should have a USB flash drive firmware install route which could be used to bring back working root certs, etc.
Of course again this depends on whether the mfg is worried about DRM bypass hacks that are found later on in the factory firmware.
I'd support legislation to issue stiff fines for devices that can't be factory reset at any time, with the only exception being for directly-consumer-benefitting anti-theft (so, iCloud lock is okay).
But can’t you? Sure, factory firmware from many years ago might have issues, but should still work well enough to allow you to fully offline upgrade to a newer working version.
I think all the OP was saying, is: Suppose you’re releasing firmware version N for some widget you make. Now, for all versions V in (0..N-1), verify that applying N to V works correctly.
> 2. A failsafe firmware reset back to factory state. Some sequence that resets the device completely back to the way it was when it came out of the box, firmware included, as a last resort.
That's a nifty mechanism that also allows downgrade attacks, so it has cybersecurity implications that may or may not be acceptable. Furthermore, it might not be practical or even be possible to restore the system to factory condition due to technical reasons.
The team next door allows its systems to downgrade to a previous minor version with a mandatory factory reset. It however refuses downgrading to a previous major version because it implies the bootloader was upgraded or the storage was repartitioned and they really don't want to rollback that.
Except when it comes to firmware, downgrade "attacks" are not attacks at all but just owners making use of THEIR devices. The real attack is the company trying to retain control over something they have sold.
This is the de facto playbook for one of the Mega-Evil Corp.'s CPE firmware (Gateways, IPTV receivers, etc...).
New firmware is pushed in phases 1%, 5%, 10%, 25%, 50% then full scale.
Each stage has some delay incorporated for acquisition/application and then for telemetry (including support contacts from affected accounts) to determine impact and allow for regression fixes.
The other reason they would phase launches is because of firmware builds being used across multiple CPE models and hardware revisions, where only a small subset of hardware could wind up being problematic, but not discovered until deployment.
When you have millions of devices deployed, even a fraction of devices having an issue can create a shit storm on the support side of things.
It all seems so obvious once you know to think about it.
> "A failsafe firmware reset back to factory state"
A failsafe firmware reset back to a safe and secure state yes. The factory state is not necessarily that, so no.
I think devices should keep a last known good state firmware but keeping a full factory state immutable firmware would be irresponsible for many usecases.
What hardware reset typically does, in my experience, is to reinstall the last firmware you installed. Many don't even have the space to keep some original and/or safe image in addition. I'm working on one device where we delete much of the existing system to make space for even downloading a new firmware image. It's wild.
iirc for computers doesn't gigabyte have some kind of patent on dual bios design (active vs backup bios chips). I'm sure there are other ways to implement it but I think thats true.
I bet, but I'm talking about devices where the manufacturer tries to shave off every cent to price their products competitively. And then you have big meetings where you have to push back on storage being reduced by a further 2 MiB. At least that's something I've seen working in the embedded space. Storing an additional firmware image, be it only a few megabytes, is unfortunately often off the table there.
Especially if there is an internal testing stage before actually rolling out to production. It's possible that the users seeing the bricked devices are in fact limited to the initial wave, but the damage is already done.
> A failsafe firmware reset back to factory state.
Or perhaps to the very first released firmware version. This way they don't have to support updating from any version to the latest, just from the first one.
Both are very reasonable features, of course. Here are (some of) the real-world challenges to their implementation:
#1: Requires competence, and/or management that isn't too focused on velocity and features to listen to their engineers' warnings about exactly the sort of problem being discussed here.
#2: Many firmware updates explicitly and specifically want to strip away features that the hardware shipped with (by introducing DRM, paywalls, etc.), so see the comment about management above.
> because some undeserving person (who is not in their group) might get it.
Surely someone, somewhere feels like this, but it’s more often a strawman argument used to make opposing arguments more easily dismissible. The discussions I have with people offline aren’t interested in these types of dismissals. I know many people who genuinely want better healthcare, cheaper education, and stronger social safety nets but who disagree with the specifics of proposals. Like most things in politics, if an argument reduces the other side to an easily-dismissible evil, it’s probably not an accurate representation of the counterarguments.
From real world anecdotes, the concern about things like student loan forgiveness (as the most recent example of an expensive social program being debated) are more about the extreme cost of the program contributing to an ever increasing list of expenditures. People are nervous about the amount of government spending and how it’s being distributed semi-randomly. This goes back to the rampant COVID loans to businesses, the stimulus programs that far overshot their target, and now proposals to give certain households with up to $250K income a free $10K.
People understand that these things do matter in a society where we’re all bidding for a limited supply of homes and such. It’s nice to imagine someone having a reduced debt load, but people still think about where that money comes from and how the uneven distribution of that money gives some people (excluding those who paid their loans off early) a financial leg up in places like the competitive housing market.
It’s all connected. The money must come from somewhere, and we’re all operating within the same markets. It’s disingenuous to pretend that there are no consequences for these programs, which IMO is where politicians fall far short of structuring them and pitching them to a wider audience.
They seem to be way less opposed for government to bail out corporations from same tax money tho.
And I definitely knew many people that oppose to tax break or something for someone that's not in their group all while they enjoy some other tax break without problem and excuse that their one is fine.
> They seem to be way less opposed for government to bail out corporations from same tax money tho.
Not really. We had nationwide Occupy protests about exactly that, from both sides of the aisle. Not only leftists were fed up, but it also spawned the Tea Party movement on the right.
Occupy was largely a Leftist movement. And while the Tea Party was created then by Ron Paul, the political machinery that swept into Federal government seats was only after Obama won in 2008. The Tea Party as an entity is completely different between the pre-Obama and post-Obama eras.
"Hands off my Medicare!" might be the prime example. Or farm subsidies. People who constantly argue that government is too expansive and always incompetent (or evil) literally seem to forget that programs benefiting them are in fact run by that same government.
I usually make the case that 60% of the US federal budget is entitlement programs with the largest portion being Medicare and Social Security.
Medicare is what I usually focus on on because a person will take out $3 for every dollar they put in.
The real welfare queens are grandpa and grandma, yet in the 80s under Reagan many of those same people were blaming the poor.
If you want to see who has power, look at what programs people are willing to touch. Medicaid is constantly under attack by Republicans. While Medicare is a sacred cow to both Dems and Republicans.
Pardon my ignorance, but how is Social Security an entitlement program? I thought it was supposed to be forced retirement savings... how often do people get out more than was taken?
It's an entitlement program because what you're entitled to it independently of what you put in. There's no isolation. You don't suddenly get cut off when "your" contribution has been exhausted. You keep drawing from the pool, indefinitely, and in fact this is the common case because official estimates of what people need for retirement (both cost per year and longevity) consistently lag behind reality. The pool "just happens" to be replenished mostly from still-working folks' contributions, but it's still a pool and not individual accounts.
Before anyone else "well actually"s me, as a retiree myself I'm well aware that there is some relationship between what you put in and what you get out. That doesn't change the system's essential nature. It's more of an anti-abuse and anti-depletion measure, similar to raising the retirement age or adding means tests. There's still a big common pool in the middle, and people can still keep drawing from that pool even if they live well beyond the point where their net contribution is negative.
It's a pyramid scheme. You put in $10 over the course of your career, but you take out $20 during retirement (funded by the next generation(s)). If US had shrinking population like Japan instead of growing population it would quickly collapse on itself because each generation takes more than it gives and relies on population growth to not collapse on itself. Or maybe social security rules would change so you have to be 75 before you get any benefits so that most people die before they qualify.
No, the social security taxes levied today go directly to the recipients today, with the plan that when you retire, the next generation will pay for your social security. So it's an entitlement program in the most literal sense.
Saving money in the bank makes sense for an individual, but not a government or a whole country. Because, they can print as much as they need. So why have a big warehouse full of cash when you can just make it later? The limitation to that is that it will cause inflation if you print too much- but if you warehoused the money and then released it later the same thing would happen.
Anyways, the reasons they print money or remove it from the economy are not because they don't have enough, it's because they're trying to moderate the boom/bust business cycle. (Not doing a very good job of it, though.)
By the way, I just want to say that having someone say "pardon my ignorance" gives me hope. I definitely support people who have an open mind! I'm sometimes afraid to ask a question because I'm worried people will downvote or attack me for not knowing the answer. There are tons of things I'm ignorant about.
> the concern about things like student loan forgiveness [...] the extreme cost of the program contributing to an ever increasing list of expenditures.
SLF is a one-off cost (of between $400Bi and 1Ti depending on which plan you subscribe to) though. It's not like, say, the DOD which is currently burning $800Bi a year and rising - I would venture that the people happy to shoot down SLF are equally happy to keep that budget going up.
> People are nervous about the amount of government spending
...going to people they deem undeserving. They're perfectly happy with the amount spend on the DOD, DHS, etc.
Student loan forgiveness is not a one-off cost unless it's bundled with a reform that prevents the same debt from being accumulated again. I support the measure, but I wouldn't have supported it if it hadn't included the income-based repayment modifications that should limit the accumulation of unmanageable student debt in the future — I don't believe this goes far enough either, but that's a different topic. My point is that it's not possible for me to countenance supporting a reform that helps some people and "pulls up the ladder" not making it available to others in the future — generally derided as "borrowing against our children's futures".
I think there's also a generational aspect to it. Student loan forgiveness is being painted as a cash transfer to younger generations who are somehow lazy, whereas moving cash from younger generations to older (e.g. social security) is sacrosanct.
That would be my family when I was growing up. A lot of their beliefs are centered people should work for a living. If someone can't work, they are lazy and working people shouldn't have to pay to support them.
My parents have since moderated their stance on this but my dad still believes this is the way things should be. :(
I've moved away from home and very much do not share their view and don't associate with people like this. I still see this belief often enough to believe it's pretty dang common in the midwest among the lower middle class.
Edit:
I should note, everyone I've talked to do make exceptions for a few people they know.
Most of them are otherwise good, kind people. They abhor the idea of anyone else getting something they haven’t earned.
Is it that they abhor the idea of someone else getting something unearned, or that they are frustrated and fed up with the things that they have worked for and earned being denied to them (and only incidentally that they are given to someone else)?
I have a disability and some of the accommodations are a regular schedule (no weekends/overtime/on call) and being able to take time off unpaid when I’m sick. This is the bare minimum I need to be able to do knowledge work.
I don’t mention this because some people don’t think I deserve special treatment just because I “claim” to have a disability. If I can take time off for “vacations” whenever I want, they should be able to as well.
If I didn’t have these accommodations, I wouldn’t be able to work. Any explanation of how being bipolar severely impacts every aspect of my life is met with “life is hard for everyone”. (Exact quote from my parents, btw.)
To put it quite simply, no one should be given anything I wasn’t given.
When it comes to taxes, they see it as the government taking what they earned and giving it to people who haven’t earned it.
This is the default point of view where and when I grew up. Based on the parable 'If you give a man a fish he is hungry again in an hour. If you teach him to catch a fish you do him a good turn.'. The independence and freedom offered by the ability to meet one's own needs rather than be at the whims of others you depend on is seen as a much greater good.
But what are the consequences? People has similar revelation when the national debt was about to surpass GDP, and yet no economy-ending consequences rose from that. In reality nobody really knows how debt works on a global scale, and those that try to understand it seem to do a fair job at keeping the economy afloat when they join FRB or the multinational conglomerates that have a stake in the US remaining economically stable.
We must live in alternate realities; I am old enough to remember polls from 2016-2018 that showed that rural conservative voters liked the Affordable Care Act and its provisions when it was labeled as such, and hated it when it was labeled as Obamacare.
I have also mostly heard "fairness"-based arguments against student debt relief.
> perhaps arguing that the cost of the something has to come from somewhere.
This is exactly what’s going on. People know that these expenditures ultimately come out of the company’s operating budget, which diminishes the funds available for other things. Installing EV chargers is a great environmental move in my personal opinion, but we can’t pretend it’s unrelated to discussions come raise/bonus time when people are told that their raises are smaller than expected because budgets are stretched this year.
The EV chargers may be a tiny contributor, but they’re still an unbalanced distribution of the company’s funds based on something completely unrelated to performance. People are keenly aware of these things.
Amusingly if it were proposed to put the EV charging stations in the worst part of the parking lot, people would probably be happier; as it is they're often right next to the handicap stalls and so you have the double whammy of someone getting the best parking space AND free "gas".
I love seeing DIY PCB manufacturing projects like this. Using the 3D printer as a plotter is a creative take on the traditional DIY process.
For any aspiring EE hobbyists: Manufacturing your own PCBs is almost never worth the effort. You'll spend days or weeks getting the process and tooling right, and you still have to manually wire any vias that connect the front and back sides of the PCB. These DIY PCB manufacturing projects are fun if you're in it for the experience, but very impractical for getting work done. It's quick and easy to order small quantity PCBs online.
OSH Park is a popular option: https://oshpark.com/#services You can get 3 boards in 9-12 days for $5/square inch. If you need the boards sooner, $10/square inch will get you a 4-5 day turnaround time. You won't save any money by buying all of the gear to DIY etch your boards, and you certainly won't save any time.
> Manufacturing your own PCBs is almost never worth the effort
I don't know, for me, since I already had an entry-level 3D printer and Dremel, I was able to mod it to engrave the negative of a PCB onto these copper plates in a few days. Now I can get from design to finished PCB in an hour.
I suspect many people getting into EE might have some of these tools already, and being able to iterate quickly with locally-available supplies is a huge advantage, especially for people new to PCB design.
Edit: I would like to add that my method was rather poorly though out and the method in article is much better. Putting a Dremel on a 3D printer is not a good idea, it was just the first thing I came up with and it happened to work ok-ish for my very basic needs.
Actually, PCB milling[1] is a thing. I suppose a MPCNC can be used for that. But it still has the huge drawback that you're limited to one layer if you're lazy (like me) or two if you fell like spending the time to add manual vias - though I was told there are tools for that, which work essentially like... Err... riveting pliers? (sorry, non native speaker)
Given that multi-layer PCBs are effectively laminations of individual boards, hobbyists have already made their own multilayer boards [1]. I don't see anything particular about milling as a means of copper removal that would stop you layering up boards?
Thanks for the link, nice to know it can be done at the hobyyist-level. But it's still a lot of work I (personally) do not enjoy spending time with. If I had the space I'd probably setup my (hypothetical) MPCNC for milling PCBs, so I can do quick single layer prototypes (maybe some simple stacking using header rows), but anything more complex would still go to a fab.
It's not a terrible idea, and I've wanted to play around with the concept myself, but I've never really had time or an acute need.
The Snapmaker (1) is a 3d printer with changeable heads. For mine I mostly use the 3d printing head or the low-power laser engraving head, but it also came with a milling head and they sell a higher-power laser head that can cut some materials.
I looked around and people have had trouble trying this with the milling head. The bit usually breaks and the X axis isn't stable enough. Maybe v2.0 will be more sable since it has a second Z axis mount.
That being said, their mounting platform is modular. There's nothing to stop you from fabricating a dremel mount and ripping apart an existing milling head to interface it with the dremel.
Those prices and lead times don't seem very competitive. At Sunstone I pay $5/sq in for 1-day service on a 2-layer board. If I was willing to wait a week it's $1/sq in.
I think people who are impressed by 3d printers often aren't very experienced with how easy it is to make or get made the things they are printing. I can draw a board in my EDA program, hit a button at 6pm and come back to work with the boards on my desk at 8am the next morning, if I want to pay for that. Same deal for parts: I can order a box full of crap from Digi-Key at 10pm Pacific Time and the Fedex guy will drop it on my desk 10 hours later. Electronics prototyping is extremely well developed, fast, and dirt cheap.
Do you have some sort of special deal on pricing? What sort of quantities are you ordering? For oshpark, it's $5/sq in for a batch of 3 boards. I just checked the prices at sunstone and they quoted ~$35 per board for a 1 square inch 2-layer board for 1-day service or $14 per board for 1-week.
You are right, we live in the golden age of electronics prototyping. I'll throw in plugs for the Octopart search engine for finding parts, and PCBEX for inexpensive prototype boards up to ten layers.
Yeah I had kind of a weird reaction to his comment.
I was inspired by Ben Eater's YouTube series about making a breadboard computer. Rather than copying his schematics, I designed my own. Took a very long time. Went on digikey and Amazon and dropped probably around $1k in parts and tools. Now I'm building the damn thing and it's very time consuming.
I also recently bought a new laptop for $200 which is many orders of magnitude more capable than my breadboard computer eventually will be. Does that fact mean my time, effort, and money spent on the breadboard computer is a waste? Hell no.
I think that just proves his point. You're spending time doing the parts of the project that interests you.
Manufacturing PCBs isn't electrical engineering, it's process engineering. I'm sure there are some process-engineering hobbyists, so if people want to spend their time doing that, then cool. But his point is that if it's the electronic engineering part you're interested in, spending a lot of time tweaking and improving your DIY PCB manufacturing when you can order dirt cheap boards from China that arrive in less than two weeks, that are far and away better than what you could do at home, that generally isn't worth it.
> Does that fact mean my time, effort, and money spent on the breadboard computer is a waste? Hell no.
Well you did spent $1k in parts and tools instead of building theses parts and tools yourself, didn't you? His argument wasn't that it's not worth the experience, in fact he even mention how the experience can be worth it if that's what you want to experience. If your goal is just to get a PCB, then paying for a professionally made PCB will worth it (just like in your case spending $1k in parts and tools will worth it).
I disagree that it won't save time, if just for the fact that 30 minutes is less than 9-12 days. I usually need neither the quality nor the quantity that these fabs will give me, so being able to quickly make a PCB is definitely worth it for me.
That makes a lot of sense to me. One of the reasons I stick mostly with software is that it allows for much quicker iteration. That increases both my ability to explore a space (when working divergently) and polish a product (when I'm in convergence mode). Hardware is, well, hard.
Iteration speed can make a huge difference in outcome. I read that the average consumer electronics company goes through 3-7 physical product iterations. Apple, on the other hand, went through more than 100 generations of prototype for the first iPod.
Obviously what you've done here is just getting started, but it points in a really interesting direction. A 3D printer with swappable heads for plotting and an add-on PCB etching kit should definitely have a market. That would mean recurring revenue for PCB supplies at a good margin, too.
> That makes a lot of sense to me. One of the reasons I stick mostly with software is that it allows for much quicker iteration.
4-5 day turnaround from OSH park is quite fast. Keep in mind that you usually need to order parts from Digi-Key or another supplier when you finish the design anyway. If you have a mistake on a PCB, it's faster to use a tiny stitch wire than to make a whole new PCB.
On a professional level: You can always buy fast turn PCBs if you need them ASAP, but you'll pay for it. Usually cheaper than paying engineers to fiddle with finicky PCB milling or etching machines, though.
> Obviously what you've done here is just getting started, but it points in a really interesting direction. A 3D printer with swappable heads for plotting and an add-on PCB etching kit should definitely have a market.
On the professional level, several manufacturers make dedicated PCB mills: https://www.lpkfusa.com/products/pcb_prototyping/machines/ These machines have helpful features to index tools to the surface and align the panels as you flip them over for two-layer designs.
I have extra parts at home, exactly because of this. But I can not order extra PCBs with my future mistakes corrected. So it's either make them myself or try a single design, spot my failure, correct, order a new one, wait a week or two until I have some more free time to mess with it.
Or avoid PCBs completely, what is clearly the best option whenever possible (what is becoming less and less common).
To be fair, modern EDA software helps a lot with not making mistakes. If your schematic diagram and tolerance settings are correct, it is pretty hard to get the PCB wrong. Every time I had a problem with one, I had made a mistake in the schematic (and once the fab house mistakenly bridged pads because my tolerances were too tight).
I wholeheartedly recommend KiCAD if you aren't already using it.
Seconding KiCAD. Their software is fairly intuitive- especially if you're used to Linux- and it's super helpful for making sure that your designs are logical and stuff, and you can also use it to figure out how to best breadboard your designs as well.
I disagree and think saying it's intuitive is doing people a disservice. It's not intuitive at all, but it's more intuitive than the alternatives and it won't take you very long to learn how to use it.
If you don't get it immediately, don't worry, that's normal. Just watch some tutorials and you'll be good to go in a few hours.
Okay, you're right. I'd forgotten that I'd A) had a failed start with it and B) only gotten past that once I had to use it for a school project. That being said, once you figure it out, it's really powerful. It just takes a bit to get there, and I'd forgotten the beginning of my learning curve.
Sorry for the confusion, and thank you for reminding me of how hard it was for me personally to start.
No problem, and I agree, it's very powerful and it's fun to use. It doesn't take very long to get a feel for what's going on, and it's very rewarding afterwards, as you know.
Digikey, Mouser, and Arrow, for the most part, give me next business day delivery for cheap (often free). From Arrow, I think the minimum is only $50 for "free" overnight shipping.
"Manufacturing your own PCBs is almost never worth the effort."
It depends on the target device. For analog or simpler digital circuits prototypes where single side boards can be used it's still very convenient, with the only annoyance being the etchant treatment and disposal. In the old days I made things easier by adding one intermediate step in which I redrew the schematic as the ICs and keyed parts were seen bottom up, that is, ICs had the 1 pin up right, etc. Also the schematic wasn't arranged functionally, like an opamp drawing for example, but it rather showed how the pcb was going to look from solder side. From then on, copying it on the actual pcb became a breeze. As for drawing on the pcb, I did all by hand, using both transfer sheets with pads where necessary and thin (1mm or less) normal water resistant markers for connections, buses etc. Faber Castell and Staedtler were my favorites, as I found those specific for pcb drawing were a lot worse quality wise, not to mention like 4x more expensive.
This method would of course not scale with circuits complexity, more than one layer and or series production, still I enjoyed a lot being able to design, etch and solder even moderately complex circuits in one evening or a week end in "mom's basement" (actually dad's attic...:). With time I also trained myself to draw my schematics directly that way, so I could get rid of one step between drawing on paper and the final drawing on the pcb. I miss those days...
> You'll spend days or weeks getting the process and tooling right, and you still have to manually wire any vias that connect the front and back sides of the PCB.
Fancy! At school we had a UV box & bath, and made single-sided PCBs. I assume you can use the same process just with boards that have copper both sides. It was great for demystifying the process, and I suspect economical in a school setting.
(Aside: I then studied EE (+CS mix) at university, and there was none if that. Many of my peers without the opportunity to take Electronics at GCSE & A Level would be nonethewiser. Maybe that's fine, lines have to be drawn, but seems to me like an easy extension of existing lab assignments.)
As a hobbyist, if I had the room I'd certainly want the kit. Nothing beats the immediacy for prototyping , or turning around fixes/modifications. The basic stuff isn't prohibitively expensive, we're not talking even hundreds of units before break-even.
I do it at home (PCB mill) for RF circuits on Rogers 5880 and similar materials. At work though (more RF), I just order it.
Seems to always be $3k-$5k per run no matter how many boards and what material. You are buying the PCB lines time yourself rather than sharing the cost.
I had twelve boards recently from OSH Park for a whopping $3.37 total, which includes shipping. They were small boards.
My mill has the cutter fully surrounded by a vacuum manifold, and the vacuum itself is bag and HEPA filtered.
The compressed paper PCBs have horrible properties. The RF laminated all have specific properties for electrical performance; dielectric constant, low loss tangent, low surface roughness, homogeneity, controller thermal expansion, thermal conductivity.
Professionally- agreed.
However I look at it as a learning exercise, which is a wonderful way of calibrating one's sense of time/money quadrants.
If you don't have time, but have money- send it to overnight PCB service.
If you have time, don't have money- do it yourself.
If you have both- send it to regular PCB service.
Now the last one is the least desirable quadrant to be in- if you don't have neither time nor money and this is the first time you're doing this- expect to find either (or even both) of those, at least until you have gotten the equipment and gone through the process a couple of times.
I've seen hobbyists use JLC PCB in the past for low quantity runs and the quality always seems astounding with relatively quick turnaround times (several days). https://jlcpcb.com/
Looks like they do SMT mount now too which wasn't the case last time I looked at their offering.
FWIW PCBWay generally gets my weekend orders to me within the week, and costs about the same as JLC ($5 for 5 boards plus $15 shipping). In some cases I've put an order in on Friday night and gotten them on Tuesday.
Yeah, shipping in the US is much faster. Here in Greece you have to pay 3 euros minimum on shipping charges and shipping from anywhere in the EU will take at least a week.
Good to hear. I led the UniFi Protect effort at Ubiquiti. Responsiveness and fast time-to-video were our top UX priorities for the system. The WebRTC direct-connect and bypassing cloud servers were big factors in delivering that responsiveness. The added security and lack of monthly fees was another major bonus.
While I have you here, please consider the option to install the software on-prem.
I was this || close to not going unifi because it wasn't clear what the longtime support for unifi video was. I'm still not super-happy with that aspect as it's not clear if the hardware is tied to protect or not.
This 75% number is not from a formal study. The number supposedly comes from quotes from 3 different psychiatrists who quoted the same 75% number.
The author verified the 75% number by asking her Facebook followers.
Do most med students require psych drugs for day-to-day
survival? I turned my question over to Facebook: “75% of
med students and residents are taking either stimulants
or antidepressants or both. True or false?”
She cherry-picked a few Facebook responses to include in the article, but even some of those don't support the author's claim:
but I have no idea if it’s 75%…I don’t know enough of my
class well enough to have that info, nor do I think anyone
does…there are usually cliques of up to 25 people, but for
people to say they know for sure details of 75% of their
class would be hard for me to believe but maybe…there is a
lot of it, I agree with that.
And
Being completely honest 75% seems a bit high, but I wouldn’t
be that surprised if it were true
The author's real point is buried at the bottom: Medical licensing requires doctors to disclose their psychiatric history and medication use. Admitting previous psychiatric problems on your licensing form requires a detailed explanation of the condition and treatment. Doctors know this, so they have a perverse incentive to hide their treatment or even avoid treatment at all for fear of risking their careers.
Ubiquiti Networks | Draper, UT or Portland, OR | ONSITE FULLTIME | ubnt.com
Ubiquiti is a high-performance networking hardware company. Our UniFi Video IP Security Camera team is hiring for several positions in our Draper, UT and Portland, OR offices. Join us to develop our next-generation security camera platform on top of Ubiquiti's industry-leading networking platform.
Senior Devops Engineer - Develop, extend, scale, and maintain our NodeJS cloud services backend. Seeking senior engineers with a track record of NodeJS development at scale.
Senior JavaScript Video Engineer - If you have significant experience with WebRTC video, WebRTC data channels, Media Source Extensions, or any other deep experience with JavaScript and video, we might have a position for you. Our stack includes React, Redux on the front-end and NodeJS with native C++ extensions on the back-end.
Computer Vision Engineer - Implement computer vision functions in our embedded camera systems. Motion detection, object tracking, person detection, facial recognition, and more. OpenCV experience is required, embedded systems experience is a plus.
Additional job listings and more company information available on our website: https://careers.ubnt.com/
Please contact me directly if interested. My e-mail address is available in my HN profile. Onsite preferred, but we do hire remote candidates who have truly exceptional track records.
> I don't understand the real value in this. They take a bunch of BGA parts + passives and put the dies in a bigger BGA that approaches the same PCB area?
It's like DRY (don't repeat yourself) for embedded systems. This package wraps the common layout tasks of an embedded systems design in to an easy, pre-packaged module. The PCB designer no longer has to do the layout work of all of the DRAM and power-supply traces, which can consume a lot of time and effort when you're just trying to do a basic design.
> If someone has the manufacturing capability to put down one integrated BGA part, then they have the capability of putting down the DRAM and processors just the same as well as the 0201s or whatever passives to support it.
That's the other great thing about this package: The BGA pitch (distance between BGA balls) is a relatively large 1.27mm. This allows for the use of much cheaper PCB processes because the designer doesn't have to use very fine vias and traces to route in between the balls. For reference, many BGA packages these days have 0.8mm, 0.5mm, or even 0.4mm pitches and require more expensive PCB processes for the entire board just to place that one chip.
> Also, how many hackers are actually putting down BGAs like this when you can buy a properly routed, tested and assembled BeagleBone for so cheap? I know a few are, but they are the 0.1% perhaps? All the makers are rocking Arduinos and Raspberry Pis and can't even spell B-G-A.
Placing BGAs is quite easy for the hobbyist these days, especially now that hot-air stations can be had cheaply. You're right though, in that this product isn't targeted at the weekend DIYer. It's for people building boards who want a CPU but don't want to mess with tight PCB tolerances and all of the high-speed routing required. It also represents a very easy path toward manufacturing for those DIYers who do start with a BBB-based design and want to move to something they can mass produce and sell.
With this, they could drop a single, easy BGA package on their PCB and skip straight to doing design and layout only for the parts of their design that differ from every other embedded systems design (e.g. not the DRAM <-> CPU interface, not the power supply layout, and so on).
Luxul | Draper, UT | JavaScript front-end engineer | ONSITE
We build top quality network hardware: high-throughput routers, high-power wireless access points, and managed switches. I'm looking for an ambitious JavaScript front-end engineer to help drive our user interface development.
Strong JavaScript knowledge is a must. Our current system uses React, Redux, some ES6 and ES7 features through Babel, and Webpack. Familiarity with those is a plus but we're also more than happy to hire ambitious candidates who are eager to learn. If you're interested in Golang there's opportunity to work on the backend components of the web architecture as well.
We're a small engineering team with a diverse array of backgrounds and deep experience in different fields. This is a good place to learn and we're always looking for experienced developers who can teach us something as well. The company continues to experience very strong year-over-year sales and sales growth. Developers get their own office with a door as we value having a good development atmosphere and we're flexible on working schedules.
1. Staged rollout of firmware updates. It’s common practice for apps and software but for some reason it’s less common with firmware. Rolling out to 1% (or less, depending on scale) of devices and waiting a day is cheap insurance. Side note: Build a good relationship with customer service people so you hear about these things immediately.
2. A failsafe firmware reset back to factory state. Some sequence that resets the device completely back to the way it was when it came out of the box, firmware included, as a last resort. In conjunction, your automated tests need to confirm that every factory firmware you’ve ever released can update to the latest firmware.