You can only get so far with a blog post. Have you filed a radar yet? I just did (rdar://9598285). Here's the text you can copy and paste (make sure you mention it's a dupe of 9598285).
*
Title: Users leave bad reviews for app that don't work on beta OS releases
Summary:
The point of the prereleases is to give developers time to modify their apps to address any compatibility issues (as well as add new features) before the public release of the operating system. Despite Apple's best intentions at keeping OS betas limited to developers, the fact of the matter remains that average users can and do get their hands on prerelease software. This leads many non-savvy users running applications not meant for the new release and often leave negative reviews on the App Store because the app is incompatible with the new OS.
One potential solution to this is to disallow app reviews for users running unreleased OSes. Another is to tighten up the developer program so that each subsequent beta release allows more and more devices to receive installs. With the current 100 devices per account right off the bat, there is widespread abuse of access to prerelease software.
I think it would be a larger engineering effort to totally block it out, but blocking it on the device would at least preventing "heat of the moment" reviews.
Luckily, most of the larger OS updates also required a new version of iTunes, so they could block it for beta releases of iTunes also.
For all itunes accounts who have a beta ios install associated with them, they cannot post reviews until a certain expiry date. Developers would not really mind the trade off and it provides a small disincentive for non developers to not install pre release versions.
After years of countless web apps and services hopping on the bandwagon of the perpetual beta, the potency of the term has become diluted. Users don't think of betas as experimental builds of software for testing purposes; they think of Gmail, which only shed its beta status as a formality two years ago with 150 million users.
Give them any path, even a tricky or expensive one, to obtain a beta of a sexy new mobile operating system and they will, expecting it to work as well as the other "betas" they've used online.
Unfortunately, beta has come to mean "Private release to a select few individuals" to many people.
As for the bug reports in the review area, I imagine this is for a couple reasons.
First, it's the easy way to give feedback to the developer. I don't know of a way to provide feedback within the App Store architecture other than through the review process. Any other method has a chance of being painful. Being forced to sigh up to some developer site bug tracking system, and then getting sent email after email notifying you of every status update. Of course, I also have to search around for how to contact this particular developer with a bug. There is no easy method that I know of to do this.
Secondly, it's a sure way of getting the attention of the developer. Reviews directly impact the developer, and an email directly to the developer might not get a response as quickly as you want.
I'm not saying either reasoning is correct, or right. I'm not judging the reasoning. I just imagine that's the two biggest reasons for using reviews for bug reports.
> I don't know of a way to provide feedback within the App Store architecture other than through the review process. Any other method has a chance of being painful.
A simple "Support" link showing up a pre-filled Mail controller in our app has proven to be very helpful with that and drastically reduced the bug reports within the AppStore.
Being forced to sigh up to some developer site bug tracking system, and then getting sent email after email notifying you of every status update. Of course, I also have to search around for how to contact this particular developer with a bug. There is no easy method that I know of to do this.
Nobody ever said beta testing was easy, and implying that it should be demeans both the testing professional and the developer.
And developer sees something like this [authentic comments to my app in Android Market, not in AppStore]:
1 star - It won't even let me login, crap
5 stars - Thank you very much! It's the best app for . *
Only info which I get is that some users don't understand information on the screen - my bad, I should write it better - but I'm even not able to help this person.
When I get info from user (mainly by special feature which let user to send "bug report" to my by e-mail) about problem I usually answer in 5-10 minutes ;-)
I think the best solution may be to give to developers ability to respond to authors of review, maybe it should be done in this way that user which will set 1 star will be asked if she agrees to get possible response from developer or something like this. Everybody will gain something.
This makes me wonder how many people ponied up the $99/yr. just to have access to the beta. Actual developers should know better. I wish Apple could figure out some way to prevent this - but if they were to prohibit reviews from people running a beta, they would also prevent legitimate reviews at the same time.
I also think there's a place for non-developers to be running the beta -- if they are responsible citizens.
I'm a dev, but not much of one yet. I'm on the beta, and I have discovered plenty of broken apps. When I discover issues, I communicate them directly to the app's developer. As a result I have also ended up beta-testing a number of apps for said developers. This is just as legitimate a use of beta-testing as developers testing their own apps.
> This makes me wonder how many people ponied up the $99/yr.
Probably not that many, but each developer gets 100 UDID to activate, and I've seen many adding random UDIDs on their accounts for people who wanted to install iOS5
> they would also prevent legitimate reviews at the same time.
There is no legitimate public review from a beta iOS device. Legitimate reviews from iOS-beta testing will be done directly to the developers, not via the AppStore.
Even worse, there appears to be at least one site that is selling access ($7) to beta iOS releases, which surely isn't allowed by the TOS: http:// iosactivations. com/
Right, if I remember properly when I downloaded it clearly said that the contents of iOS5 are confidential. That to me doesn't mean "blog it all over".
Yes, developers are supposedly bound by an NDA on matters of beta iOS versions.
However, I do not know if that NDA has clauses about adding random third-party UDIDs (though developer accounts have been banned for that in the past), and in any case it does not bind those getting their UDID added who do not own developer accounts.
I know of at least 5 people that have done this. Stupid thing is, if they had got together they could have done it for a 5th of the price and all got the same benefit!
> This makes me wonder how many people ponied up the $99/yr. just to have access to the beta.
I think it's possible to install the beta without a dev account if you know a developer who's willing to provision your device.
But still, there's gotta be quite a few people with developer accounts who really should not have them. Every time a new beta arrives, the Apple developer forums are full of kids whining that their games don't work. (Not that the dev forums are much better at any other time...)
You could restrict the beta to developers who have actually shipped an application (that wouldn't prevent them from installing it on their "friends" devices, although this could probably be audited).
I guess this could be a problem for developers whose plan is to develope their first app for a yet-to-be-released version of iOS, but I'm guessing that is an edge case.
I'm going to have to agree with him. It is a bit ridiculous that someone using the iOS 5 beta has their reviews go in along with the rest of them.
Apple should make it so reviews are only shown for the OS of the user's phone, so if I am on iOS 5 I can see the beta reviews, but most average joes still on iOS 4 would only see those relevant to iOS 4.
This also solves an issue of users still on iOS 3 with an app that may no longer be compatible for iOS 3 clogging up the reviews for they aren't relevant for most current iOS users.
No that doesn't make sense. As a developer you don't want to start at 0 reviews every time there is a OS update. Reviews are pretty important, and if it periodically looks like you don't have them it would be very bad.
Currently they roll over the reviews ever version of the App the developer releases. That makes sense because bugs get fixed and features are introduced.
You should not see any difference in the different versions of iOS. Though it might perform differently on different hardware.
I feel as if Apple hasn't really done very much to help the developer note what platforms they are targeting in an easy way. If you don't want iPhone 3g users to use the app, they shouldn't be able to download it that that device in the first place etc.
> You should not see any difference in the different versions of iOS.
Wrong. Any OS feature available only in version N and above may be used conditionally while the application still works on versions N-1 and under.
Case in point: Apple's Game Center is only available from iOS 4.2 (which excludes 1st generation iPod Touch and original iPhones), and is not available on iPhone 3G.
What if a game developer is OK with a 4.0 baseline, or even a 3.1 baseline so iPhone and Touch 1 users can still play, but he still wants users with compatible devices to have access to Game Center achievements and voice chat?
He conditionally enables the feature.
> If you don't want iPhone 3g users to use the app, they shouldn't be able to download it that that device in the first place etc.
Set UIRequiredDeviceCapabilities correctly and that's done for you. For instance, the 3G iPhone uses ARM v6 and only provides OpenGL ES 1.1. Require armv7 and opengles-2 and you've ensured your application will not be installable on 3G iPhones.
For an example of that, see Infinity Blade [0] which clearly states it is not compatible with the 3G (more precisely, it specifies that it's for 3GS and 4 in the "required configuration" section) and which will not install on a 3G.
> As a developer you don't want to start at 0 reviews every time there is a OS update. Reviews are pretty important, and if it periodically looks like you don't have them it would be very bad.
This is not true. Sure, if you were the ONLY developer with 0 reviews on a new OS version and everyone else had a bunch, it would look bad, but if every app had 0 reviews, you would be no different. And if you had a large user base on one version of the OS you'd likely have many reviews quickly.
> You should not see any difference in the different versions of iOS. Though it might perform differently on different hardware.
Certainly things are often mostly backwards compatible, but there are often changes at the API layer which break some things or add features that are available on the new version that weren't on the old one. For example, Apple added the backgrounding/multitasking mojo with iOS 4 but until an app was updated from iOS 3 and could make use of those APIs, the functionality wasn't enabled. Does it make sense for users of iOS 3 to see reviews of an app, not yet updated for iOS 4, of iOS 4 users bitching about how the app doesn't multitask right when multitasking wasn't enabled in iOS 3?
> I feel as if Apple hasn't really done very much to help the developer note what platforms they are targeting in an easy way. If you don't want iPhone 3g users to use the app, they shouldn't be able to download it that that device in the first place etc.
This is a decent idea, however most things DO work from one OS version to another, so barring people from using an app on their new device seems like it would often be an artificial limitation. Further, it is tough to beta test an OS if there are no apps to run on it. I think this concept of splitting the reviews would solve the problem of clutter in the current review system without barring people from doing the testing that ought to be done.
Further, some kind of hybrid could be done, such as maintaining the star rating across OS versions (so people could have an accurate representation of the general reliability, etc. of a given app) without clogging the review systems of different OS versions with irrelevant data and unwanted complaining about incompatibility with the new beta.
There was a thread on Reddit that people were swapping access to the iOS beta, filled with people complaining about bugs, speed, etc.
These users weren't developers, were essentially pirating the software, didn't file bug reports and clearly just thought that it was something cool to get early and didn't understand the implications of it being a beta.
> There was a thread on Reddit that people were swapping access to the iOS beta
There have been numerous threads on that, starting as soon as iOS5 became available.
> These users weren't developers, were essentially pirating the software, didn't file bug reports and clearly just thought that it was something cool to get early and didn't understand the implications of it being a beta.
And those who pointed out these issues were generally mercilessly down-voted.
The problem is to marketing twerps "beta" means "OMG cutting edge unobtainium LOL" so they not only plaster it everywhere, they don't even know what it really means.
I thought iOS and OSX had versioned frameworks etc exactly to avoid issues like this. Yet every beta seems to bring nasty crashers from core functionality. How come?
I think this should not come to any surprise to iOS developers. People use the reviews page as a bug report page. For apps I've worked on we got way more negative reviews for particular bugs than we ever got emails reporting said bugs.
There might be a warning already, but if there isn't Apple should make it very clear that every iOS developer has provided technical support contacts and other helpful links that they can use to report bugs. Bug reports don't belong in the review unless the developer never responded to your contacts or never fixed your bug after contacting them.
I also think that the developer should be able to contest reviews that solely create FUD and aren't really reviews at all.
*
Title: Users leave bad reviews for app that don't work on beta OS releases
Summary:
The point of the prereleases is to give developers time to modify their apps to address any compatibility issues (as well as add new features) before the public release of the operating system. Despite Apple's best intentions at keeping OS betas limited to developers, the fact of the matter remains that average users can and do get their hands on prerelease software. This leads many non-savvy users running applications not meant for the new release and often leave negative reviews on the App Store because the app is incompatible with the new OS.
For example, see this blog post: http://mbarclay.net/?p=1317
One potential solution to this is to disallow app reviews for users running unreleased OSes. Another is to tighten up the developer program so that each subsequent beta release allows more and more devices to receive installs. With the current 100 devices per account right off the bat, there is widespread abuse of access to prerelease software.