Tag Archives: Exchange ActiveSync

Back in September I posted an article about a problem that occurred when synchronizing iOS 7 devices against Exchange 2010 SP2. The wheels of justice grind slowly, but Microsoft has released a KB article and accompanying hotfix that describe the symptoms precisely.

I also got an odd report from a large enterprise customer; they had several hundred iOS 7.0.2 devices, all on Verizon in one specific region, that were having synchronization problems. The issue here turned out to be a network configuration issue on Verizon’s network that required some action from them to fix.

Now you’re probably starting to see the value in solutions like those from BoxTone…

from the oops-they-may-have-done-it-again department…

I just got an e-mail from a former coworker reporting a problem with synchronizing some, but not all, iOS 7.0.x devices with Exchange 2010 SP2. There are four users (Alex, Eric, James, and Peter, let’s say) with shiny new iPhone 5s devices. Two of them get the same error when syncing: the Provision verb is returning a status of 110 and throwing an exception from Microsoft.Exchange.Security.Compliance.MessageDigestForNonCryptographicPurposes.HashCore. This seems to point to a problem with crypto negotiation with the devices, but I haven’t been able to look at a trace of the conversation between the device and the server to check.

James’ device works fine. Alex’s device works fine. Peter’s device does not work, either with his own account or Alex’s. Eric’s device does not work with his account; no other accounts have been tested. This seems to indicate that the problem is not (necessarily) with the account. Peter and Eric have both wiped their devices, deleted their Exchange accounts, rebooted the devices, and done all the other stuff you might try when faced with this problem, but to no avail.

This Apple support forum thread seems to indicate that a few others who have the same problem, but none of the recommended fixes have worked for Alex or Peter. My working theory is that this is due to an unwanted interaction of some kind between Exchange 2010 SP2 and iOS 7.x, but I can’t prove that yet. As far as I can tell, Exchange 2013 CU2 doesn’t have the same problem.

I’m posting this in hope that it might come to the attention of anyone else who’s having a similar problem so I can get a sense of its scope and nature.

More news when there is news…

Well, this is gonna be fun: Microsoft just released a new native mail/calendar/contacts app (which they’re calling “OWA”) for the iPhone and iPad. A few quick notes:

• It is only supported with Office 365 wave 15 mailboxes. It may, or may not, work against on-premises Exchange 2013 mailboxes. (Update 130716 1509: Microsoft has in fact committed to on-prem support, but haven’t said when.)
• It is a native app, with separate versions for iPhone (iPhone 4 and later) and iPad (iPad 2 and later). Both versions require iOS 6. Making a native app rather than just a bound web control means that the app can include some other cool features.. including gesture controls and voice control.
• It supports Information Rights Management (and, yay, reading signed S/MIME messages). Oh, and it supports delegate access too. Oh, and online Personal Archives… and shared calendars, too!
• No support for public folders, I’m afraid.
• It uses Exchange Web Services, not EAS; to the Exchange CAS and mailbox roles, OWA on a device looks almost exactly like OWA in a browser.
• VOTING BUTTON SUPPORT YES REALLY WOO HOO.
• It has full offline functionality, powered by a local sqlite database.
• When you request a remote wipe, the wipe request removes the app and all the data from its device but leaves the rest of the device untouched. This is a huge feature.

Of course, I’ll have full coverage of the app (and how to administer and manage it) in the clients chapter of Exchange 2013 Inside Out: Clients, Connectivity, and Unified Messaging. Until then, grab the client and play with it! I was able to download, install, and use it on my iPad3 without any trouble, but the App Store refused to allow me to download it to an iPhone 4. Stay tuned…

Well, it appears that Apple has done it again: reports are starting to surface of runaway transaction log growth when mobile devices running iOS 6.1 synchronize with Exchange Server. Tony has a good synopsis here.

Those of you who have been administering Exchange for a while may think this sounds familiar– that’s because there was a very similar problem with Microsoft Entourage back in the day, as detailed by Jeremy Kelly here. Remarkably, a couple of years later, we got the same bug in a slightly different guise, as described in KB 935848. In both cases, the problem was that the client was too stupid to detect certain types of failures, so it would keep retrying the failed operation, which would keep failing. This endless loop quickly resulted in large volumes of transaction log files on the Exchange server.

Luckily, Exchange 2010 and 2013 include throttling to prevent misbehaving clients from using up an excessive share of resources. However, the throttling controls available regulate EAS based on the amount of time user requests take, the number of concurrent connections, or the number of device partnerships. None of these parameters are useful in preventing the iOS 6.1-related problem; it’s not that the individual requests take up an excessive amount of time, it’s that there are so many requests that they generate an excessive log volume. (This video may provide a useful explanation for the phenomenon.)

Exchange 2013 includes the ability to specifically block misbehaving Exchange ActiveSync devices based on “suspicious” behavior. I will have a lot more to say about that in the near future, although that spiffy feature doesn’t help anyone now suffering the problem. For now, all we can do is the following:

• Block iOS 6.1 devices using an Exchange ActiveSync device access rule
• Discourage your users from upgrading, although I expect this to be an ineffective strategy
• If you have a support relationship with Apple, report this problem to them. If you’re a developer, file a RADAR issue. If you have enterprise technical support with Apple, use it. I’ve seen reports that the ordinary consumer-level technical support (i.e. the $49 pay-per-incident support, as well as AppleCare) doesn’t have any way to report this particular problem in an actionable way.

Thoughts for another time: the rapid adoption rate of iOS devices has many benefits for users, including largely avoiding the fragmentation problems that plague Android with issues (like this “smishing” fix that virtually no one has). However, when Apple ships a buggy update, which is common, that rapid adoption multiplies the pain of the bug.

Update 1535 CST 8 Feb: Ina Fried at AllThingsD is reporting that Vodafone is telling iPhone 4S users not to upgrade to iOS 6.1.

Over the last week the Exchange community has learned a bit more about problems with Apple’s Exchange ActiveSync implementation in iOS 6; Microsoft has released a KB article outlining the problem and suggesting some workarounds, and Tony Redmond this morning pointed to a TUAW article that I hadn’t previously seen which asserts that the hijacking problem is a known issue with previous versions of iOS.

Tony’s article is titled “The emerging need for more supervision over ActiveSync implementations.” It’s certainly hard to disagree with that basic premise. Just over 18 months ago, Microsoft launched an Exchange ActiveSync logo certification program with the goal of inducing third parties who sell EAS-compatible devices or software to verify that they properly implement the client side of EAS. A quick check of this page maintained by Microsoft’s legal department shows just over four dozen EAS licensees, but only 3 vendors are listed on the EAS logo page itself. There are several possible explanations, ranging from lack of vendor interest in getting certified to a Microsoft failure to update the contents of the page.

Whatever the reason, I think Tony is right that Microsoft needs to be more proactive in requiring their EAS licensees to perform more robust testing on their clients.

Furthermore, I’m starting to think that the certification and testing program isn’t sufficient in and of itself, which brings me to Postel’s Law.

Some years ago, I worked for a company that made e-mail encryption software. This was during a time when even basic SMTP interoperability between different vendors’ systems was not assured– the Electronic Mail Association had biannual “plugfests” where vendors such as Lotus, Microsoft, and Netscape got together to ensure that their products could exchange SMTP mail properly. Whenever we ran into an interop problem, my boss would cite Postel’s Law:

Be conservative in what you do, be liberal in what you accept from others.

It seems pretty clear that Apple’s iOS EAS implementation has bugs– long-standing ones, if the TUAW article is right– and that Apple has a responsibility to fix them. I am coming to the opinion, though, that the server-side EAS implementation is too lenient about what it accepts. For years Exchange administrators had to suffer through a dance that went something like this:

1. A new version of Outlook would ship with a bug that would occasionally corrupt calendar or message items in some interesting way.
2. The corrupted items would cause the client to crash; in more severe cases, they could also crash the store process or cause backups, mailbox exports, etc. to fail.
3. The Exchange team would release a hotfix to keep that particular flavor of badly formed item from causing such a problem.
4. The Outlook team would release a hotfix that would keep that version of Outlook from emitting that particular flavor of corruption.
5. In the next release of Exchange, the team would add business logic to reject items with the specific flaws generated by Outlook.
6. GOTO 1

Over time, the Exchange business logic got better, as did Outlook’s track record of not creating bad items in the first place. This approach has served Exchange administrators and users pretty well, and it’s clear that the lessons learned from this painful process have been applied in Exchange Web Services. For example, EWS will reject attempts to create calendar items with end dates that come before start dates– something that Outlook used to occasionally do out of sheer perversity.

However, it looks as though, at least in this specific case, EAS doesn’t have business logic in place to catch the modifications that lead to the hijacking behavior. That’s something that Microsoft can fix in three steps.

• First, they can update the EAS spec so that it clearly defines which fields of each data item are to be considered read-only. This gives implementers a fair shot to understand where they may need to change their clients.
• Second, they can update Exchange itself so that it rejects client requests that violate the updated spec.
• Third, they can vigorously “encourage” their major ISV partners to test more aggressively against the updated spec. I have to wonder, for example, how much, if any, testing Apple has done against Exchange 2013…

I’ve been putting the blame on Apple for this problem, and while I don’t think that doing so is unfair or inaccurate, I also believe that the Exchange team can do a better job on the server side of blocking bad client requests, and I hope they’re madly planning how to do so in Exchange 2013 SP1.

As I hoped they would (but couldn’t mention in my previous post), Microsoft has weighed in with a KB article on the iOS 6 “hijacked meeting” bug. The article ID is 2768774, and it makes a couple of cogent points: first, that Microsoft is aware of, and is actively investigating, the bug. Second, that “Microsoft cannot mitigate this issue”; they recommend contacting Apple. The interesting part to me was the section on workarounds. Microsoft’s suggestions include:

• telling users not to make calendar changes using the iOS app
• recommending that users not update to iOS6 (as if! iOS device users are legendarily quick to update to new versions)
• blocking iOS 6 devices using Exchange ActiveSync allow/block/quarantine
• blocking users who have delegates, or who are granted delegate access, from using EAS
• using POP/IMAP instead of EAS (this made me laugh out loud– really!)

I have been testing Savvy Software’s Agenda app recently and like it quite a bit. However, it, like almost all the other calendar apps of which I’m aware,  it uses the underlying iOS calendar data store and sync mechanism, so I’d expect it to fall victim to the same bug. NitroDesk has ported its Touchdown app to iOS (though its App Store reviews are pretty poor), and Emtrace has the MoxierMail EAS client (which I still need to review.) So pick your poison: none of Microsoft’s suggested workarounds are great, but their hands are tied. It’s pretty clear this is an Apple problem and it will be incumbent on Apple to fix it… which I am sure they will, eventually.