Category Archives: UC&C

Exchange administration skills

I’m working on a project that will involve (big surprise) teaching Exchange 2010. That got me to wondering: what skills do commercial administrators and consultants actually need?

To put it in context, some explanation might be in order. I’m working on designing an Exchange 2010 curriculum that will be used with Acuitus’ Digital Tutor. The overall goal of the Tutor is to take someone with no computer skills and turn them into a competent, skilled junior sysadmin– someone who can troubleshoot and fix complex problems with Windows Server, Cisco IOS, and Exchange. We’ve already proven that we can do this with Navy students. Now we’re going to try it with a few different populations.

For Exchange 2003, which is what the Navy’s still using (yeah, yeah, your tax dollars at work), the curriculum design was simple; the Navy asked us to teach the same things they teach in their legacy courses. However, for the new course I have a free hand to include whatever I think a junior Exchange 2010 admin needs to know. 

When Tony and I put together the curriculum for the Exchange Maestro classes we focused on what we thought experienced admins needed to know about the new version. In general, all my writing and teaching has focused on explaining new features in version N to admins who were familiar with version N-1 (or N-2). That leaves me with a gap to fill, so I thought I’d ask the Exchange tribe for some feedback.

For a junior administrator— not a hotshot who’s comfortable editing EDB files with WinHex; not someone who remembers what the Exchange 5.5 IMS was for– what do you think the most important skills and concepts are? If you were hiring such a person, what would you expect them to know on day 1? What knowledge or skill would delight you if your new hire turned out to have it? What areas of Exchange 2010, in theory or practice, were most challenging for you to learn?

You can e-mail me your answer directly if you like, but I’d prefer that you leave it in the comments to hopefully stimulate some discussion.

Technorati Tags:

14 Comments

Filed under UC&C

Some MEC schedule and content updates

Today the Exchange team updated the MECIsBack.com website to share more details of what awaits us in a mere 48 days! The complete schedule is a pretty broad outline, but the session list is quite tantalizing.

Day 1 starts with an opening keynote by Rajesh Jha, but the real goodies start with a technical keynote covering the architecture of what Microsoft is calling “the new Exchange.” (It’s interesting, btw, that SharePoint, Lync, Office 2013, and Windows 8/2012 aren’t calling their products “the new X”. I like the Exchange branding.)  There are a total of 8 additional breakout sessions, all on Exchange 2013, scheduled for the rest of day 1. This is definitely a good news/bad news situation, as these 8 sessions are stuffed into three time slots so you cannot attend them all. That means that we’ll all have to choose which sessions seem most interesting. The arrangement reminds me a bit of past MVP summits when we had to make choices such as “would I rather go to the ‘what’s new in PowerShell’ or ‘storage architecture changes’ session?” This is rather jarring given how lame the last few years’ worth of TechEd content has been for Exchange, but it’s a good problem to have. Fortunately the MEC folks will have the Exchange 2013 day-1 sessions recorded for later viewing. (Personally, I think I will probably hit the high availability, security, and “Apps for Outlook and OWA” sessions.)

Days 2 and 3 are all chalk talks. Microsoft is calling them “classroom sessions” but I picture something more informal than the typical lecture sessions, with lots of back-and-forth Q&A. The preview session content list includes a bunch of sessions both on Exchange 2013 and Exchange 2010. There are some interesting tidbits hidden in the session list: “What’s New In Support Programs with Exchange,” for instance, sounds intriguing given that Microsoft has not yet publicly said anything about upcoming support changes. The sessions on site mailboxes, modern public folders, and what’s new in anti-malware (you did know Exchange 2013 includes malware filtering now, right?) look worthwhile as well.

Microsoft hasn’t yet announced exactly which speakers will be presenting the new Exchange 2013 content. However, if you look at the speaker list you can make some informed guesses. I’d expect all of the Exchange 2013 sessions to be covered by Microsoft speakers (I love it that the Microsoft product group folks are listed under the heading of “Exchange Team Personalities”– I can attest that many of the Exchange folks are, in fact, lively personalities), and if you know who does what on the product team you can probably match session titles to personalities pretty easily.

I’m presenting two sessions: E14.302, “Developing Mobile Applications with Exchange Web Services,” and E14.303, “10 Things You Didn’t Know About Exchange Unified Messaging.” Other presenters include unindicted co-conspirator Tony Redmond, fellow MCM instructor Brian Reid, the formidable Glen Scales, ex-3Sharpie Devin Ganger, and a host of others whose names you’ll probably recognize.

Interestingly, Microsoft is still looking for suggestions for sessions– drop mecideas@microsoft.com a line if there are specific things you want to talk about that aren’t covered. The exhibitors list is now up to date as well, with most of the usual suspects represented– Quest, Binary Tree, Sherpa, and so on.

One open question: there are two evening events, plus an option post-event activity… I wonder what the MEC planners have up their sleeves for us? I can’t wait to find out. See you there!

Leave a comment

Filed under UC&C

Man-in-the-middle attacks against Exchange ActiveSync

I love the BlackHat security conference, although it’s been a long-distance relationship, as I’ve never been. The constant flow of innovative attacks (and defenses!) is fascinating, but relatively few of the attacks focus on things that I know enough about to have a really informed opinion. At this year’s BlackHat, though, security researcher Peter Hannay presented a paper on a potential vulnerability in Exchange ActiveSync that can result in malicious remote wipe operations. (Hannay’s paper is here, and the accompanying presentation is here.)

In a nutshell, Hannay’s attack depends on the ability of an attacker to impersonate a legitimate Exchange server, then send the device a remote wipe command, which the device will then obey. The attack depends on the behavior of the EAS protocol provisioning mechanism, as described in MS-ASPROV.

Before discussing this in more detail, it’s important to point out three things. First, this attack doesn’t provide a way to retrieve or modify data on the device (apart from erasing it, which of course counts as “modifying” it in the strictest sense.) Second, the attack depends on use of a self-signed certificate. Self-signed certificates are installed and used by Exchange 2007, 2010, and 2013 by default, but Microsoft doesn’t recommend their use for mobile device sync (see the 2nd paragraph here); contrary to Hannay’s claim in the paper, my experience has been that relatively few Exchange sites depend on self-signed certs.

The third thing I want to highlight: this is an interesting result and I’m sure that the EAS team is studying it closely to ensure that the future attacks Hannay contemplates, like stealing data off the device, are rendered impossible. There’s no current cause for worry.

The basis of this attack is that EAS provides a policy update mechanism that allows the server to push an updated security policy to the device when the policy changes. There are 3 cases when the EAS Provision command can be issued by the server:

  • when the client contacts the server for the first time. In this case, the client should pull the policy and apply it. (I vaguely remember that iOS devices prompt the user to accept the policy, but Windows Phone devices don’t.)
  • when the policy changes on the server, in which case the server returns a response indicating that the client needs to issue another Provision command to get the update.
  • when the server tells the device to perform a remote wipe.

The client sends a policy key with each command it sends to the server, so the server always knows what version of the policy the device has; that’s how it knows when to send back the response indicating that the device should reprovision.

If the client doesn’t have a policy, or if the policy has changed on the server, the client policy key won’t match the current server policy key, so the server sends back a response indicating that the client must reprovision before the server will talk to it.

There seems to be a flaw in Hannay’s paper, though.

The mechanism he describes in the paper is that used by EAS 12.0 and 12.1, as shipped in Exchange 2007. In that version of EAS, the server returns a custom HTTP error, 449, to tell the device to get a new policy. A man-in-the-middle attack in this configuration is simple: set up a rogue server that pretends to be the victim’s Exchange server, using a self-signed certificate, then when any EAS device attempts to connect, send back HTTP 449. The client will then request reprovisioning, at which time the MITM device sends back a remote wipe command.

Newer versions of Exchange return an error code in the EAS message itself; the device, upon seeing this code, will attempt to reprovision. (The list of possible error codes is in the section “When should a client provision?” in this excellent MSDN article). I think this behavior would be harder to spoof, since the error code is returned as part of an existing EAS conversation.

In addition, there’s the whole question of version negotiation. I haven’t tested it, but I assume that most EAS devices are happy to use EAS 12.1. I don’t know of any clients that allow you to specify that you only want to use a particular version of EAS. It’s also not clear to me what would happen if you send a device using EAS 14.x (and thus expecting to see the policy status element) the HTTP 449 error.

Having said all that, this is still a pretty interesting result. It points to the need for better certificate-management behavior on the devices, since Hannay points out that Android and iOS devices behaved poorly in his tests. Windows Phone seems to do a better job of handling unexpected certificate changes, although it’s also the hardest of the 3 platforms to deal with from a perspective of installing and managing legitimate certificates.

More broadly, Hannay’s result points out a fundamental flaw in the way all of these devices interact with EAS, one that I’ve mentioned before: the granularity of data storage on these devices is poor. A remote-wipe request from a single Exchange account on the device arguably shouldn’t wipe out data that didn’t come from that server. The current state of client implementations is that they erase the entire device– apps, data, and all– upon receiving a remote wipe command. This is probably what you want if your device is lost or stolen (i.e. you don’t want the thief to be able to access your personal or company data), but when you leave a company you probably don’t want them wiping your entire device. This is an area where I hope for, and expect, improvement on the part of EAS client implementers.

1 Comment

Filed under Security, UC&C

Exchange 2013 preview ships

Yay! Microsoft has released the preview version (which us normal humans might refer to as a beta) of Exchange 2013, SharePoint 2013, Office 2013, and Lync 2013.

I don’t have time to write a full summary of all the changes, but a few highlights:

  • One big piece of news: there are now only two server roles: client access and mailbox. (Raise your hand if that reminds you of the Exchange 2000/2003 front-end/back-end split.) CAS now has a new service, Front-End Transport, that doesn’t do what you probably think. In addition, the RPC Client Access service (RCA) is now gone.
  • MAPI may not officially be dead, but the fact that Outlook 2013 can use Exchange ActiveSync sure makes it look that way.
  • The new Exchange Administration Center (EAC) is going to be polarizing; some admins will love it, while others will hate it.
  • No more multi-master replication for public folders. You should read this FAQ if you’re a public folder aficionado.
  • This is the first release of Exchange or SharePoint that really enables the “better together” story. In-place discovery searches and site mailboxes (about which, more later) will really make a huge difference in how SharePoint and Exchange are used for data management.
  • By default, malware filtering is enabled in Microsoft Exchange Server 2013 Preview.” Yay!
  • Not a ton of unified messaging changes, but a few welcome ones, including better Voice Mail Preview accuracy and some UI improvements.

There’s a list of “what’s new in this release” items, of course. Keep in mind, though, that Microsoft frequently adds features between preview releases and RTM, so there may well be additional features, or changes to existing features, between this release and the final release later this year.

Download and enjoy!

2 Comments

Filed under UC&C

Licensing change for Exchange multi-mailbox search

Microsoft today announced that you no longer need an enterprise client access license (ECAL) to use the multi-mailbox search feature. This is a welcome change, of course, since it means that it is now OK to run multi-mailbox searches against mailboxes that are licensed with a standard CAL. In essence, Microsoft is giving standard CAL holders something for free that formerly cost money. ECAL holders, of course, aren’t getting anything extra out of the deal, but I’d argue that the other ECAL features (including legal hold and the Personal Archive feature) probably make up for that.

The interesting question to me is: why this change at all, and why now? It’s common for Microsoft to adjust licensing terms with new releases of Exchange, so my guess is that we’ll see some differences in how the data loss prevention (DLP) and information management features of Exchange 2013 are licensed and which specific mix of CALs you need to use them.

Stay tuned…

1 Comment

Filed under UC&C

Stalking the wily ADAccess event 2112

Timing is everything.

A week ago, I got a late-night phone call about a problem with an Exchange server that seemed to be related to an expired certificate; the admin had replaced the expired cert on one member of a two-node DAG, but not the other. He noticed the errors in the event log when troubleshooting a seemingly unrelated problem, installed the new cert, and then boom! Bad stuff started happening.  Problem was, the reported problem was that inbound SMTP from a hosted filtering service that doesn’t use TLS wasn’t flowing, so it didn’t seem likely that certificate expiration would be involved. By the time he called me, he had installed the new certificate and rebooted the affected server, and all seemed to be well.

Fast forward to Sunday night. I’d planned to patch these same servers to get them on to Exchange 2010 SP2 UR3, in part because I’d noticed a worrisome number of events generated by the MSExchange ADAccess service, chiefly event ID 2112:

Process MSEXCHANGEADTOPOLOGYSERVICE.EXE (PID=8356). The Exchange computer HQ-EX02.blahblah does not have Audit Security Privilege on the domain controller HQ-DC01.blahblah. This domain controller will not be used by Exchange Active Directory Provider.

This was immediately followed by MSExchange ADAccess event ID 2102 with the rather ominous message that

Process MSEXCHANGEADTOPOLOGYSERVICE.EXE (PID=8356). All Domain Controller Servers in use are not responding:

However, the event ID 2080 logged by ADAccess indicated that all but 1 of the GCs were up and providing necessary services, including indicating that their SACL allowed Exchange the necessary access. I couldn’t puzzle it out in the time I had allotted, so I decided to take a backup (see rule #3) and  wait to tackle the patching until I could be physically present. That turned out to be a very, very good idea.

Last night, I sat down to patch the affected systems. I began with the passive DAG node, updating it to SP2 and then installing UR3. I half-thought that this process might resolve the cause of the errors (see rule #2), but after a reboot I noticed they were still being logged. I suspected that the reported 2102 errors might be bogus, since I knew all of the involved GCs were running and available. As I started to dig around, I learned that this error often appears when there’s a problem with permissions; to be more specific, this SCOM article asserts that the problem is that the Exchange server(s) don’t have the SeSecurityPrivilege user right on the domain controllers. However, I was still a little skeptical. I checked the default DC GPO and, sure enough, the permissions were present, so I moved on to do some further investigation.

Another possible cause is that the Exchange servers’ computer accounts aren’t in the Exchange Servers group, or that permissions on that group were jacked up somehow, but they appeared to be fine so I discounted that as a likely cause.

Along the way I noticed that the FBA service wouldn’t start, but its error message was meaningless– all I got from the service control manager was a service-specifc error code that resisted my best attempts to Bing it. Without that service, of course, you can’t use OWA with FBA mode, which would be a problem so I made a mental note to dig into that later.

A little more searching turned up this article, which is dangerously wrong: it suggests adding the Exchange computer accounts to the Domain Admins security group. Please, please, don’t do this; not only does it not fix the problem, it can cause all sorts of other tomfoolery that you don’t want to have to deal with.

Still more digging revealed two common problems that were present on this server: the active NIC wasn’t first in the binding order and IPv6 was disabled on the two enabled NICs. Now, you and I both know that IPv6 isn’t required to run Exchange.. but Microsoft does not support disabling or removing IPv6 on Windows servers. And you know what they say about what “unsupported” means! So, I enabled IPv6 on the two adapters and got the binding order sorted out, then bounced the AD topology service and…

… voila! Everything seemed to be working normally, so I ran some tests to verify that *over was working as it should, then started patching the DAG primary– only to have setup fail partway through. Upon reboot, w3svc was caught in an endless loop of trying to load some of the in-proc OWA DLLs; it kept trying endlessly until I power-cycled the server. The problem with this is that the Active Manager service was starting, so the current active node would try to sync with it before mounting its copy of the DAG databases, but it never got an answer! Net result, no mounted databases on either server, and an unhappy expression on my face as the clock ticked past 11pm.

I put the primary member server in safe mode, then set the Exchange and w3svc services to manual star and rebooted it. Rather than spend a lot of time trying to pin down exactly what happened, I ran setup in recovery mode; it installed the binaries, after which the services restarted normally. I did a switchover back to the original primary node, verified mail flow, and went home. Life was good.

Until, that is, this morning, when I got an e-mail: “OWA is down.” I checked the servers and, sure enough, the errors were back and the FBA service was again refusing to start. After some creative swearing, I once again started digging around in the guts of the server. I couldn’t shake the feeling that this was a legitimate permissions problem of some kind.

At that point, I found this article, which pointed out something critical about GPOs: you have to check the effective policy, not just the one defined in the default policy. Sure enough, when I used RSoP to check the effective policy on the DCs, the Exchange servers did not have SeSecurityPrivilege on the DCs because there was a separate GPO to control audit logging permissions, and it had recently been changed to remove the Exchange Servers group. That was easy to fix: I added the Exchange Servers group to the GPO, ran gpudate, rebooted the passive node, and found that the services all started normally and ran without error. A quick switchover let me restart the topology service on the primary DAG member, after which it too ran without errors. End result: problem solved.

It’s still not entirely clear to me why that particular service needs to have the SeSecurityPrivilege right assigned. I’m trying to find that out and will update this post once I do. In the meantime, if you have similar symptoms, check to verify that the effective policy is correct.

4 Comments

Filed under UC&C

RIP Exchange UPDATE

I am sad to report that I just got an e-mail from the lovely and talented Amy Eisenberg, executive editor at Windows IT Pro, letting me know that they are ending the publication of the Exchange UPDATE newsletter that I’ve been writing since 2002.

  The original UPDATE was delivered only via e-mail, and it was advertising-supported so that it made money for the magazine. Over time, the market shifted, with a much greater emphasis placed on delivery via RSS and a sea change in how advertisers spend (and how media outlets package and sell advertising.) Because of these changes, it doesn’t make sense for Penton to continue to publish Exchange UPDATE, so it has been moved to the great folder in the sky (and, no, I’m not talking about SkyDrive.) It’s a business decision, and not one lightly taken, but I understand the reasoning and don’t bear the magazine folks any ill will.

On the contrary– writing the UPDATE has been a terrific growth opportunity over the last 10 (!) years. When I first started writing it, I took over from the redoubtable Jerry Cochran, who had recently moved to a new role at Microsoft. At first I found the demands of writing a weekly column on Exchange to be almost insurmountable, but I soon came to realize that there’s always something going on in the Exchange world to write about. Companies introduce new products or leave the market altogether; Microsoft ships or delays products; interesting people create interesting things. This cycle will go on, of course, and I’ll still be writing about it here and for Windows IT Pro. Don’t be surprised if you also see some of my writing in new outlets as well, either…

Leave a comment

Filed under Musings, UC&C

In which I dispute Tony Redmond re Windows Phone upgrades

Microsoft just wrapped up the keynote for their Windows Phone Developer Summit. Following as it does on the heels of Apple’s Worldwide Developer Conference, comparisons are inevitable… so let me succumb to the inevitable:

  • At WWDC, Apple announced iOS 6 and talked more about the forthcoming “Mountain Lion” release of OS X, which follows OS X “Lion”‘s lead by incorporating a number of iOS-like features. Many observers believe that Apple’s long-term plan is to provide as much of a unified core between iOS, OS X, and AppleTV as possible.
  • At WPDS, Microsoft announced Windows Phone 8 (“Apollo”), which will share a common core with Windows 8– thereby beating Apple to the core OS-integration punch. They also highlighted some nifty new Apollo features that will depend on their hardware device partners, including new higher-resolution screens and support for NFC.

As they normally do, Apple was mostly quiet about the device upgrade path for iOS 6. Microsoft, on the other hand, was up front about the fact that current Windows Phone devices won’t be getting Apollo. Tony has a beef with this approach:

I think this is a brain-dead decision that looks pretty feeble when compared against Apple’s record of making sure that new releases of their O/S run on older versions of iPhones. For example, the iPhone 3GS that I used before making the now-lamentable decision to try Windows Phone 7.5, upgraded smoothly from iOS 3 to iOS5 over the time I owned the phone.

This is clearly a situation where Apple is right… well, except that they’re not.

It is true that iOS 6 will run on older devices. For example, my not-yet-two-year-old iPhone 4 will run iOS 6, as will all models of the iPad; even the relatively ancient 3GS will get the upgrade.

However, there are a number of iOS 6 features that will not work on anything older than an iPhone 4S. Among these: FaceTime over 3G and turn-by-turn directions in the mapping app, two features that exist either as App Store apps or hacks that can be applied to jailbroken devices.

Apple hasn’t said why they won’t ship these features for iPhone 4 users. It might be due to technical limitations (though I doubt it in the case of turn-by-turn directions), but I suspect it’s more likely to be as a tactic to drive sales of new devices. After all, Apple’s profit on iOS devices come from the hardware (and to a lesser extent, the attach rate of purchases from the App Store).

Microsoft, on the other hand, makes no profit on WP hardware except to the extent that they collect a license fee per handset. That raises the question of what Microsoft should have done with Apollo. You could argue, as Tony does, that at least the shiny new Lumia 900 should get the upgrade, and that ideally older devices should as well. But this approach poses some really interesting tradeoffs for Microsoft. They get to choose between two options, to wit:

  • spending a large amount of money and engineering time to get Apollo running on older WP7 devices, such as the HTC Surround or the original Samsung Focus, from which they will never derive any additional revenue and for which carriers may decide not to give their customers the upgrade anyway? or
  • spending money and effort on polishing Apollo to take full advantage of new hardware, sales of which will drive carrier efforts and directly put cash in Microsoft’s coffers?

I don’t think that Microsoft’s choice was a very difficult one. If you look at the list of Apollo features, some of them (such as NFC, new resolution support, on-device BitLocker encryption,  and the new DirectX) require upgraded hardware. Some of them (such as deep VoIP integration, which I dearly wish Apple would copy, the new Nokia map experience, and support for native C++ code) do not, and could feasibly be backported, but only to the extent that they don’t depend on the new Windows 8 common kernel. Some of the new features, in fact, are actually apps, which means that Microsoft could potentially ship them as separate standalone updates in the future.

When Tony says

After all, we’re dealing with software here and surely a few IF… THEN… ELSE conditions could be incorporated into the code to support older devices?

I’m reminded of our earlier discussion about the pros and cons of requiring an Active Directory version update for Exchange 15, in particular the observation that the test burden of software changes is a lifelong obligation. Adding a feature isn’t always hard, but once it’s added you must test it on every device and configuration for as long as you support that feature– and that is even more true here when you consider the size of the WP test matrix, which contains dozens of devices and dozens of carriers. If I were Terry Myerson, the corporate VP at Microsoft who owns Windows Phone, I wouldn’t spend the money on backporting the core, even if it were technically possible. It doesn’t make sense as an investment, nor is the ongoing cost burden supportable.

(nb. Myerson, you may recall, is the fellow who made the at-the-time extremely unpopular decision that Exchange 2007 would require x64 hardware. This was widely hailed as being an arrogant and ignorant move on Microsoft’s part, but it was actually one made for solid technical reasons, and in retrospect it has proven to be the right decision; the scalability and performance benefits of that move have been critical improvements to Exchange 2007, 2010, and 15.)

While Tony thinks of Microsoft’s move as being a “sad and arrogant indication” of Microsoft’s contempt for its mobile customers[1], I instead see it as a realistic acceptance of the fact that even the mighty Microsoft has limits on what they can feasibly accomplish. They are fighting hard to stay relevant in the mobile device market, and they’ve apparently decided that their effort is better spent on net-new development work. Based on my outsider’s view, I can’t disagree.

Having said that: yes, I’m disappointed that my two-year-old HTC Mozart (and the Lumia 800 I’m going out to buy this afternoon for a project) won’t get the Apollo upgrade.. but no more so than I am that my iPhone 4 won’t get the full flavor of iOS 6. In fact, I am probably more likely to buy a new device to run Apollo than I am to buy a new device for iOS 6. Why? Look at the delta in features; absent some major new hardware improvement, or some as-yet-hidden iOS 6 features, it doesn’t make sense to shell out for a new device just to get Siri, turn-by-turn maps, and FaceTime over 3G. But at the end of the day, this is a business decision, not one that Windows Phone customers should take personally.

[1] You know who really has contempt for their mobile customers? The carriers. Yeah, that’s right; the same ones who do things like block Windows Phone and Android updates. Of course, you can argue that as an OS manufacturer that doesn’t make its own devices, Microsoft’s customers are the carriers, not consumers. Apple’s in a different space because they sell devices both through carriers and direct-to-consumer, as do Google and the increasingly-irrelevant RIM.

4 Comments

Filed under General Tech Stuff, Musings, UC&C

Speaking at Hewlett-Packard Discover 2012

A quick reminder to those in the Hewlett-Packard-o-sphere: I’m moderating a roundtable discussion with Jeff Mealiffe from Microsoft and Donald Livengood and Stuart Ladd from H-P next week at H-P’s Discover 2012 event. Our session, RT3036, is on Wednesday morning. The abstract only hints at the Exchange-y goodness that awaits; we have an interesting list of questions to discuss, and we’ve saved a large chunk of time for audience questions. It should be a lot of fun, and I hope if you’re there you’ll drop by and say hello.

This roundtable session will assess, compare and critique the different options when deploying Microsoft Exchange 2010. Our goal is to highlight important criteria, including advantages and disadvantages, to use while planning. To provide a balanced perspective, we’ll feature technical experts from both HP and Microsoft. The session will be moderated by Paul Robichaux, a Microsoft MVP. The audience will have the opportunity to ask questions at the end of the discussion.

It’s a short trip so I won’t really have any time to sightsee, although I do plan to hit the Las Vegas Pinball Hall of Fame. H-P is holding a preview screening of Madagascar 3, which I believe was rendered on H-P hardware, but I’m going to hold off watching that until I can see it with the boys. So it’s pinball for me (though sadly they don’t have my favorite machine).

Leave a comment

Filed under UC&C

MEC 2012 call for content opens

Microsoft has just put out the call for content for MEC 2012. This gives me a bittersweet memory of the many times I sent out calls for content for Exchange Connections: the sweet because of how much fun it was to see the innovative topics and sessions proposed and to haggle over them with my conference co-chairs and the bitter because there was never a way to get every good session onto the final schedule. Now, though, I get to be on the other side of the table, proposing sessions instead of evaluating other people’s proposals.

So far I’ve proposed three sessions. I won’t find out until 25 June whether any of them have been accepted, so it’s time to settle in for the wait.

Frequently asked questions:

  • No, I’m not talking about the content of the sessions I’ve proposed because [REDACTED]
  • No, I can’t give out the link to submit session proposals; Microsoft is only soliciting content from Exchange MVPs and graduates of the MCM | Exchange program.
  • Yes, I am very much looking forward to going even if I don’t get to present.
  • Yes, I hope MEC 2013 is not in Orlando.

Leave a comment

Filed under UC&C

What "supported" really means

If I had a nickel for every time I had had a discussion like the below…

<Customer> wants to <do something>. I don’t think it’s a good idea and tried to explain that to them. They want to do it anyway. Is it supported?

The particular discussion that triggered this post was a conversation among MCMs concerning a customer who wanted to know if they could configure an Exchange 2010 server so that it was dual-homed, with one NIC on the LAN and another in their DMZ. There are a number of good reasons not to do this, most related to one of two things: the inability to force Windows and/or Exchange to use only one of the installed NICs for certain operations, or the lack of knowledge about how to configure everything properly in such a configuration. For example, you’d have to be careful to get static routes right so that you only passed the traffic you wanted on each interface. You’d also have to be careful about which AD sites your server appeared to be a member of.

The big issue for me: that configuration would add complexity. Any time you add complexity, you should be able to clearly articulate what you’re gaining in exchange. Performance, scalability, flexibility, security, cost savings.. there has to be some reason to make it worth complicating things. This is a pretty fundamental principle of designing anything technical, from airplanes to washing machines to computer networks, and you violate it at your peril.

In this case, the gain is that the customer wouldn’t need to use TMG or a similar solution. That seems like an awfully small gain for the added complexity burden and the supportability issues it raises.

You might be wondering why I’d bring up supportability in this context. The cherry on the sundae was this comment from the fellow who started the thread: “It’s not written that you can’t do it, so they assume that means you can.” This is a dangerous attitude in many contexts, but especially so here.

I’ve said it before (and so has practically everyone who has ever written about Exchange), but it bears repeating:

Just because something is not explicitly unsupported, that doesn’t mean it is supported.

Microsoft doesn’t– indeed, can’t— test every possible configuration of Exchange. Or Windows. Or any of their other products (well, maybe except for closed consumer systems like Windows Phone and Xbox 360). So there’s a simple process to follow when considering whether something meets your requirements for supportability:

  1. Does Microsoft explicitly say that what you want to do is, or is not, supported?
  2. If they don’t say one way or the other, are you comfortable that you can adequately test the proposed change in your environment to make sure that it only has the desired effects?

Point 1 is pretty straightforward. If Microsoft says something’s explicitly supported, you’re good to go. If they explicitly say something is unsupported, you’re still good, provided you don’t do it.

Brief digression: when Microsoft says something’s unsupported, it can mean one of three specific things:

  • We tested it. It doesn’t work. Don’t do it. (Example: a long list of things involving Lync device provisioning.)
  • We tested it. It works. It’s a bad idea for some other unrelated reason. Don’t do it. (Example: going backupless with a 2-copy DAG.)
  • We didn’t test it. We don’t know if it works. You could probably figure out some way to make it work.  If it doesn’t work, on your own head be it. (Example: the prior stance on virtualization of Exchange roles.)

OK, where was I? Oh yeah: if Microsoft doesn’t make an explicit statement one way or another, that is not an unconditional green light for you to do whatever you want. Instead, it’s an invitation for you to think carefully about what you’ll gain from the proposed configuration. If what you want to do is common, then there will probably be a support statement for it already; the fact that there isn’t should give you pause right there. If you believe the gain is worth the potential risk that comes from an increase in complexity, and you can demonstrate through testing (not just a SWAG) that things will work, only then should you consider proceeding.

(n.b. permission is hereby granted for all you Exchange folks out there to copy this and send it to your customers next time they ask you for something dangerous, ignorant, unsupportable, or otherwise undesirable.)

5 Comments

Filed under Musings, UC&C

MEC is back

Great news: the Microsoft Exchange Conference (MEC) is back from the dead!

During the first day of the 2012 MVP Summit, Michael Atalla (director of marketing for the Exchange team) surprised us with an announcement that the MEC was returning. We weren’t allowed to discuss it until today, but now some of the wraps are off.

I have many fond memories of attending MECs past, including taking a group of H-P’s European employees to the pistol range in Anaheim for a little American cultural acclimation. The MEC that probably stands out the most for me, though, was 1998 in Boston. It was the first MEC I attended, and at the time I knew very little about Exchange– yet I was on the hook for an Exchange exam study guide. I took feverish, furious notes and tried as hard as I could to cram knowledge into my brain. What stood out to me was that the presenters included actual engineers and support folks from the product team: Laurion Burchall gave a presentation on the internals of the Extensible Storage Engine, Daniel Chenault did an excellent troubleshooting session at Fleet Arena, and so on.

The sense of community and shared learning was palpable, and it continued on at each successive MEC. A huge amount of what I now know about Exchange, I learned there; many of the Exchange community members I now count as friends were people I first met at a MEC.

The MEC was successful in my view for 3 reasons: it focused only on Exchange; it was staffed by deeply technical presenters who could get all the way down to the source-code level to explain things; and it was well-funded and supported by Microsoft.

In its later years, MEC was renamed from the “Microsoft Exchange Conference” to the “Microsoft Exchange and Collaboration” conference, and SharePoint– then in its infancy– was invited in. Then the MEC disappeared, while SharePoint got its own conference. It long surprised me that SharePoint had a Microsoft-organized and -funded conference, while Exchange didn’t, given their relative market sizes and market shares. It also continually frustrated me that there was so little Exchange-specific content at TechEd, but that’s a function of the simple fact that Microsoft keeps adding new products, and each one has to have its day at TechEd. The same-size pie with more slices cut into it means everyone gets a smaller piece.

Sadly, the enthusiasm and commitment of the people who actually write the code for Exchange has been a long-missing ingredient in the Exchange conference world. I know that when I was the conference co-chair for Exchange Connections we tried hard, and often, to get Microsoft to send us some developers and/or support engineers. We were largely unsuccessful (although we did manage to pry Tim McMichael loose for a few visits.) This is not to demean the contributions from the many program managers, writers, and others who have carried the Exchange torch at TechEd, Connections, TEC, and so on– but as good as their participation has been, it’s not the same as having engineers presenting at a Microsoft-sponsored conference centered on Exchange. Other events have had one or two of the three factors I mentioned before, but without all three, they didn’t hit the same heights.

While Microsoft hasn’t announced any of the specifics around the MEC yet, they have announced the date and location, but you’l have to go to MECisBack.com for those details. Expect more details soon– and expect to see me there!


3 Comments

Filed under UC&C

STARTTLS, the police, and Exchange 2010

Great investigative journalism (or whatever the correct term is): security research Per Thorsheim does an experiment to see which law-enforcement organizations appear to support TLS encryption for SMTP mail. If you’re wondering why this is relevant, recall that the recent “hack” [I hate using that term since it’s not really a hack as much as it was social engineering] of the FBI-Scotland Yard conference call happened after a member of Anonymous found a conference call invitation in stolen e-mail. That kind of data can have intrinsic value that might make it attractive to an attacker, and using STARTTLS is a conceptually simple way to protect it while in transit.

TLS support in Exchange 2010 has come a long way from the bad old day of Exchange 2003, where you may recall that enabling TLS would cause the SMTP virtual server to refuse to talk to any other SMTP server that wouldn’t accept TLS. Exchange 2007 added support for opportunistic TLS, and Exchange 2010 has it too. Microsoft’s documentation makes clear how to set it up, so I encourage you to stop reading this article and just go do it.

2 Comments

Filed under UC&C

How Autodiscover works in Outlook 2011

Fellow Exchange MVP Rajith Enchiparambil, proprietor of the excellent How Exchange Works blog, asked an interesting question the other day: how does Autodiscover work in Outlook 2011? Is it different from the way Autodiscover works in Outlook for Windows?It turns out that the answer is (as you might have predicted) “it depends.” To answer that question in depth, we have to dig into the guts of Autodiscover (or AutoD, as its friends call it).

The first thing to know is that there are two parts to AutoD. One is the service that runs on Exchange 2007 and later. This service is implemented as a virtual directory named “Autodiscover” on the CAS role. When you install the CAS role, the vdir is automatically created and provisioned for you. In addition to the vdir, an Active Directory service connection point (SCP) object is created. (For probably more detail on SCPs than you’d want, see this article.)

See, in Windows Outlook, there are two primary ways that AutoD can work: domain-joined Windows machine can perform an LDAP lookup to find an AD SCP, or any machine can try to hit a predefined series of URLs. Why are there two methods? Because this design allows a computer, or device, to find the correct Exchange CAS whether it’s domain-joined or not, and whether or not it’s on the internal corporate network.

(See what I did there? I said device, because mobile devices can use AutoD also. Currently, iOS and Windows Phone 7.x devices use AutoD, as do some Exchange ActiveSync clients on some Android devices. For our purposes, we’ll treat mobile devices just like Macs insomuch as they use similar web-based queries for the AutoD vdir.)

So let’s ignore the SCP lookup process. How does Mac Outlook 2011 use AutoDiscover?

First it tries to connect to the standard AutoD URL, which is made up of the primary Exchange SMTP domain plus /Autodiscover/Autodiscover.xml. For example, https://robichaux.net/Autodiscover/Autodiscover.xml would be the first URL Outlook would try for an account in the robichaux.net domain. If that works, great. If not, it will then try tacking “autodiscover” onto the FQDN and keeping the same relative path.

If neither of those standard URL requests, both of which are made using HTTPS, bear fruit, the next attempt will be to do an HTTP request for the second URL. This request will be redirected if HTTP-to-HTTPS redirection is in use, which is what we want– if a redirection occurs, Outlook will catch the HTTP 302 response and make an AutoD request against the redirected URL.

If that check fails, the next step is to perform a DNS SRV lookup to try to find the FQDN of an Exchange CAS. If the SRV query returns a target machine, Outlook will tack on /autodiscover/autodiscover/xml to it and perform an AutoD query against the result.

Once Outlook or a mobile device gets back an Autodiscover manifest, of course, what it does with the result will vary according to its capabilities. For example, Outlook 2011 and mobile Exchange ActiveSync clients don’t (currently) use the returned URL for the target mailbox’s Exchange unified messaging (UM) server.

This process is generally pretty robust unless you’ve misconfigured the Autodiscover or service URLs on the CAS. It turns out that there’s a separate Exchange Web Services (EWS) external URL property on the CAS, and if you fail to set that properly– say, if some of your users snuck some Macs or iPads or something onto your network– then AutoD will return the EWS URL that you set, which will be wrong, so Mac Outlook won’t connect properly. The Test-* cmdlets are very useful in tracking this kind of problem down; Exchange MVP Tim Harrington has written a good primer on their use.

2 Comments

Filed under UC&C

Don’t use Symantec security software

You may know that Symantec recently admitted that its network was compromised and that the attackers got the source code to pcAnywhere, Norton Internet Security, and a few other products. Buried in their acknowledgement, however, was the fact that the source code leaked in 2006 and has thus been floating around in the community for quite a while.

Jonathan Shapiro’s response on the IP list seemed to hit the right note for me:

The pcAnywhere source code leaked in 2006, and in all that time nobody thought to do a serious security review to assess the customer exposure that this created? And now after five years in which a responsible software process would have addressed these issues as a matter of routine, they are having people turn the product off?

This is the company that ships the anti-virus and firewall software that you are probably relying on right now. A version of which, by the way, has also leaked. Do you want to be running security software – or indeed any software – from a company that fails to promptly report critical vulnerabilities when they occur and then ignores them for five years?

You can argue about whether Microsoft’s disclosure policy is perfect or not. I cannot, however, imagine a circumstance in which Microsoft became aware of a potential vulnerability and then didn’t fix it for five years.

So: if you’re running Symantec security software on your personal machine, your company’s workstations, or your servers… time to get rid of it and replace it with software from a more responsible (and, one hopes, more security-conscious) vendor.

 

1 Comment

Filed under FAIL, Security, Smackdown!, UC&C