Telco 2.0 News Review: Amazon Autopsy and Handling Streaming Video Spikes
Telco 2.0 Top Stories
- Cloud Computing: Amazonopocalypse: the lessons learned
- Online Video: Handling streaming video spikes: HOWTO and HOWNOTTO
- Broadband Connectivity: AT&T and T-Mobile - the spectrum wars
- Personal Data 2.0: Second massive data leak at Sony
- Regulation: Smartphones with “May spy on you” stickers
[Ed. The next EMEA Brainstorm is in London next week from Weds 11th to Friday 13th May, covering cloud, online video, digital entertainment, mobile internet, M2M, Apps, strategy, transformation, and personal data. See you there!]
After last week’s Amazonopocalypse, the clean-up effort is well under way, as are the recriminations. Amazon Web Services’ own report into the outage is here, basically confirming the view the last Telco 2.0 Review took on it. It seems that an internal router got misconfigured, dumping all the primary network traffic into a secondary network, which was swamped. That in turn led to a cascade failure of the control plane which manages the Elastic Block Service across multiple availability zones. Read it for much technical detail about the problems of really big cloud architectures.
It looks like Amazon is undertaking a considerable effort to explain more about the fundamentals of their systems to applications developers - their new Architecture Centre will be holding a succession of conferences on how to design services to work well with AWS. There’s more How I Won the War content here and here - the general tone of shock that AWS Availability Zones aren’t anywhere near as independent as advertised seems to be the same everywhere.
Jeff Atwood notes that Netflix, one of the heaviest AWS users to survive, uses a program called “Chaos Monkey” to randomly turn things off in order to test their systems’ stability. There’s a timely presentation on their architecture here. High Scalability’s list of articles is still being updated, but although there’s lots of How I Won the War out there, there are relatively few How We Completely Failed to Cope pieces - which is surely a missed opportunity.
The closest to that is probably this thread on the AWS Developer Forum from the customer whose cardiology monitoring application was entirely reliant on the US-East AZ. Whoops. Sensible points are made here.
Meanwhile, learn how to serve live streaming video from AWS. If you dare.
Last Friday’s UK Royal Wedding led to very heavy video streaming over the Internet. As a result, this classic NANOG discussion ensued on the challenges of Internet TV, the practicalities of running IP Multicast at very high scale, and the surprisingly complicated ways in which a very sensible technical solution interfaces with the law as regards the contracts between TV stations, content providers, and advertisers.
It seems that a major barrier in global multicast deployment - which everyone agrees would be a huge step towards solving the problems of delivering online video - is that TV stations want to count viewers and the terms of existing contracts enforce that this be done in a way analogous to cable TV. Who knew? There’s obviously an opportunity here.
The death of Osama Bin Laden lead to a very different global TV event this weekend that could have posed performance issues for major video-streaming Web sites. ZDNet UK has a detailed post on technical solutions for live video streaming of this event to its specific and very particular customer group.
Akamai reported that it caused very high levels of traffic, but nothing compared to those achieved during the 2010 World Cup. It looks like the global public prefers football to either monarchy or war, which is probably good news for the future of humanity.
Whatever the future is like, the present still has spammers.
More IP transit consolidation - CenturyLink, which already owns Qwest, just bought Savvis, making that the second merger between Tier-1 ISPs in a month. Renesys notes that this is likely to save a significant number of IP addresses. And here’s some practical experience with IPv6 and a beer festival.
Brough Turner has a useful discussion of paid peering between Google and France Telecom, although you might be forgiven for thinking that the bandwidth wars are a bit of a secondary issue after last weekend didn’t bring the Internet crashing down. Cisco thinks so too, forecasting that mobile data growth will converge down to the rates typically seen on the fixed Internet.
For its part, FTel hit its numbers for the first quarter, with a steady showing in France and good performances from its mobile operations worldwide.
In other operator news, Verizon’s LTE network experienced a major outage. Fortunately, the iPhone 4s were unaffected because they don’t have LTE…
It is, however, worth pointing out that, according to the NANOG thread we quoted, Verizon Wireless’s V-CAST media service is fully multicasted. As so often recently, VZ seems to have a distinct technical lead. (For example, they’re cross-marketing e-health applications with their FiOS fibre product.)
Meanwhile, AT&T’s HSPA uplink speeds compare poorly with T-Mobile’s, and CNET’s Marguerite Reardon has a very detailed round-up of the spectrum issues surrounding their merger. AT&T is being very bullish about the population coverage it could achieve with the additional spectrum, but are they really making use of the spectrum they already have? Also, it’s suggested that the speed gap is down to T-Mobile having been quicker to invest in backhaul fibre.
In the UK, the planned Sky/TalkTalk fibre network has had the desired effect: BT Openreach announced that its infrastructure access pricing is coming down.
Here’s some M2M news: in Australia, someone stole the USIM from a smart electricity meter and ran up hundreds of thousands of Aussie dollars’ worth of calls. Why voice calls were even provisioned is an interesting question - as is just what Telstra was charging the utility company for its connectivity. That’s a lot of calls. The case, as they say, continues. (In comments, someone asks what would happen if the thief swapped an ordinary retail SIM into the meter…)
In other Australian news, Vodafone is trying to patch up after the outages by offering everyone 12 hours of free SMS.
Outages have been a theme lately, and Sony’s PlayStation Network crisis is still in the headlines. They are cautiously preparing to restart the service, after two weeks of downtime, and are planning to give away a lot of stuff in an effort to win back the users. The rebuild includes physically moving to a new data centre, although this is less dramatic than it sounds - they were already going to move.
On the other hand, they also discovered yet a further massive data leak, with another 25 million users’ data being compromised, including tens of thousands more credit cards. Meanwhile, the hackers have put the stolen card numbers on sale. Frauds are already being reported. What a mess.
There is speculation that the hackers might have been able to control the update process and might have intended to use a botnet of PlayStations to attack SSL keys.
Meanwhile, RIM issued a profit warning, as sales were both disappointing and skewed towards cheaper devices. A general sense of tension was apparent. However, very early reports on PlayBook sales were good. There’s more at ATD, but it’s worth pointing out that we’re still in the “going to shops and looking” phase.
Phone Scoop reviewed the new Bold 9900 and was impressed. Its new OS is apparently now to be known as BlackBerry OS 7 and isn’t backwards compatible, although this isn’t the new QNX-based platform yet as far as we know.
RIM demonstrated that it has a few more tricks up its sleeve, by announcing a new version of the BlackBerry Enterprise Server that can manage fleets of iOS and Android devices as well as BlackBerries.
In the latest news from the smartphone horse race, it looks like Apple and RIM are doing well and Nokia is suffering. At a different level, that of the handset vendors rather than the platforms, HTC is seeing spectacular growth rates.
Horace from Asymco, of course, remains the world’s most ebullient Apple bull. He may well have a point - Time, Inc. agrees to let its subscribers read their magazines on the iPad after all.
Not surprisingly, after Apple and Android were caught collecting enormous volumes of user location data, it turns out Windows Phone does it as well. As a result of the whole affair, Verizon Wireless wants to put a health warning sticker on every smartphone it ships advising you that it could be spying on you and to consult the user manual for advice on how to turn it off.
It could end up like cigarette packets, although how you’d implement that remains unclear. Perhaps a picture of a typical user at the moment a really creepy targeted advert leaps out at them (well discussed here)?
Microsoft’s Q3 numbers are out and everyone noticed that consumer PC shipments fell 8%, with netbooks off 40%.
If Microsoft was the iconic company of the PC years, ARM Holdings would be the one of the embedded and mobile future. If anyone outside the trade had any idea it existed. Their Q1 is in, with revenues surging 30% and margins over 40% as the world fills up with ARM-powered smartphones, tablets, TVs, STBs, cars, you name it.
CNET’s new The Social blog, for which they hired Caroline McCarthy away from Google, reports on a new blog which is trying to come up with new features for Facebook. What strikes us, so far, is that quite a few could be implemented with something like Greasemonkey, purely on the user side (you could call it Acebook, but you won’t make any money), and quite a few others (drag-and-drop group chat) already exist in Skype. Voice, after all, is the original social application.
Thanks to the wide range of voice APIs that are now available, meet OpenVoice, an open-source clone of Google Voice that provides a hackable API and full SIP support. There’s some detail here, with links to the code. So far it uses Tropo for the voice back-end, but the developers want to make it possible to hook up any media server and any carrier.
Google has added video to the Android Google Talk application, and here’s a good discussion from Phil Wolff.
Should you trust SSDs with your data? Jeff Atwood says you shouldn’t, because the failure rate seems to be terrible, but that they’re so fast it’s worth it.
High Scalability makes the excellent point that the real opportunity to compete in the cloud is the admin process involved in building, deploying, managing, and shutting down your systems. Amazon knows this (see CloudFormation). Also in the cloud, Data Center Knowledge reports on Cisco’s new containerised data centre.
Microsoft claims that this tool will help you port iOS apps to WP7. Alcatel-Lucent, meanwhile, is giving away the OpenPlug suite that lets Adobe devs automatically generate compiled binary files for iOS, Android, and basically anything else.
Shame, Adobe and Skype. The police take over and disable a botnet. Are the specifications of mobile device speakers changing our musical culture? Driving the Arduino hardware hacker toolkit with your iPhone.