Even Google Has the Outage Blues

As has been widely noted, Google’s popular Gmail service suffered a multi-day outage.  Google says that they were rolling out a software storage update on February 27, 2011 and ran into a bug that wiped out the e-mail of 0.02% of their customers. Fortunately, the data was backed up on tape (yes, that’s right, The Cloud is made up partly of Magnetic Tape!).

Univac magnetic tape backupThis is just the kind of event that every tech person dreads. I know that firsthand. Last month, one of our company’s hosted servers went down and did not come back up.

There is nothing quite like the first few minutes after you have discovered a massive failure.  Except for the minutes after that, and the hours after that. Nothing tests a contingency plan like a real life meltdown.

Our (former) hosting provider had sent plenty of advanced notice of pending power distribution system upgrade that would require our box to be shutdown.  Re-reading those e-mails after the fact, they seem prescient.  “We are notifying you in advance to allow sufficient time to prepare.” Blah blah blah.

The box was an old one, having run for over 9 wonderful years with the same hard drive (and no RAID configuration). There’s a word for that: stupid. But we had backups of all the customer data on the box, of all of our software, and of critical configuration files. The one thing not properly planned for was the evolution of various libraries over the years. As a result, some circa 2000 C++ code didn’t want to compile properly on the modern Linux distribution running on the replacement machine. The original compiler had been far more forgiving.

New technology to the rescue.

Thanks to virtualization, we ran some of the cranky old code on an old Linux distribution in a VM on a macbook pro laptop while we cleaned up.  Thanks VMware!

And our new configuration is very different, with thoroughly modern protections and backups.

Now all we have to do is update that contingency plan …

[Image from www.computer-history.info]

Google | Comments closed

UMD Solar Project

Standard Solar of Rockville will be installing 2700 solar panels on top of the old Washington Post printing plant in College Park.  The University of Maryland recently bought the empty building, and renamed it the Severn Building.  The Office of Technical Liaison is currently housed there.

The panels will cover 6.4 acres, can crank out up to 631 kilowatts, and are expected to generate 792 megawatt-hours over the course of a year.

Never heard of the Severn Building?  It was constructed in 1958 and is tucked out-of-the-way just south of Greenbelt Road and west of the CSX tracks.  It is not part of the main campus.

University of Maryland Severn Building

The solar project is being funded by a $630,000 capital grant from the Maryland Energy Administration Project Sunburst Initiative and by Washington Gas Energy Services, which will run the panels and sell electricity to the University over a 20-year period.  The agreed kwH purchase price was not disclosed.

[From the DiamondbackOnline, picture from Bing Maps bird's eye view]

Univeristy of Maryland | | Comments closed

Verizon 4G Conundrum: High Speed Great for Watching Movies, High Price Bad for Watching Movies

Verizon 4G Coverage Map for DCVerizon launched their 4G service in the DC area this weekend.  For now, the only 4G devices the company is selling are USB modems for the Windows operating system — one from LG and one from Pantech.

Verizon’s 4G coverage map for the DC area claims fairly complete coverage inside the Beltway and along the 270, 50, and 66 corridors (4G shown in dark red).

The data plan pricing is steep.  As the Verzion web site says, “Download movies in minutes.”  But if you download more than about 4 movies in a month on their 5GB/month plan, you’ll be hit with hefty overage charges, to the tune of about $10 to $15 per movie.  Yikes!

The 5GB/month plan runs $50 per month and the 10GB/month plan is priced at $80.  The 4G pricing matches their recent 3G pricing.  There are no unlimited rate plans.

[From VerizonWireless]

Verizon | | Comments closed

Sprint Sprinkles 4G Coverage Around DC

Sprint 4G DC Coverage - December 2010Sprint is now selling 4G coverage in the Washington DC area.  But as you can see from their coverage map (4G coverage shown in blue), your mileage may vary.  I’m in the zone thanks to a tall condo building nearby, but walk one block north and its back to 3G.

We haven’t seen Verizon’s detailed LTE map yet.

[From Sprint]

Sprint | Comments closed

ICE Goes All Homeland on 82 Web Sites

ICE Seizure NoticeU.S. Immigration and Customs Enforcement (ICE) seized 82 domain names of commercial websites allegedly engaged in the illegal sale and distribution of counterfeit goods and copyrighted works.   Sites like cheapscarfshop.com and mydreamwatches.com.  But Number 80 on the list has drawn some attention, as torrent-finder.com supposedly did not actually host this kind of content.

[From ICE.gov]

U.S. Government | Comments closed

SIPRNET Way Cooler Than Internet

Washington Moscow Hotline

E.O. 12958: DECL: 12/01/2010

WikiLeaks press-released a pending leak of 251,287 U.S. diplomatic cables fetched from SIPRNET, probably via USB-based memory stick.  Only 220 of those cables were available at the time of this post, but many more have been shared with the NY Times and other mainstream media outlets.

Fantastic reading for foreign affairs junkies.  Check it out [possibly NSFW if you are US Gov't]

[Picture from NSA with enhancement by Jerry Proc shows a far more secure network -- the Washington-Moscow hotline back in the day before USB memory sticks.]

U.S. Government | | Comments closed

AC in DC: Reeves Center Gets Public Electric Vehicle Charging Station

Charging StationThe Franklin D. Reeves Center in DC now has a public, curbside electric vehicle (EV) charging station.

It’s a 240V Level II Coulomb ChargePoint station, part of the ChargePoint network of EV charging stations.

Yes, you can charge your charge to a credit card.

Installation by PEPCO and NovaCharge, LLC of Tampa.

[Photo by Coulomb Technologies.]

D.C. Government, Electric Vehicles, PEPCO | Comments closed

Ocean City Wind Farms

Ocean WindmillMaryland seeks corporate interest in a a possible wind farm off the Maryland coast, south of Fenwick Island, DE and east of Ocean City, MD.

The Bureau of Ocean Energy Management, Regulation and Enforcement published the RFI in the Federal Register on November 9, 2010 under Docket ID: BOEM-2010-0038.

There is a cool map showing where these wind farms might go.

[From Gazette; Photo from CGPGrey.com]

Wind Power | Comments closed

All GSA Staff Social Security Numbers Accidentally E-mailed Out

A GSA employee accidentally sent the names and Social Security numbers of the agency’s entire staff of over 12,000 people to a private e-mail address.

GSA notified employees of the breach via security alert e-mails, “which employees said they received frequently and often ignored.”

Sounds like a strong culture of information security at GSA!

Casey Coleman (GSA CIO, pictured) and Gail Lovelace (GSA Chief Human Capital Officer) later sent letters via the postal mail to warn employees of the privacy breach.

[From NYTimes; photo from U.S. Government]

U.S. Government | Comments closed

Fast Customs Immigration at Dulles Airport

Global Entry MachineThe U.S. Customs and Border Protection agency has rolled out their Global Entry program to speed up the customs process at some U.S. airports.  To join the program, you pay $100, pass a government background check, and get fingerprinted.  Then, you can use a kiosk instead of waiting in line for a customs agent.

At Washington Dulles (IAD), the average processing time at peak hours (using human agents) is one hour.  Using the Global Entry machines, that average time is reduced to one minute and four seconds.

If you’ve got goods to declare, you must then go through normal inspection.  Some travelers will be spot checked the old fashioned way.

[From Wall Street Journal; image from U.S. Government]

U.S. Government | Comments closed
  • Subscribe/Follow Us:
    dailyCRUD RSS Feed icon    dailyCRUD Twitter Stream icon
  • Got a topic suggestion for dailyCRUD?