|
Centralization and consolidation was and is high on any CIO's wishlist. All this centralization and consolidation offers tremendous advantages but also imposes new challenges to IT infrastructures. So moving everything to the datacenter and managing everything from the datacenter is great but what if you need to move data out of the datacenter? Say if you want to deploy a desktop image to 6500 desktops in 11 other locations? What if you wanted to do that in under four hours? Short of having 100 Gbps connetions between all locations this would prove to be impossible.
Indeed, when everything has been moved to the datacenter, it becomes a huge challenge to perform large scale deployments over a WAN connections. This should not be underestimated. Even WAN connections with plenty of bandwidth (for example 100 Mbps) will not be able to deploy a new Vista image to a hundred workstations or provide fifty desktops with a new Softgrid cache. A solution would be to place a server or “caching device” at every location but this unfortunately is very much not in compliance with the centralization and consolidation strategy that most companies are yielding. This is a problem that exists at almost every large enterprise that has several connected large locations. So what now? Well, sometimes you run into a solution that it as simple as it is brilliant. And this time the solution was found in a place that you wouldn't normally look at to solve your Enterprise IT challenges: in programs that are extreme popular under the 12-18 year olds... When you think about the problem I just described, it really boils down to this: why does that particular desktop image (or any other large piece of data) have to traverse the WAN once for every single desktop? What if it only traversed the WAN only once for all desktops? I know what you are thinking: Multicast. Well, Multicast is a possible solution but it is however very vendor / product dependant and has other distinctive disadvantages (see later on in this article).
No, the solution that I ran into to make that image traverse the WAN only once was BitTorrent. BitTorrent? Doesn’t that have something to do with updating MP3 collections and popups? Well yes, that is one way the technology has been used. And for good reason. For those of you unfamiliar with BitTorrent, here’s what it’s all about: BitTorrent is a protocol that enables fast downloading of large files using minimum bandwidth. BitTorrent maximizes transfer speed by gathering pieces of the file you want and downloading these pieces simultaneously from other devices who already have them. This process makes very large files (such as images or large directories) download much faster than is possible with other protocols. When you translate this to a deployment scenario, this would mean that the image would be sent across the WAN -ONCE- to one desktop. This desktop will then function as a source (a so called seeder) from this point on the image will be deployed to all the other desktops in “bits”. The more desktops that have (part of) the image, the faster the transfer will go. Remember, if properly setup, this last transfer is LAN only where bandwidth is usually plentiful.
Isn't this more for the computer enthusiast part-timing as a systems administrator at the local library? Well no. I've seen this technology being used in of the largest Universities in the Netherlands, "INHOLLAND ". The BitTorrent deployment system was used to distribute a 3,5 GB desktop image to 6500 desktops in 16 locations in under 4 hours. This ammounts to 22,2 TeraBytes of data! These are some pretty impressive numbers. Not only was the BitTorrent deployment system able to achieve the aforementioned results, it also was completely integrated into SMS 2003. So the deployment was initiated from SMS2003 and the status of every single desktop was also reported back to SMS 2003. So how's that for Enterprise readiness? Here are some screenshots:
I have to admit that the implementation I am referring to was completed a year ago and posed some real challenges and demanded some pretty hefty pioneering. Since then, BitTorrent Enterprise deployment has come a long way. The brain behind Enterprise deployment via BitTorrent, is a man called Leo Blom from a company called ITeleo. ITeleo & iQuinox (a Dutch company ITeleo partners with) are working together on a new product codenamed BitRain-Deployment. This product is completely redesigned and build in .NET V3.0. This product cannot be compared to the solution that was built for the INHOLLAND University. BitRain-Deployment is a product with loads of improvements and extra functionality.
Using BitTorrent for Enterprise deployment is an concept that definitely takes some getting used to. I know I needed some time to get used to the concept. To this end, Leo Blom came up with the following FAQ.
- Is the BitTorrent protocol reliable?
150 million internet users are making use of the BitTorrent protocol. The protocol itself exists since 2001. Therefore it can easily be appointed to the 'proven technology' category.
- What about the integrity of my data?
The reliability of BitTorrent is exceptionally good. The integrity of data is checked through the SHA1 hashing algorithm.
- What is the maximum network speed?
This depends on your infrastructure. The BitTorrent protocol is able to use the maximum available bandwidth that your desktops can handle. This is recommended when installing new desktops. Speeds of 11 Mbyte per second are possible on a 100 Mb Full Duplex connection. BitTorrent operates on a lower speed (typically 1 Mbit upload and 1 Mbit Download) when desktops are already installed to not disturb the end-user during production hours. This setting can be configured.
- How can a 'bare metal' desktop be installed so fast?
All surrounding desktops deliver fragments of data. There are only 10 'neighbor' desktops needed to reach a full speed installation. Another important fact is that the applications are already available (MSI's) before the OS is installed. Compare it as if we always have a 'pole position' on our competitors.
- Isn't multicast operating much better?
No......but multicast was (until now) the only alternative for massive OS-deployments. The BitTorrent protocol is more reliable because every desktop is operating autonomous. A network interruption or a complete power failure can be catastrophic during a multicast deployment. The BitTorrent solution takes care of the data deployment first. Installation of the OS or application is possible once the data is complete.
Another disadvantage of a multicast solution is that the maximum speed depends on the slowest network component. The multicast session will be negatively influenced if one NIC (network interface card) is malfunctioning (causing high delays when installing hundreds or even thousands of desktops).
Multicast solutions are not usable for application distribution.
- But what happens with a distribution if a desktop is restarted or shutdown?
BitTorrent just continues with the distribution after a restart of the desktop. Only the last fragment will be retransmitted because it is incomplete. BitTorrent just continues with the distribution after a restart of the desktop. Only the last fragment will be retransmitted because it is incomplete.
- Are Zero Touch Deployments also possible?
Yes......that is the strength of this design. The desktop can be installed in any way. Prerequisite is that a PXE server must be available at first boot.
- Can we use BitTorrent for free?
No.....BitTorrent is only free for home use. BitTorrent is not free of charge when it is used for commercial activities. (BitTorrent Trademark Guidelines).
- How many desktops can you install at once?
The record is 2000 at the INHOLLAND University. The design of the deployment method has been further improved. There are no limits anymore for the number of desktops to be deployed at once.
- Does a desktop management system have any value after a BitTorrent implementation?
Yes.....your desktop management system is now doing just where it is supposed to do.....namely making inventories and reports. The data distribution process is transferred to the BitTorrent protocol.
- Are you going to offer BitTorrent for desktop management purposes?
Yes......ITeleo & iQuinox are working together on a new product named BitRain-Deployment. This product is completely redesigned and build in .NET V3.0. This product cannot be compared to the solution we have build for the INHOLLAND University. We have added loads of improvements and extra functionality. Furthermore are the contacts intensified with BitTorrent Inc.
You can read the full version of this FAQ at ITeleo.nl. For Dutch native speakers, there's also an article published in the "Automatiseringsgids". (membership required)
|
Hello,
Just would like to correct an item in this article... Michael suggests that using Bittorrent is not free.
You need to differentiate between the functionality (ie Protocol, some applications) and the Trademark. The Trademark Bittorent is protected, but the technical functionality is Open Source and publicly licensed.
Not only is there the semi-notorious Public Torrent (note I'm not saying Bittorrent) network where practically everything is shared and shareable, many have bult private torrent networks as desribed here to take advantage of a technology millions if not billions have been spent trying to break unsuccessfully. You can make a network "private" as simply as deploying your own Tracker and restricting discovery through your Tracker, but it's also possible to design a deployment that adheres to higher/highest security standards.
I myself have built one of the relatively few networks with a modified Torrent protocol to ensure secure private distribution of content over public networks.