File sharing
|
|
File sharing is the activity of making files available to other users for download over the Internet, but also over smaller networks. Usually file sharing follows the peer-to-peer (P2P) model, where the files are stored on and served by personal computers of the users. Most people who engage in file sharing are also downloading files that other users share. Sometimes these two activities are linked together. File sharing is distinct from file trading in that downloading files from a P2P network does not require uploading, although some networks either provide incentives for uploading such as credits or force the sharing of files being currently downloaded.
| Contents |
History
Napster, originally a centralized service, was the first major file-sharing tool and popularized file sharing for the masses. Napster was an MP3-only sharing system and was successfully shut down by legal attacks from the music industry. It was openly attacked by some artists (notably Dr. Dre, Metallica) and supported by others (Limp Bizkit, Courtney Love, Dave Matthews). There was widespread media coverage of unreleased Madonna songs leaking out on to the web prior to the official commercial release. Napster was a localized index for MP3 files shared by the users logged into the system. It included IRC-like chat and instant messenger features. Almost all new major clients now follow its example in design.
Even before its legal problems, the community created an alternative: OpenNap. A reverse-engineered version of the Napster protocol, it was released as the open source server alternative for Napster users. These networks continue to exist even after Napster's collapse and many clients using this protocol have appeared, particularly with the help of the Napigator server list - an effort to centralize all of the different servers and networks.
Afterward, a decentralized network known as Gnutella appeared. This service is fully open source and allows users to search for almost any file type; users can find more than just MP3s on these networks. It was created in response to the threat posed toward any centralized body like Napster. The purpose behind decentralization is to prevent any single broken link from compromising the entire network.
Napster and Gnutella continue to define file sharing today, forming the extreme at both ends of the law in the wake of a series of civil lawsuits filed against computer users by the RIAA (which began in September, 2003). Gnutella is a free and open protocol and service while Napster has been resurrected as a commercial online music service that competes with other commercial services like iTunes and Rhapsody.
Most file-sharing systems since have sought to ride the line between these two extremes.
Today a variety of file-sharing programs is available on several different networks. Availability depends partly on operating system, and different networks have different features (for example, multiple-source downloads, different sorts of search limiting, and so on). It is common for commercial file sharing clients to contain abrasive advertising software, or spyware.
Network architecture
There are several major issues surrounding file sharing. Of these, the two most important are centralization vs decentralization and the privacy and anonymity of users. The latter takes on added importance when the legality of some file-sharing practices is challenged by copyright owners. A third issue is the collection and sale of data about users, using software referred to by its detractors as "spyware".
In the early days, client software was protocol-specific, so you had "Napster" clients, and you had "Gnutella" clients. There is an everpresent push towards making the GUI-side of things capable of using multiple protocols. It is argued: why should a user have to load up several different applications to do what is, in their mind, the same thing?
In cases where there is perceived value in collecting, some people will have lots to share and will find themselves surrounded by eager people. This can cause problems when the collector cannot keep up with demand. Decentralization is one means to alleviate this problem, especially in cases where it is possible to ensure that multiple copies of a popular item are available from multiple sources (even simultaneously, as with multi-source downloading).
Decentralization has also been pushed as a means of overcoming the threats posed to a centralized network, either by legal disputes or hostile users. A decentralized network has no body to attack; only its individual active members may be targeted, and even if a small portion of them are removed the remaining peers on the network will still be able to function.
Concepts like leeching or hoarding come about where the one centralized person will collect files and later refuse to make those available to others. Trade and ratio systems evolve in order to reduce the impact of leeching. Under these systems, a person shares when he can expect to get something in return. KaZaA, for instance, has a very simple rating system. The client calculates the user's priority and tells the sources what level of downloading priority they should give that user. Shortly afterward, however, hacked clients were released that told the sources that the user had one of the highest priority levels regardless of his actual sharing.
Another client which has a rating system is eMule. The eMule client, which uses MFTP as its protocol, tracks how much downloading and uploading has been done from individual sources and if files are downloaded locally or if other peers download files. Sometimes it seems that this rating system does not have a big impact on the download speed. A reason could be the size of the upload queue and the chunk size. If there is a free upload slot, the client takes the peer on top, transferes 8 MB to it and moves it to the end of the queue. A peer with rating of x2 would have to wait to get an upload slot for only half of the amount of time of a peer with a rating of x1. Furthermore, after the client has received an 8 MB chunk, it should upload an 8 MB chunk to the other peer as soon as possible if there is a download pending for that user. Then the other client would upload one chunk to you and your download speed and the one from the other client will increase.
BitTorrent also has a very good share rating system. The download speed is slow if a client does not upload, but it can easily be the fastest protocol if the size of the swarm is large enough.
Today we are left with a slew of clients with functionality designed around making sharing files more effective, both in the real sense of uploading and downloading (like anti-leeching functions) and in the more ethereal sense of being bulletproof toward legal issues (as with anonymity and decentralization).
Copyright issues
Anonymous Internet file sharing (such as Gnutella and Napster) grew in popularity with the proliferation of high speed Internet connections and the (relatively) small file size and high-quality MP3 audio format. Although file sharing is a legal technology with legal uses, some people (in practice, the great majority of users) have used it to download copyrighted materials. This has led to counterattacks against file sharing in general from some copyright owners.
There has been great discussion over perceived and actual legal issues surrounding file sharing. In circumstances where trading partners are in different countries with different legal codes, there are significant problems to contend with. What if a person in Canada wishes to share a piece of source code which, if compiled, has encryption capabilities? In some countries, a citizen may not request or receive such information without special permission.
Throughout the early 2000s, the entire file-sharing community has been in a state of flux, as record companies and the RIAA tried to shut down as much of it as possible. Even though they have forced Napster into cooperating against copyright violations, they are fighting an uphill battle since the community has flourished and produced many of different clients based on several different underlying protocols. The second generation of P2P protocols, such as Freenet are not as dependent as Napster is on a central server, making it much harder to shut down these systems through court actions. Another attempt (used by the maintainers of KaZaA) is to change the company's organization or country of origin so that it is impossible or useless to attack it legally.
The EFF is a donor-supported group which protects the digital rights of mankind. It is one of the most influential online human-rights bodies, and it is involved in legislation, court cases, and campaigns to make the public aware of their rights. The EFF has opposed the RIAA in its onslaught of law suits against users of file-sharing applications. The foundation supports the idea that P2P file-sharing can exist while allowing users to compensate artists for their copyrighted material.
Widely-used protocols
There are a number of filesharing protocols. Below are the most popular or most important ones.
Ares
Ares is a popular network that uses a protocal similar to Gnutella. The standard (and most popular) Ares client had the sixth best household coverage of P2P clients in the US during March 2005 (see [1] (http://www.npd.com/dynamic/releases/press_050607.html)). It is mainly used by people unaware of better networks, and those who wish to be on as many networks as possible.
BitTorrent
The BitTorrent protocol is the only filesharing protocol in widespread use that has been integrated into large-scale corporate distribution schemes. It tends to offer greater download speeds due to the fact that the tracker for each torrent contains client information for one specific file or set of files: there is a reduction in bandwidth utilization due to protocol overhead compared to other protocols, but because every peer connected to the tracker is sharing and downloading the exact same file(s) there is no search feature in any of the existing clients. BitTorrent is also less flexible and is not a purely peer-to-peer application since the tracker URL must resolve and the tracker must provide filesharer information before any file data can be found and transferred.
During early 2004 CacheLogic found that this network carried far more data than any other internet filesharing network [2] (http://www.cachelogic.com/research/slide3.php). Many people believe this is still the case.
Direct Connect
Direct Connect is a protocol commonly used on peering networks and in university residences. It is particularly good at sharing files in these and other "small network" environments. DC++ is the most common client.
eDonkey, Overnet and Kad Network
eDonkey is one network used by the popular P2P clients eDonkey 2000 and eMule. It is used to transfer a large amount of data, second only to BitTorrent [3] (http://www.cachelogic.com/research/slide3.php). It's main advantage over BitTorrent is that a comprehensive search engine is built into clients. A much broader range of files can be found with it than can be on any BitTorrent search engine. It's main disadvantage against BitTorrent is that file transfers are generally slower.
Overnet was designed as a replacement for the eDonkey network, and many eDonkey clients simultaneously use it. Unlike eDonkey, it is decentralised - there are no stand-alone servers. However, it has yet to surpass it.
The Kad Network is very similar to Overnet, and is also almost exclusively used by clients that primarily use eDonkey. Similarly, it's popularity is relatively low.
FastTrack
FastTrack, often referred to by the name of its first client, Kazaa, utilizes a protocol similar to Gnutella2. For a period around 2002-2003 it had roughly the largest userbase of all networks. However, as of 2005 it is presently third and dropping.
This drop in popularity is due to the protocal not being able to handle large numbers of users, and the network being clogged with corrupted and dummy files placed by rights-holders to disrupt the trade of copyrighted songs.
The Kazaa client installs spyware onto the user's machine. The creators of the spyware-free Kazaa Lite were ordered to cease distribution after a judge ruled that it violated copyrights held by Sharman Networks. Whether or not the presence of spyware outweighs the benefits of the client software is a decision for each individual user to make, and reading about spyware and the related security and privacy risks can only help potential users make the best choice based upon their own needs.
The popular iMesh client also uses FastTrack as it's primary protocal. The open source GiFT plugin is another way to connect to the FastTrack network.
Gnutella and Gnutella2
Developed by Justin Frankel of Nullsoft while Napster was undergoing legal battles, Gnutella was the first truly decentralized client. Since then many advances in the protocol have been made, and Gnutella has been a predominant influence in the world of filesharing.
Originally intended as a means of extending and improving upon Gnutella, Gnutella2 was developed by Michael Stokes, creator of Shareaza. It added an extensible binary packet format, UDP-based search algorithm, and many features originally proposed as additions to Gnutella, such as partial file sharing and packet compression, which alleviated many of the problems caused by outdated clients on the network. Nowadays, modern Gnutella clients support all of these features, and many more, including firewall-to-firewall transfers, are being tested.
Popular clients for these networks include:
OpenNap (was Napster)
OpenNap is a modified version of the protocol originally used by the Napster client, which was the first widely used peer-to-peer filesharing application. It is still developed and used in applications such as WinMX.
WinNY
This is an unofficial name for the protocol used by the WinNY and Share clients. This protocol is the most popular in Japan (2005). It is rarely used outside Asia.
WPNP
WPNP is the main protocol used by WinMX. This network offered the best results for finding obscure English language material for a period around 2001-02, but it's main use is for asian language material.
In March 2005, The NPD Group found WinMX was used in more US households than any other P2P client or legal music download service [4] (http://www.npd.com/dynamic/releases/press_050607.html). According to some sources, it is also the second most popular client in Japan.
Popular web sites communities
- InfoAnarchy - File Sharing / Anonymity Tools - Reviews and forums.
- ShareReactor - Was a Web Site that helped download confirmed, public files from the Multisource File Transfer Protocol used by such programs as Edonkey2000 and eMule. It was taken down by Swiss Police on March 10th, 2004 due to the suspicion of breach of copyright and trademark laws.
See also
External links
Portals
- The File Sharing Portal (http://filesharing.bz/) File sharing programs reviewed, rated, and download links.
- MP3 Newswire (http://www.mp3newswire.net) - One of the oldest digital media news sources with extensive coverage on file sharing and technology.
- Findhash.com (http://www.findhash.com/) - A Web Site indexing media content for many different filesharing client using quicklinks (AKA hashlinks)
- Hack the Planet (http://wmf.editthispage.com/) - Wes Felter comments frequently on P2P-related issues in his weblog.
- OpenP2P (http://www.openp2p.com/) - O'Reilly's Peer-to-Peer portal with news and papers. Interesting for developers and businesses.
- Nanotechnology P2P (http://www.nanop2p.com/) - Programs, News, Forums
- Peer to Peer guide for the Mac (http://www.mac-p2p.com/) with overview of p2p-clients for Mac OS X.
- Planet Peer (http://www.planetpeer.de) - Planet Peer is a comprehensive portal for mainly anonymous file-sharing programs like MUTE, Freenet and Entropy. We are hosting the official MUTE Wiki also and have a forum which deals with everything concerning issues with anonymous file-sharing tools
different filesharing client using quicklinks (AKA hashlinks)
- The File Sharing News (http://www.p2pinfo.net/)
- Slyck (http://www.Slyck.com/) - Covering file-sharing and copyright issues along with new and established clients by popularity, size, platform, and more.
- Sharelive.com (http://www.sharelive.com) - A Web Site that helps download confirmed, public files from multiple networks.
- Zeropaid.com (http://www.zeropaid.com/) - The File Sharing Portal - Programs, News, Forums
Mailing lists
- Decentralization - Implications of the end-to-end principle (http://groups.yahoo.com/group/decentralization) - A mailing list exclusively devoted to the architecture of P2P-systems, with a lot of traffic and interesting ideas passed around. Founded by Lucas Gonze, with lots of high-profile subscribers from the P2P field. Open since July 2000, and still running past Nov 2002.
- The GDF (http://groups.yahoo.com/group/the_gdf/) - The Gnutella Developer Forum is a group for discussing extensions of the Gnutella protocol. It is the largest standards body for the Gnutella network, and anyone working on a servant should be a member.
- P2P Hackers (http://zgp.org/mailman/listinfo/p2p-hackers/) - A general mailing list about peer-to-peer software development.
Newsgroups
Papers, articles, and infant projects
- The Napster Experience (http://www.napsterresearch.com) - Critical consumer research (Markus Giesler, York Universisty) analyzing file-sharing as a form of gift giving and consumer emancipation.
- Is MP3 Music a Perishable Product? (http://web.archive.org/web/20031217191730/www.mp3newswire.net/stories/2000/buycd.html) - February 2000 essay from MP3 Newswire that first explored the theory that consumers shared files to sample music before purchasing.
- Courtney Love does the math (http://archive.salon.com/tech/feature/2000/06/14/love/print.html) - An extensive article for Salon.com written in 2000 by Courtney Love, addressing music sharing, the state of the recording industry and artists' frequently poor relationships with their labels
- A Method of Free Speech on the Internet: Random Pads (http://www.eleves.ens.fr:8080/home/madore/misc/freespeech.html) (by David A. Madore) - Discusses how information can be completely separated from its creators by XORing it with chunks of random data. The resulting "pads" can then be distributed across so-called "pad archives". A pad archive neither knows what it is hosting nor does it host provably controversial data, since the data cannot be distinguished from noise. It's mainly a legal question: If the courts would outlaw hosting random data, it wouldn't work. Other than that, it's pretty safe -- interesting read, and there are already quite a lot of pad archives (thanks to Slashdot).
- The Free Haven Project (http://www.freehaven.net) - Similar goals to Freenet, with different solutions. Some interesting papers. Not much code yet.
- The Big Hack: Home of the OFF System (http://thebighack.org) - A crusade to create a peer-to-peer system of file exchage where everything exchanged is legal. All that is sent over the net is random numbers. Files are then reconstructed on your computer out of the building blocks that you downloaded. Everything comes in multi-use blocks of 128 kb. Simply explained, everything online is a number (1's and 0's). Big numbers, granted, but a number nonetheless. To keep it easy, we'll pretend the numbers are small instead of large, and we'll use base ten, as opposed to base 2 (Binary). Now, if Britney Spears copyrighted the number 7, what are the legal procedures for someone who rips 7, divides it in two, and transmits the code 3.5 + 3.5 to his buddy? Or transmits 10 - 3? Can she copyright all numbers? If so, would that make Rob Zombie liable if he created the song 14 (which happens to be 7 + 7...)? As you can see, things get confusing quickly. They're always looking for help at TBH (http://thebighack.org), so if you've the programming skill, head on over.
- OceanStore (http://oceanstore.cs.berkeley.edu/) - OceanStore is "designed to span the globe and provide continuous access to persistent information". "Data is protected through redundancy and cryptographic techniques. To improve performance, data are allowed to be cached anywhere, anytime. Additionally, monitoring of usage patterns allows adaptation to regional outages and denial of service attacks; monitoring also enhances performance through pro-active movement of data. A prototype implementation is currently under development."
- Fling (http://fling.sourceforge.net/) | Sourceforge page (http://sourceforge.net/projects/fling) - An attempt to provide anonymity on the protocol level (i.e. replace TCP/IP). Still in the planning stages as of Nov 2002.
- Ben Houston's P2P Idea Page (http://www.exocortex.org/p2p/index.html) - Ben Houston has written a lot of interesting analyses of distributed systems, among them proposals for more efficient, self-organizing and self-optimizing networks.
- The Eternity Service by Ross Anderson (http://www.cl.cam.ac.uk/users/rja14/eternity/eternity.html) - This paper is a rather simple suggestion for a redundant, anonymous storage system with payment features.
- Intermemory Project (http://www.intermemory.net/) - Aims to create "large-scale, self-organized, survivable, available, and secure widely-distributed storage". See papers.
- Who's on First Proposal (http://skuz.net/potatoware/wof/) - This page introduces the Who's On First (WOF) anonymous network, which is the working title of a proposal for a more flexible and reliable anonymous communication network than that provided by current Type I and II remailers.
- The Darknet and the Future of Content Distribution (http://crypto.stanford.edu/DRM2002/darknet5.doc) (Microsoft Word format) by Peter Biddle, Paul England, Marcus Peinado, and Bryan Willman of Microsoft.
- Peer-to-peer goes legitimate (http://news.bbc.co.uk/1/hi/programmes/click_online/4104827.stm) (BBC News, 17 December 2004) – The article explains how some new and established artists are using file sharing to market themselves and then how they can use pay-sites to make a living, all without the involvement of record companies.
- See You on the Darknet (http://slate.msn.com/id/2094336/) (Slate, 28 January, 2004).
- Research: File Traders Buy Records (http://web.archive.org/web/20031207105918/www.mp3newswire.net/stories/2002/teentrade.html) - MP3 Newswire examines pro-record industry research that claims file sharing threatens the industry and finds the numbers show just the opposite, that file sharing induces users to buy CDs.
Sources
This article was partly based on public domain material from the infoAnarchy wiki.
Template:Cyberspaceda:Fildeling de:Tauschbörse it:File sharing he:שיתוף קבצים pt:Compartilhamento de arquivos ru:Совместное использование файлов sv:Fildelning