Grid computing
|
|
Grid computing uses the resources of a many separate computers connected by a network (usually the internet) to solve large-scale computation problems. The SETI@home project, launched in the mid-1990s, was the first widely-known grid computing project, and it has been followed by many others, covering tasks such as protein folding, research into drugs for cancer, mathematical problems, and climate models. These projects work by running as a screensaver on users' personal computers, which process small pieces of the overall data while the computer is not being used.
Grid computing offers a model for solving massive computational problems by making use of the unused resources (CPU cycles and/or disk storage) of large numbers of disparate, often desktop, computers treated as a virtual cluster embedded in a distributed telecommunications infrastructure. Grid computing's focus on the ability to support computation across administrative domains sets it apart from traditional computer clusters or traditional distributed computing.
Grid computing has the design goal of solving problems too big for any single supercomputer, whilst retaining the flexibility to work on multiple smaller problems. Thus grid computing provides a multi-user environment. Its secondary aims are: better exploitation of the available computing power, and catering for the intermittent demands of large computational exercises.
This implies the use of secure authorization techniques to allow remote users to control computing resources.
Grid computing involves sharing heterogeneous resources (based on different platforms, hardware/software architectures, and computer languages), located in different places belonging to different administrative domains over a network using open standards. In short, it involves virtualizing computing resources.
Grid computing is often confused with cluster computing. The key differences are that clusters are homogeneous while grids are heterogeneous; also, grids spread out and encompass user desktops while clusters are generally confined to data centers.
Functionally, one can classify grids as:
- Computational Grids (including CPU scavenging grids), or as:
- Data grids.
| Contents |
|
|
Three definitions of Grid Computing
There are three common uses of the term:Grid computing:
- CERN, who were key in the creation of the World Wide Web, talk of The Grid: "a service for sharing computer power and data storage capacity over the Internet.".
- Pragmatically, grid computing is attractive to geographically-distributed non-profit collaborative research efforts like the NCSA Bioinformatics Grids such as BIRN: external grids.
- Grid computing is also attractive to large commercial enterprises with complex computation problems who aim to fully exploit their internal computing power: internal grids.
The Global Grid Forum
The Global Grid Forum (GGF) has the purpose of defining specifications for grid computing. The Globus Alliance implements these standards through the Globus Toolkit, which has become the de facto standard for grid middleware. As a middleware component, it provides a standard platform for services to build upon, but grid computing needs other components as well, and many other tools operate to support a successful Grid environment. This situation resembles that of TCP/IP: the usefulness of the Internet emerged both from the success of TCP/IP and the establishment of applications such as newsgroups and webpages.
Globus has implementations of the GGF-defined protocols to provide:
- Resource management: Grid Resource Allocation & Management Protocol (GRAM)
- Information Services: Monitoring and Discovery Service (MDS)
- Security Services: Grid Security Infrastructure (GSI)
- Data Movement and Management: Global Access to Secondary Storage (GASS) and GridFTP
A number of tools function along with Globus to make grid computing a more robust platform, useful to high-performance computing communities. They include:
- Grid Portal Software such as GridPort and OGCE
- Grid Packaging Toolkit (GPT)
- MPICH-G2 (Grid Enabled MPI)
- Network Weather Service (NWS) (Quality-of-Service monitoring and statistics)
- Condor (CPU Cycle Scavenging) and Condor-G (Job Submission)
Most of the grids which span research and academic communities in North America and Europe use the Globus Toolkit as their core middleware.
XML-based web services offer a way to access the diverse services/applications in a distributed environment. As of 2003 the worlds of grid computing and of web services have started to converge to offer Grid as a web service (Grid Service). The Open Grid Services Architecture (OGSA) has defined this environment, which will offer several functionalities adhering to the semantics of the Grid Service.
Grids offer a way to solve Grand Challenge problems like protein folding, financial modelling, earthquake simulation, climate/weather modelling etc. Grids offer a way of using the information technology resources optimally in an organisation. They also provide a means for offering information technology as a utility bureau for commercial and non-commercial clients, with those clients paying only for what they use, as with electricity or water.
Commercial grid computing offerings
Computing vendors have, in the 2000s, begun to offer grid solutions which are either based on the Globus Toolkit, or their own proprietary architecture. Confusion remains: in that vendors may badge their computing on demand or cluster offerings as grid computing.
Key vendors in grid computing:
- Parabon Computation (http://www.parabon.com)
- DataSynapse (http://www.datasynapse.com)
- IBM Grid Computing website (http://www.ibm.com/grid)
- Sun Microsystems Grid Computing website (http://www.sun.com/grid)
- Oracle Corp. "Oracle Grid" (http://www.oracle.com/grid)
- HP Grid Computing (http://www.hp.com/techservers/grid/index.html)
- United Devices (http://www.ud.com)
- 1st Port for Grid Computing (UK) (http://www.1stport.co.uk)
Conceptual Framework
Grid computing reflects a conceptual framework rather than a physical resource. The Grid approach is utilized to provision a computational task with administratively-distant resources. The focus of Grid technology is associated with the issues and requirements of flexible computational provisioning beyond the local (home) administrative domain.
Historical sweep
Like the Internet, the Grid concept evolved from the computational needs of 'big science'. The Internet was developed to meet the need for a common communication medium between large, federally funded, computing centers. These communication links led to resource and information sharing between these centers and eventually to provide access to them for additional users. Ad hoc resource sharing 'procedures' among these original groups pointed the way toward standardisation of the protocols needed to communicate between ANY administrative domain. The current Grid technology can be viewed as an extension or application of this framework to create a more generic resource sharing context. The SETI@home project was the first such non profit scientific cause that was designed to make use of the idle CPU cycles. These programs generally run in the background or as a screensaver when the user does not use the entire computing power of the PC. Many such projects have made progress in fields that would have otherwise taken prohibitive investment or a delay in results.
Virtual Organization
A Grid environment is created to address resource needs; the use of that resource(s) (ie. CPU cycles, disk storage, data, software programs, peripherals, etc.) is usually characterized by its availability outside of the context of the local administrative domain. This 'external provisioning' approach entails creating a new administrative domain referred to as a Virtual Organization (VO) with a distinct and separate set of administrative policies (home administration policies plus external resource administrative policies equals the VO [aka your Grid] administrative policies). The context for a Grid 'job execution' is distinguished by the requirements created when operating outside of the home administrative context. Grid technology (aka. middleware) is employed to facilitate formalizing and complying with the Grid context associated with your application execution.
Grid enabled
In addition to 1. identifying the administrative requirements and 2. identifying 'external' resource requirements, the target application must be 3. Grid-enabled. The nature and extent of this work is a reflection of the Virtual Organization context and the application itself. For example, if the ' external resource' to be used is owned by another department and resides in an adjacent building, the requirements for resource discovery, brokering, scheduling, accounting, security and communications access are minimized.
The application that is to be grid enabled commits to a Grid execution context governed by the VO policies. In addition to the overall requirements associated with submission within a VO execution context, those elements of the application that enjoin the external resources are also subject to modifications necessary to utilize VO Grid resources.
Resource utilization
One characteristic that currently distinguishes Grid computing from distributed computing is the abstraction of a 'distributed resource' into a Grid resource. One result of abstraction is that it allows resource substitution to be more easily accomplished. Some of the overhead associated with this flexibility is reflected in the middleware layer and the temporal latency associated with the access of a Grid (or any distributed) resource. This overhead, especially the temporal latency, must be evaluated in terms of the impact on computational performance when a Grid resource is employed. Web based resources or Web based resource access is an appealing approach to Grid resource provisioning. A recent GGF Grid middleware evolutionary development 're-factored' the architecture/design of the Grid resource concept to reflect using the W3C WSDL (Web Service Description Language) to implement the concept of a WS-Resource. The stateless nature of the Web, while enhancing the ability to scale, can be a concern for applications that migrate from a stateful resource access context to the Web-based stateless resource access context. The GGF WS-Resource concept includes discussions on accommodating the statelessness associated with Web resources access.
State-of-the-Art 2005
The conceptual framework and ancillary infrastructure are evolving at a fast pace and includes international participation. The business sector is actively involved in commercialization of the Grid framework. The 'big science' sector is actively addressing the development environment and resource (aka performance) monitoring aspects. Activity is also observed in providing grid-enabled versions of HPC tools. Activity in the domains of 'little science' appears to be scant at this time. The treatment in the GGF documentation series reflects the HPC (High Performance Computing) roots of the Grid concept framework; this bias should not be interpreted as a restriction in the application of the Grid conceptual framework in its application to other research domains or other computational contexts.
See also
- List of distributed computing projects
- WSRF, the Web Services Resource Framework
- Object Management Group
- CORBA
- Parallel Virtual Machine (PVM) - Message Passing Interface (MPI)
- Ganglia
- Distributed computing
- Render farm
- Semantic Grid
- Supercomputer
- computer cluster
- Sun GridEngine
- virtual organization
- SDSC Storage resource broker (data grid)
- The Java Commodity Grid Toolkit (CoG) Kit (http://www.cogkit.org/)
- information grid
- Grid and cluster computing using Java (http://wiki.java.net/bin/view/Communications/CollaborativeComputing) (LGPL)
- media grid
- OGSI
References
- Antony Davies: Computational Intermediation and the Evolution of Computation as a Commodity, Applied Economics, June 2004, Online version (http://www.business.duq.edu/faculty/davies/research/EconomicsOfComputation.pdf)
- Ian Foster, Carl Kesselman: The Grid: Blueprint for a New Computing Infrastructure, Morgan Kaufmann Publishers, ISBN 1558604758, Website (http://www.mkp.com/grids/)
- Fran Berman, Anthony J. G. Hey, Geoffrey Fox: Grid Computing: Making The Global Infrastructure a Reality, Wiley, ISBN 0470853190, Online version (http://www.grid2002.org/)
- Maozhen Li, Mark A. Baker: The Grid: Core Technologies, Wiley, ISBN 0470094176, Website (http://coregridtechnologies.org/)
- CERN: The Grid Café - What is Grid? (http://gridcafe.web.cern.ch/gridcafe/whatisgrid/whatis.html), viewed 04 Feb 2005.
- Roger Smith: Grid Computing: A Brief Technology Analysis (http://www.ctonet.org/documents/GridComputing_analysis.pdf), CTO Network Library, 2005.
External links
- News
- IEEE Distributed Systems Online, Grid Computing Section (http://dsonline.computer.org/gc/)
- GRIDtoday (http://www.gridtoday.com/gridtoday.html)
- UtilityComputing.com (http://www.UtilityComputing.com/)
- Network
- The Globus Alliance (http://www.globus.org)
- Global Grid Forum (http://www.gridforum.org/)
- ApGrid: Asia Pacific Grid (http://www.apgrid.org/)
- EU DataGrid project (http://eu-datagrid.web.cern.ch/eu-datagrid/)
- Enabling Grids for E-sciencE (EGEE) (http://egee-intranet.web.cern.ch/egee-intranet/gateway.html)
- ThaiGrid (http://www.thaigrid.net/)
- NorduGrid (http://www.nordugrid.org/)
- Portal
- Parabon Computation (http://www.parabon.com/)
- DataSynapse (http://www.datasynapse.com/)
- IBM Grid Computing website (http://www.ibm.com/grid)
- GridComputing.com (http://www.gridcomputing.com/)
- United Devices (http://www.ud.com/)
- GRIDALOGY (http://www.gridalogy.com/)
- Articles
- O'Reilly article about grid computing software (http://linux.oreillynet.com/pub/a/linux/2002/04/25/enterprise.html)
- Grid Café, the place for everyone to learn about the Grid (http://gridcafe.web.cern.ch/gridcafe)
- Tools, Frameworks, Middleware
- Globus Toolkit (http://www-unix.globus.org/toolkit/)
- Java Commodity Grid Toolkit (CoG) Kit (http://www.cogkit.org/)
- GridForge (http://forge.gridforum.org/)
- ProActive (http://www-sop.inria.fr/oasis/ProActive/) is a Java library for parallel, distributed, and concurrent computing with mobility and security
- Grid Engine (http://gridengine.sunsource.net/), open source grid engine sponsored by Sun Microsystems, runs on many platforms
- Apple Xgrid (http://www.apple.com/acg/xgrid/), an easy-to-setup grid solution for Mac
- BioSimGrid: Grid database for biomolecular simulations (http://www.biosimgrid.org/)
- The Condor project (http://www.cs.wisc.edu/condor) is a grid computing engine by the University of Wisconsin, and runs on many platforms
- Mobius Data Grid middleware (http://projectmobius.org)
- The OGSA-DAI Data virtualisation project (http://www.ogsadai.org.uk)
- Projects for end-user participation (see also the List of distributed computing projects for more)
- How you can fight against diseases using your computer (http://www.d2ol.com/).
- Einstein@Home: search data from the Laser Interferometer Gravitational wave Observatory (LIGO) in the US and from the GEO 600 gravitational wave observatory in Germany for signals coming from rapidly rotating neutron stars, known as pulsars. (http://einstein.phys.uwm.edu/)
- LHC@home: improve the design of the CERN LHC particle accelerator. (http://athome.web.cern.ch/athome/)
- Climateprediction.net: Improve the accuracy of long-term climate prediction. (http://climateprediction.net/)
- Predictor@home: Solve biomedical questions and investigate protein-related diseases. (http://predictor.scripps.edu/)
- Research
- IBM Research - Grid Computing (http://www.research.ibm.com/journal/sj43-4.html)
- Grid Research Lab at University of Vienna (http://www.cs.univie.ac.at/cta)de:Grid-Computing
es:Computación distribuida it:Grid computing nl:Grid computing ja:グリッド・コンピューティング pl:Siatka komputerowa zh:网格计算