?? index(4).html
字號:
<HTML><HEAD><TITLE>Introduction to Parallel Computing</TITLE><SCRIPT LANGUAGE="JavaScript" SRC="../tutorials.js"></SCRIPT><LINK REL=StyleSheet HREF="../tutorials.css" TYPE="text/css"><!-- BEGIN META TAGS --><META NAME="LLNLRandR" CONTENT=""><META NAME="distribution" CONTENT="global"><META NAME="description" content="Livermore Computing Training"><META NAME="rating" CONTENT="general"><META HTTP-EQUIV="keywords" CONTENT="Lawrence LivermoreNational Laboratory, LLNL, High Performance Computing, parallel, programming, HPC, training, workshops, tutorials, Blaise Barney"><META NAME="copyright" CONTENT="This document is copyrighted U.S.Department of Energy under Contract W-7405-Eng-48"><META NAME="Author" content="Blaise Barney"><META NAME="email" CONTENT="blaiseb@llnl.gov"><META NAME="Classification" content="DOE:DOE Web sites viaorganizational structure:Laboratories and Other Field Facilities"><meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1"><!-- END META TAGS --></HEAD><BODY><BASEFONT SIZE=3> <!-- default font size --><A NAME=top> </A><TABLE CELLPADDING=0 CELLSPACING=0 WIDTH=100%><TR><TD BGCOLOR=#3F5098> <TABLE CELLPADDING=0 CELLSPACING=0 WIDTH=900> <TR><TD BACKGROUND=../images/bg1.gif> <A NAME=top> </A> <SCRIPT LANGUAGE="JavaScript">addNavigation()</SCRIPT> <P><BR> <H1>Introduction to Parallel Computing</H1> <P> </TD></TR></TABLE></TD></TR></TABLE><P><A NAME=TOC> </A><H2>Table of Contents</H2><OL><LI><A HREF=#Abstract>Abstract</A><LI><A HREF=#Overview>Overview</A> <OL> <LI><A HREF=#Whatis>What is Parallel Computing?</A> <LI><A HREF=#WhyUse>Why Use Parallel Computing?</A> </OL><LI><A HREF=#Concepts>Concepts and Terminology</A> <OL> <LI><A HREF=#Neumann>von Neumann Computer Architecture</A> <LI><A HREF=#Flynn>Flynn's Classical Taxonomy</A> <LI><A HREF=#Terminology>Some General Parallel Terminology</A> </OL><LI><A HREF=#MemoryArch>Parallel Computer Memory Architectures</A> <OL> <LI><A HREF=#SharedMemory>Shared Memory</A> <LI><A HREF=#DistributedMemory>Distributed Memory</A> <LI><A HREF=#HybridMemory>Hybrid Distributed-Shared Memory</A> </OL> <LI><A HREF=#Models>Parallel Programming Models</A> <OL> <LI><A HREF=#ModelsOverview>Overview</A> <LI><A HREF=#ModelsShared>Shared Memory Model</A> <LI><A HREF=#ModelsThreads>Threads Model</A> <LI><A HREF=#ModelsMessage>Message Passing Model</A> <LI><A HREF=#ModelsData>Data Parallel Model</A> <LI><A HREF=#ModelsOther>Other Models</A> </OL><LI><A HREF=#Designing>Designing Parallel Programs</A> <OL> <LI><A HREF=#DesignAutomatic>Automatic vs. Manual Parallelization</A> <LI><A HREF=#DesignUnderstand>Understand the Problem and the Program</A> <LI><A HREF=#DesignPartitioning>Partitioning</A> <LI><A HREF=#DesignCommunications>Communications</A> <LI><A HREF=#DesignSynchronization>Synchronization</A> <LI><A HREF=#DesignDependencies>Data Dependencies</A> <LI><A HREF=#DesignLoadBalance>Load Balancing</A> <LI><A HREF=#DesignGranularity>Granularity</A> <LI><A HREF=#DesignIO>I/O</A> <LI><A HREF=#DesignCosts>Limits and Costs of Parallel Programming</A> <LI><A HREF=#DesignPerformance>Performance Analysis and Tuning</A> </OL><LI><A HREF=#Examples>Parallel Examples</A> <OL> <LI><A HREF=#ExamplesArray>Array Processing</A> <LI><A HREF=#ExamplesPI>PI Calculation</A> <LI><A HREF=#ExamplesHeat>Simple Heat Equation</A> <LI><A HREF=#ExamplesWave>1-D Wave Equation</A> </OL><LI><A HREF=#References>References and More Information</A></OL> <!--========================================================================--> <A NAME=Abstract> <BR><BR> </A><TABLE BORDER=1 CELLPADDING=5 CELLSPACING=0 WIDTH=100%><TR><TD BGCOLOR=#98ABCE><SPAN class=heading1>Abstract</SPAN></TD></TD></TR></TABLE><P><BR> This presentation covers the basics of parallel computing. Beginning with abrief overview and some concepts and terminology associated with parallelcomputing, the topics of parallel memory architectures and programming modelsare then explored. These topics are followed by a discussion on a number ofissues related to designing parallel programs. The last portion of thepresentation is spent examining how to parallelize several different typesof serial programs.<P>Level/Prerequisites: None<BR><BR><!--========================================================================--><A NAME=Overview> <BR><BR> </A><A NAME=Whatis> </A><TABLE BORDER=1 CELLPADDING=5 CELLSPACING=0 WIDTH=100%><TR><TD BGCOLOR=#98ABCE><SPAN class=heading1>Overview</SPAN></TD></TD></TR></TABLE><H2>What is Parallel Computing?</H2><UL><P><LI>Traditionally, software has been written for <B><I>serial</I></B> computation: <UL> <LI>To be run on a single computer having a single Central Processing Unit (CPU); <LI>A problem is broken into a discrete series of instructions. <LI>Instructions are executed one after another. <LI>Only one instruction may execute at any moment in time. </UL><P><IMG SRC=images/serialProblem.gif WIDTH=604 HEIGHT=250 BORDER=0 ALT='Serial computing'><P><LI>In the simplest sense, <B><I>parallel computing</I></B> is the simultaneous use of multiple compute resources to solve a computational problem. <UL> <LI>To be run using multiple CPUs <LI>A problem is broken into discrete parts that can be solved concurrently <LI>Each part is further broken down to a series of instructions <LI>Instructions from each part execute simultaneously on different CPUs </UL><P><IMG SRC=images/parallelProblem.gif WIDTH=683 HEIGHT=372 BORDER=0 ALT='Parallel computing'><P><LI>The compute resources can include: <UL> <LI>A single computer with multiple processors; <LI>An arbitrary number of computers connected by a network; <LI>A combination of both. </UL><P><LI>The computational problem usually demonstrates characteristics such as the ability to be: <UL> <LI>Broken apart into discrete pieces of work that can be solved simultaneously; <LI>Execute multiple program instructions at any moment in time; <LI>Solved in less time with multiple compute resources than with a single compute resource. </UL><P><LI>Parallel computing is an evolution of serial computing that attempts to emulate what has always been the state of affairs in the natural world: many complex, interrelated events happening at the same time, yet within a sequence. Some examples: <UL> <LI>Planetary and galactic orbits <LI>Weather and ocean patterns <LI>Tectonic plate drift <LI>Rush hour traffic in LA <LI>Automobile assembly line <LI>Daily operations within a business <LI>Building a shopping mall <LI>Ordering a hamburger at the drive through. </UL><P><LI>Traditionally, parallel computing has been considered to be "the high end of computing" and has been motivated by numerical simulations of complex systems and "Grand Challenge Problems" such as: <UL> <LI>weather and climate <LI>chemical and nuclear reactions <LI>biological, human genome <LI>geological, seismic activity <LI>mechanical devices - from prosthetics to spacecraft <LI>electronic circuits <LI>manufacturing processes </UL><P><LI>Today, commercial applications are providing an equal or greater driving force in the development of faster computers. These applications require the processing of large amounts of data in sophisticated ways. Example applications include: <UL> <LI>parallel databases, data mining <LI>oil exploration <LI>web search engines, web based business services <LI>computer-aided diagnosis in medicine <LI>management of national and multi-national corporations <LI>advanced graphics and virtual reality, particularly in the entertainment industry <LI>networked video and multi-media technologies <LI>collaborative work environments </UL> <P><LI>Ultimately, parallel computing is an attempt to maximize the infinite but seemingly scarce commodity called time. </UL><!--========================================================================--><P><A NAME=WhyUse> <BR><BR> </A><TABLE BORDER=1 CELLPADDING=5 CELLSPACING=0 WIDTH=100%><TR><TD BGCOLOR=#98ABCE><SPAN class=heading1>Overview</SPAN></TD></TD></TR></TABLE><H2>Why Use Parallel Computing?</H2><UL><P><LI>The primary reasons for using parallel computing: <UL> <LI>Save time - wall clock time <LI>Solve larger problems <LI>Provide concurrency (do multiple things at the same time) </UL><P><LI>Other reasons might include: <UL> <LI>Taking advantage of non-local resources - using available compute resources on a wide area network, or even the Internet when local compute resources are scarce. <LI>Cost savings - using multiple "cheap" computing resources instead of paying for time on a supercomputer. <LI>Overcoming memory constraints - single computers have very finite memory resources. For large problems, using the memories of multiple computers may overcome this obstacle. </UL><P><LI>Limits to serial computing - both physical and practical reasons pose significant constraints to simply building ever faster serial computers: <UL> <LI>Transmission speeds - the speed of a serial computer is directly dependent upon how fast data can move through hardware. Absolute limits are the speed of light (30 cm/nanosecond) and the transmission limit of copper wire (9 cm/nanosecond). Increasing speeds necessitate increasing proximity of processing elements. <LI>Limits to miniaturization - processor technology is allowing an increasing number of transistors to be placed on a chip. However, even with molecular or atomic-level components, a limit will be reached on how small components can be. <LI>Economic limitations - it is increasingly expensive to make a single processor faster. Using a larger number of moderately fast commodity processors to achieve the same (or better) performance is less expensive. </UL><P><LI>The future: during the past 10 years, the trends indicated by ever faster networks, distributed systems, and multi-processor computer architectures (even at the desktop level) suggest that <B><I>parallelism is the future of computing</I></B>.<P><IMG SRC=images/whoWhatChart.gif WIDTH= HEIGHT= BORDER=0 ALT='chart'></UL><!--========================================================================--><P><A NAME=Concepts> <BR><BR> </A><A NAME=Neumann> </A><TABLE BORDER=1 CELLPADDING=5 CELLSPACING=0 WIDTH=100%><TR><TD BGCOLOR=#98ABCE><SPAN class=heading1>Concepts and Terminology</SPAN></TD></TD></TR></TABLE><H2>von Neumann Architecture</H2><UL><P><LI>For over 40 years, virtually all computers have followed a common machine model known as the von Neumann computer. Named after the Hungarian mathematician John von Neumann. <P><LI>A von Neumann computer uses the stored-program concept. The CPU executes a stored program that specifies a sequence of read and write operations on the memory.<P><IMG SRC=images/vonNeumann1.gif WIDTH=275 HEIGHT=178 BORDER=0ALT='von Neumann model'><P><LI>Basic design: <UL> <LI>Memory is used to store both program and data instructions <LI>Program instructions are coded data which tell the computer to do something <LI>Data is simply information to be used by the program <LI>A central processing unit (CPU) gets instructions and/or data from memory, decodes the instructions and then <B><I>sequentially</I></B> performs them. </UL></UL><!--========================================================================--><P><A NAME=Flynn> <BR><BR> </A><TABLE BORDER=1 CELLPADDING=5 CELLSPACING=0 WIDTH=100%><TR><TD BGCOLOR=#98ABCE><SPAN class=heading1>Concepts and Terminology</SPAN></TD></TD></TR></TABLE><H2>Flynn's Classical Taxonomy</H2><UL><P><LI>There are different ways to classify parallel computers. One of the more widely used classifications, in use since 1966, is called Flynn's Taxonomy.<P>?? 快捷鍵說明
復制代碼
Ctrl + C
搜索代碼
Ctrl + F
全屏模式
F11
切換主題
Ctrl + Shift + D
顯示快捷鍵
?
增大字號
Ctrl + =
減小字號
Ctrl + -