当前位置：文档库 › Efficient communication between parallel programs with InterComm

Efficient communication between parallel programs with InterComm

Ef?cient Communication Between Parallel Programs with InterComm

Jae-Yong Lee and Alan Sussman

UMIACS and Dept.of Computer Science

University of Maryland

College Park,MD20742

jylee,als@https://www.wendangku.net/doc/402290090.html,

Abstract

We present the design and implementation of Inter-

Comm,a framework to couple parallel components

that enables ef?cient communication in the presence

of complex data distributions within a coupled ap-

plication.Multiple parallel libraries and languages

may be used in different modules of a single applica-

tion.The ability to couple such modules is required

in many emerging application areas,such as complex

simulations that model physical phenomena at multi-

ple scales and resolutions,and remote sensing image

data analysis applications.

The complexity of the communication algorithms is

highly dependent on the distribution of data across the

processes in a distributed memory parallel program.

We classify the distributions into two types-one that

represents a data distribution in a compact way so that

the distribution information can be replicated,and one

that explicitly describes the location of each data ele-

ment,so can be very large,requiring that the distribu-

tion information be distributed across processes as is

the data.

InterComm builds on our previous work on the

Meta-Chaos program coupling framework.In that

work,we showed that the steps required to perform the

data exchanges include locating the data to be trans-

ferred within the local memories of each program,

generating communication schedules(the patterns of

straightforward because each program(a component) may use a different programming language or data par-allel runtime library.

Suppose that we have two parallel programs that co-operate to solve a complex problem,and they have dis-tributed data structures ds1and ds2,respectively.To transfer a set of data elements in ds1to a set of ele-ments in ds2,all processes of the sender program and of the receiver program may need to be involved in the data transfer,because data elements in ds1and ds2 may be distributed across the processes.Thus,each sender process must determine which data elements of ds1it should send to the various receiver processes. Similarly,each receiver process must determine which data elements should be written for each message re-ceived from a sender process.This implies that each process in the sender or the receiver program must acquire information about the data distribution across processes of both ds1and ds2.

InterComm is a runtime library that achieves di-rect data transfers between data structures managed by multiple data parallel languages and libraries in differ-ent programs.Such programs include those that di-rectly use a low-level message-passing library,such as MPI.Each program does not need to know in advance (i.e.before a data transfer is desired)any information about the program on the other side of the data trans-fer.All required information for the transfer is com-puted by InterComm at runtime.As was already de-scribed,such a data transfer requires that all processes of the sender and receiver programs locate data ele-ments involved in the data transfer and that a mapping be speci?ed between the data elements in the two data https://www.wendangku.net/doc/402290090.html,ing the data distribution and mapping in-formation already described,InterComm generates all the information required to execute direct data trans-fers between the processes in the sender program and the receiver program(a customized all-to-all commu-nication pattern[18]),and stores the information in a communication schedule[6].

In this paper,we describe in Section2the algo-rithms we have developed to ef?ciently build com-munication schedules for data transfers on different classes of distributed data structures.We describe vari-ous choices that can be made in the algorithm designs, and provide experimental results that indicate which variations work best under different scenarios in Sec-tion3.We summarize our results and describe future directions for this work in Section4.

1.1Prior related work

There have been several efforts to model and man-age parallel data structures and provide support for coupling of parallel applications that use those data structures.While some of them provide similar meth-ods to distribute parallel data structures and represent the data distribution,they employ various strategies to transfer such distributed data between application components.

Parallel Application Work Space(PAWS)[3,12] provides the ability to share data structures between parallel applications.PAWS supports scalar values and parallel multi-dimensional array data structures.An application de?nes a global domain to provide each process with a global view of a data structure across all processes.The global domain is divided into sub-domains,with each subdomain assigned to one pro-cess,representing a local view of a part of the data structure.The layout(representation in PAWS)of a data structure consists of global and sub-domains. Since the shape of a PAWS sub-domain can be arbi-trarily de?ned by an application,multi-dimensional ar-rays in PAWS can be partitioned and distributed in a completely general way.A PAWS controller is a pro-cess that links applications and parallel data structures in the applications.A PAWS application registers it-self as an active application with the PAWS controller when it starts execution.The application also registers the data structures that it will share with other appli-cations.To transfer data elements between data struc-tures of two applications,the PAWS controller estab-lishes a connection between those data structures using information in its registry,and uses the parallel lay-out of both data structures to compute communication schedules for the data transfer.On the other hand,In-terComm communication schedules are generated di-rectly in the processes of the parallel applications with-out a separate controller process,allowing schedules to be computed in parallel.

Collaborative User Migration,User Library for Vi-sualization and Steering(CUMULVS)[10]is a mid-dleware library that facilitates the remote visualiza-tion and steering of parallel applications,and sup-

ports sharing parallel data structures between pro-grams.Although it supports multi-dimensional arrays like PAWS,arrays cannot be distributed in a fully gen-eral way.A multi-dimensional array is partitioned into chunks in each dimension(as in High Performance Fortran(HPF)[13]),or each data element is explicitly distributed(assigned to a process)by the application. In addition,the application programmer must export a topology for processes that represent the ownership of each data chunk.A receiver program(a visualizer)in CUMULVS is not a parallel program.It speci?es the data it requires in a request that is sent to the paral-lel sender program.After receiving the request,the sender program generates a sequence of connection calls to transfer the data.

Roccom[11]is an object-oriented software frame-work for high performance parallel rocket simula-tion.Multiple physics modules have been developed to model various parts of the overall problem to build a comprehensive simulation system.A physics module builds distributed objects(data and functions)called windows and registers them in Roccom so that other modules can share them with the permission of the owner module.A window may be partitioned into multiple panes for parallelism,and each process in a module may have multiple panes.For example,if a window has a multi-dimensional array as its data attribute,the array can be partitioned into subarrays, each of which can be a pane.

The Common Component Architecture(CCA)Fo-rum[2,4]has been developing a set of common inter-faces to provide interoperability among high perfor-mance application components.The CCA MxN work-ing group[5]has designed interfaces to transfer data elements between parallel components running with different numbers of processes in each parallel com-ponent(hence MxN).InterComm can be used as the runtime support for a general implementation of the CCA MxN services.

Model Coupling Toolkit(MCT)[15]is a system that has been developed for the Earth Systems Mod-eling Framework(ESMF)[7].ESMF has developed various earth systems simulation components and a ?ux coupler component.The?ux coupler serves to transfer data between the physics simulations compo-nent using the MCT functionality.In MCT,a glob-alSegmentMap is de?ned to describe the distribution of a data structure across processes.The globalSeg-mentMap describes each continuous chunk of mem-ory for the data structure in each https://www.wendangku.net/doc/402290090.html,ing glob-alSegmentMaps,MCT can generate a router–a com-munication scheduler that tells processes how to trans-fer data elements between a simulation component and the?ux coupler.Therefore,all data transfers between two physics components are executed through the?ux coupler.

Meta-Chaos[6]is a meta-library that interacts with data parallel libraries and languages to achieve direct data transfer between different parallel data structures. To move data between data structures in different ap-plications,Meta-Chaos locates data elements to be transferred in both the sender and receiver processes and then generates communication schedules.Since data is distributed across process,Meta-Chaos requires communication among processes to determine which process owns data elements and where the data ele-ments are located in the processes.In other words, each process needs to communicate with all other pro-cesses to obtain data distribution information.Deref-erence functions are used to request such information, and must be provided for each data parallel library or language.Meta-Chaos spends most its time in build-ing communication schedules in the dereference func-tion.In this paper,we present ef?cient algorithms that generate communication schedules by directly using data distribution information supplied by each appli-cation component via an InterComm API,without us-ing dereference functions.Therefore,InterComm can be used with any parallel program that can describe its data distribution,including explicit message pass-ing programs using,for example,MPI.We will also show,in Section3,that the InterComm algorithms per-form much better in absolute terms than those in Meta-Chaos,and also scale better.

2Scheduling Communication between Par-allel Programs

In this section we describe algorithms for generat-ing communication schedules that completely describe the pattern of communication between the processes in a source program and the processes in a destination program.More speci?cally,the schedule for each pro-cess in the source program speci?es the data elements

Source Linearization

Destination

Linearization

A Set of Regions

Figure1.Sets of regions and linearization

to send from that process and which processes in the receiver to send them to.Similarly,the schedule for each process in the receiver program speci?es which data elements will be received into,and which sender processes will send them.

2.1Background

Application programmers must be able to specify the data elements that will participate in a data trans-fer.Many common data structures allow specifying the set of data elements to be transferred in a compact way(e.g.,for an array,a sub-array requires specifying two corners of the sub-array).However,in the worst case the set can be enumerated.We call this set a re-gion[6].InterComm provides several methods to de-scribe a region,so that application programs can eas-ily specify the data elements that will participate in a data transfer.In this paper,we concentrate on multi-dimensional array data structures,since those are the only ones currently supported by InterComm.Since a single region is not always adequate to describe the data to be moved,multiple regions can be gathered into a set of regions.Figure1shows an example of a set of regions.As shown,the regions do not have to be the same in the source and destination programs.

Linearization[6]provides an implicit mapping be-tween data elements in the source set of regions and the destination set of regions,so that the mapping does not have to be explicitly speci?ed for each data trans-fer.Data elements in a set of regions are?attened into an abstract linearized space.By linearizing both the source and destination sets of regions,InterComm can compute a total ordering for the data elements in each of the set of regions,providing an implicit one-to-one mapping between the source and destination lineariza-tions.Figure1shows an example of how linearization works for two-dimensional arrays laid out in memory in column major order(i.e.Fortran-style).

Data parallel programs employ data descriptors that describe the global distribution of data structures across the processes executing the program.We say that a process in a program owns a data element if the data descriptor says that the local memory for that element is located in the process.InterComm sup-ports data descriptors that fall into two classes,both representing partitionings of multi-dimensional arrays across the processes in a parallel program.One class of descriptors supports a block array data distribution, which can be represented in a compact,essentially im-plicit way.We will take advantage of this descriptor being compact,so that it can be replicated easily and cheaply over multiple processes.The other kind of de-scriptor supports an irregular,explicit distribution of the elements in an array,and can also be easily ex-tended to non-array data structures.Such a descrip-tor cannot be described in a compact way because the descriptor must enumerate all the data elements,and therefore its size is proportional to the size of the data structure.Such a non-compact descriptor explicitly represents the distribution of the data elements,de-scribing the process that owns each data element and the memory location of the element in that process. Replicating a non-compact descriptor across processes requires a large amount of memory for a large array, therefore such a descriptor must be partitioned over the processes in a parallel program[17].

In the communication schedule building algo-rithms,three naming schemes,or views,of a data el-ement are used.A global view describes the loca-tion of a data element in the entire data structure.In this global reference space,for a distributed array data structure a data element is named with a global index in each dimension of the array.The second naming scheme is a local view,in which a given process ad-

dresses data elements within its own address space.In this local reference space,for a distributed array data elements are also named with indexes in each array di-

mension,but only for the part of the array owned by a given process.The last naming scheme is a lineariza-tion view.Since linearization determines a total order-ing of the data elements speci?ed by a set of regions, data elements to be transferred may be named by their indices in a linearization space.

Data descriptors provide the information necessary to translate data elements named in a global reference space into names in a local reference space(and vice versa).In InterComm,regions are speci?ed in the global reference space.Data elements in a set of re-gions can be mapped into the linearization space us-ing the region speci?cations and the data descriptor. Therefore we can also translate local names into the linearization space(and vice versa).The InterComm algorithms translate names across all three views to produce communication schedules from data descrip-tors and region speci?cations in both a source and des-tination program.

2.2A general schedule building algorithm

We address two types of environments for execut-ing parallel programs.One is a local cluster or dis-tributed memory parallel machine environment and the other is a heterogeneous wide-area Grid environment. Message passing is usually much faster within a clus-ter than across a wide area network(W AN).The algo-rithms we describe to generate communication sched-ules provide several options that can be used depend-ing on the computation environments of the commu-nicating programs.For example,the algorithm can attempt to reduce the number of expensive messages across the W AN in a Grid environment,perhaps at the expense of additional message passing within a cluster.

Algorithm1shows the?ve high-level steps needed to transfer data between two parallel programs.These steps will be executed in each of the processes of ei-ther or both of the source and destination programs (https://www.wendangku.net/doc/402290090.html,munication schedule building is a collective operation[18]across the processes in both the source and destination programs).The?rst four steps in the algorithm compute communication schedules needed to perform the data transfer in the last step.For steps 1:Retrieve distribution information about the source data structure

2:Retrieve distribution information about the desti-nation data structure

3:Compute partial communication schedules

4:Transfer the schedules to the processes that need them

5:Transfer data elements using the schedules

Program1Program2

(a)Method

1Program1Program2

(b)Method2

Figure2.Two methods for exchanging infor-mation

2.3Two compact data descriptors

A compact descriptor for a data structure can be cheaply replicated across all the processes that partic-ipate in building the communication schedule because of it small size.With a compact descriptor,all pro-cesses in a program have access to complete data dis-tribution information for any such data structure.The only additional information a process needs to com-pute a schedule(or a part of a schedule–a partial schedule)for a given data transfer are the data spec-i?cations for the source and destination(the sets of regions),and the distribution information for the data structure managed by the other program(the compact data descriptor of the other side).The following sec-tions describe both how processes obtain that informa-tion and how the schedule building algorithms use that information.

2.3.1Exchanging compact data descriptors and

data speci?cations

Some processes will be assigned the responsibility for computing parts of the communication schedules for a data transfer,as described in Section2.3.2.Because both the compact data descriptors and the data speci?-cations for the source(sending program)and destina-tion(receiving program)for the transfer can be repli-cated,since the replicated information is small,the processes in the two programs can exchange the infor-mation in an inexpensive way via point-to-point mes-sage passing and/or broadcast operations.Since all the processes sending information can send the replicated information,and all the processes receiving informa-

tion require the same information,we have(at least)

two options for how to send the information.Fig-

ure2shows two methods.In Figure2(a),each send-

ing process sends to the information a disjoint set of

receiving processes.Suppose we have m sending pro-

cesses and n receiving processes.Each sending pro-

cess sends the information to a disjoint set of

receiving processes.With the number of pro-cesses in each program and a process ID available at

runtime,each process can determine which processes

in the other program it has to send information to and

receive information from.Although this method re-

quires messages to be transmitted between the two programs,it is ef?cient because all the sending pro-

cesses can send messages at the same time,and also

order the messages to minimize network contention at

the receiving processes.The second method,shown in

Figure2(b),has a representative sender process send

the information to a representative receiving process.

The representative receiver process broadcasts the in-

formation to all other receiving processes.Although

this approach takes two steps,this method can reduce

network traf?c,and could perform better than the?rst

method when two parallel programs run on different

clusters with a wide-area network connection between

them.In that environment,the second method requires

only one expensive message across the W AN and one

broadcast within a cluster,while the?rst method re-

quires expensive messages between clusters.

2.3.2Responsibility for computing schedules Which processes generate communication sched-ules?Since any process can generate schedules from information about the source and destination data structures,there are(at least)two options for speci-fying which program will generate the schedules.The ?rst option is to have both sets of source and desti-nation processes compute schedules.In this case,all processes of both programs need to acquire informa-tion about the data descriptors and data speci?cations from processes of the other program.In the second op-tion,either the source(or destination)set of processes computes communication schedules for the processes in both programs,and sends the computed schedules to the processes in the destination(source)program.

The processes in the destination(source)program only send the data descriptor and data speci?cation infor-mation to the processes in the program computing the schedule.Responsible processes compute communi-cation schedules from information about both source and destination data structures while non-responsible processes send the required information to responsible processes and in return receive communication sched-ules that have been computed in the responsible pro-cesses.

What part of the schedules does each process compute?Since any process can obtain complete in-formation for computing schedules after receiving a data descriptor and data speci?cation from the other program,any process can compute any part of the schedule.For load balancing in computing the sched-ules,we must look at the linearizations for both the source and destination data structures,which implic-itly provides the mapping between elements in the two data structures.We can partition the lineariza-tion space into as many sets as there are processes that will compute the schedule(the processes in one or both of the programs,as described above),and have each process compute the schedule for one set.This approach will balance the workload evenly.How-ever,this method requires a collective communication across all processes in both programs,to send the com-puted schedules to all other processes in the program(a process may compute a partial schedule for every other process,depending on the source and destination data descriptors and data speci?cations).However,a pro-cess does not need to send computed schedules to other processes in the same program,if it only computes a schedule for data elements that it owns.However, we cannot then determine how well the workload will be balanced,because each process may own different numbers of data elements from the data speci?cation. Although this second approach may not achieve per-fect workload balance,the collective communication for sending schedules to other processes in the?rst ap-proach can be expensive.In our current implemen-tation,each process that participates in the schedule computation computes schedules for the data that it owns,so it does not need to send partial schedules to other processes in the same program.However,each such process must still send partial schedules to the processes in the other program,if only one of the two programs performs the entire schedule computation.

2.3.3Schedule Generation Algorithm

In this section,we describe how to compute sched-ules from two compact data descriptors and two data speci?cations for the source and destination data struc-tures.The algorithm runs in each process of both the source and destination program,but parts of the algo-rithm only execute in the program that does the sched-ule computations(if only one program is computing the schedules).

1:Send a data descriptor and a set of regions for the data structure in this program(the local data struc-ture)to one or more processes in the other pro-gram,if necessary

2:Receive a data descriptor and a set of regions for the data structure in the other program(the remote data structure),from a process in the other pro-gram

3:Compute the intersection between locally owned data elements and the regions speci?ed for the lo-cal data structure

4:Map the data elements in into the linearization for the local data structure

5:Compute the intersections between the data owned by each remote process and the set of re-gions for the remote data structure

6:For each,map it into the linearization for the remote data structure

7:For each remote process,compute the intersec-tion of the linearizations of and

8:For each pair(,),translate each data element name in the intersection from the global reference space into the local reference space

9:For each pair(,),generate entries into the com-munication schedules for each contiguous piece in the intersections of their linearizations(one entry for the local elements,and one entry for the corre-sponding remote elements)

10:Send the communication schedules to processes in the other program,if necessary

Linearization space

Figure3.Mapping intersections of sets of data elements and regions into the lineariza-tion space

Algorithm2shows the steps needed to generate communication schedules with two compact data de-scriptors.The steps in this algorithm correspond to the ?rst four steps in Algorithm1.Each process that is re-sponsible for building parts of the schedules receives the data distribution and data speci?cation information for the remote data structure,as was described in Sec-tion2.3.1(lines1and2in Algorithm2).In line3, each responsible process computes the intersections of the regions speci?ed for the data structure in its pro-gram with the data elements that it owns in the local data structure,because each process computes com-munication schedules only for the data elements that it owns.For the remote data structures,however,the process does not know which remote processes own data elements that correspond to the locally owned el-ements.Therefore each process computes the inter-sections of the regions for the remote data structure with the data elements owned by each process in the remote data structure,in line5.As was discussed in Section2.1,the linearization provides a total ordering of the data elements to be transferred.By mapping the intersections of the regions and data elements owned by the local and remote processes into the lineariza-tion space,the algorithm can match data elements in each responsible process with the corresponding data elements in the remote processes(lines4and6).Fig-ure3shows how the intersections of the regions and the data elements owned by a process can be mapped into the linearization space.In this example,it is as-sumed that data elements in an array are stored in col-umn major order.The algorithm then computes the intersections of the linearizations for the local and re-mote data structures for each remote process,in line 7.

Although the algorithm determines how data ele-ments in the local and remote data structures match through the linearizations,the data elements must be speci?ed in the communication schedules in the local reference space(a process ID,local address pair).The algorithm must therefore translate global references into local references using the information in the com-pact data descriptor(line8).Note that a set of con-tiguous data elements in the intersection of the local and remote linearizations is contiguous in the mem-ory layouts of both the local and the remote data struc-tures.Finally,the algorithm generates the communi-cation schedule entries for each set of contiguous data elements in the intersection of the local and remote linearizations(line9).If processes in both programs compute the schedules,then an communication entry is only for the local references,specifying which local data elements will be transferred,and to/from which remote processes.If only processes in one program compute the schedules,the processes in that program must send schedules back to the processes in the other program(line10),as described in Section2.3.2.In this case,schedule entries must be made for both the local process and the corresponding remote process for a matched pair of data elements,using the local views for those processes.

2.4One compact and one non-compact data de-

scriptor

For this case,InterComm could exchange data de-scriptors as in the compact-compact case described in Section2.3.1.However,it may not be ef?cient to send a complete non-compact data descriptor because the descriptor can be very large.We describe two algo-rithms,differing based on where schedules are com-puted.One algorithm computes schedules using the processes in the program that employs a non-compact data descriptor for its data structure.The other algo-rithm uses processes in both programs.The following sections describe the details of the algorithms and dis-

cuss their performance with respect to the workload balancing and the communication cost in Grid envi-ronment.

2.4.1Responsibility for computing schedules Which processes generate communication sched-ules?There are two algorithms,depending on which processes are responsible for computing communica-tion schedules.The?rst algorithm has the program with a non-compact data descriptor compute sched-ules.One program obtains both data descriptors,com-putes communication schedules for both programs and sends schedules back to the other program.Since only one of the data descriptors must be sent with that method,an algorithm using that method does not have to send the non-compact data descriptor.The al-gorithm sends the compact data descriptor to all pro-cesses in the program that has the non-compact data descriptor,and those processes compute the schedules for processes in both programs.The schedules com-puted in the non-compact descriptor processes must be sent back to the compact descriptor processes.With this algorithm,InterComm can avoid transferring a non-compact data descriptor between programs.How-ever,the workload for building the schedule will be imbalanced since the program with a compact data de-scriptor is idle while the program with a non-compact data descriptor computes schedules.

The second algorithm has both programs compute schedules.For the program with a compact descriptor to compute schedules,it requires the information from the non-compact data descriptor in the other program. Therefore,processes in the program with the non-compact data descriptor must transfer the information to processes in the program with the compact data de-scriptor.To minimize information to be transferred be-tween programs,InterComm extracts distribution in-formation for the data elements involved in the data transfer in the program with the non-compact descrip-tor,and sends only that information to the proper pro-cesses in the program with the compact descriptor. Therefore,InterComm requires a proportional amount of data distribution information about the non-compact data descriptor to be transferred between processes. This procedure may require that all processes with a non-compact data descriptor send information to all processes with a compact data descriptor.Although the algorithm requires more communication between two programs than the?rst algorithm,it employs the processes in both programs to compute schedules,to balance the workload.

As will be seen from the experimental result in Sec-tion3,the second algorithm performs better when both programs are run on the same cluster or on machines with a local area network connection,because both programs participate in computing schedules and the cost to transfer part of a non-compact data descriptor is not very expensive.However,the second algorithm is worse than the?rst if the two programs are executed on machines connected across a wide-area network(e.g., each program runs on a different cluster,with the clus-ters connected via a W AN)because of the high cost to transfer part of the non-compact data descriptor be-tween the two programs.

With either algorithm,the size of the schedules to be sent between programs can be much larger than for the two compact data descriptor case in Sec-tion2.3,because the algorithms must generate a sep-arate schedule entry for each individual data element to be transferred,because of the explicit nature of the non-compact data descriptor.Schedule entries for the compact-compact case can be aggregated to com-pactly describe sets of data elements,making those schedules smaller.More speci?cally,the total size of all the schedules for the compact/non-compact case is proportional to the size of the data to be transferred. To transfer the compact data descriptor and the corre-sponding set of regions,the algorithm can use either of the two options shown in Figure2.

What part of the schedules does each process compute?We?rst discuss the algorithm that has only the program with a non-compact data descriptor com-pute schedules.To compute schedules,a process must obtain information about the data distributions for cor-responding parts of the the data structures in both pro-grams.For the two compact descriptor algorithm de-scribed in Section2.3.2,each process computes sched-ules for the data that it owns.That can be done be-cause each process obtains complete data distribution information for the data structures in both programs. However,a non-compact data descriptor is not repli-cated across processes,instead it is partitioned across the processes running its program.So each process

in the non-compact descriptor program contains par-

tial distribution information about its data structure. One option for assigning the workload is for a pro-cess in the non-compact descriptor program to com-pute schedules for the data elements for which it has distribution information.This approach may cause poor load balance if the distribution information about the data to be transferred is not uniformly partitioned across the processes.Moreover,this method requires a collective communication across the non-compact de-scriptor processes,to send the schedules to the pro-cesses that own the data,as well as a collective com-munication between the non-compact and compact de-scriptor programs to send the computed schedules to the compact descriptor program.An alternative ap-proach is for a process in the non-compact descrip-tor program to compute schedules for the data that process owns.This approach eliminates the collec-tive communication across the non-compact descriptor processes to send schedules,but requires a collective communication to collect the distribution information about the data elements a process owns onto that pro-cess.This second approach may also cause poor load balance,because each non-compact descriptor process may own different numbers of data elements that are involved in the data transfer.We have therefore chosen a third alternative to balance the workload for com-puting schedules across processes.In this option,all non-compact descriptor processes compute schedules for the same number of data elements,by assigning a contiguous range in the linearization space to each process.The algorithm in which the processes in both programs compute schedules is similar to this option, with the only difference being that the linearization space is partitioned across the processes in both pro-grams.We explain the details of the algorithm in the next section.

2.4.2Schedule Generation Algorithm

In this section,we describe how to compute schedules from one compact and one non-compact data descrip-tor,and two data speci?cations for the source and des-tination data structures.The algorithm runs in each process of both the source and destination program, but parts of the algorithm only execute in the program that does the schedule computations(if only one pro-

Partitioned

Linearization space

information

Data distribution

Figure4.An example of the required distribu-tion information

gram is computing the schedules).

As we said in the previous section,workload im-balance is a major problem for the?rst two methods described in Section2.4.1.The workload imbalance comes from having each process compute schedules for different numbers of data elements in the lineariza-tion.To make the workload balanced,the linearization space can be evenly partitioned across all responsi-ble processes,and each responsible process computes schedules for its part of the partitioned linearization space.Algorithm3shows the steps to compute sched-ules with one compact and one non-compact data de-scriptor.

Since processes in the program with a non-compact data descriptor must obtain the compact data descrip-tor and corresponding data speci?cation to compute schedules,they acquire that information using one of the two methods described in Section2.3.1and Fig-ure2(line1in Algorithm3).Although a non-compact data descriptor may be very large,each responsi-ble processes needs distribution information about the data elements to be transferred in the part of the lin-earization space it is responsible for.Figure4shows how the data distribution information is partitioned across processes,and what information each respon-sible process requires to compute schedules.In this example there are four processes with a partial non-compact data descriptor and the linearization is parti-tioned into four parts.As seen in Figure4,the data distribution information that each process needs may be owned by any other non-compact descriptor pro-cess.In other words,each such process must send and receive data distribution information to/from all

1:The compact descriptor program sends its descrip-tor and corresponding data speci?cation to the program with the non-compact data descriptor;the non-compact data descriptor program receives the information from the compact descriptor program

2:The non-compact descriptor program sends data distribution information from the non-compact de-scriptor corresponding to its data speci?cation to the other processes in the non-compact descrip-tor program(or in both programs,if both compute schedules),based on the partitioned linearization space;

3:The non-compact data descriptor program(or both programs,if both compute schedules)receives data distribution information for the part of the lin-earization that each process is responsible for

4:Responsible processes(either non-compact de-scriptor processes or all processes)compute the intersections between the data owned by each compact data descriptor process and the set of regions for the remote data structure

5:For each,the responsible processes map it into the linearization for the compact data structure 6:For only the part of the linearization space each process is responsible for,compute pairs of matched data elements from the compact and the non-compact data descriptors in the linearization space

7:For each pair of matched data elements,respon-sible processes translate from the global reference space to the local reference space

8:Responsible processes generate partial communi-cation schedule entries for each pair of matched data elements from the compact and the non-compact data structures

9:Responsible processes send partial schedules back to all processes

10:Receive partial schedules from the responsible processes

Compact Non?compact (a)Processes in the program with the non-compact data descrip-tor compute

schedules Compact Non?compact (b)Processes in both programs compute schedules

Figure 5.Messages to send and receive

schedules depend on which processes are

responsibility for computing schedules

ules in both cases.When only processes with the

non-compact descriptor compute schedules,each non-

compact descriptor process sends a message with the

schedules to all compact descriptor processes,as well

as to all other non-compact descriptor processes as

seen in Figure5(a).The total number of messages

to send schedules to right processes is(inter-program)+(within program),where m and n are the number of non-compact and compact descrip-

tor processes,respectively.However,all compact and

non-compact processes must send/receive schedules

to/from all other processes when both programs com-

pute schedules,as seen in Figure5(b).This method

requires messages,with

of them are inter-program messages.As was noted

in Section2.4.1,the algorithm requires more inter-

program messages to send back schedules,if the pro-

cesses in both programs are used to compute sched-

ules.So if the two programs are run on different clus-

ters connected via a W AN,it may not provide the best

performance to employ processes in both programs

to compute schedules,because of the relatively high

message passing costs across the W AN.Figure6(b)

shows an alternative method for sending schedules

from the responsible processes,when only the pro-

cesses in the program with the non-compact descrip-

tor compute schedules.In this method,a non-compact

descriptor process sends the partial schedules needed

by all the compact descriptor processes to the

com-

Compact Non?compact

(a)Method

Non?compact

Compact

(b)Method2

Figure6.Two methods to send schedules

back

pact descriptor process that sent it the compact data de-scriptor.A compact descriptor process then distributes the received schedules to all other compact descrip-tor processes.The schedule distribution work within the compact descriptor processes is very similar to that within the non-compact descriptor processes.More-over,the compact descriptor processes do the distribu-tion work at the same time the non-compact descrip-tor processes do theirs.This alternative requires mes-sages to distribute schedules within processes with a compact data descriptor.So the total number of mes-sages to send back schedules is(inter-program)+ (within program)+(within pro-gram)while the original algorithm sends(inter-program)+(within program)messages.In a Grid environment,using the alternative may be more ef?cient because it reduces the number of messages passed across the W AN to n,instead of m n.

2.5Two non-compact data descriptors

In this case,the processes in both programs only have partial data distribution information.While nec-essary,it is very expensive for the two programs to exchange the non-compact data descriptors,since the size of a non-compact data descriptor is proportional to the size of the entire data structure.

2.5.1Responsibility for Computing Schedules

In Section2.4.2,we described an algorithm that has each responsible process extract the data distribution information necessary for other processes to compute

the schedules.The size of the data distribution infor-mation to be transferred among processes is propor-tional to the size of the actual data to be transferred. As for the other algorithms,we could assign only one program to compute schedules,as in Section2.4.1.For the processes to compute schedules,they must receive data distribution information for the data structures on both programs.Therefore,the size of the data dis-tribution information to be transferred is?xed,and is not dependent on which program computes the sched-ules.We therefore assign the workload of building the schedules to the processes in both programs.

2.5.2Schedule Generation Algorithm

1:Send data distribution information about the data elements to be transferred in the local data struc-ture to all local and remote processes,based on the partitioned linearization

2:Receive data distribution information about both the local and the remote data structures for the part of the linearization that this process is responsible for

3:Generate partial communication schedule entries for each pair of matched local and remote data el-ements

4:Send partial schedules back to all processes of both programs

Algorithm Description

Two compact data descriptors

Meta-Chaos,using dereference functions BothCom

One program computes schedules

One compact and one non-compact descriptor

Meta-Chaos

BothCom

One program computes schedules/Send schedules back to all processes OneCom/OneBack

Deref

InterComm algorithm

Table1.The methods to generate communication schedules,with labels used in the performance graphs

the same communication schedule.We experimented with two types of computation environments.The ?rst is a local cluster environment where all programs run on a single cluster connected via a local area net-work.The second is a distributed Grid computation environment where programs run on two clusters con-nected by a wide-area network(in this case,Internet2). For the experiments in the local cluster environment, we have run two parallel programs on a50-processor Linux cluster at the University of Maryland,building schedules to copy distributed data between the pro-grams.Each processor is a650MHz PentiumIII ma-chine with768MB of memory,and the processors in the cluster are connected via channel-bonded Fast Eth-ernet,providing a200Mb/sec connection for each pro-cessor.For experiments in the Grid environment,we ran one program on the50-processor cluster just de-scribed and the other program on a22-processor Linux cluster at Ohio State University.Each processor in that cluster is a933MHz PentiumIII machine with512MB of memory,and the processors are connected via Fast Ethernet.In all the experiments,a single process of a parallel program was assigned to each processor.

We employed two types of distributed datasets.One dataset is a multi-dimensional array,block distributed across processes in all dimensions,of the type sup-ported by High Performance Fortran[13]or the Multi-block Parti library[1].That dataset can be described with a compact data descriptor.For all of the exper-iments,the data is a two-dimensional array of double precision?oating point numbers of size10241024. The second dataset is an explicitly(irregularly)dis-tributed array,of the type supported by the Chaos li-

brary[17],that must be described with a non-compact

data descriptor.That data is an array of double pre-

cision?oating point numbers,with array elements ran-

domly assigned to processes so that each process owns

approximately the same number of elements.Note that

the base data type of the data(e.g.,int,?oat,dou-

ble)does not affect the time to generate communi-

cation schedules,since all operations for computing

schedules employ offsets from the beginning of the

(array)data structure,in terms of the number of ele-

ments,not the number of bytes.The operations for

moving the data using the computed communication

schedules take into account the size of the base data

type,to perform the message sends and receives speci-

?ed in the schedule.The experimental space we ex-

plore is effectively three dimensional.The?rst di-

mension is for different combinations of the types of

data descriptors in the source and destination program

(e.g.,compact-compact).The second dimension is

the number of processes in the source and destination

parallel programs.Each program was run on up to

16processes,so that we can measure times for11 through1616processes(#sender processes#re-

ceiver processes),to show the scalability of the algo-

rithms.The last dimension is the number of data ele-

ments to transfer.To show scalability with respect to

the amount of data to transfer,we computed schedules

that transfer

ements).We also measured how well the workload is balanced.For all the experiments,the execution times shown are computed from the times for?ve runs, showing the average value after removing the smallest and largest value.However,in general the execution times of those?ve runs did not vary much,because the experiments were run when the cluster(s)was(were) otherwise unloaded.As was previously discussed,In-terComm generates essentially identical communica-tion schedules as Meta-Chaos,so both systems have

the same performance for transferring the data using the communication schedules.

To make the graphical presentation of the perfor-mance results easier to decode,we summarize the la-bels used for the various algorithm options in Table1. For the InterComm algorithms with a compact data de-scriptor,there are two sets of options.One set is for how to transfer data descriptors between the programs and the other set is for where to compute schedules. When the programs are run on a single cluster,there is almost no performance difference for the two meth-ods for transferring data descriptors,because they both require the same number of messages.We would ex-pect that the method that has only representative pro-cesses exchange compact data descriptors would per-form better in a Grid environment.However,the two clusters used for the experiments in the Grid environ-ment are connected via Internet2,which provides very high bandwidth between the clusters.We therefore did not see much difference in the performance of the two methods,Thus we do not show experimental results for the representative method,to simplify the presen-tation.In the rest of this section,we present experi-mental results for building communication schedules. In Sections3.2and3.3,we show experimental results on the local cluster environment.We compare exper-imental results in the Grid environment with those in the local cluster environment in Section3.4.

3.2Scalability

3.2.1Number of processes

Figures7,8and9show the effects of scaling the num-ber of processes in the sender and receiver programs. All processes were run on the50-processor cluster de-scribed earlier,and half the data in the datasets was involved in the data transfer.

(

)

Number of processes (sender x receiver)

(a)Varying the number of receiver processes

(

)

Number of processes (sender x receiver)

(b)Varying the number of sender processes

Figure7.Varying the number of processes, for two compact descriptors,transferring half of a10241024array between the sender and receiver programs.The line labeled Deref is for the Meta-Chaos library,while the others are labeled with the algorithm variations for InterComm,as described in Table1.

Two compact data descriptors:Figure7shows the times to generate communication schedules when both the sender and receiver program have compact descriptors for their data structures involved in the transfer.

Figure7(a)shows the effect of varying the num-ber of receiver processes,?xing the number of sender

processes at four.Since Meta-Chaos computes sched-

ules only in the receiver processes,its performance im-

proves as more receiver processes are added.However,

the Meta-Chaos performance decreases when there are

too many receiver processes(the point in the graph),because Meta-Chaos requires two communica-

tion operations within the receiver processes–one to

dereference data elements and one to send computed

partial schedules to other receiver processes.

When both sender and receiver processes compute

the schedules(the BothCom line in Figure7(a)),per-

formance is determined by the program that is run with

fewer processes,because a process in that program

performs more work on average than a process in the

other program.We therefore see that performance im-

proves with increasing numbers of receiver processes

until the number of processes in both programs is the

same(44).Beyond that point,performance is lim-ited by the sender processes.When only sender pro-cesses compute the schedules(the OneCom line),per-formance is not affected by increasing the number of receiver processes.In comparing the InterComm al-gorithms to the Meta-Chaos algorithm,the best Inter-Comm algorithm takes from15%to25%the time of the Meta-Chaos algorithm.

Figure7(b)shows the effects of varying the num-

ber of sender processes,?xing the number of receiver

processes at4.Meta-Chaos performance is fairly

constant because schedules are computed on the re-

ceiver processes.Meta-Chaos performance does im-

prove slightly with more sender processes,because

more sender processes are available to perform local

dereferencing operations.However,Meta-Chaos per-

formance degrades with too many sender processes

(the164point),because the receiver processes must

send partial schedules back to more sender processes.

The InterComm algorithm that computes schedules in

both programs(the BothCom line)improves in perfor-

mance until the numbers of sender and receiver pro-

cesses are the same,again because the overall perfor-

mance is limited by the program running with fewer

processes.Beyond that point(the84and164

points),performance degrades slightly because each

receiver process must send data descriptors to multi-

ple(2or4)sender processes.The algorithm that com-

putes the schedule in one sender process(OneCom)

speeds up as the number of sender processes increases,because it computes schedules only in the sender pro-cesses.For this scenario,the best InterComm algo-rithm takes from25%to50%the time of the Meta-Chaos algorithm.

500

1000

1500

2000

2500

3000

3500

4000

4x14x24x44x84x16 T

(

)

Number of processes (sender x receiver)

(a)Varying the number of receiver processes

600

800

1000

1200

1400

1600

1800

2000

1x42x44x48x416x4 T

(

)

Number of processes (sender x receiver)

(b)Varying the number of sender processes

Figure8.Varying the number of processes, for one compact and one non-compact de-scriptor with the sender program transferring half of a10241024array and the receiver program transferring into half of a element explicitly distributed array.The line labeled Deref is for the Meta-Chaos library,while the others are labeled with the algorithm varia-tions for InterComm as described in Table1.

One compact and one non-compact data descrip-

tors:Figure8shows performance results when the

sender program data structure is a two-dimensional,

block distributed array of size10241024with a

compact descriptor,and the receiver program has an

explicitly distributed dataset of double precision

?oating point numbers with a non-compact descriptor.

As we said before,we have two sets of options.One of

them is how to send schedules back.We support only

the send-back option that all processes send schedules

back to all processes when both programs compute

schedules.As we will see later,the performance is the

worst when both programs compute schedules in Grid

environment since it requires much expensive commu-

nication between two programs while it is the best in

the local cluster environment.The send-back option

affects the performance in Grid environment.Since

the performance with both computing programs is not

good and it should be used in the local cluster environ-

ment,we provide only the send-back method that all

processes send schedules back to all processes when

both programs compute schedules.

Figure8(a)varies the number of receiver pro-

cesses,?xing the number of sender processes at

4.When both the Meta-Chaos and InterComm(the

OneCom/AllBack and OneCom/OneBack)algorithms

compute schedules in the receiver processes(in the

program with the non-compact data descriptor),they

show similar performance characteristics with perfor-

mance improving greatly as having more receiver pro-

cesses.The InterComm algorithm that has both pro-

grams compute schedules(BothCom)shows good per-

formance with small numbers of receiver processes,

since the sender processes also compute schedules.

Although sender processes compute schedules,perfor-

mance degrades with more receiver processes(48 and416)since the algorithm requires too much com-munication among all sender and receiver processes.

Although the InterComm implementation that sends

partial schedules back to the sender processes from the

receiver processes has not yet been highly optimized,

the best InterComm algorithm is always better than the

Meta-Chaos algorithm.

Figure8(b)shows the effects of varying the number

of sender processes on performance.The Meta-Chaos

and InterComm algorithms(Deref and OneCom)that

have only one program compute schedules improve in performance until the number of sender processes is the same as the number of receiver processes.Al-though the InterComm algorithms compute schedules in the receiver processes,a small number of sender processes decreases performance because each sender process must send its compact data descriptor to more than one receiver process and also receives larger schedules from the receiver processes.For the In-terComm algorithm that has both programs compute schedules(BothCom),performance improves linearly because more processes compute schedules.Although this method requires a large amount of communication among processes,the cost is not very high in the local cluster environment.However,in Section3.4we will see that this method does not perform well in a Grid environment because of high communication costs.In many cases in the local cluster environment,the Inter-Comm algorithms take less than60%of the time for the Meta-Chaos algorithm to generate the same sched-ules.

Two non-compact data descriptors:Figure9 shows communication schedule building performance when both programs have an explicitly distributed element double precision?oating point array.In this case,InterComm currently implements a single algo-rithm.

In Figure9(a),both the Meta-Chaos and Inter-Comm algorithm performance improves as the number of receiver processes increases.The Meta-Chaos al-gorithm performance improves because schedules are computed in the receiver processes.The InterComm algorithm takes less time because schedules are com-puted using all sender and receiver processes,which is why the InterComm algorithm performs better than the Meta-Chaos algorithm.In this experiment,the In-terComm algorithm takes from20%to30%of the time for the Meta-Chaos algorithm.

Figure9(b)shows the effect of varying the num-ber of sender processes.With more sender processes, InterComm performance improves,because perfor-mance depends on the total number of processes in both the sender and receiver programs.In fact,the per-formance is almost the same as in Figure9(a).How-ever,Meta-Chaos performance decreases as the num-ber of sender processes increases.The Meta-Chaos al-gorithm computes schedules in the receiver processes, but the receiver processes must send schedules back to

2000

4000 6000 8000 10000 12000

T i m e (m i l l i s e c )

Number of processes (sender x receiver)(a)Varying the number of receiver processes 0

2000

4000 6000 8000 10000 12000

T i m e (m i l l i s e c )

Number of processes (sender x receiver)

(b)Varying the number of sender processes

2000

4000

6000

8000 10000

T i m e (m i l l i s e c )

Number of processes (sender x receiver)

(c)Varying numbers of both sender and receiver processes

Figure 9.Varying the number of processes for two non-compact descriptors with both sender and

receiver programs transferring half of a

element explicitly distributed array.The line labeled Deref is for the Meta-Chaos library and the other is for the InterComm library.

the sender processes.When there are more sender pro-cesses,each receiver process must send schedules to more sender processes.In this case,InterComm takes from 7%to 50%of the time for Meta-Chaos.

Figures 9(a)and 9(b)show that the Meta-Chaos al-gorithm speeds up with more receiver processes,but slows down with more sender processes,while Inter-Comm speeds up with more processes in either the sender or receiver programs.Figure 9(c)shows an-other view of performance,when the number of pro-cesses in both the sender and receiver program in-creases.InterComm performance increases because it can make effective use of all available processes.However,Meta-Chaos performance only improves up to 44processes,and then gets worse.This result im-plies that the performance gain from having more re-ceiver processes is less than the performance loss from more sender processes when there are more than 44processes.In this experiment,InterComm takes from 4%to 35%of the time for Meta-Chaos.

8163264128

T i m e (m i l l i s e c )

Fraction of data elements to transfer (a)Varying the number of transferred data elements with two compact data descriptors 128

256

512

1024

20484096

T i m e (m i l l i s e c )

Fraction of data elements to transfer

(b)Varying the number of transferred data elements with

one compact and one non-compact data descriptors

128

256512102420484096819216384T i m e (m i l l i s e c )

Fraction of data elements to transfer

(c)Varying the number of transferred data elements with two non-compact data descriptors

Figure 10.Varying the number of transferred data elements for 4sender and 4receiver processes.The data structure with the compact data descriptor is a 10241024block distributed array and the data structure with the non-compact descriptor is a explicitly distributed array.The graphs are labeled as described in Table 1.

3.2.2Number of data elements to transfer

Figure 10shows the effects of varying the amount of data to transfer,for the three combinations of data de-scriptors using 4sender and 4receiver processes.As the amount of data that must be transferred increases,the time required to compute the schedules increases.Note that the times increase approximately linearly

with the amount of data to transfer.In results not shown,we have seen similar results for other numbers of sender and receiver processes.Both the Meta-Chaos and InterComm algorithms show good scalability with respect to the number of data elements to transfer,but InterComm has better absolute performance.

3.3Workload Balance

15 20

1x4

2x44x48x416x4

T i m e (m i l l i s e c )

Number of processes (sender x receiver)

Max-Sender Avg-Sender Max-Receiver Avg-Receiver

Figure 11.InterComm maximum and aver-age times across sender processes (Max-Sender and Avg-Sender)and receiver pro-cesses (Max-Receiver and Avg-Receiver)to compute schedules.The sender and receiver programs transfer half of a 10241024array,both with compact data descriptors.The al-gorithm is the All/Both one from Table 1.

In Meta-Chaos,the processes of the sender pro-gram are idle while the processes of the receiver pro-gram compute schedules.In other words,the work-load is poorly balanced across the programs.However,the workload within the receiver processes is well-balanced because Meta-Chaos distributes the schedule building work evenly across those processes.

Figure 11shows how well the workload is dis-tributed for one of the InterComm algorithm vari-ants for two compact data descriptors.The algo-rithm shown has all processes in the sender and re-ceiver programs exchange their compact data descrip-tors and compute schedules completely locally,so that no schedules need to be sent between processes.In Figure 11,Max-Sender and Avg-Sender show the max-imum time and the average times for the sender pro-cesses,while Max-Receiver and Avg-Receiver are the times for the receiver processes.Since the number of receiver processes is ?xed at 4,the times for max2and avg2do not change much for increased numbers of sender processes.The times for Max-Sender and Avg-Sender decrease with more sender processes.We see

that the workload across the two programs depends on the number of processes in each of the programs.When we have sender and receiver processes,a sender process has approximately

amongbetween的区别用法全

among between为近义词，皆可表示“在……之间”，但用法大不相同，现归纳比较如下：一、among一般用于三者或三者以上的“在……中间”，其宾语通常是一个表示笼统数量或具有复数意义的名词或代词。 His house is hidden among the trees．他的房子隐藏在树林之中。 She sat among the children．她坐在孩子们中间。二、between一般指两者之间，其宾语往往是一个具体数目的人(物)，或者是由and连接的两个具体的人(物)。 There was a fight between the two boys．这两个男孩间发生了一场格斗。 I am sitting between my parents．我正坐在我父母中间。三、把两者以上的为数不多的人或事物单独地看待，用and连接时，要用between；把两者以上的人或事物看成一群、一堆或一组而不是个体时，要用among。 Switzerland lies between France,Italy,Austria and Germany．瑞士位于法国、意大利、奥地利和德国之间。 The old man’s cottage lies among the trees．老人的小木屋在树林中。四、between也可用于三者以上的事物之间，强调一物与数物之间的关系。 The small village lies between the three mountains．

小村庄位于三座大山之间。 I saw something lying between the wheels of the train．我看见火车轮子之间有什么东西。五、涉及人或事物之间的区别以及人或事物之间的关系时，一般要用between。 We must find out the difference between the three companies. 我们必须查清这三家公司之间的区别。 The relations between various countries are very important. 各国之间的关系是很重要的。六、表示“由于……合作的结果”时，要用between。 Between them they landed the fish．他们协力把鱼拖上了岸。 Between the five companies the project was soon completed．在五家公司的齐心协力下，这项工程不久就完成了。七、当and连接三者或三者以上的人(物)而仍然强调两者的并列时，常用between。 The hospital lies between a river and hills．医院坐落在一条河与群山之间。 The park lies between a road and the woods．公园位于一条马路与树林之间。八、在divide,share等表示“分享”之类的动词之后。若接一个表示三者或三者以上的复数名词时，用among或between均可。 The father divided his money among/between his three sons。

四年级一件难忘的事400字作文【五篇】

四年级一件难忘的事400字作文【五篇】四年级一件难忘的事400字精选作文篇一在童年里有许许多多的事情就像在海滩上捡贝壳，有绚烂的笑，是开心的往事；有黯淡的，像是勾起一段伤心的往事，都使我非常难忘。早晨，风和日丽，我去帮妈妈和爸爸买早餐，等我去到早餐店，我发现早餐店旁有一大群人，我带着好奇心走去，让我大吃一惊，我发现那有一位老奶奶绊倒在地上。我心想：为什么一位老奶奶搬绊倒在地上，但每一个人都无动于衷，还围观老奶奶，真是不尊重老人！我气得暴跳如雷，我就问我旁边的大人：“你们为什么不扶老奶奶”那人就说：“因为最近在新闻里，有很多老人成心绊倒在地上让路人扶起她。”谁知老人就说：扶起他的人是推他的人。老人还说：赔我钱。我听完就有一点不敢拉老奶奶了。这时，一位青年人走过来，马上扶起老奶奶，旁边的人说，那个青年人真笨，青年人就说：“难道金钱比生活还重要吗？”青年人问奶奶要紧吗？奶奶说，不要紧，谢谢你，小伙子。青年人就说不用谢，说完就走了，我看到青年人的身影，我觉得很惭愧。这一件事使我非常难忘，假如社会的人都献出一点爱心，那么社会就是充满爱心的。

四年级一件难忘的事400字精选作文篇二每一个人都有自己最难忘的事，我最难忘的事就是三年级那次集体打连响儿。记得那一次我们拿着漂亮的连响儿，排着整齐的队伍，伴随着美好的音乐，踏着坚决的步伐，熟练地打了起来。看到这样整齐划一的队伍，我不由心生骄傲：看来我的努力还是没有白费呀！但放眼又一望，我们四周可以说是人山人海，有无数双眼睛正盯着我们，我心里不知从哪里来了一种莫名的紧张，我头上冒着冷汗，腿不由的发着抖。就在这一霎时，我做错了一个动作，登时，我感觉所有人的目光都投向了我一个人，我这颗本来就不平静的心，现在显得更加紧张了、更加不安了，我一下子都不知该干什么了，我真想有一个洞，让我钻到洞里去，我以前打的是那么灵敏，可现在却有气无力，仿佛有一个小精灵把我这活泼的精神给夺走了，经过漫长的煎熬，我好不容易才等到了结束的那一刻。我回家把这件事告诉我妈妈，妈妈说：想做好一件事就要加倍的努力，不要认为自己会做了就可以了，妈妈的这次教导，将使我一生受益。四年级一件难忘的事400字精选作文篇三我家有各种各样的陶瓷器皿，比如：碗，花瓶，勺子，水杯等。它们形态各异，五彩缤纷。它们站在一起，仿佛都在展示自己曼妙的

人教版四年级(上册)数学专项训练

人教版四年级（上册）数学基础知识专项训练

人教版四年级上册数学基础知识填空题专项训练１、由5个千万、4个万、8个十和9个一组成的数是（），读作（），取近似值到万位约是（）。 2、406000000读作（），这个数中的6在（）位上，表示（），改写成用万作单位是（）。 3、一周角=（）平角=（）直角。 4、367÷23把23看作（）来试商比较方便。 5、下午3：00时针和分针夹成的最小角是（）度。 6、在数位顺序表中，从右起第四位是（）位，这个数的计数单位是（），如果这个数位上的数字是8，8表示（）。 7、5个一百万、4个十万、2个千和4个一组成的数是（）。读作（），它有（）个计数单位。 8、在9、8中间添（）个0，这个数才是九千万零八。 9、一个数加上2的和比最小的五位数多1，这个数减2是（ 10、在数位顺序表中，从右起第四位是（）位，这个数的计数单位是（），如果这个数位上的数字是8，8表示（）。 11、5个一百万、4个十万、2个千和4个一组成的数是（）。读作（），它有（）个计数单位。 12、在9、8中间添（）个0，这个数才是九千万零八。 13、一个数加上2的和比最小的五位数多1，这个数减2是（） 14、120分米=（）米 540秒=（）分 72小时=（）天 132个月=（）年 15、计量角的单位是（）。（）是量角的工具。 16、角的大小要看两边（），（）越大，角越大。 17、线段有（）个端点。把线段的一端无限延长，就得到一条（），把线段的两端无限延长，就得到一条（），它（）端点。 18、过一个点可以画（）条直线，过两点可以画（）条直线。 19、按照从大到小的顺序排列下面各数 88000 80800 80008 80080 ________________________________________________ 20、把锐角、平角、钝角、直角、周角按下列顺序排列。（）＞（）＞（）＞（）＞（） 21、4293÷4口，要使商是二位数，口可以填（）

2018初中英语词汇之among与between的用法区别

2018初中英语词汇之among与between的用法区别一般说来，among 用于三者或三者以上的在中间，其宾语通常是一个表示笼统数量或具有复数(或集合)意义的名词或代词；而between 主要指两者之间，其宾语往往是表示两者的名词或代词，或者是由and 连接的两个人或物： They hid themselves among the trees. 他们躲在树林中。 There was a fight between the two boys. 这两个孩子打过一次架。 Im usually free between Tuesday and Thursday. 我通常在星期二与星期四之间有空。在下列情况，between 可用于三者： (1) 当两个以上的人或物用and 连接时： between A, B and C 在A、B和C 之间 (2) 涉及事物之间的区别或各国之间的关系时： the difference between the three of them 他们三者之间的区别 the relations between various countries 各国之间的关系 (3) 表示由于合作的结果时： Between them they landed the fish. 他们协力把鱼拖上了岸。

(4) 在divide, share 等表示分享之类的动词之后，若接一个表示三者或三者以上的复数名词时，用among 和between 均可：He divided his money among [between] his five sons. 他把钱分给了5 个儿子。

关于难忘的一件事四年级作文3篇

关于难忘的一件事四年级作文3篇【导语】每个人都有难忘的事，我当然也不会例外。下面是小编为大家整理的关于难忘的一件事四年级作文，仅供参考，欢迎大家阅读。关于难忘的一件事四年级作文一六岁那年，发生了一件令我难忘的事情——在商场里我差点儿走丢了！那天，天气很冷，我们一家人去商场买东西。买完后，我们在一楼闲逛。走着走着，我松开了妈妈的手，跑去看一款造型独特的吸尘器，当回过头时，爸爸妈妈已经走得无影无踪了。商场里人很多，我非常害怕，心想：“妈妈也一定急坏了！怎么办？” 我在商场里转了一圈，越来越害怕、恐惧，忽然，我想起爸爸的车在路边停着，干脆就“守株待兔”吧！所以就跑去爸爸的车旁边等。过了一会儿，爸爸妈妈终于找到我了，妈妈冲过来，一把把我紧紧地搂进怀中，口里不停的说：“吓死妈妈了，妈妈腿都软了！”然后又语无伦次的夸我聪明，没瞎跑，唉！我心里就像打翻了五味瓶一样，说不出的'滋味儿！从那以后，我再也不敢轻易放开妈妈的手了。关于难忘的一件事四年级作文二

一个美丽的晚霞中，那七元五角的事使我永刻不能忘怀。我背着书包回了家，对妈妈和蔼地说：“妈咪，我回来了！哦，对了妈咪，老师吩咐我们要买一支黑笔，您可不可以给我两块半。”说完，妈咪点了点头，从黑纹色的包里拿出了两块半，我拿起了钱就往商店里奔驰去。从商店里拿起了一支晶亮的黑笔，到收银台付了钱，就发现了里头还藏了个五元，我的贪婪之心有苏醒了。付了钱后就在半路上犹豫到底要不要把这五元给妈咪呢？在犹豫时就看见了一个小妹妹，她看见前面的叔叔掉了十元钱，就还给了那个叔叔，我就在想，连一个比我还小的妹妹都那么听话，我不能落后，就跑回了家，把五元还给了妈咪。做人要道德，道德只是个简单的是与非的问题，实践起来却很难，把五元还给妈咪，一个人要是从小就受到这样的教育的话，就会获得道德实践的勇气和力量。关于难忘的一件事四年级作文三在我的脑海里发生过许多难忘的事情，但是有一件事令我记忆犹新，那是发生在去年的一件事。那是一天下午我在我在学校上学的时候，我借了刘利娜的语文作业抄，下课了，我抄完了。就和女同学玩橡皮筋。正在这时候周老师来了，我们吓得腿都软了。在回家得路上我心里忐忑不安，心想这样做对不对呢？

2020年四年级数学上册解决问题专项综合练习

2020年四年级数学上册解决问题专项综合练习 1. 某小学学生向汶川地震灾区踊跃捐款，请根据下图填空．（1）______年级的捐资金额最多，是______元．（2）六个年级捐资金额这组数据的中位数是______元．（3）二年级捐资金额是四年级捐资金额的______%．（4）四年级捐资金额比五年级少______%． 2. 特快列车每小时行170千米,普通列车每小时行107千米（1）特快列车40小时行多少千米？（2）普通列车40小时行多少千米？ 3. 2010年第六次全国人口普查显示，我国人口总数大约是1339720000 人，读横线的数。 4. 如图是广本汽车销售店2013年一月至五月的销售情况统计图．

请你根据上图，完成以下的填空：（1）______月的销售量最少．（2）二月份的销售量比一月份多______台．（3）五月份的汽车销售量是三月份的______%．（4）四月份的汽车销售量比二月份增加了______%． 5. 下图显示的是某班20人在“献爱心”活动中捐图书的情况，该班级人均捐了多少册书？ 6. 小强拆了甲、乙两架纸飞机，下面是这两架飞机前5次试飞情况的统计图．（1）估一估，前5次试飞，哪一驾纸飞机飞行的距离远一些？（2）第6次试飞，甲飞机的飞行距离是21米，乙飞机的飞行距离是10米．在图中画出表示这架纸飞机第6次试飞飞行距离的直条，并计算这架纸飞机6次飞行距离的平均数各是多少．

7. 早餐店炸油饼，油锅一次最多能炸2张饼，炸熟一张饼要4分钟（正反面各2分钟）．如果一个客人要9张饼，早餐店老板最快多少分钟可以把油饼给他？ 8. 某机床厂各车间男、女工人统计图．看上面的统计图回答下面的问题．（1）三个车间共有工人______人，其中女工共______，男工共______人．（2）三个车间平均每车间有______人．（3）第三车间男工人数比第二车间男工人数多______ %． 9. 学校进行团体操表演，每行站20人，正好站24排．如果要站成16排，那么每行需要站多少人？ 10. 50枚棋子围成圆圈，编上号码1、2、3、4、…50，每隔一枚棋子取出一枚，要求最后留下的枚棋子的号码是42号，那么该从几号棋子开始取呢？ 11. 估一估，哪个算式的商最接近圈中的数，画上“√”。先估算出每个圆中的四个算式的商是多少，再比较一下，看哪个算式的商最接近圈中的数。 12. 小红在计算一个数乘2.5时，忽略了2.5的小数点，得到的结果是75，正确的结果是多少？

英语地点介词的正确使用方法

英语地点介词的正确使用方法地点介词主要有at ，in，on，to，above，over，below，under，beside，behind ，between。它们的用法具体如下： 1、at （1）at通常指小地方：In the afternoon，he finally arrived at home。到下午他终于到家了。（2）at通常所指范围不太明显，表示“在……附近，旁边”：The ball is at the corner。球搁在角落里。 2、in （1）in通常指大地方：When I was young，I lived in Beijing。我小时候住在北京。（2）在内部：There is a ball in in the box。盒子里有只球。（3）表示“在…范围之内”（是从属关系）： Guangdong lies in the south of China。深圳在中国的南部。 3、on

（1）on主要指“在……之上”，强调和表面接触： There is a book on the table。桌上有一本书。（2）表示毗邻，接壤（是相邻关系）： Canada lies on the north of America 加拿大在美国的北边（与美国接壤）。 4、to 主要表示“在……范围外”，强调不接壤，不相邻。 Japan is to the east of China。日本在中国的东面。注意：（1）at 强调“点”，on 强调“面”，in 强调“在里面”，to 表示“范围外”。（2）on the tree：表示树上本身所长着的叶子、花、果实等 in the tree：表示某物或某人在树上 on the wall：表示在墙的表面，如图画、黑板等 in the wall：表示在墙的内部中，如门窗、钉子、洞、孔 5、above

四年级难忘的一件事作文400字

每个人的每一天都充满着快乐、伤心、害怕或者难过，并且会有一些难忘的事情。你难忘的一件事是什么呢？下面是小编和大家分享的“四年级难忘的一件事作文400字”，一起来看看吧，希望大家喜欢。四年级难忘的一件事作文400字（一）虽然我才上四年级，但是我的生活里也有很多难忘的事。我就给大家讲一件让我难忘的事吧。有一次爸爸让我做大米饭，给我留下了深刻的印象。那天中午，爸爸对我说：“今天中午你来帮妈妈做大米饭吧，正好也锻炼锻炼。”我听了以后高兴地说：“好啊，小菜一碟。”说完以后我就开始动手了。我先把大米放在盆里淘干净，再倒进锅里，放在炉子上就到客厅去看我喜欢看的电视去了。在我看得正高兴的时候，忽然就闻到厨房里传出来一股烧糊的味儿来。我一看表，已经过去了半个小时了，我马上跑进厨房，把锅端下来。打开锅盖，就看见了变黑的锅底。可我没有放弃，又做了一次。我又把米淘干净倒进锅里以后，我又去写作业去了。过了一会儿，我又闻到了那股烧糊的味道。我跑到厨房一看，锅底又黑了。这一看，我的眼泪都快掉下来了，这可怎么办啊？这时候，爸爸来了，对我说：“不用哭，做什事都要专心，再来一次。”我点了点头。第三次，我终于把大米饭做熟了，看见自己的成果，我感到很高兴。这件事让我明白了：做什么事都要专心致志，不能三心二意。四年级难忘的一件事作文400字（二）今天，是我和姐姐第一次去买菜。妈妈给了我们钱，说：＂要注意安全哦。＂我们说：＂知道了。＂一路上，我的心在扑通扑通地跳。来到菜市场，我都目瞪口呆了，各种各样的蔬菜放在我面前，有西兰花、芫茜、冬瓜……还有许多嘈杂声呢！我回忆起妈妈要我买的菜，哦！要买鱼。我走进市场左看看，右望望，嘿，找到了！我走过去，正准备对卖鱼的叔叔说话，一位阿姨便先说了一步。阿姨买完了鱼，我又准备说，谁知，又有一位阿姨要买。5分钟过去了，我还没买到，我着急了，对着叔叔大声说：＂叔叔，请给我一块鲜鱼肉！＂叔叔笑眯眯地说：＂你要多少呀？＂＂4元。＂我回答说。叔叔弄好了鱼，用袋子好了给我，我付了钱并说了一声＂谢谢＂就走了。我又想到要买菜，对了，还有芫茜！我找到了买我所需要的地方，就跟那位婆婆要了一点。当我走出市场时，天空下起了牛毛似的雨，我跟姐姐说：＂我们快走吧！＂接着，我们骑着自行车，飞快地奔回家。骑到半路，雨就更大了，我们到了一棵树下遮雨，但是好久都不见天空好转。为不拖延时间，我和姐姐顶着风冒着雨冲回家。

四年级上册数学专项练习

薄弱环节专项全练全测(一) 数与代数【薄弱点一】大数读、写的准确性一、读出或写出下列各数。(8分) 30050082 读作：______________________ 3960400090 读作：______________________ 一千零四十万四千零二十写作：________________ 七千零三亿零二十万零五写作：________________ 二、求近似数。(8分) 1．省略万位后面的尾数。 16493560≈9528641≈ 2．省略亿位后面的尾数。 2709546312≈9953364778≈ 三、选择。(6分) 1．下面三个数中，一个0也不读出来的是( )。 A．50000800 B．50080 C．5008000 2．下面各数中，用9，8，7，5，0这五个数字组成的最接近8万的数是( )。 A．89750 B．80579 C．85079 3．74□7600000≈75亿，□里最大能填( )。 A．8 B．9 C．7 【薄弱点二】乘、除法计算的准确性四、改错。(12分)

五、笔算下面各题，带※号的要验算。(24分) 240×38＝207×40＝※360×50＝ 380÷70＝ 694÷72＝※633÷21＝【薄弱点三】用乘、除法知识解决问题的正确性六、解决问题。(34分) 1．一块长方形草坪的面积是120平方米，改建后，长扩大到原来的4倍，宽扩大到原来的3倍，改建后草坪的面积是多少？(8分) 2．某花店在教师节到来之际，搞促销活动：每盆月季花16元，买3盆送1盆。照这样计算，买4盆花，每盆比原来便宜多少钱？(8分) 3．一根木头长15米，把它平均分成5段，每锯一段需8分钟，锯完一共要花多少分钟？(8分) 4．一辆长途客车6小时行了348千米。照这样的速度，它12小时可以行多少千米？(10分) 解法1：先求客车1小时行驶多少千米，再求12小时可以行驶多少千米。解法2：先求12小时中有几个6小时，再求几个348千米是多少。【薄弱点四】合理安排时间七、小红家来了客人，她要沏茶招待客人。(8分) 找茶叶：1分钟沏茶：3分钟接水：1分钟洗杯子：2分钟洗水壶：2分钟烧水：10分钟应该怎样安排时间才能使所用时间最短？至少要多长时间？

四年级作文一件难忘的事

大年三十，踩气球白雪皑皑的夜晚，从一间小屋里传出了阵阵欢快的笑声。那是大年三十，人们正忙着发压岁钱。只见家人们从裤兜里掏出一张比一张大的钱，塞到满怀欣喜的孩子们的手里。我们拿着钱就去买了许许多多的气球。我们一家人开始给气球打气。我们不知疲惫地打了一个又一个。不一会儿，整整地堆满了一间大房子。气球的颜色和形状各不相同，有大的、有小的、红的、绿的、白的……在灯光的照耀下，非常漂亮。我挑了一些漂亮点儿的气球，挂了起来，把客厅装扮得像春节联欢晚会现场一样光彩亮丽。时钟刚好走到12点时，我便急得像上锅的蚂蚁一样，立即跳向气球堆。“啪啪……”气球发出响亮的爆炸声。这清脆的爆炸声吸引了爸爸妈妈和弟弟的围观。他们被我那热乎劲儿感染了，也过来跟我一起踩气球。“噼噼啪啪……”一阵响，就像放鞭炮一样，既吓人又好玩。清脆的爆炸声给热闹的夜晚增添了一份别具一格的色彩。不一会儿，整整一屋的气球就只踩剩一个了。我上前去一踩，没想到它竟然跑了，气得我张牙舞爪直瞪眼，恨不得一脚把它踩扁。我那滑稽的举动惹得爸妈们突地大笑起来。我不服气，气鼓鼓的又跑上去，对着气球狠狠地一踩，心想这回肯定能踩扁它。可谁知，我一个不小心，脚下一滑，便跌坐在气球上。“啪……”完了，气球在我的屁股上炸开了花。这下子，可把我的爸妈们逗得更乐了。欢快的笑声在小屋的四周回荡，也回荡在我生命的旋律中…

捉螃蟹记得去年冬天的一个早晨，天气晴朗，阳光明媚，我和表妹一起到河边的沙滩上捉螃蟹。河里的水非常的少，到处都是石头。螃蟹，就藏在这些奇形怪状的石头下面。我们一边沿着岸边走，一边翻着旁边的石头。突然，表妹发现了一只螃蟹，激动万分地对我说：“哥哥，这块石头下面好像藏了一只螃蟹。”我们连忙轻轻地走过去，慢慢地蹲下去，小心翼翼地把石头搬开，快速伸过手去捉它，没想到螃蟹用两只大钳子把我的手狠狠地夹住了，痛得我脸都涨红了。我赶紧用力的把手往外一甩，才摆脱掉这该死的螃蟹。“呀！流血了！”表妹惊呼道。我低头一看，才发现是自己的手被螃蟹夹了一条大口子，直流血。我淡然地擦擦手说没事，可怎知表妹被我那流血直流的手吓得大哭起来。忍者疼痛，我迅速一抓，就把螃蟹扔进了瓶子里，表妹立即破涕为笑。我们终于捉了一只螃蟹，心里有说不出的高兴，我那鲜红的手指和表妹那还挂着晶莹泪珠的脸，在阳光下，闪闪发光……

四年级数学上册专项练习

填空题 1、当除数是34时，试商时可以把除数看作( )，这样初商容易偏( )。 2、（）个26相加的和是468；（）比12个15多20。 334＝21），这时被除数是（）。 4、在括号里填上合适的数。 480秒=（）分540厘米=（）分米624时＝（）日 5、我们戴的红领巾上有一个（）角，两个（）角。 6、钟面上，分针转动360度，相应地时针转动（）度。从3：00走到3：15，分针转动了（）度。 6点时，时针和分针所组成的角是（）度，是（）角， 3点时，时针和分针所组成的角是（）度，是（）角。 7、把“78÷26=3，26+3=29”合并成一个综合算式是（）。 8、在5○1÷58中，如果商的最高位在十位上，○中最小填（），还可以填（）。如果3□2÷36的商是一位数，□里的数最大可以填（），最小可以填（）。在算式□17÷53中，要使商是两位数，□最小填（）；要使商是一位数，□最大填（）。 9、在公路上有三条小路通往小明家，它们的长度分别是125米、207米、112米，其中有一条小路与公路是垂直的，那么这条小路的长度是（）米。 10、李阳从1楼到3楼用了12秒，她从一楼到六楼需要（）秒。 11．二百零六亿八千万写作（），改写成用“万”作单位的数是（）万，用“亿”作单位这个数的近似数是（）亿。 12．2个千万、7个万、8个百和5个十组成的数是（），这个数读作（）。 13．由6、7、5、1、0组成的最大数是（），最小数是（）。14、计算除法时，错将除数36看成63，结果得到商12。请你帮他算一算，正确的商应该是（）。

15．一个边长24厘米的正方形面积是（）平方厘米。 16、把两道算式组成综合算式，再用递等式计算。 14×2=28 21×4=84 10×5=50 28-15=13 200-84=116 30＋50=80 （）（）（）选择、判断题 1、30度的角被投影仪投到屏幕上时角就变大了。（） 2、570÷40＝14……1。（） 3、在方向板上，北和西南之间夹角是135°。（） 4、在同一平面内，两条直线不是相交就是平行。（） 5、在同一平面内，两条直线不是平行就是垂直。（） 6、4个同样大的正方体可以拼成一个较大的正方体。（） 7、在10倍的放大镜下看15度的角就变成了150度。（） 8、六位数一定比七位数小。（） 9、平角就是一条直线，周角就是一条射线。（） 10、三位数除以两位数，商不可能是三位数。（） 11、三位数除以两位数，商最多是两位数。（） 12、过一点能画无数条直线。（）13．两条直线相交成直角时，这两条直线就互相垂直。（）14．观察物体时，在同一位置看到相同的形状可能有不同的摆法。（）15、两条不相交的直线叫做平行线。（） 1、在4□7÷46的商是两位数，□中的数最小是（）。 ①7 ② 6 ③5 2、想使物体从斜面上向下滚动时尽可能地快，下面的选项中，木板与地面的夹角是（）度最符合要求。 ①20 ②38 ③10 ④80

难忘的一件事作文400字四年级_1

难忘的一件事作文400字四年级难忘的一件事作文400字四年级最令我难忘的一件事，就是偷吃四色小辣椒了，那四色辣椒的滋味，真是难以形容，令人难忘。在我家楼下有一个小菜园，那里是爷爷的乐园，他每天都会去那，乐呵呵地给他种的蔬菜浇水。在这个菜园里，有白菜、罗卜和辣椒。最引起我主意的就是辣椒了。爷爷种的不是一般的辣椒，是四色小辣椒。四色小辣椒是一种能变色的小辣椒，在每个季节变一种色，春天是绿色、夏天是黄色、秋天是紫色、冬天是红色，非常神奇。一天我在楼下散步，当时正是秋天的一个。我走到菜园时，看见里面的小辣椒，紫紫的，看起来很好吃。心想：着么紫的小辣椒难道还会辣吗？肯定能好吃，心中便有了一种想法，摘一个尝一尝，反正爷爷也不在。我以飞一般的速度跑了过去。用手一拽，边摘了下来。我像得了一件宝贝似的，小心翼翼地捧着四色小辣椒，向家跑去。到了家，见没人，才放了心。跑进厨房把四色小辣椒洗干净，然后把放四色小辣椒放进嘴里嚼，不嚼不要紧，一嚼可坏了。顿时，舌头、喉喽就像要喷火一样，辣的我说不出话来。我跑到水

龙头前大口大口的喝起水来。这才好点，没想到这四色小辣椒比别的辣椒辣上十倍。这件事一直刻在我的心头，甩也甩不掉呢。四年级:戴恋佳难忘的一件事作文400字四年级最令我难忘的一件事，就是偷吃四色小辣椒了，那四色辣椒的滋味，真是难以形容，令人难忘。在我家楼下有一个小菜园，那里是爷爷的乐园，他每天都会去那，乐呵呵地给他种的蔬菜浇水。在这个菜园里，有白菜、罗卜和辣椒。最引起我主意的就是辣椒了。爷爷种的不是一般的辣椒，是四色小辣椒。四色小辣椒是一种能变色的小辣椒，在每个季节变一种色，春天是绿色、夏天是黄色、秋天是紫色、冬天是红色，非常神奇。一天我在楼下散步，当时正是秋天的一个。我走到菜园时，看见里面的小辣椒，紫紫的，看起来很好吃。心想：着么紫的小辣椒难道还会辣吗？肯定能好吃，心中便有了一种想法，摘一个尝一尝，反正爷爷也不在。我以飞一般的速度跑了过去。用手一拽，边摘了下来。我像得了一件宝贝似的，小心翼翼地捧着四色小辣椒，

on,acrossfrom,nextto或between…and练习和辨析

用on, across from, next to或between…and 完成下列句子。 1. 学校在书店的对面。 The school is ____ the bookshop. 2. 我坐在他旁边。 I sit ____ him. 3. 我们把这个书桌放在床和椅子之间。 Let’s put the desk ____ the bed ____ the chair. 4. 墙上有一些画。 There are some pictures ____ the wall. [Key: 1.across from 2.next to 3.between；and 4.on] 【辨析】这几个词语均用来表示位置，但用法有别。 ⑴on 作介词，常用于表示方位和地点，侧重指紧贴着某物，意为“在……上面”。例如：在教师的讲桌上有一些花。 There are some flowers on the teacher’s desk. There is a map of China on the wall. 墙上有一张中国地图。注意：在美式英语中，表示“在街上”用介词on，但在英式英语中，则常用介词in。例如： The Greens live on(in) Wall Street. 格林一家住在华尔街上。 Their house is on(in) that street. 他们家就在那条街上。 ⑵across from 相当于介词，在美式英语中表示“在……的对面；在……的对侧”，表示此含义时，在英式英语中也可以只用across。例如： The pay phone is across from the library. 公用电话就在图书馆的对面。 Can you see the shop across from the river? 你能看到河对岸的商店吗？ ⑶next to 相当于介词，意为“紧挨着，紧靠着”。例如： There is an airport next to the park. 有个机场紧挨着那个公园。 The pay phone is next to the post office. Go along and turn left, you’ll see it. 公用电话紧挨着邮局。你向前走，往左拐就能看到它。 ⑷between…and 意思是“在……和……之间”，用于表示两者间。between是介词，后接人称代词时，该人称代词必须用宾格形式。例如： The library is between the hotel and the supermarket. 图书馆在旅馆和超市之间。 You sit between him and me. 你坐在我和他之间。注意：between 后面也可以接代表两者的复数名词或代词。例如： The video arcade is between two shops. 电子游戏中心在两个商店之间。 They are twins. I can’t tell the difference between them. 他们是双胞胎，我看不出他们之间的区别。

四年级难忘的一件事400字作文

大家都经历过让自己难忘的事，究竟是哪件事让你印象深刻至今难忘呢？下面是小编整理的“四年级难忘的一件事400字作文”，仅供参考，欢迎阅读，希望对大家有所帮助。四年级难忘的一件事400字作文（一）我已在世上度过了十个春秋，在我的记忆里，好多事已随着时间的流逝变得模糊啦，可是有件事像是烙在了我的脑海里，使我怎么也忘不掉。记得那是一年夏天，天热的出奇，我和爸爸妈妈去文化宫玩，当时我穿着一件小背心和一条小短裤，鞋子走起路来还会发出“吱吱”的响声，我和爸爸妈妈坐在凳子上乘凉，我看到一群鸽子飞来，好美呀！有白的有灰的，可鸽子为什么又叫和平鸽呢？我疑惑的问爸爸，爸爸说：“传说在古代的时候，鸽子可以传递消息，因为鸽子象征着和平象征着祖国美好的未来，所以鸽子又叫和平鸽。”哦！原来是这么回事，那我就要犒劳犒劳它们啦，我买了一包玉米去喂鸽子，有一只鸽子很丑，好象是安徒生笔下的丑小“鸽”，可是，它并没有为它自己的丑而感到自卑，当我喂它们食物时它也和别的鸽子来抢食物，它的嘴啄一粒，就慢慢的吞下去，那样子真有趣，等它吃饱了，它就用翅膀盖住身子和头在那睡上了大觉！我喜欢上了那只“灰小鸽”，喜欢它那种自信，喜欢它那种乐观向上，喜欢它那种永不放弃。每当我一看到鸽子，我就想起那只“灰小鸽”这也就成了我记忆最深的一件事，因为它使我明白了一个做人的道理，要自强-自信-自立！四年级难忘的一件事400字作文（二）在我的成长历程里，有许多让我难忘的事，但它们都如过眼云烟，但今年暑假的这件事最让我刻骨铭心。不知道从什么时候起，我的眼睛变的斜视，虽然不影响视力，但是却不美观。到医院去检查医生说：“需要做个小手术，不然长大后就不好做了。” 于是爸爸和奶奶带我到沈阳爱尔医院去做手术。在做手术之前经过一系列的检查后，准备做手术，爸爸和奶奶都很担心，妈妈也很挂念我，一个接一个的电话打过来，问我害怕吗？我还听见她在电话里哭了。其实，我很兴奋，一点也不害怕，因为我终于等到了那一刻，走进了手术室后，医生给我打上了麻醉针，我就什么也不知道了，美美的睡了一觉，当我醒来的时候，爸爸已经守护在我的床边，问长问短，我觉得很幸福。这时候妈妈又一个电话追来问我做的怎么样，爸爸说：＂做得很成功＂，妈妈心里的一块石头终于落地了。第二天，打开纱布，我照镜子发现只是眼睛又红又肿，但是一点也不疼，在爸爸和奶奶的精心照顾下我终于出院了。我觉得我很勇敢，我想大声对爸爸妈妈说：“我长大了，不要太为我担心了。” 四年级难忘的一件事400字作文（三）今年暑假，社区组织我们参加“老少共读一本书《跨越时代的呼唤----焦裕禄》”活动。

高考英语 between…and… 的用法

------精品文档!值得拥有！------ 2012高考英语between…and…的用法（1） I’ll phone you between lunch and three o’clock. 我将在午餐后三点钟以前给你打电话。 He felt something between laughter and anger. 他既觉得好笑，又感到气愤。 (2) 由于……和……(表示原因)。如： Between the noise outside and lack of sleep, he couldn’t concentrate. 由于外面的噪音加上睡眠不足，他无法专心。注：between...and 不仅可连接两者，也可连接三者。如： Luxemburg lies between France, Germany and Belgium. 卢森堡位于法国、德国和比利时之间。 Between cooking, writing and running the farm he was kept very busy. 他又是做饭，又是写作，还要打理农场，忙得不可开交。 (3) 用于习语：between ourselves [you and me] 仅你我知道的秘密。如： Between ourselves, I don’t think he will live much longer. 咱们私下说说，他活不久了。 Between you and me, he’s not very reliable. 这是只有你我之间才说，他不是很可靠。 ------珍贵文档!值得收藏！------

最难忘的一件事四年级作文3篇

最难忘的一件事四年级作文3篇这件事，使我终生难忘。给了我鼓励与力量；给了我信心与希望，它已深深地印在我心里，伴我成长。下面是小编为大家整理的最难忘的一件事四年级作文，仅供参考，欢迎大家阅读。最难忘的一件事四年级作文一今天我在家里找东西的时候，看到了一张照片，照片上一个小男孩拉着一个小购物车站在超市门口，开心地笑着。使我陷入了回忆。哪天妈妈让我一个人去超市购物，我高兴的一蹦三尺高，因为那可是我长到现在从来没有过的事情啊！我拉着妈妈给我的小推车，走进了超市的大门。我先把我的小车车存在超市的储物台，就在超市里找了一个公用车车，推进了超市。超市里的东西真多，看得我琳琅满目。我第一个要买的东西是四盒0.5的铅，我找来找去都没找到，最和终于在文具类的下面，找到了0.5的铅。但是我又左右为难了有两种前一种是彩铅后一种不是彩铅。彩铅稍微贵一点，普通的铅不是很贵。我先来想去，最后买了彩铅，因为就贵几毛钱吗，还是一漂亮为主吧。后来我又买了许多其他的东西，就去收银台付钱了。我先把我卖的所有东西放到收银台上，等收银的阿姨算完了

钱，统一结账。这次买的钱是40元，我给了阿姨100元，阿姨给我找了60元。等我把所有东西装进我带来的小车车，就回家了通过这次买东西，我收获了自立。最难忘的一件事四年级作文二在我的记忆里，我的童年有许多许多难忘的事，但其中有一件事更是让我最难忘。这件事现在想起来，我都觉得很好笑——那就是掉牙!我换第一颗牙时，是妈妈带我去医院，医生给我打了麻药，把还没有松动的牙硬生生的给拔了下来，听医生说不拔就会影响新牙的`生长。等换第二颗牙时，我才深深地体会到了换牙的滋味。先是牙齿有点松动，吃东西时觉得有点碍事，但又好像不影响什么。于是我有事没事的就用舌头摇摇它，也不疼，心里总想它早早掉了就好了，可它偏偏好像快掉了，但又牢牢的不掉。渐渐的它越来越来松动了，我都感觉到好像就只有那么一点点肉连着它了，可它还是不掉。有一天中午，妈妈做了我最爱吃的红烧排骨，我高兴地大吃了起来，正美美地吃着，突然，我咬到了块硬硬的东西，我以为我咬到了骨头，结果吐出来一看，是带着血的小白牙，其实这样掉牙一点儿都不疼，但是它掉得可真不是时候，那香喷喷的排骨是不能吃了，真是让我遗憾。这件事虽然已经过去了很久，但这次掉牙的经历却让我

人教版四年级数学上册期末考试卷

四年级数学（上册）期末考试试题一、填空题（22分） 1. 四十五亿七千五百万写作( )，改写成用亿作单位的数是( )，省略亿后面的尾数的近似数是( )。 2. 两个数相除，商是56，余数是0.3，若被除数和除数同时扩大100倍，商是( )余数是( )。 3、写出下面各数的近似数.。（1）省略万位后面的尾数1746003≈35482954≈ （2）省略亿位后面的尾数3708500000≈9964000000≈ 4、有一个七位数，减去1就变成六位数，这个七位数是（）。 5、比较大小。 527023○4969200 160000000○16亿180÷12○180÷15 150×2○15×20 6、712÷42，除数可以看成（）来试商，它的商是（）位数。 7、如图，已知∠1=60o，∠2=（），∠3=（），∠4=（） 2 1 3 8、一个数除以73商是6，且有余数，余数最大是（）。 4 9、甲数是乙数的3倍，甲数除以乙数的商是（）；如果甲数缩小5倍，要使商不变，乙数应当（）。二、想一想，下面的说法对不对？对的打“√”，错的打“×”。（6分） 1、长方形是特殊的平行四边形。……………………（） 2、两个锐角的和一定比直角大。……………………（） 3、一个三位数除以两位数，商可能是一位数，也可能是两位数。…（） 4、估算3198÷39的结果约是100。………………………………（）。 5、638÷27=22…44 ……………………………………（）。 6、直线的长度是射线的两倍。…………………………………（）。三、选择题(8分) 1. 下面各数中，一个零也不读出来的是( ) A 202200 B 200220 C 202020 2. 在一个整数的末尾添上两个“0”，原来的数就( ) A 扩大2倍 B 缩小2倍 C 扩大100倍 D 缩小100倍 3. 在有余数的除法中，除数和余数比较( ) A 除数比余数大 B 余数比除数小 C 除数和余数一样大 4、下列图形中，具有稳定性的是（）。 A、长方形 B、平行四边形 C、三角形 5、一个数除以13，商是103，余数是11，这个数是（）。 A、1339 B、1328 C、1350 6、382770000000≈（）亿 A、38277 B、3828 C、383 7、一个数，亿位和千位上都是7，其余各位上都是0，这个数写作（） A、700070000 B、700007000 C、700700000 8、9÷3=（9×3）÷（3×3）成立的依据是（）。 A、商不变的性质 B、乘除法的关系 C、小数的性质四、计算我能行。（30分） 1、直接写出得数。（12分） 7×10= 360×2= 480÷60= 78÷13=

Efficient communication between parallel programs with InterComm

amongbetween的区别用法全

四年级一件难忘的事400字作文【五篇】

人教版四年级(上册)数学专项训练

2018初中英语词汇之among与between的用法区别

关于难忘的一件事四年级作文3篇

2020年四年级数学上册解决问题专项综合练习

英语地点介词的正确使用方法

四年级难忘的一件事作文400字

四年级上册数学专项练习

四年级作文 一件难忘的事

四年级数学上册专项练习

难忘的一件事作文400字四年级_1

最新四年级数学上册单元专项练习

on,acrossfrom,nextto或between…and练习和辨析

四年级难忘的一件事400字作文

高考英语 between…and… 的用法

最难忘的一件事四年级作文3篇

人教版四年级数学上册期末考试卷

四年级作文一件难忘的事