当前位置：文档库 › Reinforcement of polyurethane composites with an organically modified montmorillonite

Reinforcement of polyurethane composites with an organically modified montmorillonite

Reinforcement of polyurethane composites with an organically

modi?ed montmorillonite

Jiawen Xiong,Zhen Zheng,Hongmei Jiang,Sufang Ye,Xinling Wang

Department of Polymer Science and Engineering,Shanghai Jiaotong University,800Dongchuan Road,Shanghai 200240,PR China

Received 9September 2005;received in revised form 9January 2006;accepted 9January 2006

Abstract

A clay with reactive activity prepared by treatment of natural montmorillonite with Methylene-bis-ortho-chloroaniline (MOCA)was incorporated into polyurethane matrix and a series of PU/clay nanocomposites were obtained by in situ polymerization.The microstruc-ture of the nanocomposites with di?erent content of the clay was examined by atomic force microscopy (AFM).The thermal and mechanical properties of the nanocomposites with di?erent organic clay content were characterized by dynamic mechanical thermal anal-ysis (DMTA)and thermogravimetric analysis (TGA).It was found that the moduli and thermal stability of the nanocomposites were improved with augment of clay,especially,for the PU/9wt%MO-MMT nanocomposite,compared to pure PU,the storage modulus and the loss modulus were increased by about 300%and 667%at à45°C,respectively.ó2006Elsevier Ltd.All rights reserved.

Keywords:A.Polymer–matrix composites (PMCs);B.Microstructure;B.Mechanical properties;https://www.wendangku.net/doc/5e7074693.html,pression moulding

1.Introduction

Since Usuki and coworkers ?rst reported the superior nylon 6montmorillonite (MMT)nanocomposite [1,2],polymer-layered silicate nanocomposites have attracted great interest from researchers due to their academic and industrial importance [3,4].MMT,a layered silicate used widely to prepare polymer–clay nanocomposites,consists of two external silica tetrahedral sheets and a central octa-hedral sheet of alumina.The layers with high aspect ratio (about 1nm thickness and 100nm width and length)are stacked via weak dipolar force and form interlayer galleries which are generally occupied by cations (for example Na +,Ca 2+,Li +)which can be easily substituted with organic cations via ion exchange reaction in water.The special structures of MMT play important roles in improving mechanical,thermal and di?use barrier properties of polymer-layered silicate nanocomposites [5–7].Up to

now,many polymer matrices have been used to prepare polymer–MMT nanocomposites (such as polyimide,polyc-aprolactone,polypropylene,polyaniline and polyurethane)[8–12].The studies show that some properties of the nano-composites are signi?cantly improved by incorporation of MMT into polymer matrices.

Due to the poor compatibility of MMT with organic monomers and polymer matrices,it is necessary to modify MMT in the preparation of polymer–MMT nanocompos-ites for high performance.An e?ective way to modify the nature of MMT is to substitute the cations in the interlayer galleries of layers with cationic-organic surfactants.On one hand,organic modi?ers can impart hydrophilicity to the MMT,which can improve the compatibility of MMT with polymers.On the other hand,the gallery spacing of the modi?ed MMT becomes larger than that of https://www.wendangku.net/doc/5e7074693.html,an-ically modi?ed clays plays an important role in the forma-tion of the structure and morphology of polymer/clay nanocomposites,and thus signi?cantly in?uences material properties.Therefore,the choice of modi?ers used to treat clay is crucial to prepare polymer/clay nanocomposites

Corresponding author.Tel.:+862154745817;fax:+862154741297.E-mail address:xlwang@https://www.wendangku.net/doc/5e7074693.html, (X.Wang).

https://www.wendangku.net/doc/5e7074693.html,/locate/compositesa

Composites:Part A 38(2007)

132–137

with enhanced properties.Generally,three factors should be considered in forming polymer/clay nanocomposites which are:(i),organic modi?er molecules can easily enter in interlayer gallery of layers and thus enlarge the gallery spacing of MMT to ensure that polymer molecules(mono-mers)are able to intercalate into the interlayer galleries; (ii),there are of strong interactions between organic mod-i?ers and polymer matrices to enhance their compatibility, and(iii),to consider the cost and ease of use of polymer–clay nanocomposites.At present,alkylammonium and alkylphosphonium are used widely to treat MMT[13,14]. To further improve the properties of polymer–clay nano-composites,alternate functional modi?ers(namely,the modi?ers can react with polymer)are being used to prepare polymer–clay nanocomposites[15–18].

Methylene-bis-ortho-chloroaniline(MOCA)is used widely in preparing the cross-linked polyurethanes due to being an outstanding cross-linker and an excellent chain extender.In this study,MOCA was used as a functional aromatic amine modi?er to treat MMT for use in https://www.wendangku.net/doc/5e7074693.html,parisons between MMT treated with common modi?ers(such as alkyl quaternary ammonium modi?ers), the MOCA-modi?ed MMT can react with polyurethane pre-polymers,thus further improving the interaction of MMT with the polymer matrix.A series of polyure-thane–MMT nanocomposites with di?erent contents of MOCA-modi?ed MMT(MO-MMT)varying from1%to 9%by weight are prepared.The in?uences of MO-MMT weight fraction in the polymer matrix on structure and properties of the composites were characterized by atomic force microscopy(AFM),dynamic mechanical thermal analysis(DMTA),di?erential scanning calorimetry (DSC)and thermogravimetric analysis(TGA).

2.Experimental

2.1.Materials

Polypropylene glycol(PPG,M n?1000,Gaoqiao Chemical CO.)was vacuum dried at80°C for48h prior to use.Na+-Montmorillonites(Na+form,cation exchange capacity(CEC)is of110mmol/100g,diameter of MMT particle,B545l m,Zhejiang Fenghong Clay Chemicals CO.,Ltd.)were screened with325-mesh sieve prior to treatment with organic modi?ers.MOCA,(Shanghai Chemical Co.)and toluene diisocyanate(TDI,Mitsui(East Asia)&Co.Ltd.,Japan)were used as received.2.2.Preparation of organically modi?ed MMT

A uniform suspension was formed by dispersing15g screened Na+-MMTs in300ml distilled water and a MOCA solution(dissolving6g MOCA in a mixture of 0.5M hydrochloric acid and acetone(2:3by volume)) was added to the suspension.After the reaction of the MOCA with Na+-MMTs at80°C for4h,the suspension was still stirred for12h at room temperature.Then,the mixture was?ltered,and the?lter cake was washed repeat-edly with hot distilled water until the absence of Clàions were observed(no AgCl precipitate was produced using 0.1M AgNO3).Residual MOCA was removed by using acetone wash the?lter cake.The MO-MMT was dried in a vacuum oven at80°C,ground and screened with325-mesh sieve.

2.3.Preparation of polyurethane/MO-MMT nanocomposites

The procedures for preparing polyurethane nanocom-posites containing di?erent MO-MMTs were similar as shown below in Scheme1.For the preparation of5%(wt) MO-MMT content nanocomposite,100g of dry PPG plus a stoichiometric ratio of dried MO-MMT powder were put in a250ml?ask with a mechanical stirrer at ambient tem-perature,additional mixing was via an ultrasonicator (100W,nominal frequency of50kHz)for30min at ambi-ent temperature and then heated to100°C for2h to remove residual water.Then,TDI was added in a molar ratio of 1.6:1(relative to PPG)to the mixture of PPG and MO-MMT at80°C and reacted for1.5h to obtain the pre-polyurethane/MO-MMT material.The stoichiometric number of melt MOCA(depending on the content of isocyanate groups)was added to the pre-polyurethane/ MO-MMT under vigorous stirring for30–50s at80°C, and subsequently,the pre-polyurethane/MO-MMT was immediately poured in a metal mold and cured under a 10ton pressure for2h at88°C to form the polyurethane (PU)/MO-MMT nanocomposite?lm.

2.4.Characterization

Wide-angle X-ray di?raction measurements were per-formed using a Shimadzu XRD-600(Japan)X-ray di?rac-tometer(40kV,30mA)with CuK a radiation at room temperature.The scanning rate was2°/min over a range

J.Xiong et al./Composites:Part A38(2007)132–137133

of2h=2–10°for one-dimensional di?raction.AFM mea-surements were carried out in tapping mode using Scan-ning probe microscope(SPM)with a Digital Instrument Nanoscope IIIa Multimode to investigate dispersion of MO-MMT in PU matrix.DMTA was carried out in a ten-sile mode using a frequency of10Hz on Rheometric Scien-ti?c DMTA IV instrument(USA)at a 3.0°C/min temperature ramp over a scanning range fromà70to 80°C for the nanocomposites.The specimen of about 30·5·2mm was mounted on the?xture.TG analysis was carried out using a Perkin Elmer7Series thermal anal-ysis system from200to800°C at a heating rate of20°C/ min under a N2?ow.

3.Results and discussion

3.1.Structure and morphology

One-dimensional WAXD di?raction patterns of MMT, MO-MMT,pure PU and the nanocomposites with di?er-ent MO-MMT content are shown in Fig.1.Natural MMT is known to have a gallery spacing of1.1–1.3nm, while organically modi?ed MMT has a gallery spacing of 1.5–3.0nm[19].A characteristic di?raction peak appeared at2h=6.89°for MMT and5.86°for MO-MMT,based on Bragg equation,corresponding to a gallery spacing of 1.28nm and1.51nm,respectively.No obvious di?raction peak was observed from2°to10°in2h for the pure PU. The characteristic di?raction peak(001plane)of MMT disappeared in the PU/1wt%MO-MMT,indicating that oriented layers were disrupted.With an increase of MMT

content in the PU matrix,the di?raction peak for PU/ 5wt%MO-MMT and PU/9wt%MO-MMT shifted towards the small angle range as compared with that of the MO-MMT,showing that the PU molecules were inter-calated into the layers of the MMT.

AFM was used to examine the dispersion of MO-MMT in polyurethane matrix,as shown in Fig.2.Cross-section samples of AFM were obtained by brittle fracture of the nanocomposites in liquid nitrogen.For the nanocompos-ites with1wt%,AFM images showed that MO-MMTs were able to uniformly disperse in the polymer matrix with-out obvious clusters of clay particles https://www.wendangku.net/doc/5e7074693.html,pari-sons with the composites at higher clay contents,the MO-MMT in the polymer matrix for the nanocomposite with3wt%clay displayed a relatively uniform dispersion and relatively few clusters of clay particles were observed. With an increase of clay content,the MMT particles were more prone to aggregate and formed a bigger inorganic phase,meanwhile,the aggregation easily resulted in an inhomogeneous dispersion of MMT in the polymer.Obvi-ous agglomerates were observed in the AFM images for the 5wt%,7wt%and9wt%nanocomposites.Moreover,the higher the clay content,the degree of MMT increased agglomeration.The results from XRD and AFM congru-ously showed that the MO-MMT was exfoliated when the content of MMT in polymer matrix was lower,whereas aggregation of MMT particles existed in the matrix when the weight fraction increased.At the lower clay content, the sheets of the organically modi?ed MMT were easily exfoliated and dispersed in the matrix due to the reaction of MO-MMT with polyurethane pre-polymer.When the clay content reached a high concentration(>3wt%),inter-calated MMT easily aggregated together due to limited vol-ume space in the matrix,thus resulting in an uneven dispersion.

Fig. 2.AFM micrographs of cross-section of the nanocomposites at

di?erent MO-MMT contents.

134J.Xiong et al./Composites:Part A38(2007)132–137

3.2.Dynamic mechanical thermal analysis

DMTA was used to examine the viscoelastic response of the nanocomposite material to cyclic deformation and tem-perature(here in tensile mode),namely,elastic moduli (including storage modulus E0and loss modulus E00)and loss factors(tan d).Fig.3shows the temperature depen-dence of loss factors for PU and the nanocomposites at dif-ferent clay contents.The position of the tan d peak for PU (dot curve),corresponds to the glass transition temperature (T g)of the polymer system and occurred at10.7°C.For all the nanocomposites,an increase clay content resulted in a shift in tan d towards a lower temperature.The shift showed that addition of MO-MMT decreased the T g of PU matrix,moreover,the higher the MO-MMT contents, the greater the reduction of T g(Fig.3,top right).The reduction in T g for the nanocomposites may be attributed to a decrease in the content of matrix cure which may result in a lower cross-linking density by the addition of clay[20]. Another possibility is that the polymer con?ned between the MMT layers tends to lower the T g and improving the segmental dynamics of polymer chains[21].Comparisons with un?lled PU,tan d decreased,indicating that addition of MO-MMT decreased the dampening ability of the PU, and a similar result was observed by Mishra[22].However, with an increase of MO-MMT content in polymer matrix, the magnitude of the tan d was slightly increased.

The storage and loss moduli of PU/clay nanocomposites at di?erent MO-MMT contents,and as a function of tem-perature,are shown respectively in Figs.4and5.For the storage modulus,the temperature at which a decline of modulus begins corresponds to the occurrence of glass

transition of the nanocomposites.As expected,the temper-ature corresponding to the glass transition for PU was higher than those for the PU/clay nanocomposites,indicat-ing that addition of clay shifted the T g to lower tempera-tures.Moreover,the higher the MO-MMT contents in the polymer matrix,the lower the T g.

For E0and E00,the value of the moduli above and below T g was obviously enhanced with increasing the clay content in the polymer matrix.For the nanocomposite with9wt% MO-MMT content,compared with the pure PU,storage modulus and loss modulus were increased by more3-fold and about7-fold atà45°C,respectively,and also signi?-cantly improved above T g(between30and60°C),indicat-ing that incorporation of the organic clay reinforced the polymer matrix[23,24].The materials are often used at the temperature above their T g,therefore,the changes in moduli above the T g displayed more practical signi?cance than those below the T g(for example,polyurethane elasto-mer was used to produce many tyres,the improvement in storage modulus above T g is helpful to the tyres,however, the improvement in loss modulus may cause disadvanta-geous results for the tyres’s use at the temperature between 30and60°C).Generally,the enhancement in elastic mod-uli(above T g)for the polymer/clay nanocomposites is due to the mechanistic reinforcement by clay particles together with the unrestricted segment mobility at the organic–inor-ganic interface neighborhood of the PU/MO-MMT nan-composites for E00[24,25].

3.3.Thermal property analysis

The e?ect of organic clay content on T g of the polymer matrix is shown in Fig.6.DSC thermograms indicate that

J.Xiong et al./Composites:Part A38(2007)132–137135

T g decreased on addition of the organic clay into the poly-mer matrix (Fig.6bottom right corner).This may be attributable to moveable thin polymer between the interca-lated layers,thus lowering the cross-linking density and dispersion of clay.The trend in the reduction of T g by DSC for the nanocomposites is in agreement with results obtained by DMTA.The factors in?uencing T g for the polymer/clay nanocomposites are (i):the MOCA used to modify nature MMT can react with the PU pre-polymer and strengthen the interfacial interaction between the inor-ganic and organic phases.This can be advantageous,by improving the mechanical and thermal (T g and thermal sta-bility)properties;(ii)however,a reduction in cross-linking density of cured matrix can occur because of the addition of clay and non-homogeneous dispersion of clay;(iii)enhanced segmental dynamics of polymer chains con?ned between interlayer gallery of clay,which reduce the T g [26];(iv)the clay layers are like ?exible sheets than rigid plates due to their large aspect ratio and the nanometer thickness may improve the segment mobility of polymer chains,resulting in reduction of the T g [27].

The thermal stability of PU and PU-nanocomposites were investigated by TGA (Fig.7).TGA thermograms of the nanocomposites were shifted toward high temperatures

as compared to PU,indicating that addition of clay improved the thermal stability of the system and was con-centration dependant.This may be attributed to strong interactions between the organic clay and PU matrix.

4.Conclusion

A novel organic modi?er was developed to treat MMT for use as a reinforcing agent.The developed nanoparticles were characterized by AFM,showing that the functional-ized clay was homogeneously dispersed and exfoliated in the polymer matrix at low content of clay (<3wt%).At higher contents,agglomeration of the clays resulted from non-uniform dispersion.The cured nanocomposites exhib-ited excellent mechanical properties and were thermally stable compared with pure PU.Addition of the organically modi?ed MMT signi?cantly increased the moduli of the material above and below T g .Interestingly,T g of the nano-composites decreased as clay content increased.References

[1]Usuki A,Kawasumi M,Kojima Y,Okada A,Kurauchi T,Kamigaito

O.Swelling behavior of montmorillonite cation exchanged for x -amino acids by e -caprolactam.J Mater Res 1993;8:1174–8.

[2]Usuki A,Kojima Y,Kawasumi M,Okada A,Fukushima Y,

Kurauchi T,et al.Synthesis of nylon 6-clay hybrid.J Mater Res 1993;8:1179–84.[3]Bo ¨hning M,Goering H,Fritz A,Brzezinka K-W,Turky G,

Scho ¨nhals A,et al.Dielectric study of molecular mobility in poly(propylene-graft-maleic anhydride)/clay nanocomposites.Mac-romolecules 2005;38:2764–74.

[4]Choi YS,Ham HT,Chung IJ.E?ect of monomers on the basal

spacing of sodium montmorillonite and the structures of polymer–clay nanocomposites.Chem Mater 2004;16:2522–9.

[5]Uhi FM,Davuluri SP,Wong S-C,Webster DC.Polymer ?lms

possessing nanoreinforcements via organically modi?ed layered silicate.Chem Mater 2004;16:1135–42.

[6]Krikorian V,Pochan DJ.Unusual crystallization behavior of

organoclay reinforced poly (L -lactic acid)nanocomposites.Macro-molecules 2004;37:6480–91.

[7]Chaiko DJ,Leyva AA.Thermal transitions and barrier properties of

ole?nic nanocomposites.Chem Mater 2005;17:13–9.

[8]Abdalla MO,Dean D,Campbell S.Viscoelastic and mechanical

properties of thermoset PMR-type polyimide–clay nanocomposites.Polymer 2002;43:5887–93.

[9]Gardebien F,Gaudel-Siri A,Bre

′das J-L,Lazzaroni R.Molecular dynamics simulations of intercalated poly(e -caprolactone)-montmo-rillonite clay nanocomposites.J Phys Chem B 2004;108:10678–86.[10]Somwangthanaroj A,Lee EC,Solomon MJ.Early stage quiescent

and ?ow-induced crystallization of intercalated polypropylene nano-composites by time-resolved light scattering.Macromolecules 2003;36:2333–42.

[11]Bao WJ,Kim KH,Jo WH,Park YH.Exfoliated nanocomposites

from polyaniline graft copolymer/clay.Macromolecules 2004;37:9850–4.

[12]Osman MA,Mittal V,Morbidelli M,Suter UW.Polyurethane

adhesive nanocomposites as gas permeation barrier.Macromolecules 2003;36:9851–8.

[13]Fornes TD,Hunter DL,Paul DR.Nylon-6nanocomposites from

alkylammonium modi?ed clay:The role of alkyl tails on exfoliation.Macromolecules 2004;37:1793–8.

136J.Xiong et al./Composites:Part A 38(2007)132–137

[14]Yeh J-M,Liou S-J,Lin C-Y,Cheng C-Y,Chang Y-W.Anticorro-

sively enhanced PMMA–clay nanocomposite materials with quater-nary alkylphosphonium salt as an intercalating agent.Chem Mater 2002;14:154–61.

[15]Viville P,Lazzaroni R,Pollet E,Alexandre M,Dubois P.Controlled

polymer grafting on single clay nanoplatelets.J Am Chem Soc 2004;126:9007–12.

[16]Fan X,Xia C,Advincula RC.On the formation of narrowly

polydispersed PMMA by surface initiated polymerization(SIP)from AIBN-coated/intercalated clay nanoparticle https://www.wendangku.net/doc/5e7074693.html,ngmuir 2005;21:2537–44.

[17]Herrera NN,Leto?e J-M,Putaux J-L,David L,Bourgeat-Lami E.

Aqueoue dispersions of silane-functionalized laponite clay platelets.

A?rst step toward the elaboration of water-based polymer/clay https://www.wendangku.net/doc/5e7074693.html,ngmuir2004;20:1564–71.

[18]Chen TK,Tien YI,Wei KH.Synthesis and characterization of novel

segmented polyurethane/clay nanocomposites.polymer2000;41: 1345–53.

[19]Brindley G,Brown G.Crystal structure of clay minerals and their X-

ray identi?cation.London:Mineralogical Society;1980,p.227–32.

[20]Gu A,Liang GZ.Thermal degradation behaviour and kinetic

analysis of epoxy/montmorillonite nanocomposites.Polym Degrad Stab2003;80:383–91.[21]Anastasiadis SH,Karatasos K,Vlachos G,Manias E,Giannelis EP.

Nanoscopic-con?nement e?ects on local dynamics.Phys Rev Lett 2000;84:915–8.

[22]Mishra JK,Kim I,Ha CS.New millable polyurethane/organoclay

nanocomposite:preparation,characterization and properties.Mac-romol Rapid Commun2003;24:671–5.

[23]Sinha Ray S,Maiti P,Okamoto M,Yamada K,Ueda K.New

polylactide/layered silicate nanocomposites.1.Preparation,charac-terization,and properties.Macromolecules2002;35:3104–10. [24]Ray SS,Okamoto K,Maiti P,Okamoto M.New poly(butylene

succinate)/layered silicate nanocomposites:Preparation and mechan-ical properties.J Nanosci Nanotechnol2002;2:171–6.

[25]Ray SS,Yamada K,Okamoto M,Ueda K.New polylactide-layered

silicate nanocomposites. 2.Concurrent improvements of material properties,biodegradability and melt rheology.Polymer2003;44: 857–66.

[26]Kumar S,Jog JP,Natarajan U.Preparation and characterization of

poly(methyl methacrylate)–clay nanocomposites via melt intercala-tion:The e?ect of organoclay on the structure and thermal properties.

J Appl Polym Sci2003;89:1186–94.

[27]Kim BK,Seo JW,Jeong HM.Morphology and properties of

waterborne polyurethane/clay nanocomposites.Euro Polym J 2003;39:85–91.

J.Xiong et al./Composites:Part A38(2007)132–137137

如何写先进个人事迹

如何写先进个人事迹篇一：如何写先进事迹材料如何写先进事迹材料一般有两种情况：一是先进个人，如先进工作者、优秀党员、劳动模范等；一是先进集体或先进单位，如先进党支部、先进车间或科室，抗洪抢险先进集体等。无论是先进个人还是先进集体，他们的先进事迹，内容各不相同，因此要整理材料，不可能固定一个模式。一般来说，可大体从以下方面进行整理。 (1)要拟定恰当的标题。先进事迹材料的标题，有两部分内容必不可少，一是要写明先进个人姓名和先进集体的名称，使人一眼便看出是哪个人或哪个集体、哪个单位的先进事迹。二是要概括标明先进事迹的主要内容或材料的用途。例如《王鬃同志端正党风的先进事迹》、《关于评选张鬃同志为全国新长征突击手的材料》、《关于评选鬃处党支部为省直机关先进党支部的材料》等。 (2)正文。正文的开头，要写明先进个人的简要情况，包括：姓名、性别、年龄、工作单位、职务、是否党团员等。此外，还要写明有关单位准备授予他(她)什么荣誉称号，或给予哪种形式的奖励。对先进集体、先进单位，要根据其先进事迹的主要内容，寥寥数语即应写明，不须用更多的文字。然后，要写先进人物或先进集体的主要事迹。这部分内容是全篇材料

的主体，要下功夫写好，关键是要写得既具体，又不繁琐；既概括，又不抽象；既生动形象，又很实在。总之，就是要写得很有说服力，让人一看便可得出够得上先进的结论。比如，写一位端正党风先进人物的事迹材料，就应当着重写这位同志在发扬党的优良传统和作风方面都有哪些突出的先进事迹，在同不正之风作斗争中有哪些突出的表现。又如，写一位搞改革的先进人物的事迹材料，就应当着力写这位同志是从哪些方面进行改革的，已经取得了哪些突出的成果，特别是改革前后的．经济效益或社会效益都有了哪些明显的变化。在写这些先进事迹时，无论是先进个人还是先进集体的，都应选取那些具有代表性的具体事实来说明。必要时还可运用一些数字，以增强先进事迹材料的说服力。为了使先进事迹的内容眉目清晰、更加条理化，在文字表述上还可分成若干自然段来写，特别是对那些涉及较多方面的先进事迹材料，采取这种写法尤为必要。如果将各方面内容材料都混在一起，是不易写明的。在分段写时，最好在每段之前根据内容标出小标题，或以明确的观点加以概括，使标题或观点与内容浑然一体。最后，是先进事迹材料的署名。一般说，整理先进个人和先进集体的材料，都是以本级组织或上级组织的名义；是代表组织意见的。因此，材料整理完后，应经有关领导同志审定，以相应一级组织正式署名上报。这类材料不宜以个人名义署名。写作典型经验材料-般包括以下几部分： (1)标题。有多种写法，通常是把典型经验高度集中地概括出来，一

BodoiaPuranik-ApplyingReinforcementLearningToCompetitiveTetris

Applying Reinforcement Learning to Competitive Tetris Max Bodoia(mbodoia@https://www.wendangku.net/doc/5e7074693.html,) 1618Sand Hill Rd Palo Alto,CA98498 Arjun Puranik(apuranik@https://www.wendangku.net/doc/5e7074693.html,) 1618Sand Hill Rd Palo Alto,CA98498 Introduction For our project,we attempt to apply reinforcement learning to the game of Tetris.The game is played on a board with 20rows and10columns,and each turn a player drops one of seven pieces(the seven tetrominoes)onto the?eld.Points are gained by dropping pieces in such a way that all ten squares in some line(row)are?lled,at which point the line is cleared. Multiple lines may be cleared with a single piece drop and will result in more points being earned.A player loses if the height of the pieces on his board exceeds the height of the board itself. The fully observable nature of the Tetris board and the sim-ple probabilistic transitions from state to state(i.e.adding a randomly selected piece to the end of the piece queue each turn)naturally suggest the use of reinforcement learning for Tetris.Speci?cally,Tetris can be modeled as a Markov Deci-sion Process.However,the state space of Tetris is extremely large-the number of ways to?ll in a20×10board is2200, and the Tetris requirement that no row be completely?lled only reduces this to(210?1)20.As a result,the complete MDP for Tetris is entirely intractable.In our paper,we will outline the different approaches we took to dealing with this intractability.We begin by describing our initial,unsuccess-ful attempts,and commenting on the reasons why they may have failed.Next,we describe the approach that eventu-ally suceeded-?tted value iteration-and give a detailed analysis of its results.Finally,we conclude by considering the strengths and weaknesses of our implementation of?tted value iteration,and highlight other potential approaches that we did not try. Initial Attempts In this section we will outline our early approaches to the problem.In each case we describe the motivations for the ap-proach,the extent to which it was successful,and the reasons why it was ultimately unsuccessful. Block Drop As an initial pass at the problem of Tetris,we chose to imple-ment a simple game which we will call Block Drop.Block Drop is comparable to Tetris in that players drop pieces onto a rectangular grid,and gain points for?lling up an entire row (clearing a line).The main difference is that in Block Drop, the only type of piece available to players is a single unit square.Although the relationships between states in Block Drop are much more simple than in Tetris(and thus the op-timal policy is much easier to learn),the size of the state space for Block Drop and Tetris are similar.In this sense, the tractability of Block Drop can be used as a rough lower bound on the tractability of Tetris. We wrote a program to formulate an m×n game of Block Drop as an MDP.Note that a valid Block Drop gameboard never has any holes(where a certain grid square is not?lled but a higher up square in the same column is?lled).This al-lows us to represent each state by a set of n values correspond-ing to the height of the pieces in each of the n columns.These column heights can take on values from0to m?1indepen-dently of each other(columns heights are bounded by m?1 since the game ends if a column reaches height m).We also added a single terminal state to represent losing the game, so the total state space of Block Drop is m n+1.One pos-sible action exists for each column(since the player chooses which column to drop a piece in)so there are a total of n ac-tions.Transition probabilities are completely deterministic-dropping a piece in a particular column either increments that column’s value by1,or decrements the value of every column by1(in the case when a line is cleared).Our reward function assigns a reward of1for clearing a line and a cost of m for losing the game;all basic moves had0cost. In order to solve the Block Drop MDP,we used the ZMDP package written by Trey Smith(Smith&Simmons,2012). This package is designed primarily for?nding(approximate) solutions to Partially Observable MDPs,and as a result it re-quires that the input problem be described as a POMDP rather than an MDP.Unfortunately,we were unable to?nd any other software that works for pure MDPs,so we decided to con-vert our problem into a POMDP by adding a single observa-tion that is always made.We assumed that this modi?cation would not affect the tractability of the problem signi?cantly. We found that the solution to Block Drop matched the in-tuitively obvious optimal policy,and provided a good sanity check for our use of MDPs to model the problem.The result-ing policy drops a single block into each column(from left to right)until a line is cleared,at which point it repeats.How-ever,the Block Drop problem also illustrated the tractability issues that we faced:4×4Block Drop took less than a sec-ond to solve,but5×4Block Drop took16seconds and6×4 Block Drop took over a minute;all sizes above7×5were unable to complete a single solver round.Figure1shows the time in seconds required for a solution to Block Drop as a

Markov games as a framework for multi-agent reinforcement learning

Michael L.Littman Brown University/Bellcore Department of Computer Science Brown University Providence,RI02912-1910 mlittman@https://www.wendangku.net/doc/5e7074693.html, Abstract In the Markov decision process(MDP)formaliza- tion of reinforcement learning,a single adaptive agent interacts with an environment de?ned by a probabilistic transition function.In this solipsis- tic view,secondary agents can only be part of the environment and are therefore?xed in their be- havior.The framework of Markov games allows us to widen this view to include multiple adap- tive agents with interacting or competing goals. This paper considers a step in this direction in which exactly two agents with diametrically op- posed goals share an environment.It describes a Q-learning-like algorithm for?nding optimal policies and demonstrates its application to a sim- ple two-player game in which the optimal policy is probabilistic. 1INTRODUCTION No agent lives in a vacuum;it must interact with other agents to achieve its goals.Reinforcement learning is a promis-ing technique for creating agents that co-exist[Tan,1993, Yanco and Stein,1993],but the mathematical frame-work that justi?es it is inappropriate for multi-agent en-vironments.The theory of Markov Decision Processes (MDP’s)[Barto et al.,1989,Howard,1960],which under-lies much of the recent work on reinforcement learning, assumes that the agent’s environment is stationary and as such contains no other adaptive agents. The theory of games[von Neumann and Morgenstern, 1947]is explicitly designed for reasoning about multi-agent systems.Markov games(see e.g.,[V an Der Wal,1981])is an extension of game theory to MDP-like environments. This paper considers the consequences of using the Markov game framework in place of MDP’s in reinforcement learn-ing.Only the speci?c case of two-player zero-sum games is addressed,but even in this restricted version there are insights that can be applied to open questions in the?eld of reinforcement learning.2DEFINITIONS An MDP[Howard,1960]is de?ned by a set of states, ,and actions,.A transition function,: PD,de?nes the effects of the various actions on the state of the environment.(PD represents the set of discrete probability distributions over the set.)The reward function,:,speci?es the agent’s task. In broad terms,the agent’s objective is to?nd a policy mapping its interaction history to a current choice of action so as to maximize the expected sum of discounted reward, 0 ,where is the reward received steps into the future.A discount factor,01controls how much effect future rewards have on the optimal decisions, with small values of emphasizing near-term gain and larger values giving signi?cant weight to later rewards. In its general form,a Markov game,sometimes called a stochastic game[Owen,1982],is de?ned by a set of states, ,and a collection of action sets,1,one for each agent in the environment.State transitions are controlled by the current state and one action from each agent:: 1PD.Each agent also has an associated reward function,:1, for agent,and attempts to maximize its expected sum of discounted rewards, ,where is the reward received steps into the future by agent. In this paper,we consider a well-studied specialization in which there are only two agents and they have diametrically opposed goals.This allows us to use a single reward func-tion that one agent tries to maximize and the other,called the opponent,tries to minimize.In this paper,we use to denote the agent’s action set,to denote the opponent’s action set,and to denote the immediate reward to the agent for taking action in state when its opponent takes action. Adopting this specialization,which we call a two-player zero-sum Markov game,simpli?es the mathematics but makes it impossible to consider important phenomena such as cooperation.However,it is a?rst step and can be consid-ered a strict generalization of both MDP’s(when1) and matrix games(when1).

个人先进事迹简介

个人先进事迹简介 01 在思想政治方面,xxxx同学积极向上,热爱祖国、热爱中国共产党,拥护中国共产党的领导.利用课余时间和党课机会认真学习政治理论,积极向党组织靠拢. 在学习上,xxxx同学认为只有把学习成绩确实提高才能为将来的实践打下扎实的基础,成为社会有用人才.学习努力、成绩优良. 在生活中,善于与人沟通,乐观向上,乐于助人.有健全的人格意识和良好的心理素质和从容、坦诚、乐观、快乐的生活态度,乐于帮助身边的同学,受到师生的好评. 02 xxx同学认真学习政治理论,积极上进,在校期间获得原院级三好生,和校级三好生,优秀团员称号,并获得三等奖学金. 在学习上遇到不理解的地方也常常向老师请教,还勇于向老师提出质疑.在完成自己学业的同时,能主动帮助其他同学解决学习上的难题,和其他同学共同探讨,共同进步. 在社会实践方面,xxxx同学参与了中国儿童文学精品“悦”读书系,插画绘制工作,xxxx同学在班中担任宣传委员,工作积极主动,认真负责,有较强的组织能力.能够在老师、班主任的指导下独立完成学院、班级布置的各项工作. 03 xxx同学在政治思想方面积极进取,严格要求自己.在学习方面刻苦努力,不断钻研,学习成绩优异,连续两年荣获国家励志奖学金；作

为一名学生干部,她总是充满激情的迎接并完成各项工作,荣获优秀团干部称号.在社会实践和志愿者活动中起到模范带头作用. 04 xxxx同学在思想方面,积极要求进步,为人诚实,尊敬师长.严格要求自己.在大一期间就积极参加了党课初、高级班的学习,拥护中国共产党的领导,并积极向党组织靠拢. 在工作上,作为班中的学习委员,对待工作兢兢业业、尽职尽责的完成班集体的各项工作任务.并在班级和系里能够起骨干带头作用.热心为同学服务,工作责任心强. 在学习上,学习目的明确、态度端正、刻苦努力,连续两学年在班级的综合测评排名中获得第1.并荣获院级二等奖学金、三好生、优秀班干部、优秀团员等奖项. 在社会实践方面,积极参加学校和班级组织的各项政治活动,并在志愿者活动中起到模范带头作用.积极锻炼身体.能够处理好学习与工作的关系,乐于助人,团结班中每一位同学,谦虚好学,受到师生的好评. 05 在思想方面,xxxx同学积极向上,热爱祖国、热爱中国共产党,拥护中国共产党的领导.作为一名共产党员时刻起到积极的带头作用,利用课余时间和党课机会认真学习政治理论. 在工作上,作为班中的团支部书记,xxxx同学积极策划组织各类团活动,具有良好的组织能力. 在学习上,xxxx同学学习努力、成绩优良、并热心帮助在学习上有困难的同学,连续两年获得二等奖学金. 在生活中,善于与人沟通,乐观向上,乐于助人.有健全的人格意识和良好的心理素质.

增强学习Reinforcement Learning经典算法梳理

增强学习Reinforcement Learning经典算法梳理1：policy and value iteration 前言就目前来看，深度增强学习（Deep Reinforcement Learning)中的很多方法都是基于以前的增强学习算法，将其中的value function价值函数或者Policy function策略函数用深度神经网络替代而实现。因此，本文尝试总结增强学习中的经典算法。本文主要参考：1Reinforcement Learning: An Introduction；2Reinforcement Learning Course by David Silver 1 预备知识对增强学习有所理解，知道MDP，Bellman方程详细可见：Deep Reinforcement Learning 基础知识（DQN方面）很多算法都是基于求解Bellman方程而形成： Value Iteration Policy Iteration Q-Learning SARSA 2 Policy Iteration 策略迭代 Policy Iteration的目的是通过迭代计算value function 价值函数的方式来使policy收敛到最优。 Policy Iteration本质上就是直接使用Bellman方程而得到的：那么Policy Iteration一般分成两步： Policy Evaluation 策略评估。目的是更新Value Function Policy Improvement 策略改进。使用greedy policy 产生新的样本用于第一步的策略评估。本质上就是使用当前策略产生新的样本，然后使用新的样本更新当前的策略，然后不断反复。理论可以证明最终策略将收敛到最优。具体算法：

优秀党务工作者事迹简介范文

优秀党务工作者事迹简介范文优秀党务工作者事迹简介范文 ***，男，198*年**月出生，200*年加入党组织，现为***支部书记。从事党务工作以来，兢兢业业、恪尽职守、辛勤工作，出色地完成了各项任务，在思想上、政治上同党中央保持高度一致，在业务上不断进取，团结同事，在工作岗位上取得了一定成绩。一、严于律己，勤于学习作为一名党务工作者，平时十分注重知识的更新，不断加强党的理论知识的学习，坚持把学习摆在重要位置，学习领会和及时掌握党和国家的路线、方针、政策，特别是党的十九大精神，注重政治理论水平的提高，具有坚定的理论信念;坚持党的基本路线，坚决执行党的各项方针政策，自觉履行党员义务，正确行使党员权利。平时注重加强业务和管理知识的学习，并运用到工作中去，不断提升自身工作能力，具有开拓创新精神，在思想上、政治上和行动上时刻同党中央保持高度一致。二、求真务实，开拓进取在工作中任劳任怨，踏实肯干，坚持原则，认真做好学院的党务工作，按照党章的要求，严格发展党员的每一个步骤，认真细致的对待每一份材料。配合党总支书记做好学院的党建工作，完善党总支建设方面的文件、材料和工作制度、管理制度等。

三、生活朴素，乐于助人平时重视与同事间的关系，主动与同事打成一片，善于发现他人的难处，及时妥善地给予帮助。在其它同志遇到困难时，积极主动伸出援助之手，尽自己最大努力帮助有需要的人。养成了批评与自我批评的优良作风，时常反省自己的工作，学习和生活。不但能够真诚的指出同事的缺点，也能够正确的对待他人的批评和意见。面对误解，总是一笑而过，不会因为误解和批评而耿耿于怀，而是诚恳的接受，从而不断的提高自己。在生活上勤俭节朴，不铺张浪费。身为一名老党员，我感到责任重大，应该做出表率，挤出更多的时间来投入到**党总支的工作中，不找借口，不讲条件，不畏困难，将总支建设摆在更重要的位置，解开工作中的思想疙瘩，为攻坚克难铺平道路，以支部为纽带，像战友一样团结，像家庭一样维系，像亲人一样关怀，践行入党誓言。把握机遇，迎接挑战，不负初心。

主要事迹简介怎么写(2020年最新)

主要事迹简介怎么写概括?简要地反映?个单位(集体)或个?事迹的材料。简要事迹不?定很短，如果情况多的话，也有?千字的。简要事迹虽然“简要”，但切忌语?空洞，写得像?学?期末鉴定。 ?应当以事实来说话。简要事迹是对某单位或个?情况概括?简要地反映情况，?如有三个??很突出，就写三个??，只是写某???时，要把主要事迹突出出来。简要事迹?般来说，?少要包括两个??的内容。?是基本情况。简要事迹开头，往往要??段?字来表述?些基本情况。如写?个单位的简要事迹，应包括这个单位的?员、承担的任务以及?段时间以来取得的主要成绩。如写个?的简要事迹，应包括该同志的性别、出?年?、参加?作时间、籍贯、民族、?化程度以及何时起任现职和主要成绩。这样上级组织在看了材料的开头，就会对这个单位或个?有?个基本印象。?是主要特点。这是简要事迹的主体部分，最突出的事例有?个??就写成?块，并按照?定的逻辑关系进 ?排列，把同类的事例排在?起，?个??通常由?个?然段或?个?然段组成。写作时，特别要注意以下四点： 1.?第三?称。就是把所要写的对象，是集体的?“他们”来表述，是个?的称之为“他(她)”。 (她)”，单位可直接写名称，个?可写其姓名。为了避免连续出现?个“他们”或“他 2.掌握好时限。?论是单位或个?的简要事迹，都有?个时间跨度，既不要扯得太远，也不要故意混淆时间概念，把过去的事当成现在的事写。这个时间跨度多长，要根据实际情况 ?定。如上级要某个同志担任乡长以来的情况就写他任乡长以来的事迹;上级要该同志两年来的情况，就写两年来的事迹。当然，有时为了需要，也可适当地写?点超过这个时间的背景情况。 3.?点他?的语?。就是在写简要事迹时，可?些群众的语?或有关?员的语?，这样会给??种?动、真切的感觉，衬托出写作对象?较?的思想境界。在?他?语?时，可适当加?，但不能造假。 4.?事实说话。简要事迹的每?个??可分为多个层次，?个层次先??句话作为观点，再???两个突出的事例来说明。?事实说话时，要尽量把?个事例说完整，以给?留下深刻印象。

大学生先进事迹简介怎么写

大学生先进事迹简介怎么写苑xx，男，汉族，1990年07月22日出生，中国共青团团员，入党积极分子，现任xx学院电气优创0902班班长，担任xx学院09级总负责人、xx学院团委学生会科创部干事、xx学院文艺团主持部部长。步入大学生活一年以来，他思想积极，表现优秀，努力向党组织靠拢，学习刻苦，品学兼优，工作认真负责，脚踏实地，生活勤俭节约，乐于助人。一直坚持多方面发展，全面锻炼自我，注重综合能力、素质拓展能力的培养。再不懈的努力下获得了多项荣誉： ●获得09-10学年xx大学“百佳千优”(文化体育)一等奖学金和“百佳千优”(班级建设)二等奖学金; ●获得09-10学年xx大学“优秀学生干部”荣誉称号; ●2010年xx大学普通话知识竞赛中获得一等奖; ●2010年xx大学主持人大赛中获得一等奖，被评为金话筒; ●xx学院首届“大学生文明修身”活动周——再生比赛中获得一等奖; ●xx学院首届“大学生文明修身”活动周——演讲比赛中获得一等奖。一、刻苦钻研树学风作为班长，他在学习方面，将班级同学分成各个学习小组，布置每日学习任务，分组竞争，通过开展各项趣味学习活动，全面调动班级同学的积极性，如：排演英语戏剧、文学常识竞答、数学辅导小组等。他带领全班同学努力学习、勤奋刻苦，全班同学奖学金获得率达91%，四级通过率达66%。二、勤劳负责建班风

在日常班级工作中，他尽心尽力，通过网络组织建立班级博客，把班级的日常情况，班级比赛，特色主题班会等活动，及时上传到班级博客，以方便更多同学了解自己的班级，也把班级的魅力、特色，更全面、更具体的展现出来。在班级建设中，他带领全班同学参加学院组织的各项文体活动中也收获颇多： ●在xx学院首届“大学生文明修身”活动周中荣获第二名， ●xx学院首届乒乓球比赛中荣获第一名、精神文明奖， ●在xx学院“迎新杯”男子篮球赛中荣获第四名、最佳组织奖。除了参加学院组织的各项活动外，为了进一步丰富班级同学们的课余生活，他在班级内积极开展各式各样的课余活动： ●普通话知识趣味比赛，感受中华语言的魅力，复习语文文学常识，为南方同学打牢普通话基础，推广普通话知识。 ●“我的团队我的班”主题班会活动中，创办特色活动“情暖你我心”天使行动，亲切问候、照顾其他同学的生活、学习方面细节小事，即使在寒冷的冬天，也要让外省的同学感受到家一样的温暖。 ●“预览科技知识”科技宫之行，作为现代大学生，不能只顾书本知识，也要跟上时代，了解时代前沿最新科技。 ●感恩节“感谢我们身边的人”主题班会活动，在这个特殊的节日里，他带领同学们通过寄贺卡、送礼物等方式，来感谢老师辛勤的付出;每人写一封家书，寄给父母，感谢父母劳苦的抚育，把他们的感激之情，转化为实际行动来感化周围的人;印发感恩宣传单，发给行人，唤醒人们的感恩的心。三、热情关怀暖人心生活中，他更能发挥榜样力量，团结同学，增强班级凝聚力。时刻观察每一位同学的情绪状态，在心理上帮助同学。他待人热情诚恳，积极帮助生活中有困难的同学：得知班级同学生病高烧，病情严重，马上放下午饭，赶到同学寝室，背起重病同学到校医院进行

个人事迹简介

个人事迹简介我是来自计算机与与软件学院的学生，现在为青年志愿者协会的干事，在班级担任生活委员。在过去的一年里，我注重个人能力的培养积极向上，热心公益，服务群众，奉献社会，热忱的投身于青年支援者的行动中！一年时间虽短，但在这一年的时间里，作为一名志愿者，我确信我成长了很多，成熟了很多。“奉献、友爱、互助、进步”这是我们志愿者的精神，在献出爱心的同时，得到的是帮助他人的满足和幸福，得到的是无限的快乐与感动。路虽漫漫，吾将上下而求索！在以后的日子里，我会在志愿者事业上做的更好。在思想上，我积极进取,关心国家大事，并多次与同学们一起学习志愿者精神,希望我们会在新的世纪里继续努力,发扬我国青年的光传统,不懈奋斗,不断创造,奋勇前进,为实现中华民族的伟大复兴做出了更大的贡献。在学习上刻苦认真，抓紧时间，不仅学习好学科基础知识，更加学好专业课知识，在课堂上积极配合老师的教学，乐意帮助其它同学，有什么好的学习资料，学习心得也与他们交流，希望大家能共同进步。在上一个年度总成绩在班级排名第四，综合考评在班级排名第二。在工作中，我认真负责，出色的完成老师、同学交给的各项任务，所以班级人际关系良好。

此外参加了学院组织的活动，并踊跃地参加，发挥自己的特长，为班级争得荣誉。例如：参加校举办的大合唱比赛并获得良好成绩；参加了计算机与软件学院党校学习并顺利结业；此外,参加了计算机与软件进行的“计算机机房义务打扫与系统维护”的活动。在这些活动中体验到了大学生生活的乐趣。现将多次参与各项志愿活动汇报如下：2013年10月26日，参加计算机与软件学院团总支实践部、计算机与软件学院青年志愿者协会组织“志愿者在五福家园的健身公园开展义务家教招新活动”；2013年11月7日，参加组成计算机与软件学院运动员方阵在田径场参加学院举办的学校运动会；2013年12月5日，参与学校学院组织的”一二．五“大合唱比赛；2014年3月12日，参加由宿舍站长组织义务植树并参与植树活动；2014年3月23日，在计算机与软件学院团总支书记茅老师的带领下，民俗文化传承协会、计算机与软件学院青年志愿者协会以及学生会的同学们参观了“计算机软件学院的文化素质教育共建基地--南京市民俗博物馆”的活动；2014年3月26日，参加有宿舍站长组织的“清扫宿舍公寓周围死角垃圾”的活动；2014年4月5日，参加由校青年志愿者协会、校实践部组织的“南京市雨花台扫墓”活动，2014年4月9日，作为班级代表参加计算机软件学院组织部组织的“计算机应用office操作大赛”的活动。在参与各项志愿活动的同时，我的学习、工作、生活能力得到了提高和认可，丰富生活体验，提供学习的机会，提供学习的机会。