IBM Plat LSF家族安裝和配置簡(jiǎn)介.V1.0參考模板

1、_IBM Platform LSF家族安裝和配置簡(jiǎn)介V1.0版 馬雪潔2013.5.71 / 50目錄1集群結(jié)構(gòu)11.1單純LSF環(huán)境（命令行提交）11.2LSF+PAC環(huán)境（WEB提交）11.3LSF+PM環(huán)境（PM提交）32LSF安裝和基本配置舉例32.1LSF安裝步驟32.1.1獲得LSF和PAC安裝包32.1.2解壓縮lsfinstall安裝腳本文件42.1.3修改install.config配置文件42.1.4執(zhí)行安裝42.1.5配置開(kāi)機(jī)自啟動(dòng)42.1.6測(cè)試安裝42.1.7啟動(dòng)/停止LSF進(jìn)程（三種方式）52.1.8測(cè)試提交作業(yè)52.1.9使能root提交作業(yè)52.1.10修改配置

2、文件后reconfig62.1.11日志和debug62.2配置文件說(shuō)明62.3常用命令72.4配置公平競(jìng)爭(zhēng)調(diào)度策略72.4.1添加輪循調(diào)度隊(duì)列72.4.2添加層次公平競(jìng)爭(zhēng)策略82.4.3多隊(duì)列公平競(jìng)爭(zhēng)策略82.4.4使能配置92.5配置搶占調(diào)度策略92.6配置全局限制策略102.6.1限制用戶(hù)運(yùn)行的作業(yè)數(shù)目102.6.2限制節(jié)點(diǎn)運(yùn)行作業(yè)數(shù)目102.6.3限制隊(duì)列作業(yè)的運(yùn)行限制102.6.4設(shè)定General limits112.6.5使能配置122.7配置提交控制腳本esub122.8配置資源管理elim示例122.8.1匯報(bào)home目錄空閑大小122.8.2匯報(bào)root進(jìn)程數(shù)目122.8.

3、3匯報(bào)應(yīng)用程序許可證數(shù)目132.8.4測(cè)試elim腳本142.8.5添加資源定義和資源地圖142.8.6查看資源數(shù)目143LSF命令行集成應(yīng)用示例143.1CFD+集成（spooling file）143.1.1CFD+安裝和許可證143.1.2集成許可證管理elim143.1.3添加CFD+ job starter153.1.4添加CFD APP profile173.1.5CFD+命令行提交腳本實(shí)例173.2GAUSSIAN集成方式（spooling file）183.2.1Gaussian安裝和許可證183.2.2Gaussian命令行提交腳本實(shí)例183.3Abaqus的腳本集成（bsu

4、b命令）183.4Platform MPI作業(yè)193.5Openmpi作業(yè)203.6Intel MPI作業(yè)203.6.1Express版本不記賬方式203.6.2Express版本blaunch記賬方式223.6.3Standard版本PAM集成方式274安裝PAC295使用PAC進(jìn)行應(yīng)用程序集成305.1Gaussian界面集成過(guò)程315.2CFD+集成后界面和后臺(tái)腳本365.3在PAC中監(jiān)控許可證386安裝License Scheduler406.1基本安裝測(cè)試406.2基本配置舉例406.2.1添加許可證服務(wù)器地址406.2.2映射許可證feature：406.2.3使用許可證資源416

5、.2.4配置許可證調(diào)度策略417常見(jiàn)問(wèn)題418使用man page419售后技術(shù)支持411 集群結(jié)構(gòu)較大的集群都會(huì)設(shè)計(jì) 單獨(dú)的登錄節(jié)點(diǎn)，用戶(hù)只能ssh到登錄節(jié)點(diǎn)，不能直接ssh到集群的任何主節(jié)點(diǎn)和計(jì)算節(jié)點(diǎn)。同時(shí)配置用戶(hù)在計(jì)算節(jié)點(diǎn)之間的ssh互信，為了并行作業(yè)的運(yùn)行。登錄節(jié)點(diǎn)也安裝LSF，配置為L(zhǎng)SF 靜態(tài)Client或者M(jìn)XJ值為0，也即不運(yùn)行作業(yè)的客戶(hù)端。集群的WEB節(jié)點(diǎn)與辦公訪(fǎng)問(wèn)局域網(wǎng)一個(gè)網(wǎng)段。如需使用浮動(dòng)client，主節(jié)點(diǎn)網(wǎng)卡需要1.1 單純LSF環(huán)境（命令行提交）1.2 LSF+PAC環(huán)境（WEB提交）用戶(hù)通過(guò)portal提交作業(yè)：1.3 LSF+PM環(huán)境（PM提交）2 LSF安裝和

6、基本配置舉例2.1 安裝前的準(zhǔn)備工作NIS ready；NFS/GPFS ready；2.2 LSF安裝步驟Use root to install.Get NIS and NFS/GPFS ready.2.2.1 獲得LSF和PAC安裝包lsf8.3_linux2.6-glibc2.3-x86_64.tar.Z lsf8.3_lsfinstall_linux_x86_64.tar.Z pac8.3_standard_linux-x64.tar.Z 許可證文件platform_hpc_std_entitlement.dat2.2.2 解壓縮lsfinstall安裝腳本文件 Put the pac

7、kage under /root/lsfrootS2 lsf# gunzip lsf8.3_lsfinstall_linux_x86_64.tar.Z tar -xvf lsf8.3_lsfinstall_linux_x86_64.tar2.2.3 修改install.config配置文件首先添加集群管理員lsfadmin。cd lsf8.3_lsfinstallvi install.configrootS2 lsf8.3_lsfinstall# cat install.config LSF_TOP="/opt/lsf" (安裝目錄) LSF_ADMINS="ls

8、fadmin" (先創(chuàng)建lsfadmin的用戶(hù)名) LSF_CLUSTER_NAME="platform" (集群名稱(chēng)，任意指定) LSF_MASTER_LIST="s2 s3" (LSF管理節(jié)點(diǎn)) LSF_ENTITLEMENT_FILE="/root/lsf/platform_hpc_std_entitlement.dat" (安裝源許可證的地址) LSF_TARDIR="/root/lsf/" (安裝源文件包的地址)2.2.4 執(zhí)行安裝./lsfinstall -f install.config2.

9、2.5 配置開(kāi)機(jī)自啟動(dòng)/opt/lsf/9.1/installhostsetuprhostsetup2.2.6 測(cè)試安裝安裝目錄下的/conf目錄rootS2 conf# source profile.lsfAdd source profile.lsf to /etc/profileif no rsh, set ssh in lsf.confrootS2 conf# tail lsf.confLSF_RSH="ssh"2.2.7 啟動(dòng)/停止LSF進(jìn)程（三種方式）rootS2 conf# lsfstartup/lsfstop或者lsadmin limstatup/limshu

10、tdownlsadmin resstartup/resshutdownbadmin hstartup/hshutdown或者lsf_daemons start/stoprootS2 conf# lsidIBM Platform LSF Express 8.3 for IBM Platform HPC, May 10 2012Copyright Platform Computing Inc., an IBM Company, 1992-2012.US Government Users Restricted Rights - Use, duplication or disclosure restr

11、icted by GSA ADP Schedule Contract with IBM Corp.My cluster name is platformMy master name is s2You have new mail in /var/spool/mail/rootrootS2 conf# lsloadHOST_NAME status r15s r1m r15m ut pg ls it tmp swp mems2 ok 0.0 0.0 0.0 1% 0.0 1 0 151G 20G 61Gs4 ok 0.0 0.0 0.0 2% 0.0 1 2 183G 20G 62Gs6 ok 0.

12、0 0.0 0.0 3% 0.0 1 2 3734M 2G 30Gs5 ok 0.0 0.0 0.0 5% 0.0 1 2 3468M 2G 30G2.2.8 測(cè)試提交作業(yè)bsub sleep 1000002.2.9 使能root提交作業(yè)enable root to submit job: LSF_ROOT_REX=local重啟LSF進(jìn)程。2.2.10 修改配置文件后reconfig修改lsf.*配置文件后lsadmin reconfig修改lsb.*配置文件后badmin reconfig部分參數(shù)需要重啟LSF主調(diào)度或者其他進(jìn)程：badmin mbdrestart; lsadmin lim

13、restart; lsadmin resrestart; badmin hrestart2.2.11 日志和debugFind the logs under log directory. LSF will run mainly 3 processes on each node, on master node will have 2 more.Master: lim,res,sbatchd,mbatchd,mbschedCompute:lim,res,sbatchdTurn on debug in command line:Run lim -2 directly on node to check

14、 why lim not startup.2.3 配置文件說(shuō)明目錄/etc/init.d:/etc/init.s/lsf lsf服務(wù)自啟動(dòng)腳本目錄/apps/platform/8.3/lsf/conf:lsf.conf lsf配置文件lsf.cluster.cluster83 集群配置文件lsf.shared 共享資源定義文件./lsbatch/cluster83/configdir/lsb.* 調(diào)度系統(tǒng)配置文件lsb.users lsf用戶(hù)與用戶(hù)組配置文件lsb.queues lsf隊(duì)列配置文件lsb.params lsf調(diào)度參數(shù)配置文件lsb.applications lsf應(yīng)用配置文件l

15、sb.hosts lsf機(jī)器與機(jī)器組配置文件lsb.resources lsf資源配置文件lsb.modules lsf模塊配置文件2.4 常用命令bsub:提交作業(yè)；bjobs:查看作業(yè)信息；bhist:查看作業(yè)歷史；lshosts:查看節(jié)點(diǎn)靜態(tài)資源；bhosts，lsload:查看節(jié)點(diǎn)狀態(tài)和資源信息；bqueues ：查看隊(duì)列配置；blimits：查看限制limit信息；lsid:集群版本和主節(jié)點(diǎn)；bmod:修改bsub option；等等。2.5 基于資源的調(diào)度策略bsub R “ (type=LINUX2.4 && r1m < 2.0)|(type=AI

16、X && r1m < 1.0) ”或者在隊(duì)列l(wèi)sb.queues或者lsb.application文件定義：RES_REQ=select(type=LINUX2.4 && r1m < 2.0)|(type=AIX && r1m < 1.0)bsub R "selecttype=any && swap>=300 && mem>500 orderswap:mem rusageswap=300,mem=500" job1bsub R rusagemem=500:

17、app_lic_v2=1 | mem=400:app_lic_v1.5=1" job1bsub R "selecttype=any && swp>=300 && mem>500 ordermem" job12.6 配置公平競(jìng)爭(zhēng)調(diào)度策略2.6.1 添加輪循調(diào)度隊(duì)列Modify lsb.queues, add followingBegin QueueQUEUE_NAME = roundRobin PRIORITY = 40FAIRSHARE = USER_SHARESdefault,1#USERS = userGroupA

18、Define your own usergroupEnd QueueRun badmin reconfig to enable the change.Run bqueues l to check the queues configure2.6.2 添加層次公平競(jìng)爭(zhēng)策略Add following queue to add hierarchicalshare policy:Begin QueueQUEUE_NAME = hierarchicalShare PRIORITY = 40USERS = userGroupB userGroupC FAIRSHARE = USER_SHARESuserGr

19、oupB,7 userGroupC,3End Queue2.6.3 多隊(duì)列公平競(jìng)爭(zhēng)策略在lsb.queues中添加下列隊(duì)列，注意節(jié)點(diǎn)組和用戶(hù)組定義。Begin QueueQUEUE_NAME = verilog DESCRIPTION = master queue definition cross-queuePRIORITY = 50FAIRSHARE = USER_SHARESuser1,100 default,1FAIRSHARE_QUEUES = normal shortHOSTS = hostGroupC# resource contention #RES_REQ = rusageve

20、rilog = 1End QueueBegin QueueQUEUE_NAME = shortDESCRIPTION = short jobs PRIORITY = 70 # highestHOSTS = hostGroupC RUNLIMIT = 5 10End QueueBegin QueueQUEUE_NAME = normalDESCRIPTION = default queuePRIORITY = 40 # lowestHOSTS = hostGroupC End Queue2.6.4 使能配置badmin reconfig提交作業(yè)，并查看隊(duì)列的用戶(hù)動(dòng)態(tài)優(yōu)先級(jí)變化：bqueues r

21、l normal2.7 配置搶占調(diào)度策略配置最基本的slots搶占：Begin QueueQUEUE_NAME = shortPRIORITY = 70HOSTS = hostGroupC # potential conflictPREEMPTION = PREEMPTIVEnormalEnd QueueBegin QueueQUEUE_NAME = normalPRIORITY = 40HOSTS = hostGroupC # potential conflict PREEMPTION = PREEMPTABLEshortEnd Queue向兩個(gè)隊(duì)列提交作業(yè)，查看被preempt的作業(yè)的pe

22、nding原因。2.8 配置全局限制策略2.8.1 限制用戶(hù)運(yùn)行的作業(yè)數(shù)目在lsb.users文件中添加：Begin User USER_NAME MAX_JOBS JL/P user1 4 - user2 2 1 user3 - 2 groupA 8 - groupB 1 1 Default 2 - End User2.8.2 限制節(jié)點(diǎn)運(yùn)行作業(yè)數(shù)目在lsb.hosts文件中：Begin HostHOST_NAME MXJ JL/Uhost1 4 2host2 2 1host3 ! -End Host2.8.3 限制隊(duì)列作業(yè)的運(yùn)行限制在lsb.queues中添加：Begin QueueQUEU

23、E_NAME = myQueue HJOB_LIMIT = 2PJOB_LIMIT = 1UJOB_LIMIT = 4HOSTS = hostGroupA USERS = userGroupA End Queue2.8.4 設(shè)定General limits在lsb.resources文件定義全局general limits示例：Begin LimitUSERSQUEUESHOSTS SLOTS MEM SWP user1 -hostB - - 20% user2normalhostA - 20 - End LimitBegin Limit NAME = limit1 USERS = user1

24、 PER_HOST = hostA hostC TMP = 30% SWP = 50% MEM = 10%End LimitBegin Limit PER_USER QUEUES HOSTS SLOTS MEM SWP TMP JOBS groupA - hgroup1 - - - - 2 user2 normal - - 200 - - - - short - - - - - 200 End Limit 2.8.5 使能配置badmin reconfig2.9 配置提交控制腳本esub全局esub腳本在作業(yè)被提交是調(diào)用，可以被自動(dòng)的或者顯式的調(diào)用從而控制用戶(hù)作業(yè)提交的行為。編輯esub.pr

25、oject文件在$LSF_SERVERDIR下面（chmod為可執(zhí)行）：#!/bin/shif "_$LSB_SUB_PARM_FILE" != "_" ; then . $LSB_SUB_PARM_FILE if "_$LSB_SUB_PROJECT_NAME" = "_" ; then echo "You must specify a project!" >&2 exit $LSB_SUB_ABORT_VALUE fifiexit 0在lsf.conf中定義LSB_ESUB_M

26、ETHOD=”project”2.10 配置資源管理elim示例2.10.1 匯報(bào)home目錄空閑大小編輯elim文件elim.home，放置在$LSF_SERVERDIR下面。chmod為可執(zhí)行。#!/bin/shwhile true ; do home=df -k /home | tail -1 | awk 'printf "%4.1f", $4/(1024*1024)' echo 1 home $home sleep 30done2.10.2 匯報(bào)root進(jìn)程數(shù)目編輯elim.root，放置在$LSF_SERVERDIR下面。chmod為可執(zhí)行。#!/

27、bin/shwhile true ; do root=ps -ef | grep -v grep | grep -c root echo 1 rootprocs $root sleep 30done2.10.3 匯報(bào)應(yīng)用程序許可證數(shù)目#!/bin/shlic_X=0 ; num=0while true ; do # only want the master to gather lic_X if "$LSF_MASTER" = "Y" ; then lic_X=lmstat a c lic_X.dat | grep .>&2 fi # onl

28、y want training8, training1 to gather simpton licenses if "hostname" = "training8" o "hostname" = "training1" ; then num=lmstat a c simpton_lic.dat | grep .>&2 fi # all hosts including master, will gather the following root=ps efw | grep v grep | grep c

29、 root>>1&2 tmp=df k /var/tmp | grep var | awk 'print $4 /1024'>&2if "$LSF_MASTER" = "Y" ; then echo 4 lic_X $lic_X simpton $num rtprc $root tmp $tmp else echo 3 simpton $num rtprc $root tmp $tmp fi # the same INTERVAL values defined in lsf.shared sleep 60

30、done2.10.4 測(cè)試elim腳本直接運(yùn)行./elim.root查看elim輸出是否正確。2.10.5 添加資源定義和資源地圖在lsf.shared文件中添加rootprocs 定義，并在lsf.cluster resources Map中添加資源和節(jié)點(diǎn)的映射關(guān)系。使能配置：lsadmin reconfig; badmin reconfig2.10.6 查看資源數(shù)目lsload l3 LSF命令行集成應(yīng)用示例本節(jié)例舉幾個(gè)應(yīng)用的不同集成方式。使用spooling文件或者bsub命令行都可以自由轉(zhuǎn)換。3.1 CFD+集成（spooling file）3.1.1 CFD+安裝和許可證安裝路徑：l

31、n-3620-4許可證：/gpfs/software/cfdpp/mbin/Metacomp.lic 許可證服務(wù)器：ln-3620-4啟動(dòng)許可證服務(wù)器：hpcadminmn-3650 jessi$ ssh ln-3620-4Last login: Tue Mar 26 19:19:24 2013 from mn-3650.private.dns.zone hpcadminln-3620-4 $ /gpfs/software/cfdpp/mbin/lmgrd -c /gpfs/software/cfdpp/mbin/Metacomp.lic確認(rèn)許可證服務(wù)器是否正常運(yùn)行：/gpfs/softwar

32、e/cfdpp/mbin/lmutil lmstat -a -c /gpfs/software/cfdpp/mbin/Metacomp.lic3.1.2 集成許可證管理elim添加elim方法：（elim全集群只需運(yùn)行一個(gè)，因此只在頭節(jié)點(diǎn)放置elim腳本即可）在頭節(jié)點(diǎn)：cd $LSF_SERVERDIR添加如下文件：elim.lic:rootmn-3650 jessi# cd $LSF_SERVERDIRrootmn-3650 etc# pwd/opt/lsf/8.3/linux2.6-glibc2.3-x86_64/etcrootmn-3650 etc# cat elim.lic#!/bin

33、/shtotallicences=/gpfs/software/cfdpp/mbin/lmutil lmstat -a -c /gpfs/software/cfdpp/mbin/Metacomp.lic |grep "Users of CFD+_SOLV_Ser" | /bin/cut -d' ' -f7while truedousedlicences=/gpfs/software/cfdpp/mbin/lmutil lmstat -a -c /gpfs/software/cfdpp/mbin/Metacomp.lic | /bin/grep "U

34、sers of CFD+_SOLV_Ser" | /bin/cut -d' ' -f13 cfd_lic=$( $totallicences-$usedlicences )echo "1 cfd_lic $cfd_lic"/bin/sleep 30done rootmn-3650 etc# chmod a+x elim.lic修改如下的配置文件：rootmn-3650 etc# vi $LSF_ENVDIR/lsf.shared添加如下一行：cfd_lic Numeric 30 Y (CFD+ License)rootmn-3650 etc# vi

35、 $LSF_ENVDIR/lsf.cluster在resourcemap一段添加如下一行：Begin ResourceMap RESOURCENAME LOCATION cfd_lic allhostid default rootmn-3650 etc# lsadmin reconfig; badmin reconfig3.1.3 添加CFD+ job starter如果使用spooling file可不用添加。（Portal集成方式使用）添加jobstarter可執(zhí)行文件：hpcadminmn-3650 jessi$ cat /opt/lsf/jobstarter/cfd_starter #

36、!/bin/shMPI_RUN=/gpfs/software/cfdpp/hpmpi/bin/mpiruncase "$PRESSION" in SINGLE_PRESSION) CFD_CMD=/gpfs/software/cfdpp/mbin/mcfd.11.1/r4_hpmpimcfd ; DOUBLE_PRESSION) CFD_CMD=/gpfs/software/cfdpp/mbin/mcfd.11.1/hpmpimcfd ; esacCMD="$* -hostfile $LSB_DJOB_HOSTFILE $CFD_CMD"eval &qu

37、ot;$CMD"3.1.4 添加CFD APP profilerootmn-3650 etc# vi $LSF_ENVDIR/lsf.shared添加如下配置：Begin ApplicationNAME = cfdJOB_STARTER=/opt/lsf/jobstarter/cfd_starterRES_REQ="rusagecfd_lic=1"End Applicationbadmnin reconfig使得此文件生效，使用bapp l cfd查看是否成功：rootmn-3650 bin# bapp -l cfdAPPLICATION NAME: cfd -

38、No description provided.STATISTICS: NJOBS PEND RUN SSUSP USUSP RSV 12 12 0 0 0 0PARAMETERS:JOB_STARTER: /opt/lsf/jobstarter/cfd_starterRES_REQ: "rusagecfd_lic=1"3.1.5 CFD+命令行提交腳本實(shí)例hpcadminmn-3650 jessi$ cat cfd.sh#!/bin/sh#BSUB -n 12#BSUB -o %J.out#BSUB -e %J.err#BSUB -app cfd#BSUB -R &quo

39、t;rusagecfd_lic=1"cd /gpfs/software/cfd+/test/ogive/gpfs/software/cfdpp/hpmpi/bin/mpirun -hostfile $LSB_DJOB_HOSTFILE /gpfs/software/cfdpp/mbin/mcfd.11.1/hpmpimcfd然后bsub < cfd.sh 提交作業(yè)。3.2 GAUSSIAN集成方式（spooling file）3.2.1 Gaussian安裝和許可證路徑：/gpfs/software/Gaussian/許可證：無(wú)許可證版本，單個(gè)作業(yè)只能單機(jī)運(yùn)行。3.2.2 Ga

40、ussian命令行提交腳本實(shí)例下面腳本：g03.sh#!/bin/sh#BSUB -q qchem#BSUB -n 4#BSUB -R "spanhosts=1"#BSUB -cwd .#BSUB -e %J.err#BSUB -o %J.outJOB=Full_codes_112_ipr_C1_JOBNAME=basename "$JOB" .comexport g03root=/gpfs/software/Gaussianexport GAUSS_SCRDIR=/tmpsource $g03root/g03/bsd/file/gpfs/

41、software/Gaussian/g03/g03 < $JOB > "$JOBNAME.log"提交作業(yè)：bsub < g03.sh3.3 Abaqus的腳本集成（bsub命令）編輯腳本abaqus_run.sh腳本#!/bin/sh# version: 1.3.0export ABAQUS_CMD="/gpfs/software/Abaqus/Commands/abaqus"export LM_LICENSE_FILE="/gpfs/software/Abaqus/License/abq612.lic"#指定cp

42、u number,注意要與bsub命令行中-n指定的cpu個(gè)數(shù)一致export NCPU=16#指定輸入文件export INPUT_FILE=beam.inp#指定作業(yè)名export JOB_NAME=abaqus_job3$ABAQUS_CMD job=$JOB_NAME cpus=$NCPU input="$INP_INPUT_FILE"2)通過(guò)LSF提交#進(jìn)入輸入數(shù)據(jù)所在目錄，執(zhí)行bsub命令bsub -q qeng -n 16 ./abaqus_run.sh3.4 Amber作業(yè)（blaunch集成，可記賬）針對(duì)intelmpi，編寫(xiě)mpdboot.lsf腳本。變

43、為可執(zhí)行，放置在$LSF_SERVERDIR下面編寫(xiě)提交作業(yè)腳本：ymeimnis test$ cat new.sh #!/bin/sh#BSUB -q small#BSUB -n 128#BSUB -o %J.out#BSUB -e %J.err#BSUB -J IMPI#BSUB -x #export PATH=/gpfs01/software/intel/impi/24/intel64/bin:$PATH#/gpfs01/home/ymei/jessi/mpdboot.lsfmpdboot.lsfexport I_MPI_DEVICE=ssm#export I_MPI_FA

44、BRICS=shm:ofa#export I_MPI_FAST_STARTUP=1#export I_MPI_DEVICE=rdssm#mpiexec -np $LSB_DJOB_NUMPROC /gpfs01/software/intel/impi/24/test/helloword mpiexec -np $LSB_DJOB_NUMPROC $AMBERHOME/bin/sander.MPI -ng 32 -groupfile remd10.groupfilempdallexit提交作業(yè)：bsub < new.sh3.5 Platform MPI作業(yè)3.5.1 安裝Pl

45、atform MPI確認(rèn)用戶(hù)無(wú)密碼訪(fǎng)問(wèn)ssh OK。安裝Platform MPI到共享目錄下：sh platform_mpi-0-0320r.x64.sh -installdir=/opt/pmpi norpm如果缺失C Compiler，執(zhí)行：yum install gcc3.5.2 LSF外面驗(yàn)證安裝OK設(shè)置環(huán)境變量：export MPI_REMSH="ssh -x"export MPI_ROOT=/opt/pmpi/opt/ibm/platform_mpi/編譯helloworld示例程序：/opt/pmpi/opt/ibm/platform_mpi/b

46、in/mpicc -o helloworld /opt/pmpi/opt/ibm/platform_mpi/help/hello_world.crootserver3 help# /opt/pmpi/opt/ibm/platform_mpi/bin/mpirun -f ./help/hosts warning: MPI_ROOT /opt/pmpi/opt/ibm/platform_mpi/ != mpirun path /opt/pmpi/opt/ibm/platform_mpiHello world! I'm 1 of 4 on server3Hello world! I'

47、m 0 of 4 on server3Hello world! I'm 3 of 4 on computer007Hello world! I'm 2 of 4 on computer007rootserver3 help# cat ./help/hosts -h server3 -np 2 /opt/pmpi/opt/ibm/platform_mpi/help/helloworld -h computer007 -np 2 /opt/pmpi/opt/ibm/platform_mpi/help/helloworld3.5.3 通過(guò)LSF提交export MPI_REMSH=b

48、launch$ mpirun -np 4 -IBV /helloworld$ mpirun -np 32 -IBV /helloworld$ mpirun -np 4 -TCP /helloworld或者 rootserver3 conf# bsub -o %J.out -e.%J.err -n 4 /opt/pmpi/opt/ibm/platform_mpi/bin/mpirun -lsb_mcpu_hosts /opt/pmpi/opt/ibm/platform_mpi/help/helloworldJob <210> is submitted to default queue

49、 <normal>.rootserver3 conf# bjobsJOBID USER STAT QUEUE FROM_HOST EXEC_HOST JOB_NAME SUBMIT_TIME210 root PEND normal server3 *elloworld May 9 10:55 rootserver3 conf# cat 210.out Sender: LSF System <jessicomputer007>Subject: Job 210: </opt/pmpi/opt/ibm/platform_mpi/bin/mpirun -lsb_mcpu_

50、hosts /opt/pmpi/opt/ibm/platform_mpi/help/helloworld> in cluster <jessi_cluster> DoneJob </opt/pmpi/opt/ibm/platform_mpi/bin/mpirun -lsb_mcpu_hosts /opt/pmpi/opt/ibm/platform_mpi/help/helloworld> was submitted from host <server3> by user <root> in cluster <jessi_cluster

51、>.Job was executed on host(s) <4*computer007>, in queue <normal>, as user <root> in cluster <jessi_cluster>.</root> was used as the home directory.</opt/lsf/conf> was used as the working directory.Started at Thu May 9 18:49:06 2013Results reported at Thu May 9

52、18:49:07 2013Your job looked like:-# LSBATCH: User input/opt/pmpi/opt/ibm/platform_mpi/bin/mpirun -lsb_mcpu_hosts /opt/pmpi/opt/ibm/platform_mpi/help/helloworld-Successfully completed.Resource usage summary: CPU time : 0.23 sec. Max Memory : 2 MB Average Memory : 2.00 MB Total Requested Memory : - Delta Memory : - (Delta: the difference between total requested memory and actual max usage.) Max Swap : 36 MB Max Processes : 1 Max Threads : 1The