第四章 资源管理龚 斌山东大学计算机科学与技术学院山东省高性能计算中心Globus与资源规范语言RSLGlobus的资源管理Globus RMSGRAMGRAMGRAMLSFCondorSGEEEApplicationRSLSimple ground RSLInformation ServiceRSLspecializationBrokerGround RSLCo-allocatorQueries& InfoGlobus Components In ActionLocal MachinempirunglobusrunGRAMClientGSIGRAMClientGSIRemote MachineAppNexusAIXPBSMPIgrid-proxy-initX509UserCertUserProxyCertMachines GRAM GatekeeperGSIGRAM Job ManagerGASS ClientRemote MachineAppNexusSolarisUnix ForkMPIGRAM GatekeeperGSIGRAM Job ManagerGASS ClientRSL stringRSL multi-requestRSL single requestDUROCGASS ServerRSL parserGRAM(Globus Resource Allocation Manager) Overview定位：资源管理的最低层功能：远程运行作业，通过提供的API提交，检测与终止作业GRAM的具体职责处理Resource Specification Language (RSL)形式的作业请求对创建的作业进行远程监控与管理更新MDS的信息Globus Pre-WS Component Interaction DiagramGSIGSIGSIGSIFrom IBM Redbook SG24-6895-012003: Intro to Grid ComputingGRAM: Grid Resource Allocation Manager GASS: Global Access to Secondary Storage（辅助存储全局访问）MDS: Monitoring and Discovery Service GRIS: Grid Resource Information ServiceGIIS: Grid Index Information ServiceGRAMService that provides remote execution and status management of the requestWhen a job is submitted by a client, the request is sent to the remote host and handled by the gatekeeper daemon located in the remote host.Then the gatekeeper creates a job manager to start and monitor the job. When the job is finished, the job manager sends the status information back to the client and terminates.GRAM ArchitectureFrom IBM Redbook SG24-6895-012003: Intro to Grid ComputingGRAM ElementsClientsGatekeeper daemon门户监护进程Job ManagerGlobal Access to Secondary Storage (GASS)辅助存储全局访问Dynamically-Updated Request Online Coallocator (DUROC)动态更新请求在线协同分配器User Resource Specification Language (RSL)GRAM ClientsThree clients:globusrunglobus-job-runglobus-job-submitGRAM管理流程图示Client APIJob ManagerScheduler Specific PluginJob ProcessGatekeeperJob RequestJob cancelstate change callbackfork/su/execfork/exec/waitspsubmit/spqcondor,lsfgatekeeper的作用gatekeeper：A process, running as root, which begins the process of handling allocation requestsperforming mutual authentication of user and resource,determining a local user name for the remote user,starting a job manager which executes as that local user and actually handles the request.In order to start the job manager, the gatekeeper must run as a privileged program相关名词解释Resource An entity capable of running one or more processes on behalf of a userClient The process that is using the resource allocation client-side APIJob A process or set of processes resulting from a job request. Job Request A request to gatekeeper to create one or more job processes, expressed in the supplied Resource Specification Language.Job Manager One job manager is created by the gatekeeper to fulfill every request submitted to the gatekeeper.GRAM调度与状态转换模型对各个阶段的解释Unsubmitted ：The job has not yet been submitted to the schedulerStageIn ：The job manager is staging executable, input, or data files to the jobPending ：The job has been submitted to the scheduler, but resources have not yet been allocated for the job.Active ：The job has received all of its resources, and the application is executingSuspended ：The job has been stopped temporarily by the schedulerStageOut ：The job manager is staging output files from the job manager host to remote storage. Done ：The job completed successfully.Failed ：The job terminated before completion, as a result of an error, or a user or system cancel.GRAM ComponentsGlobus SecurityInfrastructureJob ManagerGRAM client API calls to request resource allocationand process creation.MDS client API callsto locate resourcesQuery current statusof resourceCreateRSL LibraryParseRequestAllocate &create processesProcessProcessProcessMonitor &controlSite boundaryClientMDS: Grid Index Info ServerGatekeeperMDS: Grid Resource Info ServerLocal Resource ManagerMDS client API callsto get resource infoGRAM client API statechange callbacksDUROC（Dynamically-Updated Request Online Co-allocator）Simultaneous allocation of a resource setHandled via optimistic co-allocation based on free nodes or queue predictionadvance reservations will also be supportedglobusrun will co-allocate specific multi-requests using DUROCGRAM ExamplesThe globus-job-run client is a sample GRAM client, using command-line arguments rather than RSL.% globus-job-run pitcairn.mcs.anl.gov /bin/ls% globus-job-run pitcairn.mcs.anl.gov s myprog% globus-job-run pitcairn.mcs.anl.gov s myprog stdin s in.txt stdout s out.txt GRAM ExamplesThe globusrun client is a more involved prototype that allows complicated RSL expressions.% globusrun r pitcairn.mcs.anl.gov f myjob.rsl% globusrun r pitcairn.mcs.anl.gov &(executable=myprog)Resource Management APIsGlobus Toolkit has APIs for RSL, GRAM, and DUROC:globus_rslglobus_gram_clientglobus_gram_myjobglobus_duroc_controlglobus_duroc_runtimeResource Specification Language可以用于说明作业要求的通用语言RSL是GRAM的核心部分，它提供了不同组件之间交换信息的手段，比如应用与资源代理之间，资源协同分配与资源管理之间的信息交换形式 (attribute=value) 需要GRAM理解这些属性attribute Globus提供使用RSL的API可以用于以上之外的更多场合RSL的一些属性(executable=string)Program to runA file path (absolute or relative) or URL(directory=string)Directory in which to run (default is $HOME)(arguments=arg1 arg2 arg3.)List of string arguments to program(environment=(E1 v1)(E2 v2)List of environment variable name/value pairsRSL的一些属性(stdin=string)Stdin for programA file path (absolute or relative) or URL(stdout=string)Stdout for programA file path (absolute or relative) or URL(stderr=string)Stdout for programA file path (absolute or relative) or URL(count=integer)Number of processes to run (default is 1)(hostCount=integer)On SMP multi-computers, number of nodes to distribute the “count” processes across(project=string)Project (account) against which to charge(queue=string)Queue into which to submit jobRSL的一些属性(maxTime=integer)Maximum wall clock or cpu runtime (schedulerss choice) in minutes(maxWallTime=integer)Maximum wall clock runtime in minutes(maxCpuTime=integer)Maximum CPU runtime in minutes(maxMemory=integer)Maximum amount of memory for each process in megabytes(minMemory=integer)Minimum amount of memory for each process in megabytesRSL Attributes For GRAM(jobType=value)Value is one of “mpi”, “single”, “multiple”, or “condor”mpi: Run the program using “mpirun -np ”single: Only run a single instance of the program, and let the program start the other count-1 processes.multiple: Start instances of the program using the appropriate scheduler mechanismcondor: Start a Condor processes running in “standard universe”RSL Attributes for GRAM(gramMyjob=value)Value is one of “collective”, “independent”Defines how the globus_gram_myjob library will operate on the processescollective: Treat all processes as part of a single jobindependent: Treat each of the processes as an independent uniprocessor job (dryRun=true)Do not actually run jobRSL 的替代符RSL supports simple variable substitutionsSubstitutions are declared using a list of pairs(rslSubstitution=(SUB1 val1)(SUB2 val2)A substitution is invoked with $(SUB)Processing order:Within scope, processed left-to-right, Outer scope processed before inner scopeVariable definition can reference previously defined variables替代符示例This&(rslSubstitution=(URLBASE “ftp:/host:1234”) (rslSubstitution=(URLDIR $(URLBASE)/dir) (executable=$(URLDIR)/myfile)is equivalent to this&(executable=ftp:/host:1234/dir/myfile)GRAM Defined RSL SubstitutionsGRAM defines a set of RSL substitutions before processing the job requestMachine InformationGLOBUS_HOST_MANUFACTURERGLOBUS_HOST_CPUTYPEGLOBUS_HOST_OSNAMEGLOBUS_HOST_OSVERSIONGRAM Defined RSL SubstitutionsPaths to GlobusGLOBUS_INSTALL_PATHGLOBUS_TOOLS_PATHGLOBUS_SERVICES_PATHGLOBUS_DEPLOY_PATHMiscellaneousHOMELOGNAMEGLOBUS_ID用于DUROC的RSL属性(subjobStartType=value)Alters the startup barrier mechanismvalues are “strict-barrier”, “loose-barrier”, “no-barrier”(subjobCommsType=value)values are “blocking-join” and “independent”if value is set to “independent”, the subjob wont be seen from the other subjobs when doing inter-subjob communication.(label=string)Identifier for this subjob(resourceManagerContact=string)(resourceManagerName=string)Resource manager to which to submit a subjobExample: (single resource for now)$ globusrun -r chi/jobmanager-pbs & (executable=/home/abose/test.exe) (host_count=2) (count=4) (arguments=“-t 100 f out.dat) (email_address=“aboseumich.edu) (queue=cac) (pbs_stagein=“morpheus:/home/abose/test.exe) (pbs_stageout=“morpheus:/home/abose/out.dat) (pbs_stdout=/tmp/stdout) (pbs_stderr=/tmp/stderr) (maxwalltime=10)(jobtype=mpi”)“get test.exe from morpheus and run it on hypnos” - submitted by Globus gatekeeper on chi using PBS job managerRSL Example Resulting PBS Submission Script on Hypnos:#! /bin/sh# PBS batch job script built by Globus job manager#PBS -S /bin/sh#PBS -M aboseumich.edu#PBS -m n#PBS -q cac#PBS -W stagein=/home/abose/test.exemorpheus.engin.umich.edu:/home/abose/test.exe#PBS -W stageout=/home/abose/out.datmorpheus.engin.umich.edu:/home/abose/out.dat#PBS -l walltime=10:00#PBS -o hypnos:/tmp/stdout#PBS -e hypnos:/tmp/stderr#PBS -l nodes=2#PBS -vX509_USER_PROXY=/home/abose/.globus/.gass_cache/local/md5/1c/fd/d3/753b9028dfec2ddd6df84cd06c/md5/0a/4b/1d/599dac54863d650c2531cb92fc/data,GLOBUS_LOCATION=/usr/grid,GLOBUS_GRAM_JOB_CONTACT=https:/chi.grid.umich.edu:58963/575/1047861360/,GLOBUS_GRAM_MYJOB_CONTACT=URLx-nexus:/chi.grid.umich.edu:58964/,HOME=/home/abose,LOGNAME=abose,LD_LIBRARY_PATH=#Change to directory requested by usercd /home/abose/usr/gmpi.pgi/bin/mpirun np 4 /home/abose/test.exe t 100 f out.dat Slides taken from NPACI Training, 2003Programming with Globus APIProgramming with Globus API Command line programs syntax: grid_* or globus_* Function calls/APIs start with globus_* Library binaries start with libglobus_*.a Includes:#include /defines most common data structures and others depending on which modules/functions are called in the program. Module Activation/Deactivation:- Functions are arranged in several modules. The corresponding modules must be activated before calling a function:- globus_module_activate(MODULE_NAME)- globus_module_deactivate(MODULE_NAME)- globus_module_deactivate_all()GLOBUS_SUCCESS (0) is returned if successful. Example Module Names: GLOBUS_GRAM_CLIENT_MODULEGLOBUS_IO_MODULEGLOBUS_GASS_COPY_MODULE Dependencies among module activations exist. Read API documentation. 评 价优点：增加了对JOB资源的描述定义了很多Attribute，支持GRAM、DUROC等多种资源管理方式缺点：也是偏重于对计算资源和资源请求的描述，不够广泛可扩展性不好目前仅用于Globus，还不被其他Grid项目所支持WWW服务描述语言WSDLWSDLWeb Service Description Language用于描 述Web服务的技术调用语法。WSDL定义了一套基于 XML的语法，将Web服务描述为能够进行消息交换的服务访问点的集合，从而满足了这种需求。WSDL服务定义为分布式系统提供了可机器识别的SDK文档，并且可用于描述自动执行应用程序通信中所涉及的细节。WSDL的当前版本是1.1，规范可以从http:/www.w3.org/TR/wsdl获得。 WSDLWSDL由Ariba、Intel、IBM和微软等开发商提出。它用一种和具体语言无关的抽象方式定义了给定Web服务收发的有关操作和消息。 WSDL保持协议中立，但它确实内建了绑定SOAP的支持，从而同SOAP建立了不可分割的联系。WSDL的信息模型WSDL信息模型充分利用了抽象规范与规范具体实现的分离，也就是分离了服务接口定义(抽象接口)与服务实现定义(具体端点)。 抽象接口规范描述了终端的处理能力，它在WSDL中表示为portType。束定机制 (binding mechanism)在WSDL中表示为binding元素，它使用特定的通信协议、数据编码模型和底层通信协议，将Web服务的抽象定义映射至特定实现。若束定结合了实现的访问地址，抽象端点也就成为可供服务请求者调用的具体端点(concrete endpoint)，WSDL的port元素表示了这一结合。抽象接口可以支持任何数量的操作(operations)。操作是由一组消息(messages)定义，消息定义了操作的交互定式。与抽象的消息、操作概念相对应的具体实现是由binding元素指定。与XML应用相同，WSDL模式定义了几个高层元素，或称为主要元素。 WSDL描述的基本属性服务做些什么-服务所提供的操作(方法)。如何访问服务-数据格式详情以及访问服务操作的必要协议。服务位于何处-由特定协议决定的网络地址，如URL。 WSDL基本元素的含义元素名含义types定义了Web服务使用的所有数据类型集合，可被元素的各消息部件所引用。 message通信消息数据结构的抽象类型化定义。使用Types所定义的类型来定义整个消息的数据结构。 operation对服务中所支持操作的抽象描述。 portType对于某个访问入口点类型所支持操作的抽象集合。 binding包含了如何将抽象接口的元素(portType)转变为具体表示的细节，具体表示也就是指特定的数据格式和协议的结合 port定义为协议/数据格式绑定与具体Web访问地址组合的单个服务访问点。 service定义服务WSDL信息模型 WSDL对象结构图 WSDL文档类型 typestypestypestypesmessagemessageoperationoperationportTypebindingbindingportportportserviceserviceWSDL文档结构WSDL工具Omniopera-图形用户界面的WSDI、XML和XSD编辑器。Microsoft的SOAP Toolkit-一种工具包，其中包括根据WSDL定义创建COM接口的向导程序，还包括根据COM接口创建WSDL的向导程序。IBM的Web Services Toolkit-一种工具包，其中包括产生WSDL和SOAP部署说明的向导程序。 资源描述框架RDFRDFResource Description Framework, RDFW3C的资源描述框架(RDF)的目的是提供一个访问网络资源元数据(metadata)的标准，因此也提供了一个描述特定资源内容的标准协议。W3C应用元数据时的推荐标准是一个模型，一种句法(syntax(es)应用在Web上时，RDF 通常用XML来编码是语义万维网 (semantic Web)的基础、支撑W3C - Resource Description Framework (RDF)http:/www.w3.org/RDF/RDF是一个用于表达关于万维网(World Wide Web)上的资源的信息的语言。专门用于表达关于Web资源的元数据, 比如Web页面的标题、作者和修改时间，Web文档的版权和许可信息，某个被共享资源的可用计划表等 为什么要使用 RDF?RDF提供共享元数据的模型(model)共享语义(meaning)元数据可以在相互了解不多或根本不了解的应用之间共享例如一个基于RDF的书目应用能够吸收基于RDF的地理空间应用的元数据并对其意义有所理解。用(X)HTML和XML置标后，软件应用必须能够理解复杂的编码RDF的基本思想用Web标识符（称作统一资源标识符，Uniform Resource Identifiers或URIs）来标识事物，用简单的属性（property）及属性值来描述资源。这使得RDF可以将一个或多个关于资源的简单陈述表示为一个由结点和弧组成的图（graph），其中的结点和弧代表资源、属性或属性值。 举例有一个人由http:/www.w3.org/People/EM/contact#me 标识, 他的名字是Eric Miller, 他的电子邮件地址是emw3.org,他的头衔是Dr. Eric MillerDr. 统一资源标识符URIURIUniform Resource Identifiers,URI是一种简单的可扩展的指定资源的方法URI的同一性尽管不同资源的访问机制可能不同，但URI允许不同类型的资源标识符在相同的上下文环境中使用URI允许用统一的语义解释跨越不同类型资源标识符的通用语法规范URI可以在不影响已有的标识符系统的情况下，引入新类型标识符URI允许在多种不同的环境中重用同一个标识符URI允许新的应用或协议采用已经存在的、广泛使用的资源标识符URI举例ftp:/ftp.is.co.za/rfc/rfc1808.txtGopher:/spinaltap.micro.umn.edu/00/Weather/Californian/LosAngeleshttp:/www.cs.sdu.edu.cn/index.htmlmailto:gbsdu.edu.cnNews:comp.infosystem.www.servers.unixtelnet:/159.226.39.252URI=URL+URNURL（Uniform Resource Locators）统一资源定位器URN（Uniform Resource Name）统一资源名字从不同角度标识一个资源URL一般形式是 :scheme:ftp,http,Gopher,mailto,news,nntp,telnet,wais,File,prospero/:/URN:=“urn:”“:”：名字空间标识符是符合名字空间规范的字符串NID:=1,31:=|“-”:= :=|Upper:大写字母，lower:小写字母，number:数字URN:=I*:=|“%”:=|“A”|“B”|“C”| “D”| “E”| “F”| “a”| “b”| “c” |“d” |“e”| “f”:= “（”|“）”|“+”| “，”| “-”| “.”| “:”| “=”| “” |“;” |“$”| “_” |“!”|“*”|“” LDAP中的资源描述LDAP用一系列“属性对”的形式来存储记录项，每一个记录项包括类型和属性值。举例dn:cn=My Computer, ou=devices, dc=sdu, dc=edu.cncn:FB Computerusage:computing resouce:866MHZresource:512M memoryresource:60GB storageresource:Linux OS资源命名资源命名的意义和作用资源名可以把资源进一步抽象，将资源的标识和资源的位置分离开来资源命名机制可以建立虚拟空间，扩大或缩小用户空间实现资源的按名访问，方便用户使用资源的命名有不同形式逻辑名称：方便用户，便于记忆物理名称：实际名字内部名称：系统内部外部名称：外部提供用户命名规则，防止冲突风格统一全球唯一