云计算技术及应用,大连理工大学计算机科学与技术学院 2010年春季,基本情况,申彦明 B810 助教：齐恒 B812 Office hour: Fri 3:30-4:30 PM Course website: 教材内容 Project 论文,教材内容,分布式系统的概况 分布式与集群基本概念 分布式数据库 分布式文件系统 GFS 分布式编程 MapReduce算法介绍 搜索引擎与PageRank 其它相关技术 Data Center BigTable AppEngine,Grading,HW：40% Final Project: 60% Final project proposal Project reports 12 teams, 4-5 students,Syllabus (Subject to change),Week 2 Mar 8: Lecture 1: Introduction Mar 10: Lecture 2: Map/Reduce Theory and Implementation, Hadoop Week 3 Mar 15: Lecture 3 & 4: Guest Speaker (8:00 AM-11:35AM研教楼102) Mar 17: Lecture 5: Distributed File System and the Google File System Week 4 Mar 22: Lecture 6 & 7: Guest Speaker(8:00 AM-11:35AM研教楼102) Mar 24: Lecture 8: Distributed Graph Algorithms and PageRank Week 5 Mar 29: Lecture 9: Introduction to Some Projects Mar 31: Lecture 10: Data Centers,Syllabus (Subject to change),Week 6 Apr 5: Lecture 11: Some Google Technologies Apr 7: Lecture 12: Virtualization Week 7 Lecture 13 & 14: Project Presentation Week 8: No class Week 9: Lecture 15 &16: Project Presentation,Gartner Report,Top 10 Strategic Technology Areasfor 2009 Virtualization Cloud Computing Servers: Beyond Blades Web-Oriented Architectures Enterprise Mashups Specialized Systems Social Software and Social Networking Unified Communications Business Intelligence Green Information Technology,Top 10 Strategic Technology Areas for 2010 Cloud Computing Advanced Analytics Client Computing IT for Green Reshaping the Data Center Social Computing Security Activity Monitoring Flash Memory Virtualization for Availability Mobile Applications,From Desktop/HPC/Grids to Internet Clouds in 30 Years,HPC moving from centralized supercomputers to geographically distributed desktops, clusters, and grids to clouds over last 30 years R/D efforts on HPC, clusters, Grids, P2P, and virtual machines has laid the foundation of cloud computing that has been greatly advocated since 2007 Location of computing infrastructure in areas with lower costs in hardware, software, datasets, space, and power requirements moving from desktop computing to datacenter-based clouds,What is Cloud Computing?,1. Web-scale problems 2. Large data centers 3. Different models of computing 4. Highly-interactive Web applications,1. “Web-Scale” Problems,Characteristics: Definitely data-intensive May also be processing intensive Examples: Crawling, indexing, searching, mining the Web Data warehouses Sensor networks “Post-genomics” life sciences research Other scientific data (physics, astronomy, etc.) Web 2.0 applications ,How much data?,Google processes 20 PB a day (2008) “all words ever spoken by human beings” 5 EB CERNs LHC will generate 10-15 PB a year,640K ought to be enough for anybody.,What to do with more data?,Answering factoid questions Pattern matching on the Web Works amazingly well Learning relations Start with seed instances Search for patterns on the Web Using patterns to find more instances,How do I make money?,Petabytes of valuable customer data Sitting idle in existing data warehouses Overflowing out of existing data warehouses Simply being thrown away Source of data: OLTP User behavior logs Call-center logs Web crawls, public datasets Structured data (today) vs. unstructured data (tomorrow) How can an organization derive value from all this data?,2. Large Data Centers,Web-scale problems? Throw more machines at it! Centralization of resources in large data centers Necessary ingredients: fiber, juice, and land What do Oregon, Iceland, and abandoned mines have in common? Important Issues: Efficiency Redundancy Utilization Security Management overhead,3. Different Computing Models,Utility computing Why buy machines when you can rent cycles? Examples: Amazons EC2 Platform as a Service (PaaS) Give me nice API and take care of the implementation Example: Google App Engine Software as a Service (SaaS) Just run it for me! Example: Gmail,“Why do it yourself if you can pay someone to do it for you?”,4. Web Applications,What is the nature of future software applications? From the desktop to the browser SaaS = Web-based applications Examples: Google Maps, Facebook How do we deliver highly-interactive Web-based applications? AJAX (asynchronous JavaScript and XML) A hack on top of a mistake built on sand, all held together by duct tape and chewing gum?,Some Cloud Definitions,Ian Foster et al defined cloud computing as a large-scale distributed computing paradigm, that is driven by economics of scale, in which a pool of abstracted virtualized, dynamically-scalable, managed computing power, storage, platforms, and services are delivered on demand to external customers over the internet(云计算是一种商业计算模型。它将计算任务分布在大量计算机构成的资源池上，使各种应用系统能够根据需要获取计算力、存储空间和各种软件服务。) IBM experts consider clouds that can: Host a variety of different workloads, including batch-style backend interactive, user-facing applications Allow workloads to be deployed and scaled-out quickly through the rapid provisioning of virtual machines or physical machines Support redundant, sel