Azure Academic Pilot - Recently Updated Pages

The NIST Definition

-Krishna.Kumar- — Wed, 10 Mar 2010 09:05:19 CST

The National Institute of Standards and Technology NIST is an agency of the U.S. Department of Commerce and was founded in 1901 as the nation's first federal physical science research laboratory. One of the major research components of the NIST, their Information Technology Laboratory (ITL) accelerates the development and deployment of information and communication systems that are reliable, usable, interoperable, and secure; advances measurement science through innovations in mathematics, statistics, and computer science; and conducts research to develop the measurements and standards infrastructure for emerging information technologies and applications.

In this document, they have brilliantly outlined the definitions, attributes and characteristics of cloud computing:
http://csrc.nist.gov/groups/SNS/cloud-computing/cloud-def-v15.doc

Creating an online Azure account

-Krishna.Kumar- — Mon, 08 Mar 2010 15:54:51 CST

There is no abstract available for this page revision.

Introduction to the Windows Azure Platform

-Krishna.Kumar- — Sun, 07 Mar 2010 17:54:40 CST

Table of Contents:

Overview
Introducing Windows Azure

OVERVIEW:

Microsoft's Windows Azure Platform is an internet-scale cloud platform (cloud computing platform as a service - PaaS) offering that provides an operating system and a set of developer services that can be used individually or together. Azure’s flexible and interoperable platform can be used to build new applications to run from the cloud or enhance existing applications with cloud-based capabilities. Its open architecture gives developers the choice to build web applications, applications running on connected devices, PCs, servers, or hybrid solutions offering the best of online and on-premises.

Windows Azure: Windows Azure is a cloud services operating system that serves as the development, service hosting and service management environment for the Windows Azure Platform. Windows Azure provides developers with on-demand compute and storage to host, scale, and manage internet or cloud applications.

Live Services: Live Services is a set of building blocks within the Windows Azure Platform for handling user data and application resources. Live Services provides developers with an easy on-ramp to build rich social applications and experiences, across a range of digital devices that can connect with one of the largest audiences on the Web.

Microsoft SQL Azure: Microsoft SQL Azure extends the capabilities of Microsoft SQL Server into the cloud as a Web-based, distributed relational database. It provides Web services that enable relational queries, search, and data synchronization with mobile users, remote offices and business partners. It can store and retrieve structured, semi-structured, and unstructured data.

Microsoft .NET Services: Microsoft .NET Services make developing loosely coupled cloud-based applications easier. .NET Services includes access control to help secure your applications, as well as a service bus for communicating across applications and services. These hosted services allow you to easily create federated applications that span from on-premises environments to the cloud.

Microsoft SharePoint Services & Dynamics CRM Services: In the future, developers will have access to SharePoint & CRM functionality for collaboration and building stronger customer relationships. With the flexibility to use familiar developer tools like Visual Studio, developers will be able to rapidly build applications that utilize SharePoint and CRM capabilities as developer services for their own applications. Developers can expect a breadth of SharePoint & CRM capabilities across the spectrum of on-premises, online & the Windows Azure Platform.

Benefits of the Windows Azure Platform:

The Windows Azure Platform is designed to help developers easily create applications for the web and connected devices. The services platform offers the greatest flexibility, choice, and control in reaching users and customers while using existing skills.

Easy developer on-ramp to the cloud - Millions of developers worldwide already use the .NET Framework and the Visual Studio development environment. Utilize those same skills to create cloud-enabled applications that can be written, tested, and deployed all from Visual Studio. In the near future developers will be able to deploy applications written on Ruby on Rails and Python as well.

Enables Agile & Rapid Results - Applications can be deployed to the Windows Azure Platform with the click of a button. Changes can be made quickly and without downtime, making it an ideal platform for affordably experimenting and trying new ideas.

Imagine and Create New User Experiences - The Windows Azure Platform enables you to create web, mobile, or hybrid-applications that use the cloud with on-premises applications. Combined with Live Services ability to reach over 400 million Live users, new opportunities exist to interact and reach users in new ways.

Standards-Based Compatibility - The services platform supports industry-standard protocols, including HTTP, REST, SOAP, RSS, and AtomPub, for consuming, exposing, and integrating with third-party services. You can easily integrate applications built on a variety of different technologies and operating systems.

As you may remember from a couple of modules back, the following figure illustrates the typical cloud computing platform architecture:

The following figure maps the Windows Azure platform to the generic layered cloud architecture above:

This platform provides a set of services to application developers. These services can be used both by applications running in the cloud and by applications running on local systems. Windows Azure, operating system for the cloud, is the foundation of Microsoft's cloud platform offering. As shown in the above figure, the Windows Azure Platform provides a set of shared services: SQL Data Services, .NET Services, and Live Services, which can be used individually or collectively. In addition, Microsoft offers various finished cloud applications including Exchange Online, SharePoint Online, and CRM Online. Here, however, our focus will be on the Windows Azure operating system and related patterns.

The following whitepaper provides a comprehensive overview of the Windows Azure Platform:

Cloud Protocols, Standards and Wire Formats

-Krishna.Kumar- — Sun, 07 Mar 2010 17:45:02 CST

Cloud computing relies on the ability of users to reach across the internet and access the compute and storage resources offered in the cloud. And therefore, to enable this interaction, it relies on a number of open standards around areas like communication and data representation.

In this section, we will touch upon the key communication, service and data representation standards.

Cloud computing, most commonly exposes its resources as a service over the Internet, and therefore the HTTP and HTTPS protocols act as the backbone communication infrastructures supporting cloud computing. All the other standards for exposing compute and storage resources, therefore are technologies that have evolved to closely interoperate with HTTP around issues like scalability and statelessness. In the following section, we will look at the REST architecture for exposing services, the AtomPub protocol for creating and updating web resources and the XML, Atom and JSON data representation formats.

Pre-REST: Prior to REST, there was a long evolution of attemping to retrofitting existing client/server-based Inter-process communication systems paradigms over to the web. Early in the Internet's history, circa 1995, the Common Gateway Interface (CGI) was created to handle computational and data-centric requests that didn't necessarily always fit into the hypertext-based document model. for eg. payment processing.

One of the most common distributed IPC paradigms that had taken hold in the client-server space was Remote Procedure Call with implementations such as Java RMI, DCOM and CORBA. Initial implementations to port the above model over to the web in a literal sense resulted in technologies like XML-RPC which used XML to encode its calls and HTTP as a transport mechanism. HTTP was chosen as a framing protocol mostly because it was entrenched enough so that people already had port 80 open in the firewall. XML was chosen because the bet was that XML was going to be the platform-neutral meta-schema for data.

As new functionality was introduced, the standard evolved in in late 1999 to what is now SOAP — the Simple Object Access Protocol. SOAP defined a wire protocol that standardized how binary information within the computer's memory could be serialized to XML. Initially there were no discovery and description mechanisms either, but eventually the Web Service Description Language (WSDL) was settled on. WSDL enabled SOAP-based service designers to more fully describe the XML their service would accept as well as an extensibility mechanism. Using this extensibility mechanism, the Web service community came up with a collection of extensional specifications that together are referred to as the WS-* ("WS-star") specifications. These include WS-Security, WS-Reliable Messaging, WS-Addressing, and WS-Policy, to name just a few. These specifications all essentially added layered metadata to the SOAP payload by modifying the header while keeping the message body intact.

But there is a pattern here we should more closely examine. In the beginning we had simple hypermedia documents. However, as soon as we added the capability to access server-based application data processes, the basic architecture of our applications began moving away from the pure hypermedia model. CGI introduced a "gateway" where none had existed. SOAP introduced a novel use of XML, but this then required servers to accept and handle SOAP requests with arbitrarily complex payloads or negotiate lowest common denominators. Then came WSDL. With WSDL it became easier to modify the client's SOAP request to match what the service provider required, but then the server needed to formulate the WSDL, adding more processing requirements and moving the data communication architecture even further away from a pure hypermedia basis. And when the incredible complexity of the WS-* specifications is added, the entire data communication process incorporates processing and metadata transfer completely unforeseen only a few years ago.

The gist of all of this is that somewhere along the way, all of us associated with designing and developing RPC-based Web services diverted from the basic architecture of the Internet. SOAP and WS-* started from examining various platform-specific solutions like DCOM and CORBA, and their ultimate goal was to build those same types of systems in a platform-neutral way. To accomplish this goal, they built on the substrate of some of the emerging Internet protocols of the time - HTTP & XML. All the advanced WS-* specifications (Security/Reliability/Transactions) were taking application server features du jour and incorporating those protocols into this platform-neutral protocol substrate.

REST: In 2000, which is about the time the SOAP protocol started to take root, Roy Fielding, a co-designer of the HTTP specification, finished his doctoral dissertation, titled "Architectural Styles and the Design of Network-based Software Architectures" (www.ics.uci.edu/~fielding/pubs/dissertation/top.htm). His dissertation revolves around the evaluation of network-based architectures, but his final two chapters introduce the concepts of Representational State Transfer, or REST. REST essentially formalizes services using concepts inherent in the Internet.

The principal concept in REST is the existence of resources (sources of specific information), each of which is referenced with a global identifier (e.g., a URI in HTTP). In order to manipulate these resources, components of the network (user agents and origin servers) communicate via a standardized interface (e.g., HTTP) and exchange representations of these resources (the actual documents conveying the information), which is typically an HTML, XML or JSON document of some kind, although it may be an image, plain text, or any other content.

The following short video provides a very good overview of REST:

Atom / AtomPub: The name Atom applies to a pair of related standards. The Atom Syndication Format is an XML language used for web feeds, while the Atom Publishing Protocol (AtomPub or APP) is a simple HTTP-based protocol for creating and updating web resources. The following article: “Atom Publishing Protocol” by James Snell introduces in detail AtomPub, Atom Syndication Format and curl commands.
http://www-128.ibm.com/developerworks/library/x-atompp1/

ADO.Net Data Services: The goal of the ADO.NET Data Services framework is to facilitate the creation of flexible data services that are naturally integrated with the web. As such, ADO.NET Data Services use URIs to point to pieces of data and simple, well-known formats to represent that data, such as JSON and ATOM (XML-based feed format). This results in the data service being surfaced as a REST-style resource collection that is addressable with URIs and that agents can interact with using standard HTTP verbs such as GET, POST, PUT or DELETE. In order for the system to understand and leverage semantics over the data that it is surfacing, ADO.NET Data Services models the data exposed through the data service using a model called the Entity Data Model (EDM), an Entity-Relationship derivative. This organizes the data in the form of instances of "entity types", or "entities", and the associations between them. For relational data, ADO.NET Data Services supports exposing an EDM model created using the ADO.NET Entity Framework. For all other (ie. non relational) data sources or to use additional database access technologies (ex. LINQ to SQL) a mechanism is provided which enables any data source to be modeled as entities and associations (ie. described using an EDM schema) and exposed as a data service.

Many of the Microsoft cloud data services (Windows Azure tables, SQL Azure Data Services, etc.) expose data using the same REST interaction conventions followed by ADO.NET Data Services. This enables using the ADO.NET Data Services client libraries and developer tools when working with hosted cloud services.

The following video goes into details on how to consume ADO.Net Data Services from client side code.

In case you are interested, the other videos from this series provide additional details on creating and consuming ADO.Net Data services.

How Do I: Getting Started with ADO.NET Data Services over a Relational Database

How Do I: Getting Started with ADO.NET Data Services over a Non-Relational Data Source

How Do I: Consuming an ADO.NET Data Service in a Silverlight Application

How Do I: Consume an ADO.NET Data Service in a .NET Application

credits: Kenn Scribner

Virtualization

-Krishna.Kumar- — Sun, 07 Mar 2010 17:32:07 CST

As we saw in the previous section, the principles of cloud computing include:

Virtualization and automation
Interchangeable (fungible) resources such as servers, storage and network
Management of these resources as a single fabric
Elastic capacity (scale up or down) to respond to business demands
Applications (and the tools to develop them) that can truly scale out
Focused on the service delivered to the business

A key enabler for many of the above principles is the concept of virtualization.

A host computer runs an application known as a hypervisor; this creates one or more virtual machines, which simulate real computers so faithfully that the simulations can run any software, from operating systems to end-user applications. The software "thinks" it has access to a processor, network, and disk drive, just as if it had a real computer all to itself. The hypervisor retains ultimate control, however, and can pause, erase, or create new virtual machines at any time. By providing multiple VMs at once, this approach allows several operating systems to run simultaneously on a single physical machine.

Rather than paying for many under-utilized server machines, each dedicated to a specific workload, server virtualization allows those workloads to be consolidated onto a smaller number of more fully-used machines. Virtualization means that e-mail, Web, or file servers (or anything else) can be conjured up as soon as they're needed; when the need is gone, they can be wiped from existence, freeing the host computer to run a different virtual machine for another user. Coupled with management software and vast data centers, this technology allows cloud providers to reap massive economies of scale. And it gives cloud users access to as much computing power as they want, whenever they want it. Virtualization reduces IT costs, increases hardware utilization, optimizes business and network infrastructure, and improves server availability.

Preface

-Krishna.Kumar- — Sun, 07 Mar 2010 15:44:40 CST

Table of Contents:

Welcome
Evolution of Hardware & Software Paradigms

Welcome:

Tools

-Krishna.Kumar- — Sun, 07 Mar 2010 15:41:58 CST

While Azure projects can be created and packaged using a generic text editor and the command line, good tooling makes the process a whole lot more smoother and productive.

We recommend using Visual Studio 2008 for developing applications on Azure.

Getting Started with Windows Azure

1. Get Visual Studio
You can either use the free Visual Web Developer Express edition or download the professional version of Visual Studio from MSDN AA or Dreamspark.

Download Visual Studio 2008 from MSDN AA
or Download from DreamSpark
or Visual Studio 2008 Trial Versions
or Free Download: Visual Web Developer 2008

Make sure SP1 is installed.
2a. Enable IIS7 with ASP.NET and WCF HTTP Activation

Steps for Vista / Win 7 / Win Svr 2008 here

2b. Get the SDK and Tools
You do not need to download the Azure SDK separately if you download the following package:

Windows Azure Tools for Microsoft Visual Studio (includes the Windows Azure SDK)

3. Get an Azure Deployment Token
Required only to upload your solution to the Cloud.

Introduction to Cloud Computing

-Krishna.Kumar- — Sat, 06 Mar 2010 11:17:43 CST

Table of Contents:

OVERVIEW:

Cloud computing represents perhaps the biggest paradigm shift in the delivery architecture of information services, since the invention of timesharing or the introduction of the client/server network.

Source: Forrester

The term cloud is used as a metaphor for the Internet, based on how the Internet is depicted in computer network diagrams and is an abstraction for the complex infrastructure it conceals. And thus, cloud computing is the style of computing in which functionality [compute (applications) & storage (data)] is provided as a service over the internet or over an internal/external, location-transparent, centralized facility. No elaborate construction or development program is needed to kick-start such technology--the infrastructure is already in place and making money. Existing data centers, built to support the likes of Amazon and Microsoft, can rent spare capacity, creating a collection of services that provide the illusion of infinite computing power and storage on tap.

The key motivation behind the growth of cloud computing are the factors of cost and agility. In 2000, according to commerce department statistics, the average % of capital equipment budget spent on IT was 45%, which is a significant share. On the other hand, the % of utilized server capacity on average is 6% (Economist Survey on IT) mainly due to the overprovisioning of hardware due to the necessity to engineer for peak load. What cloud computing attempts to do is to provide access to the said infrastructure as a utility service thereby making the above problems obsolete. Additionally, unlike in-house or hosted investments in IT, cloud computing allows the provisioning of the resources to scale up and down with demand with almost no up-front lead time requirements. This allows a company to be as nimble as its operations require, resulting in increased agility and decreased cost.

The following 2 videos, between them, expound on the above topics further and provide a very high level overview of the concept of cloud-computing and contrast it against the traditional hosting methodologies as shown in the above diagram.

5 dimensions of the Cloud

-Krishna.Kumar- — Sat, 06 Mar 2010 11:17:10 CST

The cloud creates opportunities and responsibilities
The cloud learns and helps you learn, decide and take action
The cloud enhances your social and professional interactions
The cloud wants smarter devices
The cloud drives server advances that drive the cloud

On Mar 4, 2010, Steve Ballmer spoke to a group of students and faculty at the University of Washington to discuss how cloud computing will change the way people and businesses use technology. During that talk, he outlined the above dimensions of the cloud. Here is the full speech.

http://www.microsoft.com/cloud/ is the location of the official Microsoft Cloud Services effort.

IIT Cloud Workshop

-Krishna.Kumar- — Tue, 02 Mar 2010 15:49:08 CST

Lab Files

Welcome to the Azure Academic Pilot

-Krishna.Kumar- — Fri, 05 Feb 2010 11:38:39 CST

Windows Azure is the development, hosting, and management environment of the Microsoft Cloud Platform that enables you to run applications at Internet scale while leveraging the skills and tools you use today.

The goal of this light-weight, free-form program is to get early hands-on access to the Windows Azure Platform and shape the content surrounding this exciting technology for use by selected faculty and students around their research and teaching needs.

Please use the links on the left to browse through the content modules on this site.

Other Cloud Considerations

-Krishna.Kumar- — Mon, 14 Dec 2009 04:14:03 CST

Table of Contents:

This section goes into the other details of cloud computing that were not covered so far. We will look at the Azure Fabric that hosts the compute and storage components, the fabric controller that manages the health of fabric and the applications running in the fabric. We will also look at best practices for development on the cloud keeping in mind some of its idiosyncrasies and finally, we will end with a section on Cloud Economics and ROI on the cloud vs. traditional on-premises offerings.

Physical Design and Architecture of the 'Cloud' - Datacenter Design

-Krishna.Kumar- — Mon, 14 Dec 2009 00:24:51 CST

The 'Cloud' in cloud computing, where our applications end up running, more often than not is a a data center owned and operated by the vendor providing the cloud service.

A data center, also called a server farm, is a facility used to house computer systems and associated components, such as telecommunications and storage systems. It generally includes redundant or backup power supplies, redundant data communications connections, environmental controls (e.g., air conditioning, fire suppression) and security devices.

A data center can occupy one room of a building, one or more floors, or an entire building. The traditional design of equipment in a data center is often in the form of servers mounted in 19 inch rack cabinets, which are usually placed in single rows forming corridors between them.

Other Resources

-Krishna.Kumar- — Sat, 26 Sep 2009 17:14:27 CDT

Azure Case Studies:

http://www.microsoft.com/azure/casestudies.mspx

Azure Pricing and SLA:

http://www.microsoft.com/azure/pricing.mspx

Azure Technology Whitepapers:

http://www.microsoft.com/azure/whitepaper.mspx

Microsoft Cloud Security Whitepaper:

http://www.globalfoundationservices.com/security/documents/SecuringtheMSCloudMay09.pdf

Presentations:

AMCIS 2009: Slide Deck

AMCIS Cloud Computing Workshop - Collateral

-Krishna.Kumar- — Thu, 10 Sep 2009 15:35:17 CDT

Please find the slide deck for the AMCIS 2009 Cloud Computing workshop attached here.

Evolution of (Hardware &) Software Paradigms

-Krishna.Kumar- — Mon, 03 Aug 2009 14:44:36 CDT

Pre 1970s:
Mainframes roamed the earth

When the concept of the commercial computer was relatively rare and in its infancy, computers were hard-wired and then later programmed using binary code that represented control sequences fed to the computer CPU. This was difficult and error-prone. The downside was that the code executed on these computers was in a machine-specific "language" with the program coupled to the hardware itself. Code developed for one machine could not run on another. Programs written in binary are said to be written in machine code, which is a very low-level programming paradigm.

As the use of computers spread and the need for various business applications was felt, assembly languages were developed to make programming easier. These replaced machine code functions with mnemonics and absolute memory addresses with symbolic labels. Assembly language decoupled the code from specific machines, enabling it to run on multiple computers. That code, however, was now coupled to the machine architecture: code written for an 8-bit machine could not run on a 16-bit machine, let alone withstand differences in the registers or available memory and memory layout. Assembly language programming is considered a low-level paradigm although it is a 'second generation' paradigm.

The next advance was the development of compiler based higher-level languages such as COBOL and FORTRAN in the 1960s. These third-generation languages (the first described as high-level languages) use vocabulary related to the problem being solved. For example,

COBOL (Common Business Oriented Language) - uses terms like file, move and copy.
FORTRAN (FORmula TRANslation)

The developer would write in an abstraction of machine programming (the language), and the compiler would translate that into actual assembly code. Compilers for the first time decoupled the code from the hardware and its architecture. The problem with those first-generation languages was that the code resulted in nonstructured programming, where the code was internally coupled to its own structure via the use of jump or go-to statements, resulting in hard to maintain code whose control flows resembled a bowl of spaghetti. Minute changes to the code structure often had devastating effects in multiple places in the program.

1970s:
The rise of the Minis

With the advent of the mini-computer, programming as an activity became more interactive and democratized and consequently, the next advance was the development of procedural languages, based upon the concept of the procedure call. Procedures, also known as routines, subroutines, methods, or functions simply contain a series of computational steps to be carried out. Additionally, the modular and structured nature of these procedural languages such as C and Pascal, decoupled the code from its internal layout and structure using functions and structures and significantly reduced the bowl of spaghetti problem that was faced by the previous generation of sequential languages.

Mid-1980’s:
Proliferation of the PCs:

The explosion of the PC revolution resulted in the demand for a wide variety of commercial and open source software while completely democratizing the developer experience thereby enlisting cadres of programmers into the fold. This resulted in an urgent requirement for a reusable paradigm of software development after finding that traditional structured programming resulted in compromised quality as hardware and software became increasingly complex, due to the fact that the basic unit of reuse is the function. But the problem with function-based reuse is that the function is coupled to the data it manipulates, and if the data is global, a change to benefit one function in one reuse context is likely to damage another function used somewhere else. Researchers studied ways to maintain software quality and developed object-oriented programming in part to address common problems by strongly emphasizing discrete, reusable units of programming logic. The methodology focuses on data rather than processes, with programs composed of self-sufficient modules (objects) each containing all the information needed to manipulate its own data structure. The only way that a user can access the data is via the object's 'methods' (subroutines). Thanks to this, the internal workings of an object may be changed without affecting any code that uses the object. The mechanism of reuse was class-based, enabling both direct reuse and specialization via inheritance.

Additionally, with the proliferation of PCs, came the concept of commercial networks and network programming. At this stage creating what could be vaguely described as a distributed application meant network programming using sockets, since OOP languages required direct memory references and did not support distribution.

1990s:
Explosion of the Networked Server

As the need for software and code reuse mushroomed in the 90s amidst the proliferation of interconnected PCs, developers started discovering the following problems with the Object Oriented Model of software development:

The generated application was a single, monolithic application. Languages did not concern themselves with binary representation of the generated code, and so, every time a minute change was made to the code, the entire application had to be redeployed.
While the basic unit of reuse was a class, it was a class in source format. Consequently, the application was coupled to the language used. Language and source based reuse lead to a host of other problems.
Inheritance turned out to be a poor mechanism for reuse, often harboring more harm than good because the developer of the derived class needed to be intimately aware of the implementation of the base class (which introduced vertical coupling across the class hierarchy).
Oblivious to issues like deployment, versioning, fault isolation, authorization and authentication of client objects, scalability, responsiveness and throughput.

The solution for the problems of object-orientation evolved over time, involving technologies such as the static library (.lib) and the dynamic library (.dll), culminating in 1994 with the first component-oriented technology, called COM (Component Object Model). Component-orientation provided interchangeable, interoperable binary components. With this approach, instead of sharing source files, the client and the server agree on a binary type system (such as IDL) and a way of representing the metadata inside the opaque binary components. The client only programs against an abstraction of the service: a contract called the interface. As long as the interface is immutable, the service is free to evolve at will. A proxy can implement the same interface and thus enable seamless remote calls by encapsulating the low-level mechanics of the remote call. The availability of a common binary type system enables cross-language interoperability.

The above paradigm lent itself readily to distributed programming and many technologies were created for communication among software components distributed across networked computers. The most notable ones are DCOM, CORBA, RMI and .NET Remoting.

The following were the most common architectures of distributed programming that were in vogue:

Client-server — Smart client code contacts the server for data, then formats and displays it to the user. Input at the client is committed back to the server when it represents a permanent change.

3-tier architecture — Three tier systems move the client intelligence to a middle tier so that stateless clients can be used. This simplifies application deployment. Most web applications are 3-Tier.

N-tier architecture — N-Tier refers typically to web applications which further forward their requests to other enterprise services. This type of application is the one most responsible for the success of application servers.

Early 2000s:
The Internet comes of Age

As the Internet grew in popularity and critical mass towards the end of the 90’s, it became an application platform in its own right and the distributed technology that evolved to bridge the heterogeneous nature of the Internet was Web Services. It provided the same component oriented functionalities as DCOM, CORBA and RMI, but did so in a more interoperable fashion. Web Services usually used Extensible Markup Language (XML) messages that follow the Simple Object Access Protocol (SOAP) standard and have been popular with traditional enterprise. In such systems, there is often a machine-readable description of the operations offered by the service written in the Web Services Description Language (WSDL), similar to IDL from earlier technologies.

Late 2000s:
Present Day

Combining the concept of Web Services and loose coupling led to the introduction of the ‘Service Oriented Architecture’ in the late 2000s that has emerged as the answer to the shortcomings of component-orientation. In a service-oriented application, developers focus on writing business logic and expose that logic via interchangeable, interoperable service endpoints. The business logic in each of these services comprises intrinsically unassociated, loosely coupled units of functionality that have no calls to each other. Clients consume those endpoints (not the service code, or its packaging). The interaction between the clients and the service endpoint is based on a standard message exchange, and the service publishes some standard metadata describing what exactly it can do and how clients should invoke operations on it. The metadata is the service equivalent of the C++ header file, the COM type library, or the .NET assembly metadata, yet it contains not just operation metadata (such as methods and parameters) but also plumbing metadata. Incompatible clients—that is, clients that are incompatible with the plumbing expectations of the object—cannot call it, since the call will be denied by the platform. In many respects, a service is the natural evolution of the component, just as the component was the natural evolution of the object, which was the natural evolution of the function.

One additional deployment technology that has taken the IT world by storm recently is “Cloud Computing”. Cloud computing is a style of computing in which dynamically scalable and often virtualized resources are provided as a service over the Internet. The concept generally incorporates combinations of the following that we shall explore in detail in the next module.

infrastructure as a service (IaaS)
platform as a service (PaaS)
software as a service (SaaS)
software + services (S+S)

Cloud computing customers do not generally own the physical infrastructure serving as host to the software platform in question. Instead, they avoid capital expenditure by renting usage from a third-party provider. They consume resources as a service and pay only for resources that they use. Many cloud-computing offerings employ the utility computing model, which is analogous to how traditional utility services (such as electricity) are consumed, while others bill on a subscription basis. Sharing "perishable and intangible" computing power among multiple tenants can improve utilization rates, as servers are not unnecessarily left idle (which can reduce costs significantly while increasing the speed of application development). A side effect of this approach is that overall computer usage rises dramatically, as customers do not have to engineer for peak load limits. Additionally, "increased high-speed bandwidth" makes it possible to receive the same response times from centralized infrastructure at other sites.

Summary

Here is a brief recap of the evolution of the computing models from above:

Period	Platform	Software Paradigm	Examples	Distributed Paradigm	Examples
Pre 1960s	Mainframes	Low Level Programming	Machine Lang, Assembly Lang.	None
Pre 1970s	Mainframes/Minis	High Level Sequential Programming	COBOL, FORTRAN	None
1970s	Minicomputers	Structured Programming	C, Pascal	None
1980s	Personal Computers	Object Oriented Programming	Smalltalk, C++	Network programming using Sockets	TCP Socket Programming
1990s	Networked Computers	Component oriented programming	Frameworks like COM	Homogeneous Client Server, 3-Tier, n-Tier Architectures	RPC based technologies like DCOM, CORBA, RMI
Early 2000s	Heterogeneous Networks (Internet)	Component oriented programming	.NET	Heterogeneous n-Tier Architectures, Web Computing	Message passing technologies like Web Services, SaaS
Late 2000s	Heterogeneous Networks (Internet)	Service Oriented Architecture	.NET, WCF	Web Computing, Cloud Computing	WS*, REST, IaaS (EC2), PaaS (Azure)

Credits & References: Juval Lowy, Wikipedia

Storage in the Cloud

-Krishna.Kumar- — Thu, 23 Jul 2009 11:00:44 CDT

Table of Contents:

Overview
Samples and Hands On Labs

OVERVIEW

Applications work with data in many different ways. Accordingly, the Windows Azure Storage service provides several options. To use Windows Azure Storage, a developer must first create a storage account. To control access to the information in this account, Windows Azure gives its creator a secret key. Each request an application makes to information in this storage account—blobs, tables, and queues—carries a signature created with this secret key. In other words, authorization is at the account level. Windows Azure Storage doesn’t provide access control lists or any other more fine-grained way to control who’s allowed to access the data it contains.

Windows Azure storage can be accessed by a Windows Azure application, by an application running on-premises within some organization, or by an application running at a hoster. In all of these cases, all three Windows Azure storage styles use the conventions of REST to identify and expose data. In other words, blobs, tables, and queues are all named using URIs and accessed via standard HTTP operations. A .NET client might use the ADO.NET Data Services libraries to do this, but it’s not required—an application can also make raw HTTP calls.

The following figure shows the various storage options available in Windows Azure today:

BLOBS:
The simplest way to store data in Windows Azure storage is to use Binary large objects - blobs. A blob contains binary data and they can also have associated metadata, such as information about where a JPEG photograph was taken or who the singer is for an MP3 file. To use blobs, a developer first creates one or more containers in some storage account. Each of these containers can then hold one or more blobs. Blobs follow a simple hierarchy: A storage account can have one or more containers, each of which holds one or more blobs.

To identify a particular blob, an application supplies a URI of the form:
http:// <StorageAccount>.blob.core.windows.net/<Container>/<BlobName>

<StorageAccount> is a unique identifier assigned when a new storage account is created, while <Container> and <BlobName> are the names of a specific container and a blob within that container. Containers can’t be nested—they can contain only blobs, not other containers—so it’s not possible to create a hierarchy of blobs.

Blobs can be big—up to 50 gigabytes each—and so to make transferring them more efficient, each blob can be subdivided into blocks. If a failure occurs, retransmission can resume with the most recent block rather than sending the entire blob again. Once all of a blob’s blocks have been uploaded, the entire blob can be committed at once.

Containers can be marked as private or public. For blobs in a private container, both read and write requests must be signed using the key for the blob’s storage account. For blobs in a public container, only write requests must be signed; any application is allowed to read the blob. This can be useful in situations such as making video, photos, or other unstructured data generally available on the Internet.

TABLES:
Blobs are just right for some situations, but they’re too unstructured for others. To let applications work with data in a more fine-grained way, Windows Azure storage provides tables. Unlike the name suggests, these aren’t relational tables. In fact, even though they’re called “tables”, the data they hold is actually stored in a simple hierarchy of entities that contain properties. The following figure illustrates how the parts of a table fit together.

Each table holds some number of entities. An entity contains zero or more properties, each with a name, a type, and a value. A variety of types are supported, including Binary, Bool, DateTime, Double, GUID, Int, Int64, and String and a property can take on different types at different times depending on the value stored in it. Furthermore, there’s no requirement that all properties in an entity have the same type—a developer is free to do what makes the most sense for her application.

Whatever it contains, an entity can be up to one megabyte in size, and it’s always accessed as a unit. Reading an entity returns all of its properties, and writing one atomically replaces all of its properties.

Windows Azure Storage tables are different from relational tables in a number of other ways. Such as, they can’t be accessed using ordinary ADO.NET, nor do they support SQL queries. And tables in Windows Azure Storage enforce no schema—the properties in a single entity can be of different types, and those types can change over time. The reasoning behind this design decision around Azure Tables stems from the primary Windows Azure goal of supporting massively scalable applications. In fact, a single Windows Azure table can contain billions of entities holding terabytes of data. To allow this, the storage mechanism needs to get simpler: What’s needed is the kind of structure provided by Windows Azure tables. It frees developers from worrying about scale—just create new tables, add new entities, and Windows Azure takes care of the rest. It also eliminates much of the work required to maintain a DBMS, since Windows Azure does this for you. The goal is to let developers focus on their application rather than on the mechanics of storing and administering large amounts of data.

And rather than using SQL, tables are accessed RESTfully, like everything else in Windows Azure Storage, again for the purpose of achieving scale-out storage. A .NET application can use ADO.NET Data Services or Language Integrated Query (LINQ) to do this, both of which hide the underlying HTTP requests. Any application, .NET or otherwise, is also free to make these requests directly. For example, a query against a particular table is expressed as an HTTP GET against a URI formatted like this:

http:// <StorageAccount>.table.core.windows.net/<TableName>?$filter=<Query>

Here, <TableName> specifies the table being queried, while <Query> contains the query to be executed against this table. If the query returns a large number of results, a developer can get a continuation token that can be passed in on the next query. Doing this repetitively allows retrieving the complete result set in chunks.

Windows Azure tables aren’t the right choice for every storage scenario, and using them requires developers to learn some new things. Still, for applications that need the scalability they provide, tables can be just right.

QUEUES:
While tables and blobs are primarily intended to store and access data, the main goal of queues is to allow communication between different parts of a Windows Azure application. A primary function of queues is to provide a way for Web role instances to communicate with Worker role instances.

Like everything else in Windows Azure Storage, queues are accessed RESTfully. Both Windows Azure applications and external applications reference a queue by using a URI formatted like this:

http:// <StorageAccount>.queue.core.windows.net/<QueueName>

As described above, a common use of queues is to allow interaction between Web role instances and Worker role instances. The following figure shows how this looks.

In a typical scenario, multiple Web role instances are running, each accepting work from users (step 1). To pass that work on to Worker role instances, a Web instance writes a message into a queue (step 2). This message, which can be up to eight kilobytes, might contain a URI pointing to a blob or entity in a table, or something else—it’s up to the application. Worker instances read messages from this queue (step 3), then do the work the message requests (step 4). It’s important to note, however, that reading a message from a queue doesn’t actually delete the message. Instead, it makes the message invisible to other readers for a set period of time (which by default is 30 seconds). When the Worker instance has completed the work this message requested, it must explicitly delete the message from the queue (step 5).

The reason for making instances explicitly delete messages is that it allows handling failures. If the Worker role instance that retrieves a message handles it successfully, it will delete the message while that message is still invisible, i.e., within its 30 second window. If a Worker role instance dequeues a message, however, then crashes before it completes the work that message specifies, it won’t delete the message from the queue. When its visibility timeout expires, the message will reappear on the queue, then be read by another Worker role instance. The goal is to make sure that each message gets processed at least once.

Regardless of how data is stored—in blobs, tables, or queues—all information held in Windows Azure storage is replicated three times. This replication allows fault tolerance, since losing a copy isn’t fatal. The system provides strong consistency, however, so an application that immediately reads data it has just written is guaranteed to get back what it just wrote.

Here is an overview video that brings all of the above together:

Here are quick start videos for programming Azure Storage blobs, tables and queues:

For additional information, please check out the whitepapers on:

Enabling Technologies of Cloud Computing

-Krishna.Kumar- — Thu, 23 Jul 2009 10:42:25 CDT

Table of Contents:

OVERVIEW:

As we saw in the previous sections, Cloud Computing was not born in vacuum; instead there has been a steady evolution of enabling technologies and paradigms that came together under the right circumstances to give rise to the concept of cloud computing.

The following are some of the precursor technologies that enabled cloud computing as it exists today:

Inexpensive and plentiful storage and CPU bandwidth
Sophisticated communication, security and syndication formats to communicate with applications like HTTP, OpenID, Atom
Established data formats and flexible protocols to enable message passing like XML, JSON and REST
Sophisticated client platforms, such as HTML, CSS, AJAX
The right solution stacks like .NET and LAMP
SOA (service-oriented architectures) and SaaS
Commercial virtualization
Large datacenter infrastructure implementations from Microsoft, Yahoo, Amazon and others that provided real-world, massively scalable, distributed computing

Out of these, we shall explore the main building blocks of Datacenter design, Virtualization and Cloud protocols in the following sections.

Cloud Taxonomy

-Krishna.Kumar- — Thu, 23 Jul 2009 10:40:36 CDT

While cloud computing as a concept, is uniformly characterized by a few common attributes as we saw in the previous section, there are a host of various ways in which it is implemented and offered as a service to the consumer, as detailed in the following discussion.

To examine the taxonomy of cloud computing, let us draw on our understanding of the traditional "on-premises" application platform, which can be thought as typically comprising of these three parts:

A foundation or Infrastructure
Middleware or Runtime Platform
Application Services

If the complex on-premises computing applications of today are to move successfully to "the cloud," all three types of service will need to precede them, and furthermore, even new kinds of services may be required.

Transferring the above pieces of infrastructure to the cloud gives us the following 3 main widely-accepted classifications of cloud computing architectures:

Software-as-a-service offerings [SaaS]. Software as a Service is a model of software deployment whereby a provider licenses an application to customers for use as a service on demand. SaaS services include complete applications that can be customized by clients and delivered over the Internet such as salesforce.com, Gmail, and enterprise applications delivered as a service.

Platform-as-a-service offerings [PaaS]. Platform as a service offerings are externally hosted services providing complete computing platforms and solution stacks as a service to create, run, and operate applications, including development tools, administration and management tools, runtime engine(s), data management engine(s), security facilities, and user-management services. These services include offerings such as the Windows Azure Platform, Google App Engine and Force.com.

Infrastructure-as-a-service [IaaS]. Infrastructure as a Service is the delivery of computer infrastructure (typically a platform virtualization environment) as a service. IaaS includes both full infrastructures on which clients can deploy applications (like Amazon Elastic Compute Cloud) as well as separate infrastructure services (e.g., virtual computing capacity, virtualized hosting, utility storage, etc.). These services are typically billed on a utility computing basis and amount of resources consumed (and therefore the cost) will typically reflect the level of activity. It is an evolution of web hosting and virtual private server offerings.

The following are the key characteristics of each of these variants of cloud computing:

Software-as-a-service offerings [SaaS]

network-based access to, and management of, commercially available software
activities managed from central locations rather than at each customer's site, enabling customers to access applications remotely via the Web
application delivery typically closer to a one-to-many model (single instance, multi-tenant architecture) than to a one-to-one model, including architecture, pricing, partnering, and management characteristics
centralized feature updating, which obviates the need for end-users to download patches and upgrades.
frequent integration into a larger network of communicating software - either as part of a mashup or as a plugin to a platform as a service.

Platform-as-a-service offerings [PaaS]

Services to develop, test, deploy, host and maintain applications in the same integrated development environment
Web based user interface creation tools
Multi-tenant architecture
Integration with web services and databases (via SOAP and REST)
Support for development team collaboration
Utility-grade instrumentation
Financial billing and instrumentation

Infrastructure-as-a-service [IaaS]

Resources delivered as a service including servers, network equipment, memory, CPU, disk space, data center facilities,
Dynamic scaling of infrastructure which scales up and down based on application resource needs
Variable cost service using fixed prices per resource component
Multiple tenants typically coexist on the same infrastructure resources
Enterprise grade infrastructure allows mid-size companies to benefit from the aggregate compute resource pools

Here is a more holistic representation of the above concepts with reference to the cloud computing platform architecture figure from the previous module:

Credit: Joshy Joseph

In this model, each layer abstracts the layer below it, exposing interfaces that layers above build upon. There is no hard dependency between layers and each layer provides composable or Plug and Play architecture with services from other layers. Each layer provides horizontal scalability as needed.

As you can see, a cloud platform is composed of a number of subsystems. Let's look at each one next.

A hosting platform The hosting platform provides the physical, virtual, and software assets. These assets include physical machines, operating systems, network systems, storage systems, power management, and virtualization software. Bare metal and other operational resources are abstracted as virtual resources to the layers above.

Cloud infrastructure services The most important function of this layer is to abstract the hosting platform as a set of virtual resources and manage those resources based on scalability and availability needs. Fundamentally, this layer provides three kinds of abstract resources: compute, storage, and network, and exposes a set of APIs to access and manage these resource abstractions. Thus you gain access to the underlying physical resources without knowing the details of the underlying hardware and software and can control these systems efficiently through configuration. Services offered by this subsystem are classified as Infrastructure as a Service (IaaS).

Cloud platform services Developing and managing software for cloud computing is complex. It becomes really complex when you integrate on-premise software with hosted services. Platform services provide a set of capabilities exposed as services to help with such integration. Availability of platform services may differentiate one cloud provider from another. Services provided by this layer can be classified as Platform as a Service (PaaS).

Cloud applications This layer houses applications that are built for cloud computing. These applications expose Web interfaces and Web Services for end users, enabling multitenant hosting models. Some functions include connecting disparate systems and leveraging cloud storage infrastructure to store documents. These services fall under the umbrella of Software as a Service (SaaS).

Security services Security services ensure token provisioning, identity federation, and claims transformation. These services are built on the open standards, WS-Security, WS-Trust, WS-Federation, SAML protocols, and OpenID, for greater interoperability.

Management services Management interfaces cut across all the layers described above. The hosting platform leverages management interfaces and agents for automated scalability and availability administration. Even though the cloud is hosted and managed in a datacenter, customers may need functions that allow them to easily control their application and post deployment configurations, get analytics about service usage, and connect their enterprise management systems.

Tools Tools help you build, test, and deploy applications into the cloud. These tools may be extensions to existing tools (Visual Studio Tools for Windows Azure, for instance) or hosted tools from a specific cloud provider.

The following is yet another representation of the above concepts:

Credit: Tommy McCall, Erica Naone

Outlined as under is a mapping of the various technologies and their classification per the taxonomy outlined above:

Authors: Peter Laird, Kent Dickson of BEA Systems

Computing in the Cloud

-Krishna.Kumar- — Thu, 23 Jul 2009 02:08:03 CDT

Table of Contents:

OVERVIEW

As we saw in the previous section, the Windows Azure Compute service can run many different kinds of applications. The building blocks of these applications are the two instance types provided by Windows Azure - Web role instances and Worker role instances.

The following figure illustrates this idea:

As its name suggests, a Web role instance can accept incoming HTTP or HTTPS requests. To allow this, it runs in a VM that includes Internet Information Services (IIS) 7. Developers can create Web role instances using ASP.NET, WCF, or another .NET technology that works with IIS. Developers can also create applications in native code—using the .NET Framework isn’t required. And as the above figure shows, Windows Azure provides built-in hardware load balancing to spread requests across Web role instances that are part of the same application. By running multiple instances of an application, Windows Azure helps that application scale.

Worker role instances aren’t quite the same as their Web role cousins. For example, they can’t accept requests from the outside world. Their VMs don’t run IIS, and a Worker application can’t accept any incoming network connections. Instead, a Worker role instance initiates its own requests for input. It can read messages from a queue, for instance, as described later, and it can open connections with the outside world. Given this more self-directed nature, Worker role instances can be viewed as akin to a batch job or a Windows service.

A developer can use only Web role instances, only Worker role instances, or a combination of the two to create a Windows Azure application. If the application’s load increases, he can use the Windows Azure portal to request more Web role instances, more Worker role instances, or more of both for his application. If the load decreases, he can reduce the number of running instances. To shut down the application completely, the developer can shut down all of the application’s Web role and Worker role instances.

For developers, building a Windows Azure application looks much like building a traditional Windows application. Windows Azure also provides Visual Studio 2008 project templates for creating Web roles, Worker roles, and applications that combine the two. One obvious difference, however, is that Windows Azure applications don’t run locally. This difference has the potential to make development more challenging. To mitigate this, Microsoft provides the development fabric, a version of the Windows Azure environment that runs on a developer’s machine.

The development fabric runs on a single machine running either Windows Server 2008 or Windows Vista. It emulates the functionality of Windows Azure in the cloud, complete with Web roles, Worker roles, and all three Windows Azure storage options. A developer can build a Windows Azure application, deploy it to the development fabric, and run it in much the same way as with the real thing. (In fact, it’s entirely possible to create a Windows Azure application without ever using Windows Azure in the cloud.) Once the application has been developed and tested locally, the developer can upload the code and its configuration file (described below) via the Windows Azure portal, then run it.

Service Definition

The shape of a service is determined by the service definition file. The service definition file indicates whether a service implements a web role, a worker role, or both types. For a web role, the service definition file specifies whether it exposes an HTTP endpoint or an HTTPS endpoint, or both. It also may establish configuration parameters for the service and define a local storage resource for caching purposes.

The service definition can't be changed while the service is running in Windows Azure. To change the shape of a service, you must republish the service. The service definition file has a default extension of .csdef. It is packaged with the role binaries when the service is packaged for publication. The following diagram shows the possible shapes for a service, as determined by the service definition file.

The possible shapes for a service, as determined by the service definition file

Service Configuration

The service configuration file sets values for the service that can be configured while the service is running in the fabric. The values you can specify in the service configuration file include the number of instances that you wish to deploy for each role, and the values for the configuration parameters that you established in the service definition file.

The service configuration file has a default extension of .cscfg. The service configuration file is not packaged with the service, but is uploaded to the Windows Azure fabric as a separate file. You can upload a new service configuration file without republishing your service. The following diagram shows a published service running multiple instances of a web role and a worker role.

The published service running multiple instances of a web role and a worker role

Quick Start:

More Detailed Overview:

Direct Download Link to the Video