Cloud Front Office

Cloud Computing Is Moving So Fast, Traditional Management Process and Tools Will be Under Pressure.

This morning I got my Amazon EC2 newsletter outlining all the changes they are introducing this month. 

Three things struck me about the changes. 

1) The speed at which improvement and new offerings are coming to market. It's really going to put pressure on all MSP's but also in-house cloud users. It's not just a matter of matching the offers, but having to match and beat them every month.  This requires a deep bench of product marketing expertise and process.

2) Continued downward pricing of instances.Which means more pressure on margins for MSP's. This is going to require more financial savvy  than we usually see in IT departments.

3) The IT storefront will need to be dynamic and fast moving.  The changes Amazon introduces require a continuous evolution of the service catalog and console interfaces.  I think this is going to strain TCO for companies trying to build their own portal with traditional tools ("What? I can't order 16G of Ram because it takes 3 months to change the form?" or "I can't do reserved instances until the end of the year?") At some point the rate of change is such that

This is something we have worked on a lot with our cloud service catalog--our traditional customers make lots of changes to the catalog, continually updating and improving it.  This is possible by the non-programmatic nature of our ServiceDesigner tools  ("New offer, change this here, voila! ready to go.") 

For example, we introduced dynamic pricing in our last release. This allows a cloud product manager to set different prices depending on the options chosen. For example, "Select cloudwatch" as an option, varies your hourly pricing. Now mind you, a web page designer could do that. The trick is to maintain that day in day out against a dynamic market environment.

Drinking My Own Champagne and Reading "Compute Cloud Abstraction APIs ... who needs 'em?"

Adrian Cole, who is Mr. Jclouds to me, has an interesting blog post on cloud API standards. Specifically, do we need them?

This is relevant to newScale as we already connect to a variety of provisioning systems and Run Book Automation tools. And clouds. Specifically, we use our RequestCenter product internally to provision and manage our growing Amazon Cloud offering.

We call this, drinking our own champagne! Other vendors call it "eating your own dog food;" I guess they know more about what they are selling.

All joking aside, we also have products for VmWware. This don't work on Amazon. I've been looking for a cloud provider to host them and so far have not had much luck. But once I do, I need to provide the same level of automation so we are planning on vCloud.

So it would make my life easier if we had a standard API to all these clouds.  Alas, we are not quite there, though for some limited functionality, there is some very encouraging progress. 

This is for a couple of very good and valid reasons.

First, the offerings are evolving at a very fast pace. There's a lot of great innovation. So any API that tries to abstract offerings is always going to be a behind going native. When Amazon added spot pricing, that is not a trivial API to implement. And as API developer, Amazon doesn't give you any notice. So Sunday your API is good, Monday is far behind. 

On the other hand,  from what I've seen, there's pretty much agreement on calls like "start instance," "stop instance" But not on "snapshot instance" because not all cloud providers offer snapshots (yet). I like Adrian's Turing Test:

1. Instance lifecycle. The lifecycle defines the basic commands to provision, start, stop, and terminate instances. The bare-bones of a compute service.
2. Shared images. While it is possible to bootstrap from plain OS images created by the cloud service provider, the ability to build your own customized image, and crucially to share it with others provides a social element that helps drive adoption of a cloud platform.
3. Instance metadata. This feature allows you to inject small amounts of user-specific metadata to each instance at boot time - e.g. secret keys, or custom boot parameters - which allows another level of customization. This feature works well in conjunction with shared images: the common non-user specific code is baked into the shared image, and the user-specific code is supplied at launch time as metadata.
4. Network controls. Cloud providers need to think about the network environment that the user's instances run in. Offering such services as DNS, firewalling, VPNs (and exposing it via an API) makes it easy for developers to get started quickly without having to build this infrastructure themselves.

Second, compute is but one element of cloud automation, as someone (Rightscale CTO?) writes on Jclouds.

The real benefit of standards would be around the semantics of resources like vlans, security groups, reassignable IP addresses, network attached block storage devices, etc. But vendors today barely support these and if they do they have quite different semantics from one another that reflect different underlying architectures and also use-cases.

Which is right. There's a lot more to providing cloud computing than just providing compute power.

Start instance is but one element of the API call we use at newScale. We also need to decide which offering (we use our service catalog), for which customer and where (we use our LifeCycle Center product for that and integrate that data from SF.com) and various other configurations questions like domain name, IP, security group, back up schedule, monitoring level, and of the most important one for us -- Length of subscription! 

As cloud matures and a set of services becomes baseline, I believe we'll see cloud API's increasingly able to describe enough major elements to be useful. But it can never be 100% -- vendors will differentiate (which is good for them) and innovate (which is good for us).

Now this API discussion does presume there's one way to do this. At newScale, we've worked out a different approach.

This approach, nicely benefits from standardization too, but does not require API standardization to support multiple clouds. I'll write about in my next blog.

Meanwhile, I've just received notice from Amazon EC2 that I've been upgraded to 100 instances in EU!  So more champagne drinking will happen today.

Using a Service Catalog and Lifecycle Management to Increase Capacity or How Many VM Fit Into a Server?

A colleague sent me this CIO article this morning, How Many Virtual Machines Fit on Your Server?

And it's a very interesting read, because the answer seems to be depends on your workload and how well you manage your lifecycle.  Here's the first quote on best practices.

But making sure the hardware can support that additional load is a real trick because of the almost infinite variety of the software that runs within the virtual environment — each application making a slightly different set of demands on the host OS and the hardware, says Chris Wolf, analyst at The Burton Group.

In my experience, the data center folk rarely know what's running on the boxes they manage. They also have little visibility into the workloads (with a few exceptions like SAP) or even owners. 

So one of the key things we are recommending to newScale service catalog customers is to make part of the request process, the capture of workload sizing and business information.

This can be very difficult and involved.  Or it can be as simple as "small, medium, large, xl" for everyday requests. 

Second, we add to the form, a wizard which helps the request translate what they need into actionable Quality of Service instructions to the data center.  The other good practice is to detail the workload in the portfolio so it can be mapped to risk, criticality, hours of operation, supported business process, etc.

This information provides the data center the right information about the workload to know when it's ok to over commit capacity, what's the criticality of the application if performance is constrained. Or as one admin said to me, "I wish I'd known you are only support to work on that app on Saturday nights." 

Third, because we've enabled self-service (we provide the best IT Storefront), even if the user makes a mistake in estimating workload size, they can quickly fix it.  Here's another quote from the article that makes this point:

"If you put five VMs on a server, you're running six operating systems and all the applications, so you have to ramp up to be able to handle that and keep the service levels, the performance, high for the applications," Scanlon said. "We ended up having to put on a lot more memory than we figured during the capacity planning."

This is not a failure of capacity planning. I was at a Morgan Stanley CTO event a few months back, and they were enabling self-service for their users to request servers.  One of the attendees said, "But won't they make mistakes?  The answer was "The users never get it right. With all the planning, architecture review, it's always wrong." But, he continued, they can self-correct immediately.

And the users don't get to blame the data center, which is a nice bonus.

The other interesting aspect we've brought into play is lifecycle management. As we move from hard assets to virtual assets, we've lost the ability to look at what's running and ask who owns it, and for how long do they need it. 

Here's the recommendation from the article.

Second: build a fence. VMs are easy to launch and hard to see, so server sprawl - having too many VMs running without actually being used - is appallingly common. Killing off all the unused servers and the disk space they reserved saved gave Computacenter back so many resources it was able to put off a major upgrade until the following budget cycle, Scanlon says

That's a pretty impressive statistic. And the key to killing unused servers is to have a good life-cycle manager tied to the self-service storefront. 

We've introduced these new capabilities in newScale 9. The service catalog details the offerings, and helps drive standard configurations. This removes a large part of the "idiosyncrasies" of workloads. 

The Wizard and configurator guides the user to make a request that actually works. This includes detailing the quality of service, SLA's, and portfolio elements (like criticality, disaster recovery, storage tier, etc, etc).  And it includes a subscription expiration.

Here's an example of the questions (of course, it's all customizable)

All this information drives the workflow for provisioning, but also is captured in the lifecycle database which syncs with the VM infrastructure.   This is what the lifecycle looks like:

The result is unprecedented technical and business visibility about the workloads running in the data center.

The real hard dollar results come from reducing costs due to hoarded capacity, rogue servers, and zombie VM's. When you think about increasing virtualization density 20-30% by JUST KNOWING what's running and what can be done to the workload... That's a mind-boggling return on investment.

SiliconRepublic.com: Consumerisation and self service IT will explode #Cloud

More people getting wise to the gig. This trend is central to cloud computing success, after all, if you can't order it, you can't sell it.

Consumerisation and self service IT will explode

The smart phone, particularly the phenomenal growth of the iPhone, is now increasingly joining the laptop and the netbook as a business tool, Gilmore continued.

“The on-demand service era will really take root in business as it becomes clear that major cost reductions are possible through the introduction of ‘anytime, anywhere,’ personalised services that reflect the hugely successful Apple iTunes mode of application access and availability."

via www.siliconrepublic.com

Great coverage on newScale 9 #cloudcomputing #itil

We launched newScale 9 last week.  Here's some coverage from last week:

 Can newScale Make IT Service Catalogs Sexy? – Data Center Knowledge

newScale Revs Its Service Catalog Software – Cloud Computing Journal (this same article was also covered in Virtualization Journal, Client/Server News, and other Sys-Con online publications)

Changes in the Datacenter and Cloud – newScale Allows Pioneers in Cloud to Work with IT  – Silicon Angle

Companies Take to Service Catalogs – IT Business Edge

newScale 9 Takes Self-Service IT and Lifecycle Management to a New Level – Press Release (also covered by VMblog.com and several other online publications)

 
And my own blog post here

SiliconANGLE — Blog — Changes in the Datacenter and Cloud - NewScale Allows Pioneers in Cloud to Work with IT

Great coverage on newScale 9 release.

The vision of NewScale is all about the "consumerization" and "commoditization" of IT.  It's a relevant model today.  For example, IT is dealing with massive commoditization of components that are being "mashed up" into a set of services more often then not.

Simply put the enterprise is seeing "mashups" as the norm not the exception.  Ultimately the user within an enterprise demands consumer like services and experiences.  This is the new IT model.

We are moving from users to consumers within the enterprise.  What NewScale is doing is providing a platform to enable this.

via siliconangle.com

Reading: "How cloud computing changed internal IT in 2009" IT Shops Planning "Run IT as a Cloud"

David at Infoworld www.infoworld.com

Strangely enough, cloud computing has turned out to be more about reviewing and modernizing internal IT than about how internal IT systems can be extended to the clouds. More simply put, we now try to replicate the value of cloud computing by modernizing and reimplementing our existing IT architectures using cloud computing concepts, such as self-provisioning, virtualization, elasticity, multitenancy, and yes, SOA.

Maps to what I'm seeing in our customers. The cloud is moving them to define a cloud operating model to increase business agility and reduce costs.

As they mature beyond virtualization, they run into the need to:

• standardize services
• deploy an actionable service catalog
• manage, report, track consumption (and maybe costs)
• deploy an IT storefront for self-service.

I'm Trying to Save Some Jobs. Cloud Computing Impact on IT Folk. What Should I Emphasize

I'm preparing a talk on cloud computing and the impact on IT service management folk and careers for the Pink Elephant conference in Vegas.  I've noticed that many data center folk don't know about ITIL, and the many ITIL practitioners don't know about cloud computing. But I'm pretty sure they need each other.

So I'm trying to unite these two streams and maybe save some jobs.  I'd love to have your input for this talk.  

The thesis is this:  The growing interest in Cloud Computing and Software As A Service (SaaS) models is providing yet another push towards defining the value of IT in the context of Services versus Technology. Cloud Computing focuses on the Business Outcomes and Value elements rather than the technology components they are comprised of.

So I'm doing  session to explain to IT practitioners the important role they will play in the adoption of cloud computing strategies, the changing emphasis from inward looking IT processes to Business customer process automation.

My hope is help IT service management folk understand the types of roles, knowledge and costing/pricing approaches that will be valuable.

Reading: "Can I get a price check on this AMI?" Catalog, Order Management, Subscription and Metering

William Vambenepe has a nice post that illustrates some of the challenges in the catalog side of cloud computing. First a quote, then my comments.

Of course, by the time my account usage page was updated (it took a few hours) I had found the price list which in retrospect wasn’t that hard to find (from Amazon, not IBM).

So maybe I am not the brightest droplet in the cloud, but for 20 bucks I consider that at least I bought the right to make a point: these prices should not be just on some web page. They should be accessible at the time of launch, in the console. And also in the EC2 API, so that the various EC2 tools can retrieve them. Whether it’s just for human display or to use as part of some automation logic, this should be available in an authoritative manner, without the need to scrape a page.

The other thing that bothers me is the need to decide upfront whether I want to launch a Tivoli instance to manage 50 virtual cores, 200 virtual cores or 600 virtual cores. That feels very inelastic for an EC2 deployment. I want to be charged for the actual number of virtual cores I am managing at any point in time. I realize the difficulty in metering this way (the need for Tivoli to report this to AWS, the issue of trust…) but hopefully it will eventually get there.

This anecdote shows the disconnect between the flat-static catalog, the order system, and the subscription / metering system that will need to be addressed.

The other issue raised, is that of units of measure. Units of measure vary today depending on where the customer sees value. This needs to be also managed in the cloud catalog/order/subscription manager trinity.

Now can you imagine trying to call your cloud provider to explain that you didn't mean to launch 1000 servers for a week? 

Reading: Products for Cloud Ops vs. Traditional Ops

Don't know how I missed this one. Highly recommended. I completely agree that there are new winners and losers because of cloud. For example, self-service and the catalog come to the forefront. Lifecycle and subscription management also become important.

On the other hand the traditional CMDB? Not so relevant since so many services become abstract; what elements do you need to track? 

As for assets? They become subscriptions, which are financial liabilities. 

Anyway here's the rest... I recommend it.

I see roughly five dimensions for how ITSM product use might shift within a cloud model:

• Overall increase in use, due to IT Operational needs created by the automation and dynamics within a cloud infrastructure. With more dynamic/unpredictable resource requirements, some ITSM tools may become more valuable than ever. For example, take Billing/Chargeback. Clearly any provider of a public (or internal) cloud will need this to provide the pay-as-you-go economic model, particularly as individual resource needs shift over time. Same clearly goes for tools such as Dynamic Workload Brokering, etc.
  
• Overall decrease/obviation of need, due to the the automation/virtualization within a cloud infrastructure. As automation begins to manage resources within the cloud, certain closely-monitored and managed services may simply be obviated. Take for example application-specific Capacity Planning; no longer will this matter to the degree it used to - now that we have "elastic" cloud capacity. Similarly, things like event correlation _might_ no longer be needed -- at least by the end-user -- because automation shields them from need to know about infrastructure-related issues.

via fountnhead.blogspot.com

Governance and Service Catalogs in the Cloud

Interesting and challenging article. One thing I'm noticing is that there's a lot of re-invention of concepts going in the cloud space. Work that's been done in the ITSM community is very applicable.

These two quote allude to the need of service catalog manager.

The concept of governance and its relationship to cloud computing has been discussed but those conversations tend to revolve around the need for more generalized governance. That is, a service catalog of all cloud provider offered services as well as a cloud-application service catalog for consumers.

Maybe this is a job for “Cloud Coordinator”, or whatever you might want to call such a group or individual. SOA eventually came up with a similar notion, often called the “Registrar” or the “Librarian”, depending on what governance solution was being implemented. Someone or some team that is responsible for (a) understanding what services are available in the apposite cloud computing environments and (b) managing those services with an eye toward sharing or at least ensuring consistency of solutions across applications deployed in a public cloud computing environment.

via devcentral.f5.com

CloudComputing Self-Service Will Be MUCH, MUCH, Harder Than People Expect

At this point we have about 81 server instances up and running in the cloud. A few over at dedicated hosting company and of course our internal servers. 

Because newScale supports a variety of platforms, I've spent the last few weeks evaluating and trying different cloud providers and it's been super useful to truly assess the state of cloud offerings today.

And one thing is struck me: The "self-service" consoles are just not serviceable.  So my insight is

Self-Service in the cloud will be MUCH, MUCH, MUCH harder than people expect. 

This is due to two factors.

One, the need for configuration self-service means that the front end will have to be more intelligent, powerful and configurable to truly drive the assembly of useful workloads.  Appliances / Images are too gross a deployment unit for real applications, so these assemblies will require a lot of "over-writing" configuration scripts. We will need to have constraint rules (this option clashes with that option).

For example, in our cloud deployment, the image is not enough. We need to set up DNS directions, decide if there's a volume for back up or not, which data center and region to use. We also need to specify end-dates for subscriptions, and service level option; this goes to the customer account record -- part of which is in SalesForce.  And this is a trial architecture, never mind trying to set up n-tier stuff.

Second, because te cloud is so new, there will be a lot of volatility as to exactly what is the right offering package, service tier, configuration, admin actions, etc. This means the self-service front end better be very flexible to account for new actions. For example, if the user need to have limited Guest Admin VM privileges, that self-service better have good Role Based Access Control and a flexible set of configurations functions to allow some fine gradations of privilege assignment.

Why is this important?  Because in every cloud architecture I've seen, there's a box that says self-service catalog on top of 50 other boxes.  Then they speaker goes to talk about all the burly, manly infrastructure underneath.  

Funny story: . One of the cloud providers I tested did try to offer an application request in addition to infrastructure. The ugliest, loneliest little form with what seemed 58 unintelligible check boxes. Selecting anything in it required a 24 hour turn around and probably a call. That's so NOTcloud.  I moved on.

If you can't see the offer, configure the request and order it, your cloud will be empty of users.  

The Mysterious Case of the $0.72 Or Why #Cloudcomputing Should Not Be Like Telco

I've been trying different cloud providers in the last few weeks. This required me to give my credit card out to 8 different cloud providers (CTO at newScale is a dangerous job). This is a story of one of the

To protect the innocent, I've given them code name based on fruit.  I was working with cloud provider GRAPE, learning their console and trying to figure out how to upload my machine image.

Suddenly I noticed that I had been charged $0.72 for bandwidth charges. I had not uploaded or downloaded anything yet, so I was curious as to why I had been charged for consuming 4 megabytes. And the charge was per Gig transferred, so like the famous minutes in my phone bill, 1 byte over the limit and it seem to trigger a 1 Gigabyte charge.

So, I send an e-mail to GRAPE's help desk and fun and games begin.  It goes like this:

Day 1 - Send query to help desk

Day 2 - Receive email that says I need to go to the billing department and provides e-mail. (Why can't they send e-mail to billing department, and cc me?).  So I send the email.

Day 5 - Follow up from Helpdesk: "Did the billing dept" return my inquiry. My answer: no

Day 6 - Case resolved.Maybe. "You will not be charged for the bandwidth charges as you will be given the 1st 1GB of data transfer for free."  My response was but what are you charging me for? And what gigabyte is free?

This event raises all sorts of uncomfortable questions for me.

First, it's $0.72 so in a larger bill, I'm very likely to miss it. How am I going to track my consumption and then check it against their records? 

Second, it's not clear what behavior I engaged in that results in charges, so I'm unlikely to be able to manage it.

Third, I'm all of a sudden in the same damn predicament that I have with my cell phone bill, which is I don't understand the charges at all so I'm paying for all sort of fees. 

I want another telco relationship like I want to be mauled by bears while listening to a dentist drill symphony

Without standardization, packaging and product management, you won't succeed with a private or public #cloud

The last few days, I've been trying to get a VMWare appliance loaded to a cloud service to no success.

As I wrote in "This is so NOT cloud.." and "I'm Still Waiting" it's been a bumpy ride and I still haven't achieved what I wanted to achieve.

Which has me reflecting on why is this happening? And my thinking so far is this:

They truly do not understand what customer self-service is; the result is endless rounds with support that create failure points. For example, wrong cost quotes. For example, login info that is lost in the mail and can't be recovered.

Oh, they all have this shiny consoles to turn an instance on and off. It's not sufficient, they need to figure out the whole product experience!  For example, one has a nice console, but you need a command line to upload. Which requires JDK 1.6. Which cannot be added to a locked machine. Said command line has little to no documentation. After 5 hour upload process, it fails with cryptic error.  As I said, truly don't understand self-service.

What I'm seeing is the need for whole product thinking and much better packaging of services. This is the job of the product manager.  These hosting providers theoretically have them; but most IT organizations don't.

And that's why I think, without standardization, packaging and product management, you won't succeed with a private or public #cloud 

Further (Mis) Adventures in Cloud Computing. Like Bob Marley, I'm Still Waiting. #cloud

Yesterday I wrote about my effort to find a cloud provider that I can use to upload an image.

I tried three.  Each one is in some state of beta-ninnes, with complex sign up procedures, no clear self-service catalog.  You see, I'm pretty sure they've done a mahvelous job putting their infrastructure in place, using the best tools, hardware out there. 

But no thought to the user experience.  So here's the countdown.

Cloud One, the upload takes 5-6 hours, fails, then gives me a cryptic error message. So every cycle of trial and error is a day. Status: Not working

Cloud Two, after scary, long sign up process, chat sessions, disconnected chat sessions ("I was talking to Randy about.... sigh...."), appeals to product manager ("let me get back to you"). I'm finally provisioned!  But it doesn't work with VMDK. Status: Not working

Cloud Three, Same as two, but I can't log in... so I call support and they ask me two questions: account # and domain name.  Me: "I don't have them, this is the only number I got from sign up" Support: "I can't find you." Me: "Here's my name, address,credit card, blah" Support: "I can't find you." 

Much later... We'll get back to you soon.  That was yesterday. Status: Not working. 

So I'm still waiting... Lest you think I wasted the whole day in this foolish quest without gain, it did remind me of how much I love Bob Marley's "I'm Still Waiting"  So enjoy!

This is so NOT #Cloud. Why Amazon Still Wins.

I have a VMware appliance I want to test in the cloud. So I've looked at several providers. Turns out so far I'm having no luck. 

I want to share one experience that I had today, but it's representative of the other experiences today.  These all companies that claim to have a cloud offer, so off I go to their websites to sign up and get going. 

No one has self-service ready to go.  No one has a catalog that I can just see service descriptions, SLA's and pricing.  NO ONE!!! 

I find nothing.  So I click on the chat.. This is a transcript of a very nice conversation.

Hello. Your Sales Professional is Salesperson
Welcome to [REDACTED] Salesperson will be right with you.
Salesperson
Welcome to [REDACTED}, how can I help you and what would you like to accomplish today?

Me
Use cloud hosting. I have a VM appliance I'd like to use

Salesperson
Are you a current customer?

Me   
no

Salesperson
Alright, you are looking to run a VM app in our Cloud environment?

Me
yes. I have VMDK / OVF file i want to upload and run

Salesperson
Great! Could i have a cloud specialist contact you to go over our offerings?

Me
yes.

Me
now?

Salesperson
It will be soon, within the hour if that is OK?

Me
ok

Salesperson
May I receive your contact info, i.e., full name, company name, phone, email?

Me
Rodrigo Flores, newscale, etc, etc

Salesperson
Thanks Rodrigo, do you know how many VM's this is going to be running across or is it just one VM?

Me
1 to test. more later if it works

Salesperson
Sure, do you know RAM and Storage you are looking for?
Me
2 G RAM, 5 G storage

Salesperson
Great, I will have a specialist contact you ASAP Rodrigo.

Salesperson
Is there anything else i should know?

Me
no. thanks

Salesperson

Great, we will be in contact shortly and thanks for chatting in. Have a nice day!!!

A while later I do get a call. We have a conversation about memory, disk, what I'm trying to do, etc, etc.  I want to upload a VMDK.  So that requires another internal discussion and I'll get a call later.

I use EC2 for a whole bunch of servers; unfortunately, I need to test this very specific configuration so it has to be VMware hosting. 

I'll report on what I find. 

Update: One called me back and told me their cloud package starts at $2000+ a month. Still don't know what I get for that or if I can use my appliance. FAIL!

There Goes the Job of the DB Admin! Or Does it? Amazon Launches Database Cloud Service #cloudcomputing

I'm a big fan (and user) of cloud computing because of the potential efficiencies it brings to the enterprise IT operating model. So this announcement, via www.techcrunch.com, from Amazon is interesting.

Amazon has launched a hosted relational database service, Amazon RDS, as part of the suite of services available at AWS. The new service is a hosted MySQL database instance with the full capabilities and access rights as a normal self-hosted DB.

As a hosted solution, instances are easily created and available almost immediately. Pricing stars at $0.11c per hour for the smallest scale specification, and is available now on the AWS site.

So my question is: does the adoption of this cloud service remove the need for a database administrator? Because if it does,

it's a radical change to the IT operating model. If it doesn't, then it's more complicated to find value because I still need to have a DB admin.

What do you think?

Cloudy inquiry trends from CloudPundit.

When Gartner's clients start asking about cloud computing, a real market is about to appear.

Organizations have really absorbed the hype — they genuinely believe that shortly, the cloud will solve all of their infrastructure issues. Sometimes, they’ve even made promises to executive management that this will be the case. Unfortunately, in the short term (i.e., for 2010 and 2011 planning), this isn’t going to be the case for your typical mid-size and enterprise business. There’s just too much legacy burden. Also, traditional software licensing schemes simply don’t work in this brave new world of elastic capacity.

The enthusiasm, though, is vast, which means that there are tremendous opportunities out there, and I think it’s both entirely safe and mainstream to run cloud infrastructure pilot projects right now, including large-scale, mission-critical, production infrastructure pilots for a particular business need (as opposed to deciding to move your whole data center into the cloud, which is still bleeding-edge adopter stuff). Indeed, I think there’s a significant untapped potential for tools that ease this transition. (Certainly there are any number of outsourcers and consultants who would love to charge you vast amounts of money to help you migrate.)

via cloudpundit.com

Meet Your New Cloud, Same as the Old Cloud

This is ugly. Saas really requires a new discipline if it's going change internal apps. This story from www.networkworld.com makes it really clear why

Even senior developers can get lost in the weeds trying to figure out how a feature can be, or might have been implemented, in Force.com. Typical "gotchas" include:

* With numerous configuration options and powerful programming tools there are many ways to implement the same features. Configuration experts and developers run the risk of reinventing the wheel if they do not collaborate closely on solution design.

* Force.com deployment tools do not currently support critical items such as data sharing rules, picklist fields on standard data objects, lead and sales processes, and the management role hierarchy.

* Apex unit tests are impacted by actual org data, so code that passes unit test requirements in the sandbox may not deploy. For example, data queries on objects with more than 100K of data require querying an external ID field. Unit tests that pass in a sandbox can fail in production, killing your deployment.

Survey says! Lack of management solutions creates hurdle to Cloud Computing #cloudcomputing

Interesting survey

The survey found that, while adoption is increasing, CIOs are concerned about how to manage and control applications and resources in Cloud environments. More than half worry about a lack of cost control (57 per cent) and more than a third (35 per cent) fret over scalability. But the biggest concern is being locked-in to one vendor for Cloud services, according to 63 per cent of organizations. The results show a need for more sophisticated tools to monitor and control Cloud usage and the movement of applications from Cloud to Cloud. Over half of enterprises surveyed (53 per cent) also say introducing government standards and legislation to control Clouds would boost adoption rates.

via www.zeus.com

Do Clouds Need Admins? Yes.

Recommended reading.

The programmer-managed infrastructure suffers from a death by a thousand cuts. The programmer is competent with technology and fully capable of setting up a system that can support the application being built. The programmer, however, lacks a detailed understanding of ongoing infrastructure management. Consequently, the programmer-managed infrastructure ultimately leads to an environment incapable of adjusting to changing demands and potentially opens vulnerabilities to hackers through discreet channels.

via broadcast.oreilly.com

Reading: Is Virtualization Enabling Legacy Sprawl – Another Y2K in the Making? #Cloudcomputing

I read this with interest because I'm seeing the danger in my own use of Amazon EC2 services:  Instances go up, but unless you are very determined, they don't come down on their own and every month they cost money.

Just a Thought – Is VMware Enabling Legacy Sprawl – Another Y2K in the Making?

One of the hidden beauties of any virtualized state is the clear disaggregation of hardware and software, the logical separation of traditionally tightly coupled environments, allowing us much greater flexibility in deciding what applications to run where, and for what reasons.  This flexibility in and of itself is a good thing, and we all benefit from it, but I suspect that if we’re not careful in how we use virtualization some apparently intelligent decisions today may in fact turn out to be significant problems in the future.

via blogs.gartner.com

Data Center Dialog: Multiple virtualization vendors in one IT shop? If so, the management challenge changes

Recommended.

Why would organizations want so many types of server virtualization in house? ESG's Stephen O'Donnell blogged that he didn't think this was a result of maturing products from Citrix and Microsoft, but rather because of licensing moves by vendors and license management practices by customers. ESG was also running a second poll while I was viewing their survey results, asking "Why is your organization running more than one hypervisor (e.g., Citrix/Xen, KVM, Microsoft Hyper-V, VMware ESX)?" The most common answer when I looked (24%) said that pressure from people like Oracle and Microsoft is pushing them to support multiple hypervisors. 13% say it's just departments doing their own thing. Both contribute, I'm betting.

Another reason is likely to be acquisitions, something that's becoming more common with the downturn. Sure, you didn’t set out from the start with more than one hypervisor vendor, but mergers tend to mean you get plenty more than you bargained for. Why should server virtualization be any different?

via datacenterdialog.blogspot.com

And beyond that, people will still need physical hardware. And they'll add cloud services as well. This is what newScale tools are all about.

Bazillion Cloud Servers up and Running. Maybe Not. But We're "Wasting" Servers with Singular Delight

So how real is cloud computing? Pretty real by the number of servers being deployed by Amazon, reports Bernard Golden in an excellent CIO article. 

I don' know what the number is and it's probably a lot less than what the article implies, yet it is a staggering number. But there are some differences worth discussing. More after the quote.

Tue, September 29, 2009 — CIO — There's been a lot of discussion the past couple of days about an analysis by Guy Rosen, in which he estimates that Amazon Web Services (AWS) is provisioning 50K EC2 server instances per day. He created this estimate by examining EC2 resource IDs (if you read his post, you'll see how he broke down resource IDs to understand their meaning) and doing a time-series analysis on how much the IDs are incremented per hour. From this analysis, Rosen concluded that AWS is provisioning around 50,000 EC2 instances per day.

At newScale we have been using Amazon EC2 for about year. During that time, we have spun up lots of instances. We also shut them down when no longer needed. So that leads to a large number of instances I suppose, as the article implies, but a lower total number of actual servers running all the time.  Since our software is entirely web-native, it's easy to give one of our customers a new version, trial, proof-of-concept, dev, qa, and production servers in the cloud. This bypasses the months of waiting for hardware.

The other thing is that we tend to "waste" servers on purpose. For example, we have a bunch of images, each specialized for certain workloads. It's easier, and more convenient to spin up two or three images that are purpose built rather than spending a day or two installing SQL, Oracle, Java, and our software on a server. In other words, I'm wasting servers, because labor hours are precious and scarce. That's a theme that I've observed with other companies using cloud computing. 

It's a bit like what happened with CPU power: programmer resources and time are expensive, so we use higher level languages like Java rather than Assembly and "waste" processor. I remember when Windows took off -- people were concerned that all those "graphics" were wasting transistors. Turns out we didn't have to worry.

In the cloud, if I spin up 20 servers this week, and shut them down in a couple of days, and then spin up them later, I only pay for what I consume.  I couldn't do that with physical servers, even virtualized. 

Working with some amazing sys admins that come from traditional hosting, it's been a hard change at first. Traditionally, server uptime and server scarcity is what they care about. In my project, I care about their time.

So what's the key to this magic?  We've had to do three things:

1) Establish clear standards, policies and images.  For example, only use Small Windows with SQL Express for these type of workloads. Here are the two images that fit that kind of scenario.

2) Tight control over the lifecycle to prevent sprawl.  There are lot of parts to it, but all servers have an expiration date as well as other attributes that allows us to map it to customers, groups and usage. And a draconian termination policy: servers that are orphaned, get terminated.

3) Put some controls on entitlement: who can order, what level, how do we account for expenses.

At first, the Amazon console was good enough, but around 5 servers and 3 internal requesters, the ease of use became a problem leading to cloud sprawl.

We are about to probably quintuple our usage of cloud. So we are setting up all of this in our service catalog and our service lifecycle manager. I'll report more on our journey, if there's interest.

It's service world: More thing companies buying service companies

So Xerox is getting into the business process outsourcing world. Essentially they want to cloudify their existing products. Same thing Dell needs to do.

The trend is clear as hardware vendors look at the large-scale computer outsourcers and see in them a captive audience of customers for their gear, but also a growth opportunity in delivering data and information technology assets via the web.

Last week Dell said it would buy Perot Systems and the year before HP purchased EDS, another outsourcer. Both moves were efforts to get deeper into the cloud, providing everything from software to infrastructure as a service.

via gigaom.com

VMworld.com: newScale's VMworld Blog: Will Provision For Food: The Tale of Steve the Server Guy

This is hilarious.  It's Friday, enjoy!

VMworld.com: newScale's VMworld Blog: Will Provision For Food: The Tale of Steve the Server Guy.

Paul Maritz, VMware CEO - General Session Keynote Mentions Service Catalog

Paul Maritz presented their view of the Front Office of IT, that includes Service Catalog, Self-service, charge back and service profiles.  This is all about Offer a Cloud.

In other words, they have adopted newScale's Front Office vision; which is great.

However, based on the Lifecycle Manager lab that I attended last night, their 'catalog' is still very much focused on the back-end mechanics. It's a good step but has a few years to mature.

So overall, I'm very pleased by this development. It validates what I've been writing for a while. IT needs to be able to make offers to manage demand. Cloud is all about standard offerings.

No Catalog, No Cloud. Reading vCloud API, VMware Agrees.

I'm catching up on my reading of VMware vSphere API and sitting right there are service catalogs as a core enabler for the VMware cloud. 

Essentially, for cloud computing to work at all, you need to have very well defined set of standards, policies and configuration dials.  Without these, it's all manual, error-prone processes that take forever. The very anti-theses of cloud computing.  The back and forth requirements process has to give way to very standardized IT "services" you get from the catalog. 

So having a catalog is critical. I've written before about this in No Catalog, No Cloud.

vSphere (and imagine Microsoft, Citrix and others) are introducing a new set of objects that need to be discovered and brought into the catalog.  A few concepts to ground my assesment (directly from the vSphere API guide):

Catalogs
An Organization can support one or more Catalogs, which contain references to entities such as vApp templates and Media files such as ISO images of installable software. A vApp template can be instantiated and deployed from a catalog. Similarly, a deployed vApp can be saved back into a catalog as a template.

vApps
A virtual application (vApp) is a software solution, packaged in OVF containing one or more virtual machines.  A vApp can be authored by Developers at ISVs and VARs or by IT Administrators in Enterprises and Service Providers.

Virtual Applications
A virtual application (vApp) is a software solution comprising one or more virtual machines, all of which are deployed, managed, and maintained as a unit. Each virtual machine in a vApp typically contains a pre‐installed, pre‐configured operating system and an application stack optimized to provide a specific set of services. A vApp can be as simple as an individual virtual machine installed with a specific operating system, or as complex as a complete corporate Web site.

Essentially, these three concepts are building from the bottom up an IT technical service, and maybe a Business Service as well.  Now the notion of catalog here is on the one hand simplistic given my experience, but on the other hand, it's impressively rich compared to other areas of IT. 

Being the glass-half full kind of guy I am, color me impressed.

This is why newScale added support for VMware vCenter first and next vSphere.  There's an emerging operating model here where there's a service catalog and lifecycle front end that needs to map to the infrastructure standard definitions in that catalog.

Remember: No catalog, no cloud. You heard it first here, and now everwhere.  Even Paul Maritz, CEO of VMware, was praising the virtues of service catalog at vmworld this morning's keynote.

If you are curious as to how all this is going to work in real life, you may want try our free trial version for VMware. It's an VM appliance, with great built in videos. You'll get a good sense of what your cloud is going to look like.

Server virtualization – what could possibly go wrong? Answer: VM Sprawl

Server virtualization – what could possibly go wrong? • The Register.

..server virtualization has a lot going for it. The challenge does not lie in its faults, given that no technology is perfect. Rather, when we researched this topic we found that the level of competence and/or experience around virtualization is not particularly high.

So what could go wrong? Top of the list - given the gung-ho way that some organisations are rolling out server virtualization - is the potential for proliferation of virtual machines, and subsequent virtual server sprawl.

As one Reg reader has pointed out: “Virtualization does save a kicking from the Finance Director.” But we have also been told how the cost savings may well be storing up manageability issues down the line.

Part 9: Preventing Sprawl and Shaping Demand with Showback - 10 Things VMware Admins Should Know About Self-Service

Part 9: Preventing Sprawl and Shaping Demand with Show back

The service catalog can be used to shape demand by making clear the cost of resources; then you can use that information to shape demand and better plan capacity.

You can starting by putting clear prices and costs for resource consumption. Whether you do charge back or "showback" (showing costs, but not charging) is not that important initially.  Showback can be a stepping stone to full charge back at a future moment.   What's important is that EVEN WITHOUT charge back, communicating costs starts to change behavior.

One of newScale's customers a few years ago put a $200 for a service they needed an external company to provide and watched requests drop 40%.

Just understanding that "Large Linux" is 3X small Linux, helps people, many for the first time, to understand the differences in cost to IT.  For example, most people have zero idea of the huge difference in cost between 4 nines and 5 nines of availability. Showing what's included and the costs helps you communicate VALUE.

Show back also helps shape demand.  Demand shaping is a very useful concept for DataCenter management. Here's an explanation from an EE times article:

"Demand shaping" is a demand-driven, supply-constraining customer-centric approach to planning and execution.

... It also helps optimize use of resources, reducing excess inventory and improving inventory turns. At the strategic level, the emphasis is on aligning customers' long-term demand patterns to long-term resource and capacity constraints.

At the tactical level, the focus is on understanding demand patterns and then influencing customers' demand toward available supply, using the levers of price, promotion and products/services bundling.

Pragmatically this might mean showing the cost of "Small Linux VM instance for qa/testing for a month" as effectively $0, but the same configuration in physical form to be $nnn.  This pricing sends the signal that the VM is a better deal. 

We can also add other demand shaping info like lead times.  VM's delivered in 5 minutes, physical in 5 weeks.  

The basic tenet of demand shaping is that it starts with understanding the products the customers are requesting. This is where the standardization and packaging that a service catalog provide are so central to this process.  Understanding which packages and what other services are bundled, then tracking consumption helps us come back with a better offer that is better aligned with our existing capacity and architecture. 

Unless you've defined your "products" you can't do the kind of reporting needed to understand demand. This is what I see broken with most PPM tools -- it's all labor. You have no visibility to demand.

Applying the concept of optimizing resources and reducing excess inventory to Virtualization, translates to gaining higher density and utilization of our existing infrastructure. Which means we need to understand demand patterns in order to better plan capacity.

People really do respond to cost information. It can be a powerful tool to manage capacity.

Real CloudComputing at #VMware #VMWorld will Happen Here: Join us Tue Night 9pm

I'm arranging a tweet up for people attending VMWorld at the Montgomery at the Cigar Bar and Grill in San Francisco on Tuesday at 9 pm. This is going to be a casual meet up for people who are interested in both cloud computing and blowing clouds.  I call it the real cloud computing summit (drums!). 

At any rate, it's  a very nice yet casual place, with a beautiful heated patio and great selection of cigars and drinks.  Several people will be there, but it's just a casual meet up. Show up, grab a drink / cigar and say hello.

For those not familiar with San Francisco, the place is not too far from the hotels, but if you want to cab it, it's about 5 minutes. SF is not that big.

Also, @ericsiebert and @jasonboche are headed out Monday night at 10pm for a stogie at the Occidental Cigar Club where I plan to join them as well.  I'm not familiar with the Occidental but it's also close by.

2 In 3 IT Managers Have Cloud Funding -- Cloud Computing -- InformationWeek

Interesting quote in Information Week's 2 In 3 IT Managers Have Cloud Funding

As IT managers adopt cloud computing, they are looking to fill in the missing pieces behind the simplified services now available. Among other things, 90% said they want control over who can access their cloud resources. "Who has access? How is it controlled? Who can self-provision a server," said MacVittie.

Wow. It's what newScale does!  Send them my way.

.

Part 8: 10 Things Vmware Server Admins Should Know About Self-Service

Part 8 of the series "10 Things VMware Server Admins Should Know About Self-Service Catalogs and Lifecycle Management" that I'll be publishing over the next couple of weeks.

8. The service catalog software can communicate directly with the underlying support systems to orchestrate workflow

Once we have created our standard templates, published them in the service catalog with nice pictures and text descriptions and some simple forms we are ready to automate. 

Now comes the time to make this actionable. 

The service catalog platform should integration with either your management console or your server provisioning tool to kick off the workflow and build the environment.  Whether this is VMware vCenter, a tool like Opalis, or another orchestration product it's not that important. 

The catalog tool needs to connect to a variety of back end tools so it should have a nice integration tool built in. 

A nice use of the service catalog is that you can create bundles of services, which may need to be provisioned by different groups or tools. A common one is identity provisioning: you my need to add specific test id's, grant access, or change firewall rules. These would be separate services. They could be bundled or the user could select them individually into their shopping cart.

By the way, are there other things VM admins should know about self-service? 

Kicking up the racket on "Rackspace Launches Cloud Tools"

James Water has an interesting take on the Rackspace announcement.  My comments follow:

My Take: Its early enough to grow their own ecosystem, but it will require sincere focus and value-ad to justify the differentiation

Rackspace’s API is different for a reason. There are certain features (like static/unique IPs) they wanted to support that EC2 did not.

All the same that puts the onus on them to continue to create value ad differentiation for their cloud until there are numerous ISV’s building natively on the Rackcloud instead of porting over to it—that is until they have a unique market footprint, and their own cluster of long tail users who prefer their cloud for those architectural features. This is a partial departure from their service and quality branding—its compute quality and feature centric.

I agree with James in that is too early to declare winners and losers.  But I'd say that the window is closing very fast. You got VMware, Amazon, Microsoft in the leading spots for 1-2-3. The fight now is to be #4 or to unseat one of the incumbents.

The issue of compatibility vs innovation is one of the real difficult ones any platform provider faces.  What I see is this: why not just embrace the EC2 API's?  Launch a server is launch a server.  You can always extend to support differentiated value.  This would get you a whole bunch of ISV's for free, then let the customer drive extensions by beating up the ISV's.

Embrace and extend.  That's a strategy you don't see working often. Oh wait.

Part 7: 10 Things Vmware Server Admins Should Know About Self-Service. Role Based Access Control.

Part 7 of the series "10 Things VMware Server Admins Should Know About Self-Service Catalogs and Lifecycle Management" that I'll be publishing over the next couple of weeks.

7. It's safe to offer your services in a catalog as long as there's robust role-based access controls down to the granular level and policies.

At larger companies, there's often a concern that if the users can see it, they'll order it. "We'll be inundated with requests and we can barely keep up as it is," I've heard some folk say.

This is where role-based access control (RBAC) comes in.  RBAC allows you to create rules so that some service classes, or even individual services are viewed by some groups, departments or even individuals.

Your service catalog helps you control:

• Who can see, order or approve what services
• Who can change what services once operational

For example, certain mainframe environments are available only to application teams that work on the mainframe; while linux test servers could be available to all of QA and Development groups. 

The same could be for certain VM environments. Some developers can do some basic lifecycle (start, reboot, terminate) while others may get more services (add more memory, snapshot).

The most extreme case of RBAC use in a service catalogue was the car buying service available to executives in a Mexican company.  Pretty sweet.

Our service catalog at newScale can provide access control from a complete service category, to a service definition, to form component, down to a field.  These get quite a bit of use. For example, you may only want the requestor to see the server password. 

Having these type role based access control actually makes it easier to deploy self-service since you don't need to worry about security.

Cloudcomputing is a batttle for the API's

I was reading this news about Rackspace adding a partner development portal. And this brought to mind that a large part of the cloud battle is a battle over the API's to the cloud.

At newScale, we have support for VMWare out of the box which means their API's; and we have Amazon EC2 running in the cloud, which means those API's.

Now comes Rackspace with their API's and Gogrid is out there also. Can we add support for yet one more cloud provider? Maybe if the demand is there.  But a fourth, fifth, sixth?  At some point, you simply can't support all these environments out of the box equally well.

So if cloud is the new OS, the winner is the one with most developers, no? In that sense, Rackspace is making a good move in trying to draw developers in.  But 15 is a drop in the bucket.

My question is: why not just adopt the Amazon API's?  It has tons of developer support already. Result: instant eco system. 

Part 6: 10 Things Vmware Server Admins Should Know About Self-Service

Part 6 of the series "10 Things VMware Server Admins Should Know About Self-Service Catalogs and Lifecycle Management" that I'll be publishing over the next couple of weeks.

6.There's a lot more to self-service than a web form.

I'm shocked by what some people call self-service. A 1996 style web form painted against someone's workflow API's with all the appeal of a lipsticked pig without a date on a Saturday night.  No offense to pigs.

There's a heck of lot more to self-service than that.  We need to think about the customer experience and what we want to achieve rather than the technology. Some useful concepts to consider are:

Navigation categories that guide a user to find a particular server or configuration. For example, some users will think from the point of view of the use they'll give it (say, Intranet or App Server), others will think in terms of performance (High, Medium), yet others may think in terms of cost.  The same server should be available from multiple categories. 

Consider using a catalog to deliver pre-packaged environments into appliances so it's not a server that's requested, but an Oracle App Test environment or MS SQL development system.  This a growing trend and many good tools are out there from the likes of VMware and rPath.

Comparison guides that help answer the question "am I making the right choice from what's available?" These guides need to have different vectors of comparison so the user can see the 2-5 most important differences in a configuration

Good content needs to be developed. Feature, benefit and functions.  With good pictures.  Not the "Win2kSrvrx86FullImg"  This include links to other documents, FAQ's, or videos.

Good search. Many young application people are well, young.  They are used to Google.  So this means they want to search and click.  So really need to work up keywords and mis-spellings so you get that hint of "Did you mean....?  Try it on Google or Amazon. Search for Adobee.  It's really simple to do and trust me it'll bring a smile to your user's face.

More advanced but even more important for server administration are Configuration Wizards .  So after I've picked up my my general server or environment, a wizard will guide to a final recommendation; it will also let the user tweak their choices so they can make adjustments her and there.  

These type of wizard needs to also be present in the forms to allow finer grain control.  For example, one of our forms asks about data privacy requirements, if you select yes, it switches from a cloud environment to an internal environment. 

A self-service wizard can help the user do some rough calculations about capacity, availability and security. It doesn't need to be exact, but it provides Rough Order of Magnitude estimates that help that user make a better choice. The wizard should also prevent users from mixing things that can't go together.

Provide self-service control over lifecycle processes such as start, stop, snapshot, and configure.  Recently, I attended a Morgan Stanley CTO symposium where the topic of self-service came up.  An interesting quote was: "Use self-service.  Don't worry about the users getting it wrong. They always do, no matter how much we work with them, except they always blamed us before.  Just make sure they can correct it on their own."  Smart. 

Your service catalog should  let a user see all the items they have ordered, the status of service delivery and ... the lifecycle requests that can be made against them.   For example, a server request could be pending approval, and another operational.  The operational server would have requests for termination, reboot or reconfiguration as options that can be requested.  This is lifecycle management.

Don't skip on the graphics. Have a graphic strategy so the colors, fonts and pictures are all in the same family. 

Use service bundles to communicate value. This allows a form to be simpler because it reuses other services. 

I could go on, but I'll stop here.

I've written about this here, here and here.

Part 5: 10 Things Vmware Server Admins Should Know About Self-Service Catalogs and Lifecycle Management

Part 5 of the series "10 Things VMware Server Admins Should Know About Self-Service Catalogs and Lifecycle Management" that I'll be publishing over the next couple of weeks.

5. A service catalog will help VMware admins get ready for cloud computing, public or private.

When I first came up with the concept of a service catalog to drive fulfillment process back in 1999 (Yep. 10 years ago. Time flies when you are having fun.) it was obvious that internal shared services like IT needed to emulate the likes of Amazon.

Well, here we are in 2009 and the wheel of time has brought us back to the same place.  Now it's the data center that is being disrupted rather than end user services.   Customers are beginning to ask:  Why can't you be more like Amazon EC2? Why can't you provision fast, at guaranteed cost?

Let's look at how Amazon EC2 uses the concept of a service catalog and lifecycle management to deliver cloud computing in consumer-like experience.

There's a lot of talk about the technical aspects of cloud computing, and little the customer side:  Amazon communicates with its customers through a service catalog and lifecycle system.  The brochure part of the catalog is found here.  (I wrote this in more detail in my post: Amazon has written your technical services catalog).

To see the full functionality of this service catalog in action, I broke it down into Structure, Benefits, Pricing and Actionable for simplicity.

Structure

The whole structure looks like this:

• Functionality
• Service Highlights (Benefits)
• Instance Types
• Operating Systems and Software

• Pricing
• Resources
• Detailed Description
• Intended Usage and Restrictions

It covers what it does, what benefits (hightlights), details, major options and pricing! Then what I call the fine print (aka SLA's).

Benefits

It doesn't skimp on benefits.  In fact, benefits and outcomes are front and center. We can do the same with with our virtualization offerings. 

They tout their unique differentiators are variable (elastic) cost, while re-assuring that you have complete control, flexibility and of course, it's inexpensive. In fact, if you read that section, it draws a comparison against an internal data center!  And it gets to heart of what customers don't like about IT costs; highly fixed, over-bought, hard to plan for, etc. 

It also covers the OS, database software and middleware choices. This is an example of going beyond the server.

What are your benefits? What are your unique differentiators?

Pricing

Next, the catalog outlines the main packages: Standard and High CPU.  Two choices, and then some three sub-choicess.

There's a lot more description, links to explanation, FAQs, etc.  It's the way they standardize these formerly complicated configurations that is a useful take away.

Pricing follows and there three aspects to highlight.  First, it's completely and easily understandable as a unit of measure. They use per hour.

Standard Instances	Linux/UNIX	Windows
Small (Default)	$0.10 per hour	$0.125 per hour
Large	$0.40 per hour	$0.50 per hour
Extra Large	$0.80 per hour	$1.00 per hour

Think of all the complexity of running a datacenter: people, machines and facilities, etc. Amazon gets it down to controllable unit of of measure, hours. As a customer, I can choose to consume and hour or not.  That's a level of control that's appealing to me.  Is this the right unit of measure for every customer? No. It will depends on your customer and the benefit they want to buy. (More in future postings).

Second, they include all the pricing units for network, storage and servers.  Your complete datacenter (almost) configuration.

Third, some charges like data transfer charges are harder to map to controllable costs, so Amazon provides a pricing calculator to help translate these costs into the potential bill. And they provide sample configurations and estimates. 

Except for chargeback, which you are doing or not, every leson is directly applicable to how we present virtual environments.  

How does the catalog play a role? In two ways, it establishes the standards which enable self-service and then uses those to meter and report to your account what your consumed.

Actionable!

Finally, this catalog is NOT STATIC. It's completely actionable.   If you have an account and log in, Amazon provides:

• Self-service ordering, configuration and deployment. This request management against known, vetted standards is core to making cloud computing work. Think if Amazon had to go back and forth for weeks with a user about their configurations? 
• Account management functions. The customer can perform a variety of actions on their own to manage the lifecycle of virtual instance.
• Consumption management and billing.  The customer gets clear, hourly consumption metrics.

In other words, Amazon delivers a very complete service catalog tool set to enable cloud computing. I like that they have brought the ease of their regular catalog to a more complex environment. And ease wins.

Amazon has redefined the expectations and pricing for data center services. Make no mistake, they are your competitors. Now the challenge is to respond with your own service catalog and differentiated service definitions.  

So if your plans are to provide private cloud computing to your users, or at least behave as one, you need to consider a service catalog very early on to help you establish standards, service levels, and provisioning processes.

This time, we ought to know one thing: No Catalog, No Cloud. 

Part 4: 10 Things Vmware Server Admins Should Know About Self-Service Catalogs and Lifecycle Management

Part 4! of the series "10 Things Vmware Server Admins Should Know About Self-Service Catalogs and Lifecycle Management" that I'll be publishing over the next couple of weeks.

4. There's a lot more to setting up environments than provisioning servers, the service catalog provides access to all the other required services.

It's great that you can quickly get a server instance going. Awesome, really.  But that's not an application hosting service. It's important to understand the "whole product" requested. Is it a raw computing power w/ an OS? or is it an application stack? Or particular integration points, network, storage and security?  

There's a need to really think from a whole product perspective. What ancillary services need to go be coordinated to deliver an environment. Typically we'll need to consider

1. Network
2. Storage
3. Security
4. Middleware
5. Data
6. Applications

Sometimes we can create complete application stacks such as: "Small Linux / JBoss / Oracle for standard development." Other times, these items required hand-offs between teams.

In talking with VM admins, sometimes there's a bit of the "not my problem" mentality -- it's those other jerks who are slow. But if the think about our job as delivering environments that can work in a data center or in the cloud and as we virtualized the network and storage, there's more and more need for having a catalog of individual server request as well as complete environment

The service catalog contains all the other services that the customer needs in to deploy their application.

What We Have Here is a Failure to Analyze: Gartner Consulting is in the Cloud Collision #FailBucket

Silicon angle wrote an interesting observation (rant?): Gartner Consulting is in the Cloud Collision #FailBucket

 Is there a bigger intellectual #FAIL in current technology analysis than Gartner consulting? Watch them closely, because in my opinion they are the definition of dissonance

My take:

The cloud represents the consumerization of IT infrastructure services and a new delivery model for IT services. Thus it represents both a threat to parts of the IT, while representing an opportunity to IT and the business overall. 
Gartner magic quadrants are traditionally not used for consumer stuff, but rather for enterprise buying decisions. The cloud today, and particularly Amazon, represent freedom from IT operations for many organizations, as I've written before in 16 Digits to Freedom.

Gartner is probably right about the leadership quadrant from an enterprise point of view. A CIO might be curious about Amazon, but will call those their traditional suppliers in Gartner's magic quadrant when it comes time to explain cloud computing and how to use it.

The question is whether the cloud is out of the barn already, and no vendor can stop -- only hope to ride it.

Part 3: 10 Things Vmware Server Admins Should Know About Self-Service Catalogs and Lifecycle Management

This is part 3 of the series "10 Things Vmware Server Admins Should Know About Self-Service Catalogs and Lifecycle Management" that I'll be publishing over the next couple of weeks.

3. The catalog system is more than a document, it's also used to manage the lifecycle.

What's great about VM's is how fast and easy they are to provision, but sometimes they are hard to kill.
I see the emails going around that say: "no one is touched that instance, who owns it?"

Back when resources were scarce, our hunter gatherer customers in Application Development and QA learned to never let go of a server.  Like woolly mammoth's they were hard to catch and came only sporadically; in the summer of ROI funding, or when great migrations came. Most of the time, QA was starved for resources.  So they hoarded.

And while executing the initial request for a server environment through the service catalog gives you a nicely documentation and speed, over time changes happen and configurations drift.

This process of managing a server or environment from "as offered," to "as agreed," to "as built," and then managing the change requests against it, is what I mean by lifecycle management. 

The service catalog, being the source of "as offered," "as requested" and "as built," contains the whole lifecycle for your VM, plus information on who owns it, for how long they need it, and any other relevant data that went into the build sheet. 

Unlike a static spreadsheet, when looking at a server, you can see what the maintenance hours, SLA's and OLA's are. The lifecycle system an tell you what types of requests can be made against that VM (like add memory, for example).That server can be started, stopped, snapshotted, upgraded.  Notice they are all verbs against a thing, the VM instance.

The result is we have complete business context information about the server, the history of requests about it, subscription information and of course the proper technical build sheet, including workload requirements.  As one VMware admin recently said, "I wish I'd known that you can only work on that server on Saturdays after 5pm."

If you are curious as to what this type of system looks like, the best way to learn is to try it for free.

Part 2: 10 Things Vmware Server Admins Should Know About Self-Service Catalogs and Lifecycle Management

This is part 2 of the series "10 Things Vmware Server Admins Should Know About Self-Service Catalogs and Lifecycle Management" that I'll be publishing over the next couple of weeks.

2. The service catalog is the place where your user can document (communicate) their request

Let me show you an example. 

If you to the Dell storefront and choose to look at servers,Dell has broken down their servers into classes (Rack, Blade, etc), which then provides diferent models, which can then be customized within the parameters allowed for that model.  I'm not saying this makes sense for your environment, but the break down between classes, models, and then self-service configuration is a useful construct for thinking about your templates.  

What are your standard classes of environment you provide?  Could it be production, development, QA?  What about models? Could those be on-line transaction processing, extranet, intranet HR, basic web server, basic database? 

We would want to ask entirely different set of questions and configuration options for an extranet, high transaction database than for a personal development environment, wouldn't we?  

It'd also make our job much simpler and faster if we know what parameters were involved for that particular request.

The service catalog is key to enable your customers to:

1. Discover what's available to
2. Guide me based on my high level needs,
3. Help me compare models, then
4. Assist me in customizing my request.

And of course all the tracking, workflow and lifecycle management that the service catalog enables.  This is what makes a service catalog different from a "web form front-end" to help desk.

By the way, the best way to learn is to try it for free.

Wary of public cloud, CIO builds private cloud and transition plan [but he goes!]

Half empty or half full?  This story tells the how Intercontinental Hotels is building out a private cloud because he is not ready for a public cloud. Makes sense. 

The more interesting part is that he is architecting it so they can move to a public cloud when ready..

Wary of public cloud, CIO builds private cloud and transition plan.

"We needed to architect our inner cloud so we could easily slide into the public cloud when we are ready,"

Peer said. That has meant building applications that could live on cloud configurations through a service-oriented architecture approach and starting to take data out of two mainframes that predate the 1960s.

A year into its four-year plan to move aspects of IT to the public cloud, 60% of the data on its hotel loyalty/reward program mainframe has already been moved to servers on its inner cloud (see sidebar) and about 5% of the data has been moved off its reservation system mainframe. Whether or not sensitive data such as customer information will one day be moved to the public cloud remains to be seen, he said.

A definite move will be IHG's application testing and development and if an application passes muster in the test public cloud environment -- and doesn't contain sensitive customer data -- it will be pushed into production mode on the public cloud.

Part 1: 10 Things Vmware Server Admins Should Know About Self-Service Catalogs and Lifecycle Management

This part 1 of the series, "10 Things Vmware Server Admins Should Know About Self-Service Catalogs and Lifecycle Management" which I'm featuring in my other blog about cloud and virtualization, Cloud Front Office that I'll be cross posting here.  The advice is applicable to all technical services catalogs.

1. The service catalog is a tool for driving users to standard configurations.

To get the operational efficiencies we hope to achieve from virtualization and / or cloud computing, we need to establish standard configurations. This is tough, for a couple of reasons.

First, the challenge is the gap between the language of the customer, and the detail needed by the operations group typically generates a lot of back and forth during the "server engineering" process.  Instead of having "pre-packaged" configurations, every thing is bespoke. Instead of having useful abstraction layers and levels, the customer has to invent their own little bit of the data center. This made sense when the new app meant a whole new hardware stack to which the app would be fused to and the concrete poured on it. It doesn't make sense now.

Second,  there's resistance from customers to adopt standard VM builds.  Sometimes the reasons are valid, other times less so. The issue arises because the technical configurations have not been abstracted to a level the user can understand what they get and what's available for configuration.  Nor can they compare one template to another in ways that are meaningful to them.  

The service catalog is the tool to help deal with these two obstacles.  The service catalog is a useful tool to communicate, in the language of the customer, the different options available from IT for hosting environments.  A service catalog will support multiple views (customer, technical, financial, etc) so that when the customer selects "small Linux" for testing, this generates both a bill of materials and standard configuration options.  Once that base is selected, self-service configuration wizards provide both guidance and gutter-rails so the customer is both helped to the right thing and prevented from making errors.

From this customer configuration, the environment build sheet is generated which will drive provisioning and configuration activities or to execute any policy automation in place.

And the catalog allows the VM admins to figure out what their "market" is buying; which is very useful for capacity planning. 

10 Things Vmware Server Admins Should Know About Self-Service Catalogs and Lifecycle Management

As the CTO of newScale, I have the benefit of meeting with many customers and prospects. Recently, I've been talking to VMWare administrators, Server managers about their challenges as they introduce virtualization in the enterprise.  They all seem to face a core set of common issues, which despite the different sizes of the organizations, were remarkably coherent.

For example, a small company may not need self-service for their users to request servers, but they need a system of record to capture the workloads, SLA's, configurations, and change process.  For example, one said "I wish someone had documented that server can only be touched Saturdays between 9 and 11 pm." Or, "I've been here a year, and was wondering why that machine had almost no usage. When I found the user, they said, "That thing? We haven't done anything for a year." 

And there were many more stories like that.

This led me to think a short guide to 10 things the server team should know about service catalog and life cycle management might be useful. Over the next few days I'm going to write more extensively about this.  Here's my first pass at the ten things VMware admins should know about self-service catalogs and lifecycle management.

1. The service catalog is a tool for communicating offers to your user community. These offers represent the server standards and configurations you want to encourage.  Make them easy to find, understand, compare, and order makes them likely to be followed.

2. It's the place where your user can self-configure their request.  Rather than argue with the user, you can publish baseline configurations and allowed changes. For example, Large Linux can be configured with 2,4 or 8 Gigs of RAM.

3. The catalog system is more than a document, it's also used to manage the lifecycle of the request. This means that you know what was requested, by whom, for how long, etc.  Modern service catalog software actually lets you model all the parameters required to describe a workload, keep a record of them.

4. There's a lot more to setting up environments than just a server, the service catalog provides access to all the other required services such as security, network, storage. 

5. The service catalog software can communicate directly with the underlying support systems to orchestrate provisioning workflows, change management processes or any other process that it's needed. 

6. The service catalog and lifecycle manager are your best friend to track the "as requested" to "as configured" to as "as built" and "as changed" for the inevitable moment something breaks.

7.There's a lot more to self-service than a web form.  When you look at the Dell site, you'll see guidance and configuration wizards. These are important to truly enable self-service.

8. It's safe to offer your services in a catalog as long as there's robust role-based access controls down to the granular level and policies.

9. A service catalog will help you get ready for cloud computing, even a private cloud. How? By separating the offers, tracking and charging from the underlying provisioning system.

10. Providing self-service will not risk job security. The opposite is true. The easier you make it for your customers to do business with you, the more security you have.

I came up with a few more as I was writing these, but I'll save them for another time. Let's see if I have the stamina for 10 days of writing! 

By the way, the best way to learn is to try it for free.

Showback is a great first step to charge back. And it shapes behavior positively #cloudcomputing

I was reading the article about VMware chargeback and saw this quote.

On the other hand, he noted, showback "maintains the existing power relationship, but cost accountability becomes a positive thing instead of a negative thing." If VMware focuses on showback rather than actual chargeback, "it will have marginal success," Mann said. Otherwise? "I'm skeptical."

And I have to say I agree with it. In my experience, show-back gets you tremendous value without having to do a heck of a lot of work. One customer, we at newScale worked with years ago, reported 40% reduction in a certain type of request by just showing the cost to company.  It's hugely encouraging that so many people want to do the right thing if given the information.

As we virtualize our data center, the issue of who pays for shared infrastructure becomes thorny.  I fear we may be devolving back to paying for CPU cycles; and that didn't work.

Instead, I think the first step is to understand what is the customer buying from us, then explain it in market terms. That's the role of a service catalog. Putting the metering and chargeback ahead of the "what" is a mistake.

The rest of the article:  Price, politics working against VMware vCenter Chargeback.

I Agree: Cloud Requires Mature IT Service Management

Does a cloud computing infrastructure obviate the need for IT Service Management systems? | The IT Skeptic.

Clouds are just another form of outsourcing. Like any other outsourcing, if you outsource the governance you should be fired. Somebody still needs to own IT within the organisation, to manage the providers of the cloud. And somebody still needs to own service within the organisation, to manage the service delivery. This much is self-evident to me. the more you externalise an activity, the better you must define, manage and measure it. And the more you externalise the harder it is to get the transparency and ready access to data that you need to manage and measure it. With cloud, we'll need ITSM more in order to protect our interests and our customers.

Love in the cloud is cheap!

Pretty cool use of the cloud by eHarmony. Your next important someone maybe found through cloud processing. 

Online dating site eHarmony is also making good use of Elastic MapReduce, processing tens of gigabytes of data representing hundreds of millions of users, each with several hundred attributes to be matched. According to an article on SearchCloudComputing.com, they are doing this work for $1,200 per month, a considerable savings from the $5,000 per month that they estimated it would cost them to do it internally.

Amazon Elastic MapReduce now available in Europe | Cloudiquity.

So the cloud makes love cheap? Or love is in the cloud?

I'll stop now.

Understanding cloud computing costs in the enterprise « Yet Another (ex-)VC Blog

Understanding cloud computing costs in the enterprise « Yet Another (ex-)VC Blog.

Although cloud computing provides financial benefits like reduction of CAPEX and the ability to pay-as-you-go, organizations will still need a reasonable amount of granularity in the reporting of cloud usage and the ability to map that usage into a financial chargeback model that makes sense. Knowing which applications and departments are driving IT expenses is critical now, and will continue to be critical as cloud computing goes mainstream in the enterprise. Therefore, any cloud chargeback solution should integrate with the chargeback framework that the company uses to manage their physical assets.

What's the difference between private cloud and public clouds?

There's a nice discussion going on over at Cloud Use Cases about the difference between private and public clouds. Below is my comment.

I tend to think of cloud computing as a business model underpinned by virtualization. Just as the virtualization layer abstracts the underlying hardware, there's a front-layer that abstracts the cost, sla, workload definition from the underlying computing infrastructure. 

We,and many startups, are using EC2 because the low, standard, predictable and variable cost with elastic provisioning of resources in minutes. And this is all accessed on a self-service basis with full control of the life cycle.  This change is hugely revolutionary vs what we faced before with dedicated hosting. 

Now when I hear customers talking about building a private cloud, generally they mean the ability to deliver all or a subset of the "standard, predictable variable, elastic, self-service, etc" value.  So where's the difference between private cloud and public cloud? It is wholly dependent on your role.  If you are building the technology,the technologies and issues are very similar.  If you are a consumer, you'd like to treat them both the same, 

But if you are the owner or operator, the private cloud is very challenging. You need to think of a couple of things, One, what are the standard environments and workload variations I will allow? How do I condense everything we do into a predictable unit cost?  And, how do I provide elasticity?  There are many other challenges as well, but those three are biggies for your typical IT geek.