IOPS Quality of Service for Amazon EBS Cloud Storage

Define the pipe!

You can now have EBS optimized EC2 instances with e.g.  500 or 1000 Mbit/sec throughput to EBS.

According to AWS a standard EBS volume will cope with 100 IOPS on average allowing some burts. As I mentioned before there was plenty of complaints and discussion about EBS performance workarounds though. Here is the news: AWS is offering provisioned IOPS EBS volumes. You can create up to 20 TB of provisioned IOPS volumes with a total of 10,000 IOPS per AWS account (but apply to extend your AWS account for more).

Seems like a lovely idea for a db data file, doesn’t it?

Does that make you run your EBS I/O benchmark again? Let me know about the results!

Amazon / Oracle Cloud Workshops: Mission Completed

I am just back from a successful, long and intensive trip to Australia. If you are waiting for a reply to an email there is a good chance you will get it soon now.

I delivered an  Amazon and Oracle public cloud workshop series on behalf of AUSOUG, based on my cloud computing book: 6 cities, 6 workshops of 4 h each, almost 200 attendees, hands-on, live development in the cloud (WebLogic with 61 GB heap deployed live on 3 continents, cloud storage, auto scaling, transformation of a classical Java EE app in the cloud), 16 flights (including a 1 week trip to the outback, flying 2,5 h north of Perth and driving 2200 km for shooting the cover image for my upcoming WebLogic 12c: Advanced Recipes book).

Check out the flattering reviews for the event.

 

 

thanks to everybody who attended!

 

Amazon Web Services (AWS) EC2 Autoscaling

 

How many blog posts have you read that were written on a ferry? Well this is post makes it one more. Live from famous Manly ferry since I am currently offering a number of Amazon and Oracle Cloud Computing Workshops in Australia.

Elasticity is a key criteria for cloud computing. Autoscaling is automated elasticity. Interesting enough is widely unknown how this will be implemented on the Oracle Public Cloud for the Java Service.

Autoscaling on AWS can only be configured with lengthy commands from the command line but not from the web cased AWS console. Getting all the parameters right can be tricky, so here is one of the easiest examples and a hands-on screen cast using it:

 

 


as-create-launch-config surfLaunch --region ap-southeast-1 --key access --image-id ami-b83374ea --instance-type t1.micro
as-create-auto-scaling-group surfScale --launch-configuration surfLaunch --region ap-southeast-1 -availability-zones ap-southeast-1a,ap-southeast-1b --min-size 2 --max-size 4
as-describe-auto-scaling-groups --headers --region ap-southeast-1

as-describe-auto-scaling-instances --headers --region ap-southeast-1
as-put-scaling-policy ScaleDown -auto-scaling-group surfScale --adjustment=-1 --type ChangeInCapacity --region ap-southeast-1
as-put-scaling-policy ScaleUp -auto-scaling-group surfScale --adjustment=+1 --type ChangeInCapacity --region ap-southeast-1

 

as-execute-policy ScaleUp --auto-scaling-group surfScale --region ap-southeast-1

as-describe-scaling-activities -g surfScale --region ap-southeast-1 --show-long

 

as-execute-policy ScaleUp --auto-scaling-group surfScale --region ap-southeast-1

 

mon-put-metric-alarm HighCPUAlarm --comparison-operator GreaterThanThreshold --evaluation-periods 1 --metric-name CPUUtilization --namespace "AWS/EC2" --period 600 --statistic Average --threshold 80 --alarm-actions "XXX" --dimensions "AutoScalingGroupName=surfScale" --region ap-southeast-1

 

mon-put-metric-alarm LowCPUAlarm --comparison-operator LessThanThreshold --evaluation-periods 1 --metric-name CPUUtilization --namespace "AWS/EC2" --period 600 --statistic Average --threshold 20 --alarm-actions "XXX" --dimensions "AutoScalingGroupName=surfScale" --region ap-southeast-1

 

REMOVE ALL:

as-update-auto-scaling-group surfScale --min-size 0 --max-size 0 --region ap-southeast-1
as-delete-auto-scaling-group surfScale --region ap-southeast-1

as-delete-launch-config surfLaunch --region ap-southeast-1

 

The comments section is open now for your suggestions how this will be done with the Oracle Public Cloud.

Oracle DB with OEM in Amazon Cloud

Since today Oracle EM is available with the Relational Database Service in Amazon Web Services.

RDS instances come with a free trial for 60 days and there is no additional cost for OEM.

I recommend to read my Cloud Databases whitepaper to get started, follow the discussion of Oracle DB instances in this previous posting and give it a try yourself.

Here is a screencast that explains how to create an Oracle DB instance in AWS, how to enable OEM (just in case you are an admin) and how to connect to your cloud instance with a local installation of NetBeans (in case you are a developer).

 

AWS Cloud: Use Same Access Key in Different Regions (or in All Regions)

In the Amazon cloud you require an access key to connect to your instances. This key is can be generated when you create your first instance. It’s then downloaded to your client and you specify it when connecting to the instance. Typically you need one key per AWS region.

However, you can use the same key also for different or all regions. You have to connect to a running instance and then copy it from the instance’s authorized_key file under .ssh/and import it as a key pair in the new region as shown in the following video.

Cloud Computing Workshop 2011: Oracle, Rackspace and Amazon

This year I really kept the best until the end! Last week I was running a 2-day cloud computing workshop with a 2-hour hands-on management presentation the night before the workshop for Contribute in Belgium. Contribute is an Oracle Platinum partner and being surrounded by Oracle Fusion Middleware experts, DBAs, application architects and senior level management the technical level of the workshop was very high with many interesting discussions.

We covered Oracle Public Cloud (OPC), Amazon Web Services (AWS) and Rackspace (RS). OPC is not available yet, but the overall functionality including its limitations for the first release is more or less known and quite interesting compared to let’s say running WebLogic on AWS.

To prove the point I was running WebLogic 12c on AWS cloud with 30GB of heap on a high-mem 4xl instance with 8 cores. Proving the point cost me a bit more than US$2.

Typically I expected that the more tech savvy audience prefers AWS over Rackspace, yet this time people were impressed by the easy setup of Rackspace and the way they handled a minor problem with their web console file-upload feature during a live chat session.

Among hundreds of other details we looked at the I/O performance. The performance of Amazon’s EBS is known to be interesting (you may want to read this as ‘difficult’). See Adrian’s posting for a thorough explanation, some benchmarks here, and some more details there.

The out-of-the-box performance looking at Rackspace Cloud is more consistent and there is a surprisingly high throughput which is almost independent of the data size. Here is some data comparing a local laptop disk, to the disks attached to the Rackspace Cloud servers to my brand new consumer SSD (not sure if a 512 GB SSD still qualifies as ‘consumer’). All numbers refer to a READ-benchmark with increasing data size.

Laptop HD (500GB SATA): 80 MB/s

Laptop SSD (Crucial m4): 281 MB/s

Rackspace (SAN): 302 MB/s

 

I am only posting the screenshot for one of the Rackspace I/O measurements since quality isn’t perfect. There is some older data with graphs available in a previous post of mine.

 

 

 

 

 

 

 

 

 

Now I am still curious about the dip on the left part of the graph which is consistent over several instances and measurements. Any comments?

Oracle Technologist of the Year Cloud Architect Award for Frank Munz

Last week I received the Oracle Technologist of the Year Award, Cloud Architect.

It makes me feel flattered and it’s of course a great honor for me being on this list of fame together with companies such as Dell, TurkCell and others! The award is part of Oracle’s Excellence Awards program. The winners were selected by a panel of judges that scored each entry across multiple categories.

I know there was a tremendous amount of support for the nomination of my Oracle Middleware and Cloud Computing book by my customers, workshop participants, individual book reviewers, Oracle user groups, middleware experts and even some people at Oracle HQ – many thanks to all of you!

Read the full story in the Oracle Magazine:

 

 

 

 

 

 

 

Make VirtualBox shared folder directory read / write for a non root user

This is just a note to myself. The command adds user oracle to the vboxsf  group. Don’t even try chmod or chown, it won’t help. Replace oracle with your own uid.

sudo usermod -a -G vboxsf oracle

Oracle Cloud Computing Buch zu gewinnen!

(Posting in GERMAN ONLY)

Während der DOAG2011 Konferenz können Sie ein Examplar des Oracle Cloud Computing Buches gewinnen:

✘✘✘ Unterstützen Sie die Oracle Cloud Computing Buch Seite und klicken Sie auf “gefällt mir”.

✘✘✘ Oder melden Sie sich bei der munz & more Info-Newsletter an. Es erscheinen ca. 4 Ausgaben pro Jahr mit Informationen über Vorträge und Workshops zu Cloud Computing, Oracle WebLogic, Service Bus und SOA Suite.

Die Gewinner werden Ende November benachrichtigt. Der Rechtsweg ist ausgeschlossen. Link zur Amazon-Seite mit Kritiken zu “Middleware and Cloud Computing”.

Artikel: Oracle WebLogic Server und Fusion Middleware in der Cloud

German only. Heute exklusiv auf deutsch ein Artikel den ich für das DOAG Magazin im Früjahr 2011 geschrieben habe:
Download: Oracle Fusion Middleware und WebLogic Server in der Cloud (PDF)

  • Cloud Dienste oder Fusion Middleware Features?
  • Was zeichnet eine echte Cloud aus?
  • Architektur Blueprint für die AWS Cloud und Java EE Anwendungen.

Teile des Artikels sowie zahlreiche Grafiken sind aus meinem “Middleware and Cloud Computing” entommen. Viel Spaß beim Lesen!

Oracle announcing Oracle Public Cloud – First comments.

I am at S.F. at the Oracle Open World conference right now where Larry Ellison announced the Oracle Public Cloud in an entertaining and rather fun presentation just an hour ago. To see some more photos of the event and my paparazzi shot of Sting who already showed up for some 30 seconds: check out the Facebook site of my Oracle Cloud Computing book.

Larry picked up many ideas that I published earlier this year in my cloud computing book:

He was talking a lot about migrating from one cloud to another (mostly using AWS as an example, so they seem to be on the friend list). Also he emphasized that simple multi-tenant SaaS offers such as Salesforce.com with a shared DB are not real clouds and risky (because of the shared DB :) ).

When Oracle’s position about clouds was rather fluffy (should I say cloudy?) even one year ago, I now hear them talking more about elasticity, self-service, chargeback etc.

What I didn’t like: So far this does not include pay-per use yet (one of my 4 criteria of cloud computing). Larry mentioned a monthly subscription during his keynote which was confirmed in the Thu morning keynote. Yet Oracle Enterprise Manger 12c is announced to provide metering at various levels.

I will post an update here as soon as there will be more details out tomorrow.

Apart from announcing the Oracle Public Cloud also Oracle Social Media (a part of Fusion Applications) was announced. See fotos on Facebook.

AWS outage destroys EBS-based AMIs in Europe region

I always recommend to create your own EBS-based AMIs (e.g. for running complex software such as Oracle Fusion Middleware). This hold true for the classic AMIs as well as for the converted Oracle VM templates. Never rely on the existence of AMIs provided by Oracle because:

- Oracle can change or update (or remove) them any time.

- They often don’t exist for certain AWS regions, they are S3-based or only exist based on 32-bit OEL instead of 64-bit.

- Also, the AMIs provided often don’t exist for a specific version of Oracle products.

So always create your own copy! Yet here is something to consider:

AWS broke an EBS-based AMI of mine by deleting arbitrary block in the image. This is particularly annoying since there is no easy way to create an offline copy an EBS-based AMI. You could rsync the running image to local computer but there is absolutely no support to get this done in a user-friendly way from the AWS console.

The good: They informed me in time (being in Sydney if something happens in the EU regions gives you an advantage) and sent an apology. They also replaced the deleted blocks with empty blocks.

The bad: It cost me several days to create this AMI which was an OEL EBS-based, full-blown installation of Oracle SOA Suite 11.1.1.5 (I still have to check if it will be usable after a file system check).

For a more detailed explanation of what happened take a look at Amazon’s summary of the events. It summarizes to an error in the EBS software that was overlayed with a power outage in Dublin.

Hello Amazon: Why don’t you provide an easy way to have an offline backup of EBS-based AMIs for disaster recovery?

DOAG 2011 Konferenz Presentation

Looking forward to meet you all at DOAG2011. I’ll be giving a presentation as usual which was just accepted by the program committee. So don’t miss the lovely city, Nürnberger sausages, the Sauerkraut and of course my talk.

WebLogic JMS with SAF and JMS bridges or SQS : Legacy Integration in the Cloud with Oracle WebLogic, WebSphere and OSB / Apache Camel

An interesting question popped up on my Oracle Middleware and Cloud Computing book site which I like to answer here for the benefits of all the others puzzling at similar integration questions. In the context of using JMS as an integration technology I’d like to summarize the usage scenarios for Oracle WebLogic JMS Store-and-Forward and JMS-bridges (both are included in WebLogic server).

Hi Frank – [ ...] We have a requirement to build 2-way asynchronous integration between an application running on WLS in AWS and a legacy J2EE app running on IBM WebSphere in our Data Centre. From your excepts my understanding is that SQS is intended for use only between AWS apps – is this correct ? I think we need to be looking at a full JMS solution for our integration – perhaps using WLS JMS Store-And-Forward – Thanks, Peter D

Hi Peter,

Based on your comment I cannot go into great detail or even provide a solid architecture that anwsers you question (one that will save you from more reading) but here are some important points to consider:

- Amazon’s SQS is not restricted to be only used from AWS instances. SQS is purely based on web services (or language bindings that encapsulate those WS calls) so you can use it from any computer. E.g. you can read or write to SQS queues from remote.

- WLS Store-and-Forward (SAF) can only couple WLS instances of the same version and does not bridge to other JMS providers. You cannot use SAF to transfer from WLS JMS to IBM MQSeries (or whatever Websphere might use). JMS is a pretty bad integration technology which requires to have the right messaging classes in your classpath. E.g. when writing messages from Websphere to a WebLogic JMS queue you are required to have the WLS JMS classes in Websphere classpath.

-  You can use the WebLogic’s JMS bridge to solve the somehow messy classpath issues. WLS JMS bridge has to be deployed as JCA adapter (still the jar file from the other provider is required but it is not used in custom code). The bridge will automatically forward from e.g. WLS JMS to MQSeries and even supports transcations. However there is no support to bridge between WLS JMS and AWS SQS.

- Unlike let’s say Oracle Service Bus, if you are looking at Apache Camel there is support to convert incoming JMS messages to outgoing SQS. Note to Oracle’s product manager of OSB: we would appreciate to have SQS as a supported transport protocol or possibly as an SOA Suite JCA adapter. Thanks for considering it :)

regards,

Frank

Oracle InSync2011 Conference in Sydney

 

  • It’s confirmed now. I will give a presentation at AUSOUG’s InSync2011 conference 16th / 17th August 2011. My talk is about cloud services. Really looking forward to be in Sydney again.
  • Right after the conference I will be offering an Amazon Cloud workshop/training centred around Oracle Fusion Middleware. Learn how to do real cloud computing with WebLogic right now (including elasticity, load balancing and database as a service). Bring your laptop – no need to bring your Exalogic machine for this…