Building an Oracle Image on AWS EC2

The effort behind the AWS button push does not appear accidentally. RDS makes it appear so simple. The only reason for me to build on EC2 is the ability to shut down certain configurations depending on the project.

This post documents what it takes to build a fresh image of an operational, usable instance of Oracle 12c plus Enterprise Manager on a real Oracle operating system (not available on RDS) on an AWS EC2 instance.

Ideally, the steps taken in this document would be automated and as readily available as on RDS. Nobody said anything was ideal. So, yes, all of this should be automated, and this become a button push. For future Oracle12c installations, the Amazon Machine Image (AMI) we create will skip this work.

Oracle database administrators are responsible for installing the product on the host provided by the employer or customer. When local hardware was the only choice, it boiled down to spec, order and wait, or install on an already crowded machine. The job could take days, and in some cases, weeks.

AWS changes all of that with Oracle on RDS. Put licensing issues aside and a full compliment of Oracle may be installed on Linux in less than an hour. The installation includes a version of Cloud something (EM) Express. It is full featured on RDS if you license the management packs. Installing the full version of EM is a separate post to come. For this installation, we will install Express.

Relational Database Service (RDS) still uses a pay for what you use model; however, it cannot be turned off. In order to stop the billing, you must delete the instance.

With the same database built on an EC2 host, when you turn it off the billing stops.

You will then only be paying for storage and any elastic IPs you want to maintain. I spent several weeks two months ago trying to find something that I could not do on RDS and was unsuccessful.

No longer having to do backups and patches is worth a bit of extra expense for RDS.

RDS is great, there’s no question about that. I will use and recommend it where appropriate; however, today, I need an instance with an Oracle operating system.

I want the best for this image. That is Oracle12c on Unbreakable Linux.

Here’s how to do it…

Open the AWS EC2 Dashboard and Launch an Instance

AWS EC2 Dashboard

Launch Instance

Searching for Oracle Linux brought back a list and this is the one I selected

Oracle Linux from the AWS Marketplace

Oracle Linux from the AWS Marketplace

This AMI (Amazon Machine Image) has a separate license fee of 6 cents an hour… per hour used. When it is down, it does not cost you.

There are other places to obtain your OS; but, this one comes with patches and some amount of support from the vendor. It also has flaws which we will get into later.

Step 2. My favorite instance type is m3.medium since it is approximately what I would have purchased at home. Obviously, pick what you need. Remember that you can change it later in a matter of minutes.

Choose an Instance Type

Choose an Instance Type

Probably the biggest commitment will be the storage and the subsequent cost of storage of snapshots and backups. Next we will provide configuration details. The Network and Subnet fields need to be set… and obviously, you will need to have these items built already, or build them. They exist in my account, so let’s go.

It is also important to establish any IAM roles ahead of time and if, like in this example, you want to include the rules for ec2-s3, it must be done now. Rules cannot be added to an existing instance later.

Do Not Forget to Add Roles...

Do Not Forget to Add Roles…

Step 4: Add Storage. A trick taught by Kevin Epstein of the LA Users Group was that adjusting the size from whatever default to 100 increases the IOPS numbers…

IOPS with default size device

IOPS with default size device

in this example, from 45/3000 to 300/3000

IOPS with 100 G size device

IOPS with 100 G size device

However, when you go to use the additional storage, it is not partitioned or formatted.

Under traditional circumstances today this is not a big deal. One does not get very far into being a Linux administrator without learning how to detach drives and re-attach them to a second instance to be fixed… whatever the fix is.

In this case, the fix is partitioning, and then extending of the file system. AWS site gives step by step instructions on how to proceed. The attachment is to be done while the second instance is running. The error message says it cannot be attached because the instance is running.

Cannot attach while not in stopped state

Cannot attach while not in stopped state

For the record, while it can be attached to a stopped instance, in this particular case, it was not bootable. AWS documentation warns you against booting while another bootable disk is attached… and for good reason.

I have spent way too much time on this aspect as I typically refuse to accept that something cannot be done; but, in this case, it was a long, waste of my time. The topic will come up again, shortly.

An alternate solution is best for me. My customers are of the type who will not allow their data to be unencrypted and the primary disk on an EC2 instance cannot be encrypted. I opted for the following solution which includes 100 GB of encrypted storage with the increased IOPS burst.

two volume setup

two volume setup

The image above is the configuration. The first device is properly formatted with a matching file system. The second device is encrypted. The keys required for encryption are automatically provided by AWS. There is no wasted space that I cannot get to… paid for needlessly.

For my next AMI, I will use three devices. This way, the first device can be the default 15 G. The second can be big enough for Oracle. The third device can be added later after I determine the required DB size. This way, the stored AMI can be smaller.

The image below is the before and after… one disk, vs. two.

On the top notice that xvda is 100G, and the partitioned device is only 15G. Also notice on the second host that the full device is partitioned.

one disk vs. two

one disk vs. two

I was over it. Then came time to mount the disk permanently. That requires an entry in the Linux OS file named fstab. This is where you mess up and the system will not come back. This is probably the number one place to learn about detaching and reattaching disks. Since it cannot be done with a Marketplace image… restore!

A sane pre-requisite to this entire post is to back up your system. I will admit to restoring it three times… with the second device properly attached the 3rd time.

Once we are up and running with an Oracle Linux OS and some storage here are some other simple tasks completed:

  • create users and groups required for Oracle operation
  • create minimal structure for Oracle with permissions, as required
  • entry in /etc/hosts for this server
  • set time zone
  • obtain software for the Oracle 12 enterprise database & enterprise manager
  • upload software
  • unzip software into a staging area
  • install the latest sqldeveloper (on your pc)
  • configure putty and Xming (X11) server (also on pc)
  • install Oracle… database first

It takes X11 to use the Oracle installer. X11 is very slow and you should resist the temptation to click ahead while in the installer. The password setting page and the global name fields are particularly awful to deal with in X.

Silent mode is used after you have perfected things. RDS uses silent mode. Because this is an Oracle OS distribution, I was expecting things to be a lot more “ready” for Oracle. It took considerably more work than expected. It is recorded now.

Snapshots along the way, and restores as problems were addressed to cleanly test the change, were necessary to get this gold image. This way one does not induce tools and unnecessary components on the image that is produced at the end. No unnecessary packages. No Adobe or third-party stuff here. Oracle Linux and Oracle only.

This is Oracle12c on Unbreakable Linux. There will be around 40 screens to follow. Each has a title and flyover (often the same words)… minimal comments, if I can….

Here is how to install Oracle12c on the AWS instance built above:

Logon to host server

Logon to host server

Notice in this example, the SID, UNQNAME and HOSTNAME are not the same values that I choose when I built the instance. dbhome_1 should not exist on a fresh host… if it does, the next available number will be used by the installation.

Having these values right now will save having to manually enter them later. The defaults will be based on what you have set here, and you may modify them at that time… from these defaults.

Set your bash profile

Set your bash profile

Execute the following statement as root to meet prerequisites:

yum install oracle-rdbms-server-12cR1-preinstall

Output of that command:

Dependencies Resolved

Dependencies Resolved

Following depends on where you unzipped the OTN files…

Start the installer with an ampersand to run in background

Start the installer with an ampersand to run in background

Clear email address and un-check the box to receive updates

Clear email address and un-check the box to receive updates

Answer Yes to remain uninformed

Answer Yes to remain uninformed

Create and Configure a Database

Create and Configure a Database

Change the following from desktop to server class…

Select Server Class

Select Server Class

We are not installing the grid software. Shared storage does not happen on AWS.

select single database installation

select single database installation

Advance install is required here…

Select Advanced Install

Select Advanced Install

Select your language - English here...

Select your language – English here…

This represents a change in the OTN distribution. This is an Enterprise only edition with a group of advanced (extra cost) features. Oracle Standard Edition and Standard Edition One are licensing restrictions… and have previously shared the same distribution package.

Enterprise Edition is the only choice...

Enterprise Edition is the only choice…

Default locations if all your environment variables are set right

Default locations if all your environment variables are set right

All of this can be changed later…

General Purpose

General Purpose

I should have used an AWS defined domain instead of .world. It will have to be corrected before EM Express can be used on the network.

The global database name field is particularly hard to set using X11.

Global Name, SID and deselect container DB choice

Global Name, SID and deselect container DB choice

More memory would be nice. 1506 is not 40% of what is available and it cannot be adjusted beyond 1882. This is acceptable for a desktop. Not for a server.

Specify Configuration Options - Memory

Specify Configuration Options – Memory

The default here is a terrible choice. Use AL32UTF8. While the default is acceptable, it is not compatible with a lot of US built databases. Internationalization may change all that; but, the default has never been a good choice here.

Specify Configuration Options - Character Set

Specify Configuration Options – Character Set

If this were a production server these would never be installed. I have a particular purpose and these schemas are my main reason for building this image.

Specify Configuration Options - Sample Schemas

Specify Configuration Options – Sample Schemas

Express comes by default… use it here.

Express or full Enterprise Manager?

Express or full Enterprise Manager?

We do not want ASM any more than we wanted the grid software above…

Enable Recovery - Default location if environment variables set

Enable Recovery – Default location if environment variables set

We did not initially create an operator group… leave blank or assign to dba. Do not assign to oracle even if it is in the drop down choices.

Privileged Operating System Groups

Privileged Operating System Groups

 

Dependencies Resolved

Dependencies Resolved

Summary of Components to be Installed Next

Summary of Components to be Installed Next

Product is being installed

Product is being installed

Product is being installed - looks close...

Product is being installed – looks close…

Separate windows installing clone database (examples)

Separate windows installing clone database (examples)

The string for the URL here should have been used above instead of .world.

Configuration Info - EM Express's URL, too

Configuration Info – EM Express’s URL, too

Nothing to change here, just showing you the users which are installed…

only two of forty accounts unlocked

only two of forty accounts unlocked

Install at 100%

Install at 100%

Success!

Success!

Now, let’s make that final golden image of Oracle12c on Unbreakable Linux.

Here is the freshly created instance... down.

Here is the freshly created instance… down.

Action. Image. Create Image...

Action. Image. Create Image…

I was wrong when I wrote the Image description in this shot… it cannot be changed.

Specify New Image Details Here

Specify New Image Details Here

Your image is being built now

Your image is being built now

AMI page while building... watch here and on snapshots page

AMI page while building… watch here and on snapshots page

Enter your own info for name field

Enter your own info for name field

Launch the new image...

Launch the new image…

It is not safe to delete the old instance until the new instance created by the new AMI is tested. You definitely want to check out the newly launched host (instance) and add labels on the storage page and anywhere else they are not consistent. Those steps are complete on my VPC. You now have a golden image of Oracle12c on Unbreakable Linux.

Don’t forget to update /etc/hosts as root on the instance built by your gold image of Oracle12c on Unbreakable Linux. Snapshots stay with the new AMI. Delete any other snapshots, volumes or instances that are now no longer needed.

30 year career in data, analysis, architecture, transformations, migrations, replication, using Oracle, Sybase, SQL Server, DB/2, XDB, Informix, Ingres, Seed, SQL DB/4381, and others. Nobody buys a database. People buy solutions. AWS is my network.

Leave a Reply