Some time ago I uploaded on GitHub the code to build Docker images for OBIEE 12c (all the current 3 releases). In the meantime, as Docker is now my main (and only with the exception of Sample Applications) way to execute OBIEE, I improved and fixed things around the images and I finally uploaded the new version.
If all you look for is to have an OBIEE Docker container up and running in 10-15 minutes follow the notes on the GitHub page.
If, on the other hand, you are curious to understand a bit more how OBIEE can be installed in a Docker container this post is for you! It is time to have a closer look at how the image is built, how OBIEE can be “dockerized”.
For this post, I will focus on the 12.2.1.2.0 version, but the others are quite the same (12.2.1.1.0 is the exact same thing with different binaries and 12.2.1.0.0 is just missing some validation steps because not supported at the RCU level).
Dockerfile : the base of everything
The Dockerfile is the key element to build a Docker image, and the Docker documentation explains it quite clearly:
Docker can build images automatically by reading the instructions from a Dockerfile. A Dockerfile is a text document that contains all the commands a user could call on the command line to assemble an image. Using docker build users can create an automated build that executes several command-line instructions in succession.
In few words the Dockerfile contains all the operations Docker must perform to build an image, it is composed by a set of commands telling Docker what to do.
Each command will be a step during the build process and will produce a new slice, a new layer, of the image. A Docker image is a set of slices of filesystem put together, one on top of each other to produce a “virtual filesystem” the container will use as base when executed.
This concept is important when “size matter …”.
The final size of the image is defined by the sum of the size of all the slices composing the image. Because of that it’s a key concept with building Docker images to find a good balance between readability and maintainability of the Dockerfile vs. reduced number of commands producing a smaller image (from a disk size point of view).
The file on GitHub focus more on readability than size. The final OBIEE image is around 18.6Gb, while another image I use for my developments is more focused on minimal size and is only 9.2Gb with exactly the same software and functionalities (actually it has few extra pieces like GIT etc.). When you will be comfortable with Dockerfiles you will easily find how to adapt it to reduce the final size of the image.
Let’s dive into the Dockerfile for OBIEE 12c and go through every single step …
FROM: Base image on which to build the OBIEE image
1 2 3 4 |
# Pull base image # --------------- FROM oraclelinux:7-slim |
The beginning of every Dockerfile is a reference to an existing image which will be used as base. It would make no sense to try to create a new Linux image from scratch all the time, so it always starts from an existing image, ideally the smallest and minimal as possible for the need.
Oracle published a set of Oracle Linux images freely accessible on the Docker Hub. As OBIEE 12c is certified with OEL7 the simplest is to start by that one.
As the idea is still to have only OBIEE and only what is required to make it works I use the “7-slim” version of Oracle Enterprise Linux, which is the smallest one available, with a really minimalist set of things installed by default.
MAINTAINER: Who puts together the image file
5 6 7 8 |
# Maintainer # ---------- MAINTAINER Gianni Ceresa <gianni.ceresa@datalysis.ch> |
The MAINTAINER is now deprecated and replaced by LABEL which is a more generic command to add metadata to the image. As I just find out this now, I will have to update the Dockerfile at some point to move to the new command before they remove support of the MAINTAINER command and prevent the build failing on an error.
The idea of this command is to identify the author of the image.
ENV: Environment variable available during build and execution
9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 |
# Environment variables required for this build (do NOT change) # -------------------------------------------------------------- ENV INSTALL_FILE_JDK="jdk-8u101-linux-x64.rpm" \ INSTALL_FILE_WLS="fmw_12.2.1.2.0_infrastructure_Disk1_1of1.zip" \ INSTALL_FILE_BI_1="fmw_12.2.1.2.0_bi_linux64_Disk1_1of2.zip" \ INSTALL_FILE_BI_2="fmw_12.2.1.2.0_bi_linux64_Disk1_2of2.zip" \ OBIEE_VERSION="12.2.1.2.0" \ INSTALL_FILE_RSP_WLS="weblogic.rsp" \ INSTALL_FILE_RSP_BI="obiee.rsp" \ INSTALL_FILE_RSP_CONFIG="bi_config.rsp" \ RUN_FILE="runOBIEE.sh" \ ORACLE_BASE=/opt/oracle # Use second ENV so that variable get substituted ENV INSTALL_DIR=$ORACLE_BASE/install \ ORACLE_HOME=$ORACLE_BASE/product/$OBIEE_VERSION \ DOMAIN_HOME=$ORACLE_BASE/config/domains |
Like any script using variables make it more flexible and dynamic as generally you only have to change values in a single place on top of a script and everything else will then adapt. In Docker it’s the same thing except that variables aren’t Docker variable but Linux environment variable, accessible by any command or script executed later.
This is of course also a limitation as if you store a secret password as environment variable it isn’t really secret anymore. Anyone with access to the image will be able to see it.
In this case, you can see there are only two commands ENV but several variables set. As each command is a slice of the final image it’s a good practice to group together commands when possible (and when making sense) and using a backslash ” \ ” to separate them.
The only reasons to have two ENV and not a single one is because to reference another variable it must exist first. In the second ENV I reuse variables set in the first one.
From an OBIEE point of view you can see that some variables start defining the structure on the disk of the OBIEE setup.
ORACLE_HOME and DOMAIN_HOME will be the location where the product and config will be stored. The main reason for splitting it into two different folders (by default the config would be inside the ORACLE_HOME in the “user_projects/domains” folder) is to make upgrades easy.
If you upgrade from 12.2.1.1.0 to 12.2.1.2.0 you will install the new product (the newer version of the code) but keep the configured domain which will be deployed on the new OBIEE. By adopting this structure, the Docker image will allow you to practice and test product upgrades the same way as it will be done on your real production environment, making this image a good sandbox to practice scripts and processes.
COPY: Load files from the host inside the image
27 28 29 30 31 |
# Copy binaries # ------------- COPY $INSTALL_FILE_JDK $INSTALL_FILE_WLS $INSTALL_FILE_BI_1 $INSTALL_FILE_BI_2 $INSTALL_DIR/ COPY $INSTALL_FILE_RSP_WLS $INSTALL_FILE_RSP_BI $INSTALL_FILE_RSP_CONFIG $RUN_FILE _configureOBIEE.sh _dropRCU.sh _validateRCU.sh $ORACLE_BASE/ |
COPY is used to make the binaires and other required files available inside the image. All these files are located next to the Dockerfile on the host and with this command I make them available on the image filesystem at the specified path.
From an image layers point of view this step is the first one producing a big slice as it will contain all the binaries required for the installation.
RUN: The command executing operations
32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 |
# Setup filesystem and oracle user # Adjust file permissions, go to /opt/oracle as user 'oracle' to proceed with Oracle Business Intelligence installation # Install pre-req packages + Oracle JDK # Make sure the run file is executable # ----------------------------------------------------------------------- RUN chmod ug+x $INSTALL_DIR/$INSTALL_FILE_JDK && \ groupadd -g 500 dba && \ groupadd -g 501 oinstall && \ useradd -d /home/oracle -g dba -G oinstall,dba -m -s /bin/bash oracle && \ echo oracle:oracle | chpasswd && \ yum -y install oracle-rdbms-server-12cR1-preinstall unzip wget tar openssl && \ yum -y remove java-openjdk java-openjdk-headless && \ yum -y install $INSTALL_DIR/$INSTALL_FILE_JDK && \ yum clean all && \ rm $INSTALL_DIR/$INSTALL_FILE_JDK && \ touch $ORACLE_BASE/oraInst.loc && \ echo inventory_loc=$ORACLE_BASE/oraInventory > $ORACLE_BASE/oraInst.loc && \ echo inst_group= >> $ORACLE_BASE/oraInst.loc && \ mkdir -p $ORACLE_HOME && \ mkdir -p $DOMAIN_HOME && \ chown -R oracle:dba $ORACLE_BASE && \ chmod ug+x $ORACLE_BASE/$RUN_FILE |
By default, the oraclelinux:7-slim image start with the root user. It’s the good user to perform generic actions like adding groups and users, perform updates or install required packages by using YUM. All the actions you would first perform when connecting to a clean Linux host before to install your tool.
For OBIEE I add the dba and oinstall groups first and then create the “oracle” user as member of these groups. These groups are generally used for database installations. Here they are used as I first copied the Oracle Database Dockerfile to create the first OBIEE one. It doesn’t really matter the name of the groups and the new user is mainly because you are never supposed to install and run OBIEE as root, so there is no reason to not follow the same rules here.
YUM will also install the database prerequisites mainly because with these you do not have a single problem of missing packages or requirements checks when installing OBIEE. It’s a nice way to keep the installation of prerequisites simple even if it will probably install few extra pieces I don’t really need (simplicity vs. size in this case).
For Java, OBIEE has some precise requirements based on the certification matrix and that’s why the official Oracle JDK is used instead of other versions. The Docker image will be as close as possible to a certified install (except for Docker itself which is currently, at time of writing, still not supported).
Last steps are the creation of the required folders structure where OBIEE will be installed and making sure the “oracle” user will have the required ownership and permissions on these folders.
Jumping back to layers of the image and use of as few commands as possible you see that all the commands are grouped into a single RUN and using ” && \ ” to concatenate them. It would make no sense to use RUN for each row having a slice of image for each one. As all the operations generate the requirements at the OS/environment level before the install, they make sense to be grouped in a single step.
RUN: Replace some placeholders to make install “dynamic”
55 56 57 58 59 60 61 62 63 64 65 66 |
# Replace place holders (and force /dev/urandom for java) # --------------------- RUN sed -i -e "s|###ORACLE_HOME###|$ORACLE_HOME|g" $ORACLE_BASE/$INSTALL_FILE_RSP_WLS && \ sed -i -e "s|###ORACLE_HOME###|$ORACLE_HOME|g" $ORACLE_BASE/$INSTALL_FILE_RSP_BI && \ sed -i -e "s|###ORACLE_HOME###|$ORACLE_HOME|g" $ORACLE_BASE/$INSTALL_FILE_RSP_CONFIG && \ sed -i -e "s|###DOMAIN_HOME###|$DOMAIN_HOME|g" $ORACLE_BASE/$INSTALL_FILE_RSP_CONFIG && \ sed -i -e "s|###ORACLE_HOME###|$ORACLE_HOME|g" $ORACLE_BASE/$RUN_FILE && \ sed -i -e "s|###ORACLE_BASE###|$ORACLE_BASE|g" $ORACLE_BASE/$RUN_FILE && \ sed -i -e "s|source=file:/dev/random|source=file:/dev/urandom|g" /usr/java/default/jre/lib/security/java.security && \ sed -i -e "s|source=file:/dev/urandom|source=file:/dev/./urandom|g" /usr/java/default/jre/lib/security/java.security |
This one is a separate RUN exclusively for readability, it could perfectly be included in the previous one.
This step will only perform replacement of values inside the response files which are going to be used for the installation of Weblogic and OBIEE as well as a really important step pointing Java to /dev/urandom instead of /dev/random.
Installation and execution in virtual machines, and even more in containers, often suffers the issues of /dev/random blocking the configuration or start process because of the lack of entropy. As the container isn’t performing anything else than running OBIEE there isn’t anything generating entropy like on a normal server.
There are tons of articles online about how good or bad it is to point to /dev/urandom, the key element is that without this change there are lot of chances your container will never finish the configuration of OBIEE, so it’s a fact that /dev/urandom helps. The little trick of using /dev/./urandom is because various version of Java wanted to be smarter and automatically replace /dev/urandom by /dev/random, bringing back the issues.
Alternative solutions could exist like deleting /dev/random in the container and make a symbolic link to /dev/urandom or map /dev/urandom of your host to /dev/random of the Docker container etc.
By doing the change in the java.security file there isn’t anything else to care about: it will work!
USER+RUN: Switch to a different user before to install
67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 |
# Start installation # ------------------- USER oracle RUN cd $INSTALL_DIR && \ unzip $INSTALL_FILE_WLS -d ./tmp_wls && \ rm $INSTALL_FILE_WLS && \ java -jar $(find $INSTALL_DIR/tmp_wls -name *.jar) -silent -responseFile $ORACLE_BASE/$INSTALL_FILE_RSP_WLS -invPtrLoc $ORACLE_BASE/oraInst.loc && \ rm -rf $INSTALL_DIR/tmp_wls && \ unzip $INSTALL_FILE_BI_1 -d ./tmp_bi && \ rm $INSTALL_FILE_BI_1 && \ unzip $INSTALL_FILE_BI_2 -d ./tmp_bi && \ rm $INSTALL_FILE_BI_2 && \ $(find $INSTALL_DIR/tmp_bi -name *.bin) -silent -responseFile $ORACLE_BASE/$INSTALL_FILE_RSP_BI -invPtrLoc $ORACLE_BASE/oraInst.loc && \ rm -rf $INSTALL_DIR/tmp_bi && \ rm -rf $INSTALL_DIR |
The USER command tells Docker to switch to a different user and perform the following commands as this new user. In my case, as the environment is ready, it’s time to start using the freshly created “oracle” user.
The installation is quite straightforward and well documented: unzip, install, delete install files.
First Weblogic, which is installed by calling a Java JAR, then OBIEE by executing the “bin” file.
As you can see all the installations are done in “silent mode”, requiring no interactions or wizards as all the parameters are provided in the response file. This method of installation is the one you would be supposed to use when installing your normal OBIEE environments as it’s the best way to guarantee all your environments are aligned with the same settings and properties.
Sadly, too often users aren’t aware of this method, or just ignore it, and keep doing visual installations by using X-window emulators and things like that.
I will never repeat it enough: the more you automate / script steps, the less human errors you will have.
WORKDIR+EXPOSE: Set default directory and ports the container can listen to
84 85 86 87 88 89 90 |
# Set work directory & Expose ports #-------------- WORKDIR $ORACLE_BASE EXPOSE 9500-9514 9799 |
The WORKDIR command will set the base folder for any following operation. As there aren’t any other RUN command this folder will be the default one when you connect to a running container or execute commands on it. In my case I set the main folder where I can easily find configuration, product and logs in underneath folders.
EXPOSE is, in the case of OBIEE, extremely important as it tells which ports the container will listen to. Without this command OBIEE will be up and running inside your Docker container, but you will have no way to connect to it from outside, OBIEE will be accessible only from inside the container itself, making it a bit useless.
Exposing ports doesn’t mean your container will use them, you can decide if you want to bind these ports (one, many, all, none) when creating the container.
OBIEE uses many ports and, in 12c, by default the range 9500-9514 used. That’s why this range is exposed as well as 9799 (Essbase 12c installed with OBIEE).
CMD: The default command to execute when starting a container
91 92 |
# Define default command to start Oracle Business intelligence. CMD $ORACLE_BASE/$RUN_FILE |
Last step and the most important to make your container simple to use. By default, a container has a command defined, the command which will be executed when starting the container as long as not overwritten by the “docker run” command itself.
Because a container run only as long as the command it executes is running, this command must be a kind of “infinite loop”, something which can run forever if needed but will also listen for stop commands and perform a clean shutdown of OBIEE before to exit.
For this reason it’s a custom script: runOBIEE.sh . Here again I used the Docker Oracle Database GitHub code as example to start setting up my own script.
An additional reason for being a custom script is that it must have a dual logic: at the first execution OBIEE isn’t configured and instead of starting the tool it must first configure it (which also means the RCU will create the schemas, the domain is created and finally OBIEE started). At any other execution, when restarting an existing container, it must only start OBIEE as it is already configured.
The final result: a set of layers composing the image
When building an image based on this Dockerfile the result is a 13-steps image composed by various slices. If I inspect the layers this is the output (by adding –no-trunc to docker history you can get the full list of commands for every single layer):
When inspecting the “history” of the Docker image you clearly see every single step and also where the space on disk is lost. But you also see one of the key concept of Docker, which is the way images are managed as a sum of layers. A powerful concept if you use it well but it also has an impact on the storage driver you choose etc. For more details about this topic there is an interesting page in the Docker documentation.
TL;DR : A Dockerfile is just all the steps you would do by hand …
As you can see there isn’t anything really special other than the runOBIEE.sh script as it must manage various things. All the other steps are things you would do by hand when installing an environment in a controlled way (response files and not by wizards).
In the end the Dockerfile can also be used as “documentation” for a silent install in a VM or a physical server: response files are provided, values are all described and order of execution clearly defined.
If you aren’t a fan of Docker and prefer Ansible or any other similar tool allowing you to script/automate environments provisioning you can easily adapt the Dockerfile steps to generate OBIEE environments.
Fantastic write-up. Thanks for the clear and detailed steps.
Docker support coming soon to a cloud near you !
I heard the official support for Docker is coming.
Now if that “cloud near me” could have a more dev-friendly trial … would definitely help 😉
Great article! I’m new to docker but was wondering how would one go creating multiple containers for an OBIEE image and it is possible to add or kill containers based on the load?
To have multiple containers you execute the docker run as many times as you needs. If you want to increase the number of instances for the same OBIEE to manage an increase of load (so scaling up and then down once the load is decreasing) it’s something else: you need to create a container which will manage automatically to join an existing OBIEE cluster and work in a cluster mode. Or you will need a load balancer which will remember which user has a session on which container as the containers based on this docker file are independents. Docker doesn’t replace the cluster functionalities of OBIEE.
Hello Gianni, I have a short time working with OBIEE and from what I noticed some settings are strongly coupled with the hostname or IP of the machine. So, I wonder if the containerization solution would be feasible when it is necessary to raise container instances with different IPs and hostnames, but using the same image as a base?
Hi Diego, no problem at all because the image doesn’t contain any configuration. All the configuration of the system is done when you create a container. A container will, by default, get a dynamic hostname and at the first start OBIEE will be configured (using that hostname). Next time you stop and start the same container the hostname will be the same (it’s the docker ID of the container), so no issues there.
The price for this flexibility is that the first start of the container take 5-10 minutes, the time to create the RCU schemas in the DB and configure OBIEE, while any following start will be faster as it’s “just” starting OBIEE itself.
Hi Gianni,
Great article. Is there any way to add the security realms configuration as part of the docker image? In my case, I have some custom BI-SQL Authentication Provider, Custom User Groups etc. I am able to setup Custom Catalog, RPD & Security Model via BAR File Import Process but couldn’t find any easy way to add the security realms configuration to the container after it is up and running.
Thanks
Raj
Hi,
Of course there is a way: Docker is just an automated way to do things, so everything you can script can be added to the image (I have one myself which does deploy and configure a custom skin when I create a container the first time so I have my own “branded” OBIEE).
All the things about security can be scripted, don’t focus on the Docker aspect but just on how to automate the activity itself. If you google that aspect you find commands on how to export/import the security realm which is maybe what you look for. You then just add the required code inside the “$ORACLE_BASE/$RUN_FILE” so it’s executed at the first start of the container (and just once like the OBIEE configuration).
How to add security realm in yml file for weblogic
The same as you would do in a normal OBIEE, it isn’t different because that’s a configuration which happen once the system is fully configured, not before.