Using Vagrant, Chef and IntelliJ to Automate the Creation of the Java Development Environment

One of the challenges faced by software developers today working on cloud applications and distributed systems is the problem of setting up the developer workstation in a development environment comprised of an increasing number of services and technologies. It was already hard enough to configure developer workstations for complex monolithic applications, and now it’s even harder as we start to break down the application into multiple microservices and databases. If you are starting to feel like your developers’ workstations have become fragile beasts that are able to generate builds only by the grace of God and through years of mystery configuration settings, then you are facing trouble. Seek medical help immediately if you are experiencing any of the following symptoms:

• The onboarding of new developers takes days or even weeks because getting a new development machine configured is a time-consuming and error-prone process.
• The words “But the code works on my machine” are uttered frequently within your organization.
• Bugs are often discovered in production that don’t occur in development or staging.
• The documentation for deploying the application to production is a short text file with a last modified date that’s over a year old.

But making big cultural change is hard. Very hard. Nonetheless, you have to start somewhere if you want to keep software development efficient in this new cloud world. The Chinese philosopher Lao-tzu suggests a simple solution to this dilemma: “The journey of a million miles starts with a single step.” For me, the first step toward DevOps enlightenment starts with the developer workstation. If you want developers to start coding infrastructure, then you have to make it easy and painless for them, and that starts at the IDE.

The Workflow Blueprint

Here are the business requirements for our new workflow:

• The creation of the application environment on the developer workstation should be automated.
• The same automation scripts used to provision the developer workstation should be able to be reused to provision the staging environment and even production.
• The underlying automation technologies must support cross-platform application development.
• Development needs to remain fast. Code-compile-debug cycles need to be as fast as developing on a bare-metal workstation.
• The solution must allow for remote debugging, because developers have to be able to step through code during debugging.
• The developer should be able to use the IDE of her choice, configured exactly the way she likes it.

• Vagrant is a light-weight command line tool that leverages existing virtualization technologies (e.g. VirtualBox, VMware, Amazon AWS) and infrastructure provisioning technologies (e.g. Chef, Puppet, shell scripting) to orchestrate the management of virtual servers on your local workstation. For our solution, Vagrant is primarily responsible for managing the bare-bones virtual CentOS Linux server (a.k.a. the “box”), and running Chef scripts to install and configure software packages on the server. In this example, we’re creating a Linux server, but Vagrant also supports Windows VMs.

• Chef is a powerful infrastructure automation tool that allows you to manage your servers, routers and workstations through a collection of centrally managed and componentized configuration scripts. We’ll use it here simply to install and configure the application server on our virtual box.

• IntelliJ is an industrial-strength, commercial Java IDE than has support for remote Tomcat deployment, remote debugging and the ability to hot swap classes. These are all requirements to keep code-compile-debug cycles fast. You could also use Eclipse here if you prefer an open source option.

• Tomcat is used in our example as the application server. In the project, we are developing a trivially simple application with a couple of JSP pages and a servlet.

The basic steps to setup the workflow are:

1. Install Vagrant and Chef on the developer workstation.
2. Create the IntelliJ project.
3. Create the virtual server using Vagrant.
4. Provision the virtual server with Chef.
5. Configure IntelliJ for remote deployment to the virtual server.
6. Check in the application source code, Vagrant script and Chef scripts into source control.

1. INSTALL VAGRANT AND CHEF

Installing Vagrant is easy and painless so I won’t spend any time going over it here. The documentation is pretty good, and the O’Reilly book Vagrant Up and Running is a quick read and well-written.

When you install Chef, make sure to install the Chef Development Kit. This gives you all the latest development tools. Fortunately, installing Chef is also painless. However, it’s a much more robust software package than Vagrant, so if you’re like me, you’ll want to read-up on it. Learning Chef is a decent book on the subject.

2. CREATE THE INTELLIJ PROJECT

Next, create a project folder and setup the proper directory structure. For this project, we’re using the Maven-recommended directory structure.

Make a new project folder (MyApp in this example) and create a new IntelliJ project in that folder also called MyApp. IntelliJ’s default directory structure is not the Maven convention, so we have to tweak the folder structure a bit to get it there. The Maven convention is not required, but it’s a good standard so we’ll use it in this example.

If you are changing the directory structure to match the Maven convention, you’ll need to designate your Java source code directory so that IntelliJ knows where your Java class files are so it can compile them. Go to File > Project Structure > Modules > Sources. Select the src/java directory in the right-hand pane and select the Mark as: Sources button above the directory browser—this designates it as a Java source code directory. If there are any other directories listed in the right-hand pane that are marked as blue source directories, then select them and hit the Mark as: Sources button to un-set them as a source directory.

Also, make sure you’ve set your Java SDK properly for the module. From the same Modules dialog box, click on the Dependencies tab and select the correct SDK. You should also set the Project SDK in File > Project Structure > Project.

Configure the Web Facet

We need to make sure that the Web Facet is setup correctly in IntelliJ. The Web Facet tells IntelliJ a little bit about our web application so it can deploy it properly to Tomcat. Go to File > Project Structure > Facets and select the Web Facet in the middle pane. In the right-hand pane, make sure that the Web Module Deployment Descriptor is mapped to the web.xml file in the project. In this example, the web.xml file is located in [Project Directory]/src/main/webapp/WEB-INF/web.xml.

Also, make sure that the Web Resource Directory is mapped to the web application directory in the project. In this example, it’s located at [Project Directory]/src/main/webapp. The Path Relative to the Deployment Root should be “/”.

Check the Source Roots checkbox at the bottom of the dialog box. It should point to the Java source folder that you set earlier.

Finally, make sure to set the Language Level to the proper version of Java that you are compiling against (version 8 in this example).

The Application

Now that the project is setup, you’ll want to create a simple application to test the workflow. It should include web resources (e.g. JSP pages) and classes so you’ll be able test the speed of the code-compile-debug cycle time. The application in this example is simple. It consists of an index.jsp page that asks the user a question and displays two links, each representing a possible answer. The links target the servlet and they pass it a birthYear parameter in the URL. The servlet simply reads the birthYear parameter and generates a fortune based on the value. It then passes the fortune to a JSP view to be displayed to the user.

Below are links to the code for our simple web app:

• src/main/webapp/index.jsp
• src/main/webapp/WEB-INF/web.xml
• src/main/webapp/WEB-INF/jsp/view/revealFortune.jsp
• src/main/java/net/randomant/FortuneServlet.java

• pom.xml

Now would be a good time to test the project by running it against a local instance of Tomcat. Even though we’ll eventually be running the application against a remote instance, you still need a local instance of Tomcat installed on your machine. I won’t cover how to setup and run against a local instance of Tomcat here, but it’s a relatively straight forward process. If you don’t know how to do this, checkout the online documentation for help.

After you run the project, it should create an output directory called target in the project directory root. Double check to make sure this folder has been created. We’ll be syncing this folder with our virtual server and we’ll get an error if it doesn’t exist. Also, you’ll probably want to add the target folder to the Excluded directories list in IntelliJ. To do this,go to File > Project Structure > Modules > Sources, select the target directory, and click the Exclude button at the top of the directory browser.

3. CREATE THE VIRTUAL SERVER USING VAGRANT

Now let’s setup our VM. Make a new directory called vagrant in the root project folder, and then cd to the vagrant directory. Now, we need to initialize the new folder as our Vagrant directory. Type in vagrant init at the command line. This creates a default Vagrantfile, which we’ll use to setup our VM.

MyApp$ vagrant init A 'Vagrantfile' has been placed in this directory. You are now ready to 'vagrant up' your first virtual environment! Please read the comments in the Vagrantfile as well as documentation on 'vagrantup.com' for more information on using Vagrant.

Open the default Vagrantfile in IntelliJ. It’s well commented and provides some pretty useful information to help get you started. Vagrant uses Ruby scripts to manage the creation of VMs, so you’ll notice that the Vagrantfile is written in Ruby. Fortunately, you don’t really need to know Ruby to use Vagrant, since mostly we’ll just be setting attributes within an existing template.

Replace the contents of the default Vagrantfile with the code below. This sets up our basic, CentOS box. You’ll want to replace the first argument for config.vm.synced_folder on line 5 with the path to the target directory on your local workstation.

Vagrant.configure(2) do |config|

  config.vm.box = "chef/centos-6.5"
  config.vm.network "private_network", ip: "192.168.33.77"
  config.vm.synced_folder "~/projects/MyApp/target","/webapp"

end

In this file we’re telling Vagrant to do three simple things:

1. Download a pre-made, bare-bones CentOS box and use it for our VM. By default Chef/CentOS box uses VirtualBox as the VM provider. You can change this to VMware in the Vagrantfile or through the command line. VirtualBox is open source and easy to use, which is why we’re using it here. In a real production setting, however, you’re better off with a commercial-grade VM provider like VMware. It’s also worth noting here that there are lots of pre-made, open source boxes you can use for your base box, so you’re not limited to CentOS.

2. Setup Private Networking on the VM and give it an IP address of 192.168.33.77. Make sure this IP address doesn’t conflict with another IP address in your network. If it does, choose another IP address. Private networking is one of three networking options available in Vagrant. With Private Networking, all ports on the VM are open to the host machine, but they are not accessible beyond the local host. This is a safe and low-friction configuration setting. The other two options are Port Forwarding, which opens specified ports on the guest VM, and Bridged Networking which allows you to connect the VM to the world outside the host machine.

3. Sync the target directory on our host development workstation with the webapp directory located at the root level of the VM. Vagrant will create the directory for us on the guest VM if it doesn’t already exist. It is worth mentioning here that Vagrant does not like it when you delete a synced directory on the host workstation. If this does happen, the synced directory on the guest VM will break, requiring you to destroy and re-provision the VM. Because of this, you would never want to sync a folder inside target, because the folders in there are volatile, and can get deleted when you're doing a clean build. It's also worth noting that Vagrant automatically creates a synced directory linking the vagrant directory on the host machine with the /vagrant directory on the guest VM.

It will take a few minutes the first time you run this because Vagrant needs to download the box image. After that, it will use a cached version on your workstation. These box images can get pretty big, so be mindful of the available space on your workstation as you start downloading more of them.

To SSH into the box, type in vagrant ssh and Vagrant will log you in automatically as the user “vagrant”. The password for this user is also “vagrant” should you need it. By default, the sudoers file is configured to give the “vagrant” user full root access without having to type in its password.

4. PROVISION THE VIRTUAL SERVER USING CHEF

Now we have a virtual Linux server, which is great, but it doesn’t do much. We need to install Tomcat on it, configure it to host our web app, and allow it to talk with IntelliJ on our host workstation.

We’ll start by making a new chef directory under the vagrant directory on the host workstation. The path should look like this: [Project Directory]/vagrant/chef. All of our Chef scripts will live in subdirectories within the chef folder. But before we start building out our Chef directory structure, let’s briefly discuss how Chef works.

The most basic unit in Chef is the Recipe file. A Recipe is a simple script that specifies the desired end-state of the Node we are managing. The term “Node” is used by Chef instead of “Server” because Chef can also manage routers, workstations and a bunch of other infrastructure components. Chef Recipes are Ruby scripts, but we don’t really have to know much about Ruby to start working with Chef because—much like Vagrant—we’re primarily setting configuration values in Recipes. As you start doing more advanced work with Chef, however, having some solid Ruby skills will serve you well.

In Chef, Recipes are pretty low-level artifacts. Related recipes are grouped together in modules called Cookbooks. When you want to install something like Java or Tomcat on a server, you’ll be downloading a Chef Cookbook, not individual Recipes.

Cookbooks also contain other useful things like Files, Attributes and Templates. Files are static documents that Cookbooks copy onto the Node during the provisioning process. Attributes are key/value pairs that are used in Recipes. When you use pre-made, open source Cookbooks, you’ll generally want to avoid changing any of the Recipe code and will instead rely on setting Attributes to enable customization. Templates are similar to Files except that they contain dynamic elements within the document that can be set using Attributes.

Download Cookbooks for Tomcat and Java

Now that we understand Chef a little better, let’s continue by making a cookbooks directory under our chef folder on the host workstation. Our game plan here is to download pre-existing Cookbooks to do the heavy lifting of provisioning the virtual server, and then create a simple Cookbook of our own to do the final fine-tuning.

There are hundreds of open source Cookbooks in the Chef Supermarket (supermarket.chef.io) so, more likely than not, someone has already created the Cookbook you need. We’ll go here to find ourselves a Tomcat Cookbook. Type “tomcat” in the search field and sort by Most Downloaded. When I run the search, 10 entries are returned. The first Cookbook is named tomcat. It’s authored by Opscode/Chef and it has had over 15 million downloads. This appears to be the gold standard Tomcat Cookbook. A word of caution though: anyone can upload Cookbooks, so you’ll want to review the code before blindly running it on your systems. By selecting Cookbooks authored by Opscode/Chef with a high number of downloads, you’re reducing the odds of choosing a Cookbook with malicious code.

The tomcat cookbook is very robust and well-written, but, as per the documentation, it only supports Tomcat 6. If you poke under the hood you’ll find that it relies on the CentoOS package manager (yum) for installation, and, at the time of this writing, yum’s catalog only provided installation for version 6. Given the way the recipe is written, it would be a lot of work to tweak the code it get it to install a newer version. Since we don’t want to start down the path toward DevOps enlightenment with an old version of Tomcat, we’ll choose another Cookbook.

After some review, the cookbook tomcat_latest looks like the best option. It officially supports Tomcat 7, and because it doesn’t rely on yum for installation, it will be easier to modify it in the future to work with Tomcat 8. With over 5 million downloads, tomcat_latest seems pretty popular too. The Readme describes how to use the Cookbook and it lists the Attributes available for customization. It also identifies the java Cookbook as a dependency, which means we’ll need to download that Cookbook too.

Download the tomcat_latest and java Cookbooks, decompress them, and move the uncompressed folders into the [Project Directory]/vagrant/chef/cookbooks directory.

Set Cookbook Attributes

We’ll want to change some of the Attributes used in the java and tomcat_latest Cookbooks. There are a number of ways to do this in Chef, but we’re going to use a Role to do it. In Chef, a Role is a single job function that could apply across Nodes in an organization. An example of a Role might be “Nginx Web Server,” “Firewall,” or “MySQL Server.” We are going to create a “Java Development Workstation” Role.

Make a roles directory in the chef directory. In the roles directory, create a file named java-dev-workstation.rb and insert the following code:

name "java-dev-workstation"

default_attributes( 

  # We want version 8 of Java. At this time, only Oracle provides this (not OpenJDK). 
  # Because it's Oracle, we need to agree to the terms and conditions.
  :java => { 
    :install_flavor => 'oracle',
    :jdk_version => '8',
    :accept_license_agreement => true,
    :oracle => { "accept_oracle_download_terms" => true } 
  }, 

  :tomcat_latest => {
    :tomcat_install_loc => '/usr/share',
    :java_options => '${JAVA_OPTS} -Xmx128M',
    :auto_start => false
  }

)

run_list(
  "recipe[java]",
  "recipe[tomcat_latest]"
)

This script creates the role, sets the version of Java we want to install, and tells Chef where to install Tomcat. The java-dev-workstation.rb script also performs one other important task: It sets the run list which tells Chef which Cookbooks to run in which order. In this case, since the tomcat_latest Cookbook depends on the java Cookbook, we’ll run the java Cookbook first.

Update Vagrantfile to Call Chef

Now we need to tell Vagrant to run our Chef Cookbooks. Open the Vagrantfile and add the lines 7 - 11 to it.

Vagrant.configure(2) do |config|

  config.vm.box = "chef/centos-6.5"
  config.vm.network "private_network", ip: "192.168.33.77"
  config.vm.synced_folder "~/projects/MyApp/target","/webapp"

  config.vm.provision "chef_solo" do |chef|
    chef.roles_path = "chef/roles"
    chef.cookbooks_path = "chef/cookbooks"
    chef.add_role "java-dev-workstation"
  end

end

This new code instructs Vagrant to use Chef for provisioning and it tells it where our Role script is located. Now, let’s test what we have so far. Make your vagrant directory the current directory, and tell Vagrant to provision the virtual server by typing vagrant reload –provision from the command line.

Once the provisioning process is done, log into the machine by typing in vagrant ssh from the command line. Confirm that the Tomcat directory was created at /usr/share/tomcat7/apache-tomcat-7. Also confirm that the server is responding to HTTP requests by typing in http://192.168.33.77:8080/ in the browser on your host workstation. If you see the default Tomcat page, it’s up and running.

Create a Cookbook to Finish the Job

The bulk of the provisioning is now done, but there are a few more tweaks we need to make to the server before our job is done. We’ll create our own Cookbook and Recipe to accomplish this. Our Cookbook will handle the following remaining tasks:

• Set additional configuration settings in a setenv.sh file to allow Tomcat to support remote deployment and debugging.
• Configure the tomcat-users.xml file to allow us to login to the Tomcat manager web application.
• Create a symbolic link that connects the synced folder on the guest VM (/webapp) to the web app directory of our application in Tomcat home.
• Set Tomcat to automatically start after the server reboots.

[Project Directory]/vagrant/chef/cookbooks directory your current directory. Then type chef generate cookbook java-development at the command line.

This creates a Cookbook directory for you with a skeleton of the files and directories you need to make a Cookbook. List the contents of the java-development directory to see the anatomy of a Cookbook:

java-development$ ls -la total 48 drwxr-xr-x 10 jakeb 581267880 340 Jan 14 20:45 . drwxr-xr-x 3 jakeb 581267880 102 Jan 14 20:45 .. drwxr-xr-x 10 jakeb 581267880 340 Jan 14 20:45 .git -rw-r--r-- 1 jakeb 581267880 126 Jan 14 20:45 .gitignore -rw-r--r-- 1 jakeb 581267880 203 Jan 14 20:45 .kitchen.yml -rw-r--r-- 1 jakeb 581267880 51 Jan 14 20:45 Berksfile -rw-r--r-- 1 jakeb 581267880 52 Jan 14 20:45 README.md -rw-r--r-- 1 jakeb 581267880 974 Jan 14 20:45 chefignore -rw-r--r-- 1 jakeb 581267880 234 Jan 14 20:45 metadata.rb drwxr-xr-x 3 jakeb 581267880 102 Jan 14 20:45 recipes

We need to do three things at this point to make our Cookbook operational:

1. Create the Templates for our Cookbook
2. Create an Attributes file that establishes the key/value pairs referenced in our Templates
3. Write a Recipe that applies our Attributes and Templates to the virtual server

Create the Cookbook Templates

We’ll start by making our Templates. Within the java-development directory, make a templates directory, and a directory named default underneath that. So, it should look like this: [Project Directory]/vagrant/chef/cookbooks/java-development/templates/default/.

Chef Templates are basically static files, like an httpd.conf file, with bits of Ruby code inserted in them to allow Chef to dynamically set certain values during the provisioning process. The file name of the Template is just the standard name of the file with an “.erb” appended to the end. In our example, we’ll create three Templates inside our templates directory.

1. setenv.sh.erb This template sets the Tomcat config values that enable remote deployment and debugging. It sets the agentlib option for the JVM to enable debugging on the virtual host and sets the JMX flags that enable IntelliJ to remotely manage Tomcat. Security note: the jmxremote.ssl and jmxremote.authenticate options are set to false which means this configuration is not secure. This is acceptable for a local development environment that is only accessible from the host workstation. However, it is not acceptable for staging and definitely not production.

my_java_opts="-Djava.rmi.server.hostname=<%= node['java-development']['internal-ipaddress']%> "
my_java_opts+="-agentlib:jdwp=transport=dt_socket,address=<%= node['java-development']['jdwp-socket'] %>,"
my_java_opts+="suspend=n,server=y"
export JAVA_OPTS="$JAVA_OPTS $my_java_opts"

my_catalina_opts="-Dcom.sun.management.jmxremote "
my_catalina_opts+="-Dcom.sun.management.jmxremote.port=<%= node['java-development']['jmx-remote-port'] %> "
my_catalina_opts+="-Dcom.sun.management.jmxremote.ssl=false "
my_catalina_opts+="-Dcom.sun.management.jmxremote.authenticate=false "
export CATALINA_OPTS=$my_catalina_opts

2. tomcat-users.xml.erb This template creates an admin user with rights to manage the Tomcat web applications on our virtual server.

<tomcat-users><br>  
<role rolename="manager-gui"/> 
<user username="<%= node['java-development']['tomcat-user'] %>" password="<%= node['java-development']['tomcat-user-password'] %>" roles="manager-gui,admin-gui"/>
</tomcat-users>

3. tomcat7.erb This is an init script that allows us to configure Tomcat to restart automatically after rebooting.

### BEGIN INIT INFO
# Provides:        tomcat<%= node['tomcat_latest']['tomcat_version'] %>
# Required-Start:  $network
# Required-Stop:   $network
# Default-Start:   2 3 4 5
# Default-Stop:    0 1 6
# Short-Description: Start/Stop Tomcat server
### END INIT INFO

PATH=/sbin:/bin:/usr/sbin:/usr/bin

start() {
 sh  <%= node['java-development']['tomcat-home'] %>/bin/startup.sh
}

stop() {
 sh <%= node['java-development']['tomcat-home'] %>/bin/shutdown.sh
}

case $1 in
  start|stop) $1;;
  restart) stop; start;;
  *) echo "Run as $0 <start|stop|restart>"; exit 1;;
esac

In all three Templates, you can see the special syntax used to insert dynamic values:

<%= node[‘java-development’][‘tomcat_user’] %>

Create the Cookbook Attributes

Now we need to create our Attributes. Make a new attributes directory in our Cookbook, and create a file named default.rb within it. So, it should look like this: [Project Directory]/vagrant/chef/cookbooks/java-development/attributes/default.rb. Insert the following code inside the new file:

# Sets the user name and password for the Tomcat administer for the web-based Tomcat management too
default['java-development']['tomcat-user'] = "admin"
default['java-development']['tomcat-user-password'] = "secret"

# Sets the Tomcat JXM port. This is used by IntelliJ to manage Tomcat remotely.
default['java-development']['jmx-remote-port'] = "1099"

# This is the internal (192.168.*) address assigned to the virtual host. The host also has a 10.* IP address
# which we don't want.
default['java-development']['internal-ipaddress'] =
    node[:network][:interfaces][:eth1][:addresses].detect{|k,v| v[:family] == "inet" }.first

# This is the directory under the "webapps" directory in Tomcat that is used for our application
default['java-development']['app-dir'] = 'myapp'

# The JDWP socket is used for remote debugging in IntelliJ. This should match with the port specified in the Debug
# Run Configuration in IntelliJ.
default['java-development']['jdwp-socket'] = '49174'

# This is a convenience variable so we can easily access the Tomcat home directory our Recipes and Templates
default['java-development']['tomcat-home'] = "#{node['tomcat_latest']['tomcat_install_loc']}" +
    "/tomcat#{node['tomcat_latest']['tomcat_version']}/apache-tomcat-#{node['tomcat_latest']['tomcat_version']}"

This creates our Cookbook Attributes and establishes default values for them. It’s worth mentioning here that not all the Attributes available to us in Recipes are established here. Internally, Chef uses a program called OHAI to catalog a huge amount of information about each Node under its management, and it makes this information available to use via Attributes. So, for example, you could write a Recipe that determines a Node’s operating system and applies different configurations to a Windows Server versus a Linux Server. Security note: The password for the Tomcat administrator is not very secure in this example. You should change it to something more secure, especially in staging and production.

Create the Cookbook Recipe

With the Templates and Attributes in-place, we can now write our Recipe to pull everything together. When we ran the command chef generate cookbook, Chef created a default, blank recipe for us. We’ll use this file now to create our recipe. From the java-development directory, open the recipes/default.rb file and replace its content with the following code:

#
# Cookbook Name:: java-development
# Recipe:: default
#

tomcat_version = node['tomcat_latest']['tomcat_version']
tomcat_home    = node['java-development']['tomcat-home']

# Link the synced /webapps directory on the guest VM to the Tomcat web application directory on the guest VM
link "#{tomcat_home}/webapps/#{node['java-development']['app-dir']}" do
  to "/webapp"
end

# Upload the XML file that governs users and access to the Tomcat admin application
template "#{tomcat_home}/conf/tomcat-users.xml" do
  source 'tomcat-users.xml.erb'
  mode '0600'
  owner 'root'
  group 'root'
end

# Upload the config file that sets the Tomcat config settings to allow IntelliJ to communicate with Tomcat via JMX
template "#{tomcat_home}/bin/setenv.sh" do
  source 'setenv.sh.erb'
  mode '0755'
  owner 'root'
  group 'root'
end

# Place the Tomcat startup script in the init.d directory so we can set Tomcat to start automatically on reboot
template "/etc/init.d/tomcat#{tomcat_version}" do
  source "tomcat#{tomcat_version}.erb"
  mode '0755'
  owner 'root'
  group 'root'
end

# Run chkconfig to configure our Tomcat startup script to execute at run levels 2-5
execute 'chkconfig --level 2345 tomcat7 on'

# Start Tomcat
execute "#{tomcat_home}/bin/catalina.sh start"

The code is fairly straightforward and you can see it accomplishes the four tasks we set out to do. Chef Recipes are structured to take certain Resources (e.g. Files, Templates, Services, Links) and apply the desired configuration to them on the guest VM. In the Recipe above, you can see that we simply identify various resources (e.g. a link in the Tomcat webapp directory) and set various properties of that resource in the associated code block (e.g. the source of the link). We use this same pattern to manage all of the configurations we need to finish up our provisioning.

Activate the New Cookbook

Our Cookbook is now done. To put it into action we just need to add it to the Chef run list and re-provision the VM. Open our Role script ([Project Directory]/vagrant/chef/roles/java-dev-workstation.rb) and add the new java-development Cookbook to the run list on line 25:

name "java-dev-workstation"

default_attributes( 

  # We want version 8 of Java. At this time, only Oracle provides this (not OpenJDK). 
  # Because it's Oracle, we need to agree to the terms and conditions.
  :java => { 
    :install_flavor => 'oracle',
    :jdk_version => '8',
    :accept_license_agreement => true,
    :oracle => { "accept_oracle_download_terms" => true } 
  }, 

  :tomcat_latest => {
    :tomcat_install_loc => '/usr/share',
    :java_options => '${JAVA_OPTS} -Xmx128M',
    :auto_start => false
  }

)

run_list(
  "recipe[java]",
  "recipe[tomcat_latest]",
  "recipe[java-development]"
)

Then make the vagrant directory your current directory and run the following command: vagrant reload –provision.

The command vagrant reload reboots the machine and configures it as per the Vagrantfile. You usually have to reload the VM before applying provisioning changes. But don’t worry if you forget to do this—Vagrant will give you a nice message to remind you.

The –provision flag tells Vagrant to re-provision the server. It will apply all of the Chef configurations that have not already been applied.

After the command has finished, test to make sure that Tomcat is running. Point your browser to http://192.168.33.77:8080/ and confirm that the default Tomcat page is still being served. Now, let’s check to see if Tomcat will restart after rebooting the VM. Type in vagrant reload at the command line to reboot the machine. After the command has completed and the VM is backup and running, refresh your browser. If you still see the default Tomcat page then Tomcat restarted after the reboot, indicating that our new Cookbook is working.

Also, confirm that the JMX port is open. Login to the virtual server and type ps aux | grep :1099 at the command line. You should see an entry appear from the list of running processes. Also, check for the agentlib debugging port by typing in ps aux | grep :49174. Again, you should see a process entry displayed.

5. SETUP INTELLIJ FOR REMOTE DEPLOYMENT TO THE VIRTUAL SERVER

The last piece of the puzzle is configuring IntelliJ to do remote deployment to our new virtual server. For this we have to create a remote Tomcat Server configuration in IntelliJ. To get started, go to Run > Edit Configuration. In the left-hand pane, press the plus button at the top of the pane and select Tomcat Server > Remote from the list. The right-hand pane now displays the settings for the remote Tomcat Server.

---

Add a Remote Tomcat Run Configuration in IntelliJ

We’ve now created a new remote Tomcat Server configuration. Configuring the remote Tomcat Server can be a little tricky as IntelliJ is extremely finicky about these settings. Every setting is important to getting the workflow up and running. The image below displays the dialog box that we’ll use for configuration.

---

Configuring the Remote Tomcat Server

1. Change the name of the Remote Server to "Virtual Tomcat Server”.
2. Select the Application Server from the drop down list. IntelliJ is looking for a locally installed instance here. Even though we’ll be deploying against a remote Tomcat server, we still need to have Tomcat installed locally. I have Tomcat 8 installed on my workstation but it works fine in conjunction with Tomcat 7 on the virtual server for this project.
3. Set the URL that IntelliJ will open after the build is complete to http://192.168.33.77:8080/myapp. Note that Tomcat on the remote server is running on port 8080 in our example.
4. Set the On Update action to Hot Swap Classes. This allows us to hot deploy updated compiled classes without restarting the server.
5. The JMX port is used by IntelliJ to remotely manage Tomcat. The default port of 1099 is fine because our Chef scripts configured Tomcat to listen for JMX on this port.
6. Set the Remote Staging Type to Local or mounted folder. Our synced folders have been configured to support this option.
7. Click the ellipsis button on the right of the Host field to create a new host. This brings up the Deployment dialog box which allows us to add and configure the remote server. Click the plus button at the top of the left-hand pane to create a new server. Name it "VagrantServer" and select Local or mounted folder for the type. Set the Upload/download project files field to [Project Directory]/target. This is the directory on our local workstation that is mapped to virtual server. Finally, type in the URL to the root of the Tomcat web server: http://192.168.33.77:8080. Click OK and return to the Run/Debug dialog box.
8. I’ll admit: This field is a bit of a mystery to me. It specifies where the output directory for the build should be for the remote deployment (remote-out in this example). The thing is, I’m not entirely clear why there needs to be a separate output directory, since IntelliJ already does a build under [Project Directory]/target. But what I can tell you with absolute certainty is that you do NOT want to set this to be the root of [Project Directory]/target (which is naturally what you would expect it to be). If you do, you’ll end up with very strange recursive behavior: every time you “Run” your project, it will take a copy of the build and tuck it away in a subdirectory of the new build. So, every time you “Run” the project, you’ll keep adding nested build directories. Very annoying.
9. Set this to the document base of the web app on the virtual server: /usr/share/apache-tomcat-7/webapps/myapp
10. Set the Remote Connection Settings field to the IP address of the virtual server. Don’t preface it with “http://”. Set the port to 8080.
11. Click the plus button in the Before Launch section and add Make and the MyApp:war exploded Artifact to the list of steps that IntelliJ will perform when you “Run” the project.

Now let’s configure the Deployment tab.

---

Configuring the Deployment Tab in the Run/Debug Configurations Dialog

The exploded WAR Artifact should appear in the Deploy at the server startup list. If not, add it by pressing the plus button at the bottom of the list. Also, make sure to set the Application Context to /myapp.

Finally, we’ll configure the settings on the Startup/Connection tab.

---

Configuring the Startup/Connection Tab in the Run/Debug Configurations Dialog

Click the Debug configuration from the list at the top of the tab. Then, set the port to 49174. This was the port number we set in the JAVA_OPTS variable in our setenv.sh.erb Template.

And with that, we’re done configuring IntelliJ. Whew! A lot of configuration!

Now go to Run > Run Tomcat-Vagrant (or press Shift+F10) and watch the magic happen. It should compile your code, deploy your code to the virtual server and open up your browser to the home page of the application.

---

The Fortune application running from our virtual server--Don't expect to win a Webbie

After you confirm that the application works, try making changes to the FortuneServlet class, and hit Run again. The changes will be made quickly, without a reboot of the server. Finally, set a breakpoint in the FortuneServlet class and go to Run > Debug Tomcat-Vagrant. When you hit the FortuneServlet in your browser, it will trigger the breakpoint, allowing you to step through your code during debugging.

And with that, we’ve now met all of the requirements we established for our workflow.

Troubleshooting

If you run into problems, here are a few tools that you can use to help diagnose the problem:

• Jconsole: If you are having trouble connecting to Tomcat via JMX or you want to see the application settings that IntelliJ is making to Tomcat, use Jconsole. Jconsole is GUI JMX tool. Type in jconsole & from the command line on your workstation to fire it up. Then go to MBeans > Catalina > WebModule to view the settings (e.g. document base) for each application managed in Tomcat. You can also set Tomcat settings using Jconsole. Here’s a good blog post to help you start working with it.
• Tomcat Logs: Many of the problems that can arise are the result of incorrect directory-related settings. Check the Tomcat logs if you’re getting an error message in IntelliJ like this: “Error during artifact deployment. See server log for details.” The logs are very helpful in diagnosing the problem. You can can find them on the virtual server at: /usr/share/tomcat7/apache-tomcat-7/logs/catalina.YYYY-MM-DD.log
• Refresh Your Browser: If you're getting a 404 Error or a blank page when you view the page, try refreshing your browser. Sometimes when IntelliJ launches the browser after you Run the project, it loads the URL in the browser before Tomcat has finished reloading. Refreshing will fix the problem.
• Double check the IntelliJ settings: When in doubt, check the settings in IntelliJ—especially the Run/Debug Settings. An incorrect settings is very likely the culprit.

6. CHECK-IN THE CODE AND EXHALE

At this point, you should commit the project to source control, including the Vagrantfile and Chef scripts. One of the biggest advantages of automating the provisioning of the development environment is that the configuration of infrastructure is now comprehensively documented, and can be maintained and shared via source control. Now, when you onboard new developers to a project, they simply need to download the source and call vagrant up to setup the development environment. Easy as pie. But perhaps even more important, your software developers are now thinking about infrastructure automation and have access to the automation scripts in their solution. And with that small change, we are now one step closer to reaching DevOps Nirvana.