Like many open source projects, VERDI has a development version and one or more stable versions active at any given time. By active, we mean that the project developers are working on new features, improvements and bug fixes.
Source code for the stable branch(es) can be found in the branches folder of the VERDI github repository. For example, at the time of writing the active stable branch is https://github.com/CEMPD/VERDI/tree/master
Source code for the development versions are in individual branches.
Branches contain the bleeding edge code where the latest features are being worked on.
- Introduction ============
This manual contains instructions on how developers can set up, run, build, and obtain updates from the software repository for the Visualization Environment for Rich Data Interpretation (VERDI) software. Developers are encouraged to develop and contribute code for VERDI. Anyone who experiences a bug should check and, if appropriate, update the existing issues list on the VERDI github site: https://github.com/CEMPD/VERDI/issues. If the bug is new please submit a new issue report along with available test datasets and screenshots that demonstrate the problem. Requests for enhancements are always welcome.
Initial development of VERDI was done by the Argonne National Laboratory for the U.S. Environmental Protection Agency (EPA) and its user community. Argonne National Laboratory’s work was supported by the EPA through U.S. Department of Energy contract DE-AC02-06CH11357. Further development has been performed by the University of North Carolina at Chapel Hill’s (UNC) Institute for the Environment under U.S. EPA Contract Nos. EP-W-05-045 and EP-W-09-023, by Lockheed Corporation under U.S. EPA contract No. 68-W-04-005, and by Argonne National Laboratory. VERDI is licensed under the GNU Public License (GPL) version 3, and the source code is available through GitHub: https://github.com/CEMPD/VERDI. VERDI is supported by the CMAS Center under U.S. EPA Contract No. EP-W-09-023. The CMAS Center is located within the Institute for the Environment at UNC.
VERDI 1.6.0 alpha is distributed for the following computing environments:
- Windows 7, 64-bit
- Windows 7, 32-bit
- Linux, 64-bit
- Linux, 32-bit
- Mac, 64 bit
- Install Developer Environment =============================
To install VERDI 1.6.0 alpha on Windows 7, you no longer need administrator privileges. You should exit all programs before installing software. NOTE: VERDI 1.6.0 alpha was developed under Java 7 and has been minimally tested under Java 8.
The remainder of this chapter discusses how to install the Java Development Kit, the Eclipse Integrated Development Environment (IDE), and Subversion on your development system.
Check to see if the Java Development Toolkit (JDK) and the Java Run-time Environment (JRE) 7 are already installed and properly configured on your computer. VERDI v1.6.0 alpha was developed using the JDK version 7u55. If your version of Java is at least that release of Java 7, you can skip to section 2.2.
- Under Windows 7, open a Command window to get to a Command prompt.
- Type the command “java –version” as shown below Figure 2‑1. If you see the proper response from that java command, then your java version is already installed and included in your PATH.
- Make certain that your Java version is at least that shown above. If you have an earlier Java version (e.g., 1.6 or before) you need to download and install an update.
Download the JDK and the JRE for your operating system from the current repository for Java. Follow the installation instructions provided by the download site. http://www.oracle.com/technetwork/java/javase/downloads/index.html
- For Linux 64:
- Download the gzipped tar file <version>-linux-x64.tar.gz file
- Run the command tar –xzvf *jdk-8u91-linux-x64.tar.gz to give it executable permissions
- Run the alternatives program to tell the system about the existence of your new installation:
Alternatives –config java
(this will list how many versions are installed. If there is only one then install the JDK as number 2) alternatives –install /usr/bin/java java /opt/jdk1.6.0_20/bin/java 2 Run the alternatives program again, to choose the new installation alternatives –config java, select number 2
- Verify that your computer is finding the correct version of Java: $ java –version
- Repeat the above steps for javac:
Alternatives –install /usr/bin/javac javac /opt/jdk1.6.0_20/bin/javac 2
Alternatives –config java, select number 2
- For Windows 64 be certain to download the installer (.exe) version of the Java Development Kit.
- Click on Start Button> Control Panel> System > Advanced System Settings> Environment Variables
- In the System Variables Window highlight Path and click on the Edit button
- If you already have a version of Java in your path, update it’s version, otherwise add it.
- Edit the PATH environment variable to add the fully qualified path to your java executable (e.g., C:\Program Files\Java\jdk1.8.0_91\bin).
- Note that the list is semicolon-delimited.
- Follow the instructions at the beginning of this section and shown in Figure 2‑1.
- For Windows 64 with Powershell: Use the following command to set the path, then exit Powershell and restart it for the path to be set.
- setx PATH “$env:path;\the\directory\to\add” –m
- You should see SUCCESS: Specified value was saved.
- Restart your session and the variable will be available. setx can also be used to set arbitrary variables type setx /? at the prompt for documentation.
- Use your Web browser to go to: http://www.eclipse.org/downloads/packages/eclipse-ide-java-developers/mars2 If the URL has changed, go to http://www.eclipse.org/ and navigate to IDE and Tools, Desktop IDEs, Java IDE.
- Look for Download Links compatible with your development system.
- Download and install the Eclipse IDE for Java Developers.
- Windows: Select a mirror or click Direct link to file under Other Options to download Eclipse.
- Select a convenient directory where you have permission to install software; that is, you do not have to install Eclipse under one of the “Program Files” directories or on the root file system on one of your drives.
- Unzip the downloaded file into your chosen directory.
- Linux/Mac: Install to local directory and extract using tar –xzvf
- If you are running Eclipse on Microsoft Windows, you can put the Eclipse icon where it will be convenient for you. Using the Windows Explorer, navigate to where you installed Eclipse. Go to the eclipse subdirectory and right-click on the eclipse.exe file. Windows displays a context menu associated with eclipse.exe. Select one or more of three main options.
- Pin to Taskbar: Place the icon in the taskbar at the bottom of your main window. Once there, you can click and drag the icon along the taskbar to place the icon at a logical place for you.
- Pin to Start Menu: Place the icon in the Start menu. The icon will be placed as the last item in your list of pinned items (i.e., just above the horizontal line that divides your pinned items from your frequently used items).
- Place on Desktop: Hover over “Send to”, move your mouse into the menu that opens to the right and select “Desktop (create shortcut)”. You can then move your shortcut to a different location on your desktop or rename it.
- Using git from the command line ===============================
VERDI is available through the GitHub repository under the CEMPD organization, see Figure 3‑1.
If you do not have a GitHub account, register for it on the following website: https://github.com/ and click on Sign Up to create a personal account.
To assist with development, please send an e-mail to [email protected] with your github ID and you will be added as an outside collaborator to the VERDI GitHub repository. https://github.com/CEMPD/VERDI
- Install git by following the links for your operating system on http://gitimmersion.com
- Use the default settings selected by the Git installer
- On windows after running the installer you will see Git Bash on the Start Menu
- Follow the instructions starting on step 3 (as you have already installed git for the command line) following this link https://help.github.com/articles/set-up-git/
- On Windows, Start from the Git Bash command window
- Set your user name and e-mail using git config
- Connect to the GitHub repository over HTTPS
- Skip create a repository, as VERDI is already available on GitHub
- mkdirSkip Fork a repository, instead follow the directions below to clone a branch of VERDI from GitHub
- To obtain a local cloned copy of VERDI use the following commands to download the source code from GitHub to the directory local_git_repository/VERDI
> mkdir local_git_repository
> cd local_git_repository
> git clone https://github.com/CEMPD/VERDI.git ./VERDI
> cd VERDI
(note: if you get a message that you could not read from the remote repository please contact us and provide us with your GitHub ID).
- First go to the VERDI directory using:
> cd VERDI
- To see a list of the branches that are on the local repository use the command
> git branch
Output: * master
- To fetch a branch from the GitHub repository use the command:
> git fetch origin
- To see a list of all branches use the command:
> git branch -a
Shapefiles master * shapefile_mpas remotes/origin/HEAD -> origin/master remotes/origin/Shapefiles remotes/origin/gh-pages remotes/origin/master remotes/origin/shapefile_mpas remotes/origin/verdi_1.6 remotes/origin/verdi_1.6_v1 remotes/origin/verdi_config_v1 remotes/origin/verdi_mpas
- To checkout a branch other than the master branch and switch to using that branch use the following command, for example to obtain and switch to using the Shapefiles branch:
> git checkout shapefile_mpas
Branch shapefile_mpas set up to track remote branch shapefile_mpas from origin.
Switched to a new branch ‘shapefile_mpas’
Make changes, check status, stage change, commit change to the local repository, and push changes to GitHub
- After you edit a file on the local repository git will keep track that changes have been made to the files.
- To see files that have been modified on your local repository use the command:
> git status
- To add the files to the index for staging before a commit use the command:
> git add directory/filename
- To commit the file to the local repository branch that you are using:
> git commit –m “commit message”
- To push the changes from the local repository branch to the remote GitHub branch
> git push
- One line history> git log –pretty=format:’%h %ad | %s%d [%an]’ –graph –date=short
* 0ca8b20 2016-04-19 | Temporarily disable Add Layer and Edit Layer buttons in Configure GIS Layers dialog (HEAD -> Shapefiles, origin/Shapefiles) [Catherine Seppanen] * 5bbd76b 2016-04-18 | Set world bounds on MapContent viewport before rendering [Catherine Seppanen] * 1455e91 2016-04-18 | Specify no ellipsoid shift in assumed datum for datasets [Catherine Seppanen] * 93145c5 2016-04-18 | Fix inconsistent datum and ellipsoid for world map shapefile [Catherine Seppanen] * 2ebbdf3 2016-04-18 | Merge branch 'Shapefiles' of https://github.com/CEMPD/VERDI into Shapefiles [Catherine Seppanen] | | * 4107a02 2016-04-18 | removed 3 old *.sld files no longer being used. [Jo Ellen Brandmeyer] * | 9a6b7bd 2016-04-18 | Ignore QGIS spatial index files (*.qix) in Git [Catherine Seppanen] |/ * 6f37689 2016-04-18 | Merge branch 'Shapefiles' of https://github.com/CEMPD/VERDI.git into Shapefiles [Jo Ellen Brandmeyer] | | * 79a6222 2016-04-17 | Use current timestep and layer when building Area Information table [Catherine Seppanen] | * e5ca158 2016-04-17 | Replace invalid characters in Area Information dockable identifier [Catherine Seppanen] * | c058216 2016-04-18 | Changed US Counties coverage from tl_2015_us_county to cb_2014_us_county_500k (smaller file, highest resolution of the 3 sets of cartographic boundary shapefiles US counties from www.census.gov/geo/maps-data/data/cbf/cbf_counties.html [Jo Ellen Brandmeyer] |/ * c9d9bac 2016-04-14 | When user enters a pattern for formatting values shown in the legend, changed code such that a "0" is appended only with the modifier ends with "E" and not "0". [Jo Ellen Brandmeyer] * de70b4a 2016-04-13 | Merge branch 'Shapefiles' of https://github.com/CEMPD/VERDI.git into Shapefiles [Jo Ellen Brandmeyer]
If you plan to add a new feature or make changes to VERDI please create a new branch.
Note: VERDI new development is started by cloning a new branch, rather than forking the VERDI code. By cloning rather than forking, you avoid creating an independent repository.
- Go to the GitHub website: http://github.com/CEMPD/VERDI
- Click on the Branch button.
- Type in the new name of your branch, for example: new_branch in the text box.
- Click on the Blue “Create branch: new_branch from ‘master’ (see Figure 3‑2).
The GitHub site allows you to create a tagged Released Version of your code. GitHub allows you to indicate that it is a pre-release and whether it is production ready or not. Binary assets (such as compiled executables and documentation) can be added to a GitHub tag released version. Once published, the release details and assets are available to anyone that can view the repository.
- Git Desktop Client ==================
As an alternative to using git command line, the Git Desktop Client allows you to view changes that were made to files in eclipse and then to synchronize those changes to the remote server. In the middle of Figure 4‑1 you see two tabs, one that is labeled “No Uncommitted Changes”, and the other that is labeled “History”. Download the Git Desktop client and follow the set-up instructions from the following website: https://desktop.github.com/
If you make changes to a file in Eclipse and then view the VERDI project in the Git Desktop Client, then you see a list of the number of files that were changed with the filenames changed on the left side, and the changes in the file highlighted in green on the right side (Figure 4‑2). At the bottom of the left panel is a comment box for you to add a message about the commit and the “Commit to Shapefile Branch” button that you use to make the commit. After you commit the change, click on the Sync button in the upper right to synchronize the local shapefile branch with the remote GitHub Server.
- Start Eclipse =============
Using Windows: Select the Eclipse icon on your taskbar, start menu, or desktop, or go to the directory where your installed Eclipse (e.g., C:\Program Files\eclipse directory and double-click on eclipse.exe.
Using a Mac: Go to the applications directory, to the Eclipse folder, and click on the Eclipse icon.
Using a Linux machine: Go to the directory where Eclipse was installed and run the eclipse executable.
Figure 5‑1 shows the startup window for Eclipse. A progress bar is displayed at the bottom of the window to indicate how Eclipse is being configured. Eclipse requires more startup time as you add tools into the Eclipse environment.
Next, select the workspace for Eclipse (Figure 5‑2). If you have not yet used VERDI on your computer, you can select the Browse button to select where you want to put the workspace directory. Eclipse will create the directory for you. Warning: The location of the workplace directory should be different than the location of the local git repository directory. In fact, it will look empty (even after you import the VERDI local git repository into Eclipse) but will contain a .metadata directory that contains a version.ini file. Eclipse’s startup screen is once again displayed while more tools are loaded.
Depending on the version of Eclipse to enter the developer workspace, click on the link under the Welcome screen; titled “ Go to the workbench” (Figure 5‑3) or click on an arrow that is labeled Workbench (Figure 5‑4). The Eclipse workbench contains several windows that allow you to view source code, edit, and build within a single developer environment (Figure 5‑5).
The Eclipse IDE window as shown in Figure 5‑6 has a title bar at the top showing the name of the file currently being edited, menus and icons below the title bar, the Package Explorer down the left-hand side, multiple tabbed panes in the central file editor with Java keyword highlighting, messages along the bottom, and the Ant build environment in the bottom right-hand corner. These and other windows may be added, closed, moved, and resized as-needed for the work being performed.
- Import VERDI into Eclipse from your local git repository ========================================================
Click on the Git Folder to open it, and then select Checkout Projects from Git by clicking on it (see Figure 6‑3). Then click next.
Click on the Existing local repository to open it, and then select Next (see Figure 6‑4). Then click next.
Click on Add.. and type in the directory location of your local git Repository into the search field (Figure 6‑5), then click on the box next to the local git repository that was found and then click Finish.
Click on the box next to the local Git repository that was found and then click Finish (Figure 6‑6).
To load the software into eclipse click on VERDI and then click next and then select Import Existing Eclipse Projects. Click the Finish button (Figure 6‑7).
Eclipse then checks out VERDI from the local repository. The workspace and the directory where the VERDI software is installed should not share the same location. Figure 6‑8 shows that the code has been successfully imported into the workspace. A red X by one of the folders, on the other hand, indicates a problem. The six VERDI projects are shown in the workspace in Figure 6‑9. Note that the Git branch name for each project is listed next to the project’s name.
Some of the libraries that VERDI uses are open source and have their source code readily available. The source code to many of these libraries are distributed with VERDI and are linked within the Eclipse project. Now if your debugging session needs to go into a library for which the source code is distributed, Eclipse should be able to display the source code for you.
All of this source code is included in the verdi_core/lib_src directory of your VERDI source code installation. As shown in Figure 6‑10, the library source files are provided as jar or zip files.
Each library is cross-referenced within Eclipse from its executable jar file to its source file. To see the libraries used by one of the projects, verdi_data_loaders for example, right-click on verdi_data_loaders in the Eclipse Package Explorer. This brings up the Properties box. Select Java Build Path on the left-hand side and the four tabs then open – Source, Projects, Libraries, and Order and Export (Figure 6‑11).
As shown in Figure 6‑11, the library gt-data-12.2.jar within the directory verdi_core/lib is cross-referenced to the gt-data-12.2-sources.jar within verdi_core/lib_src. Note that both the class and source libraries are located in verdi_core directory structure, although the properties for the verdi_data_loaders project are shown. The libraries that are used for multiple Eclipse projects within VERDI are stored under verdi_core, which is the largest project. All library source code that is distributed with VERDI is located within verdi_core/lib_src.
- Configure Apache Ant to Use tools.jar from the JDK ==================================================
Apache Ant is a software tool for automating the software build process. It is provided with Eclipse.
You need to edit the General Ant Preferences to add the tools.jar from the JDK. To do this, select Window> Preferences as shown in Figure 7‑1.
Next, select Ant> Runtime> Global Entries as shown in Figure 7‑2.
Then select Add External JARS and navigate to the location where the JDK is installed on your computer. Next, browse to the lib, select tools.jar and click the Open button (Figure 7‑3). Finally, press the Apply button followed by the OK button.
- Set Eclipse Preferences =======================
This chapter provides recommended Eclipse settings for building VERDI.
Eclipse can be set up to build the projects automatically after a developer makes local changes to the Java source code. To automatically build after source code changes are made, enable this preference using the Eclipse menus (Figure 8‑1):
Window > Preferences > General > Click on Workspace >
NOTE: Any options that you set here are for all of the projects in this workspace.
If you want Eclipse to automatically rebuild your projects as you change a file, check the Build Automatically checkbox (Figure 8‑1). Note that this option can slow down your development if you are making several changes because your projects will rebuild after each change.
There is also a setting to automatically recognize files that are added to the workspace. To automatically synchronize the workspace with the underlying file system, check the Refresh on Access option (Figure 8‑1.
If your code is to be used on multiple platforms, go to “Text File Encoding” near the bottom of the window. Click the radio button to the left of “Other” and select “UTF-8”. Click the Apply button and then the OK button.
In the Package Explorer view, right click on verdi_core and select Properties at the bottom of the pop-up menu. A pop-up window titled Properties will appear for verdi_core (Figure 8‑2).
Select Java Build Path. The right side of the window then shows four tabs, containing information on the Source folders, the required Projects, the Libraries (Java ARchives (JARS) and class folders on the build path), and the Order and Export (entries that are selected for export to dependent projects) (Figure 8‑3).
Figure 8‑3 shows that the verdi_core project depends upon three other projects – verdi_bootstrap, verdi_saf_core_runtime, and verdi_saf_ui. Therefore, these projects must be built and available to the Java compiler when verdi_core is built. Also, note that the latter two projects are part of the Repast Simphony library. You should not need to change these dependencies.
From the verdi_core Properties window, select Java Compiler (Figure 8‑4). The panel on the right side shows the version of the JDK that is currently being used by VERDI. Note that the compliance settings are all set to Java 1.7. Also, the check mark at the top enables project-specific settings for the Java Compiler.
The Classfile Generation section, which is beneath the compliance settings, has several options that are used by the debugger. If you plan on running VERDI in the Eclipse debugger, check these settings.
After verifying all of your settings, click the Apply button and then the OK button.
- Build the NetCDF-Java Library with Modifications for VERDI ==================================
You may skip this chapter unless you need to make changes to the NetCDF-Java library. Note that VERDI uses a modified version of the netcdfAll library.
Follow these steps to download version 4.5.5 of the NetCDF-Java Library from GitHub. Git and maven are required.
- Check if mvn is installed on your machine using
- mvn –version
- The output should be something close to: Apache Maven 3.2.3
- If mvn is not found, install Maven on your machine by following the Installation Instructions on the Maven Download site: http://maven.apache.org/download.cgi
- Determine if git is installed on your machine using
- git –-version
- The output should be something close to: git version 2.8.1
- If git is not found, install Git on your machine following instructions in Chapter 3.
- To obtain the thredds source code from the Unidata/thredds GitHub site: https://github.com/Unidata/thredds
- Download the 4.5.5 version using the command:git clone -b target-4.5.5 git://github.com/Unidata/thredds.git
- Within Eclipse, select File>Import>Maven>Existing Maven Projects
- Browse to the directory where git downloaded the 4.5.5 version. Use the top level thredds directory as the Root Directory to import.
- Eclipse will import and use maven to build the class files for thredds.
- There will be some errors in the opendap and ui projects.
- Change package import statement on files that did not compile correctly from opendap.util.gui to opendap.tools.gui.
- Under ucar.nc2.dods remove *#*import ucar.nc2.DODSNode;
- For CoordSysTable.java use import ucar.ma2.DataType to solve the issue of the unresolved DataType.
- The following link has tips on solving these type of errors: https://meteo.unican.es/trac/wiki/TutorialMaven
- There are three files that have been modified for VERDI: WRFConvention.java, M3IOConvention.java and Stereographic.java
- The versions specific to VERDI are found under: verdi_data_loaders/src/ucar/nc2/dataset/conv/M3IOConvention.javaverdi_data_loaders/src/ucar/nc2/dataset/conv/WRFConvention.javaverdi_data_loaders/src/ucar/unidata/geoloc/projection/Stereographic.javaThese versions need to be copied from the above directories to replace these files under thredds:cp Stereographic.java thredds/cdm/src/main/java/ucar/unidata/geoloc/projectioncp M3IOConvention.java thredds/cdm/src/main/java/ucar/nc2/dataset/conv/cp WRFConvention.java thredds/cdm/src/main/java/ucar/nc2/dataset/conv/
- Refresh the cdm project. Note the WRFConvention.java uses the logger log4j, so you need to add the log4j to the build path.
- After the files have been compiled in eclipse, do a mvn install from the command line (outside of eclipse) to build the jar file needed by VERDI.
- This will create a netcdfAll-*.jar under thedds/ui/target
- Copy the netcdfAll-*.jar to /verdi_core/lib/netcdfAll-4.5.5-SNAPSHOT.jar
- Refresh the verdi_core project and rebuild VERDI.
- Test VERDI Using Scripts within Eclipse =======================================
Scripts are available for testing VERDI within Eclipse for several plot types.
From the Eclipse Main Menu, select either Run>Run Configurations or Run> Debug Configurations if you want to run the script within the debugger. Then, select Java Application to view the scripts (Figure 10‑1).
The script names include verdi_script, verdi_script_batch, verdi_script_camx, verdi_script_two_bars, and verdi_script_vertical_crosssection. Select verdi_script, and then click on the arguments tab to view the command-line arguments that are passed to VERDI in the script (Figure 10‑2). When you run the script, VERDI automatically loads the data and creates plots using the script commands specified in the arguments tab. Setting up and running scripts shorten the time required to debug plot issues because plots can be reproduced more quickly. (Note: The pathnames are specified relative to the distfiles/data directory in the arguments tab. This allows developers to run the test scripts on different platforms [Windows, Linux, or Mac] without having to edit the pathname to load the data correctly.)
Before you run the script, you need to add the VERDI_HOME environment variable to point to a location with the distribution files (the files that are available after you build VERDI within eclipse). Click on the Environment Tab shown in Figure 10‑3.
Add the environment variable VERDI_HOME and have it point to your eclipse workspace VERDI (see Figure 10‑4). At this point you are able to debug VERDI within Eclipse using the verdi_script.launch file.
- Build the VERDI Distribution =============================
Once VERDI has been checked out of the repository, the folders will be displayed in the Project Explorer Window on the Workbench (see Figure 6‑9). The next step is to edit the appropriate build.properties file.
If you are building VERDI for the Windows platform, open and edit either the 32-bit build.properties.win32 or the 64-bit build.properties.win64 file that matches your JDK; double-click on the appropriate file to open it in the text editor (Figure 11‑1). Edit the build.properties file to specify the JDK used to compile VERDI and the directory where Eclipse will build the VERDI distribution. Save your new file both under its initial name and as the new build.properties file. An example JDK location is:
If you are building VERDI for a Linux platform, open and edit either the 32-bit build.properties.linux32 or the 64-bit build.properties.linux64 file that matches your JDK. Edit the file to specify the JDK used to compile VERDI and the directory where Eclipse will build the VERDI distribution. Save your new file both under its initial name and as the new build.properties file. An example JDK location is:
If you are building VERDI for a Mac, open and edit the build.properties.mac file. Specify the JDK used to compile VERDI and the directory where Eclipse will build the VERDI distribution. Save your new file both under its initial name and as the new build.properties file. An example JDK location is:
There are five build_dist xml files available for Ant building, each for a specific version of JDK: build_dist_win32.xml, build_dist_win64.xml, build_dist_linux32.xml, build_dist_linux64.xml, and build_dist_mac.xml. These XML files provide the instructions for how to build the respective Windows, Linux, and Mac OS X distributions of VERDI. After editing a file, save it both under its initial name and as the new build_dist.xml file.
The build_dist.xml file obtains the local directory settings from the build.properties file. Although the changes to specify these directories could be made in the in the build_dist xml file, the build.properties file has been created to clearly identify what settings are dependent on the local computer configurations. Also, this structure should reduce errors that might be incurred by a user editing the build_dist xml file.