When I first started learning about JD Edwards I asked about the Business Function Documentation and was told that "there isn't any." Since then one of the big questions in my mind is, "Why would people say such things?" At this point I have a couple of guesses:
- The main thing is that what little documentation can be found is not very exciting - especially if you are not seriously into C++ software development.
- If you do know C & C++, especially if you are comfortable with the command line tools used for working with binary object files and DLLs, you will find that the "documentation" is really a few notes in the source code. In that sense it's all hiding in plain sight. Of course, if you don't know C++, it becomes invisible.
In any case it's necessary to generate the available documentation and take the time to poke around to get a sense of the available functionality. If you are trying to write business functions and/or business services (in any language,) you may find it difficult to get started but, after a while, you'll get comfortable and find that you are able to get the job done.
In this article we will go through two exercises:
First we'll just run the Business Function Documentation Generator
script provided with JD Edwards. UBE R98ABSFN, batch version XJDE0001,
is already listed in the Solution Explorer menu tree. All we have to
do is submit the job and wait while it generates the 13k+ HTML
pages. In there we also get a 400+ page PDF index of all the
functions. I'm not saying that anybody will be excited about this -
I'm just saying that it's available so we might as well generate it
and keep it handy.
We will also take a quick look at the related Business Function Documentation Viewer. It, too, is listed in the Solution Explorer. It's kind of handy in the sense that it's easier to use than the 400+ page PDF file. It can be handy from time to time.
- The actual source code for the business functions can be found in the Enterprise Server. When you look at the code you see right away that the authors are using Doxygen to generate indexed reference documentation. Unfortunately there are over 14M lines of code. Doxygen has a hard time crunching it all. We will look at the process of generating the documentation using GNU Parallel and Doxygen TAG files. Once you see the results you will find yourself confident that this must be the documentation the software authors are using at Oracle.
Built-in BSFN Documentation
When I first started trying to learn about JDE I downloaded lots of Oracle ebooks and copied them into my Kindle. Unfortunately there are too many books and they are written in a circular fashion, ie: Each one expects that you already know and understand the contents of the others. This forces the reader to skim very quickly through hundreds of pages. While skimming, of course, we miss details like this:
Development Tools: APIs and Business Functions Guide
"Run UBE R98ABSFN, batch version XJDE0001 to generate all business function documentation. The system creates a hypertext markup language (HTML) link for each business function for which you generated documentation. It also creates an Index HTML file. These HTML files appear in the output queue directory."
UBE R98ABSFN, batch version XJDE0001
In fact, it turns out, there is an entry in the Solution Explorer for this script. It's in Application Development -> Object Management -> Business Function Documentation and it's called Generate BSFN Documentation.
(Throughout this article you will find that the images are
lightbox-enabled. Click on an image to see it the full-sized
version. Press the ESC key to dismiss the full-sized image and return
to the article text. Scroll down for more text.)
To run this batch job: Click on the Generate BSFN Documentation menu item. This will open a form called Generate BSFN Documentation. It contains a form called Work with Batch Versions - Available Versions.
Click on the XJDE0001 version in the list of rows.
This will bring-up a small window called Version Prompting. Click the Submit button.
Next, select a Report
Destination and click the OK button to submit the job.
The Viewer: P98ABSFN
Use FastLink P98ABSFN or click on the Application Development -> Object Management -> Business Function Documentation -> Business Function Documentation Viewer menu item in the Solution Explorer menu tree. The viewer is nice to have in that it allows quick access to the basic info that we need from within the Solution Explorer.
The UBE Script generates the HTML files into the PrintQueue folder (c:\JDEdwards\E910_1\PrintQueue in the Standalone v9.10 demo,) but they will not stay there. After a while they disappear (the P98ABSFN viewer will continue to function.)
While you've got the files I recommend that you zip them for future use. You can open the bsfnindx.htm file to access the JDE documentation through any browser. Unzip the files into any convenient web server in your office for convenient access. If you generate the Doxygen docs below you can unzip the PrintQueue folder into the Doxygen html folder for access to both sets of documentation.
Next, we'll generate Doxygen documentation.
Cross Referenced and Indexed Doxygen docs
On the JD Edwards Enterprise server there are a number of environments that are stored in the Packages folder (C:\JDEdwardsPPack\E910_1\packages\ on my Enterprise Server - your paths may be different.)
Within each of the environment folders you will find an Include folder and a Source folder. The Include folder contains almost 20k header files all together in that one folder. The Source folder contains 48 subfolders - each one containing a collection of related C and C++ source files (over 12k files, total.) The grand total is 14M+ lines of source code - and it appears to have been written with the expectation that Doxygen will be used to generate cross-referenced and indexed documentation.
At the time I first saw the code I didn't realize that it was so huge. I quickly ran Doxygen against the entire API. After about five or six hours I discovered that I'm not patient enough to wait for the results.
Fortunately the folks who wrote Doxygen added a feature to solve this specific problem: TAG files. A TAG file is an XML file containing information about everything that Doxygen found while scanning a collection of files. Doxygen can quickly generate a TAG file. It's much faster than generating the full documentation.
Doxygen can then read the TAG files very quickly before generating documentation for another collection of source files. Any references to the items in the TAG files are quickly resolved in the TAG data. The resulting documentation is mostly the same output that we would otherwise get - but it is generated much more quickly.
After a bit of tinkering I wrote a script, Gendocs, that automates the two phases:
- In Phase I:
We need to generate TAG files for each subfolder of the source folder. I used Doxywizard to create a Doxygen configuration file for this phase called tags-template. As the name implies: This configuration file tells Doxygen to scan all the source code in a subfolder and generate a TAG file. The Gendocs script copies the tags-template once for each of the 48 subfolders.
- In Phase II:
We can generate the documentation for the source code in each subfolder. That code, of course, contains references to the other subfolders. As such, when we process one of the subfolders, we need to begin by loading the 47 TAG files that were generated for the other subfolders.
Again, I used Doxywizard to create a Doxygen configuration file for this phase; this one is called main-template.
Of course you can enable or disable lots of Doxygen features in these configuration files. As we know that the code we are scanning contains little in the way of english explanation, I enabled every feature I could: Hyperlinked call graphs, source code embeded in the documentation pages, etc.
The Gendocs script copies all the necessary template files, updates each one with the necessary folder paths and TAG file names, etc. It generates a script with the Doxygen commands necessary for Phase I and another script with the commands necessary for Phase II.
The next problem is that Doxygen uses a single thread. If you request lots of graphs using DOT you will start to see some activity on the other CPU threads in your system but, generally, Doxygen will pin one thread at 100% until it finishes scanning and generating output for the one subfolder.
To get things moving faster we want to keep each of the CPU threads on our system busy. GNU has a nice utility for this purpose called Parallel which is available in Ubuntu and other Linux distributions. It takes, as input, a script containing a list of commands to execute. It then figures out for itself when to spawn a command so as to keep all CPU threads busy until all the commands have finished executing. It's easy to use and the results are excellent.
One warning: It really works. The CPU temperature on my notebook jumps up 30+ degrees Celsius while generating the Doxygen documentation. Use a can of compressed air to clean the dust out of your cooling fins before you run this script.
Here's the process tree that you can see on a quad-core Intel i7 CPU while the Gendocs script is generating tag files:
/usr/bin/time -v -a -o doc/doxyfiles/doxyfile-tags.par.log /usr/bin/parallel -v \_ perl /usr/bin/parallel -v \_ /bin/bash -c /usr/bin/time -v -a -o doc/doxyfiles/ccrin-tags.log /usr/bin/doxygen doc/doxyfiles/ccrin-tags &>doc/doxyfiles/ccrin-tags.log | \_ /usr/bin/time -v -a -o doc/doxyfiles/ccrin-tags.log /usr/bin/doxygen doc/doxyfiles/ccrin-tags | \_ /usr/bin/doxygen doc/doxyfiles/ccrin-tags \_ /bin/bash -c /usr/bin/time -v -a -o doc/doxyfiles/cdist-tags.log /usr/bin/doxygen doc/doxyfiles/cdist-tags &>doc/doxyfiles/cdist-tags.log | \_ /usr/bin/time -v -a -o doc/doxyfiles/cdist-tags.log /usr/bin/doxygen doc/doxyfiles/cdist-tags | \_ /usr/bin/doxygen doc/doxyfiles/cdist-tags \_ /bin/bash -c /usr/bin/time -v -a -o doc/doxyfiles/cloc-tags.log /usr/bin/doxygen doc/doxyfiles/cloc-tags &>doc/doxyfiles/cloc-tags.log | \_ /usr/bin/time -v -a -o doc/doxyfiles/cloc-tags.log /usr/bin/doxygen doc/doxyfiles/cloc-tags | \_ /usr/bin/doxygen doc/doxyfiles/cloc-tags \_ /bin/bash -c /usr/bin/time -v -a -o doc/doxyfiles/cmfg-tags.log /usr/bin/doxygen doc/doxyfiles/cmfg-tags &>doc/doxyfiles/cmfg-tags.log | \_ /usr/bin/time -v -a -o doc/doxyfiles/cmfg-tags.log /usr/bin/doxygen doc/doxyfiles/cmfg-tags | \_ /usr/bin/doxygen doc/doxyfiles/cmfg-tags \_ /bin/bash -c /usr/bin/time -v -a -o doc/doxyfiles/cmfgbase-tags.log /usr/bin/doxygen doc/doxyfiles/cmfgbase-tags &>doc/doxyfiles/cmfgbase-tags.log | \_ /usr/bin/time -v -a -o doc/doxyfiles/cmfgbase-tags.log /usr/bin/doxygen doc/doxyfiles/cmfgbase-tags | \_ /usr/bin/doxygen doc/doxyfiles/cmfgbase-tags \_ /bin/bash -c /usr/bin/time -v -a -o doc/doxyfiles/copbase-tags.log /usr/bin/doxygen doc/doxyfiles/copbase-tags &>doc/doxyfiles/copbase-tags.log | \_ /usr/bin/time -v -a -o doc/doxyfiles/copbase-tags.log /usr/bin/doxygen doc/doxyfiles/copbase-tags | \_ /usr/bin/doxygen doc/doxyfiles/copbase-tags \_ /bin/bash -c /usr/bin/time -v -a -o doc/doxyfiles/crmsfa-tags.log /usr/bin/doxygen doc/doxyfiles/crmsfa-tags &>doc/doxyfiles/crmsfa-tags.log | \_ /usr/bin/time -v -a -o doc/doxyfiles/crmsfa-tags.log /usr/bin/doxygen doc/doxyfiles/crmsfa-tags | \_ /usr/bin/doxygen doc/doxyfiles/crmsfa-tags \_ /bin/bash -c /usr/bin/time -v -a -o doc/doxyfiles/cruntime-tags.log /usr/bin/doxygen doc/doxyfiles/cruntime-tags &>doc/doxyfiles/cruntime-tags.log \_ /usr/bin/time -v -a -o doc/doxyfiles/cruntime-tags.log /usr/bin/doxygen doc/doxyfiles/cruntime-tags \_ /usr/bin/doxygen doc/doxyfiles/cruntime-tags
The parallel command executes one command for each available CPU thread. As each command finishes it executes the next command until they're all done. The result is that the system is getting the job done at the maximum possible speed. On an i7 CPU at 2ghz Gendocs will generate the documentation for the 14M+ lines of JDE BSFN code in about 75 minutes. Uncompressed it's about 6gb of output.
Installation & setup of gendocs
You can download a copy of the Gendocs script, including the template files, here:
If you are working using a Linux development environment you can simply just download my Gendocs script and run it. You will need to install the php-cli, parallel and doxygen packages.
If you are working in Windows you will probably have a hard time finding something like GNU Parallel. That's okay - it will take a bit longer to generate the documentation but it will work. Of course you will need to edit the Gendocs script a little to get it to run.
I think the biggest problem in Windows is that php is not commonly installed in that operating system. You might want to install XAMPP or try a work-around like manually creating the 96 doxygen configuration files for the 48 subfolders - that would be tedious but it wouldn't take too long and it would work.
In all cases you will want to get Doxygen up and running on your system. Don't forget that you might want Doxygen to run DOT which is part of the GrahpViz package.
Next, copy one of the environments out of your Enterprise Server and into your development workstation. I used the DV910FA environment at C:\JDEdwardsPPack\E910_1\packages\DV910FA on my Enterprise Server. Note that your server may use different folder names and you might choose to pick another environment - it really shouldn't matter which one you pick.
The source code contains lines that include the jde.h header file - but there is no jde.h in the environment include folder. A copy of this file is available in a folder called system/include in the Enterprise Server. Find this folder and copy it to the folder into which you copied the environment folder.
Click on the lightbox-enabled image on the right to see the full folder tree.
If you are working in Linux there will be a case sensitivity problem with the file name, JDE.H, in the system folder. In Linux the references to jde.h will not be found so I added a symlink:
$ cd ../system/include $ ln -s JDE.H jde.h $ ls -l jde.h jde.h -> JDE.H
Next: Create a doc folder within the environment folder. Unpack the gendocs files into that folder. The Gendocs script, as it exists now, will create a folder for tagfiles and another folder, doxyfiles, for the various templates and log files that it will generate. It will also create an HTML folder into which Doxygen will write the doc files.
Finally, check the gendocs, tags-template and main-template files:
- Check the various references in the gendocs script. There are a number of defines at the top of the script which should be fine on many systems. Unfortunately I've only tested this script on K/Ubuntu 13.10 so it's a good idea to check for any adjustments that might be needed for your workstation.
- Check the main-template and tags-template file settings to make sure they meet your needs. Adjust the settings and fix any path references that you need for your system.
Finally, don't forget to set the eXecute bit on the gendocs script in the doc folder. The script should be ready to run at this point so go ahead and try it:
$ ./doc/gendocs ------------------------- gendocs ------------------------- Sanity check...done. Usage: ./doc/gendocs [tags] [docs]
If all is good it should display a usage note and stop. If you get any error messages you will want to correct the problems before you proceed.
To generate the documentation run the following command from the environment folder. The command and the screen output is shown here (edited for brevity:)
$ ./doc/gendocs tags docs ------------------------- gendocs ------------------------- Sanity check...done. Generating Tagfiles: Executing /usr/bin/time -v -a -o doc/doxyfiles/doxyfile-tags.par.log /usr/bin/parallel -v <doc/doxyfiles/parallel-commands-gentags.txt &>doc/doxyfiles/doxyfile-tags.par.log: /usr/bin/time -v -a -o doc/doxyfiles/cconvert-tags.log /usr/bin/doxygen doc/doxyfiles/cconvert-tags &>doc/doxyfiles/cconvert-tags.log ... /usr/bin/time -v -a -o doc/doxyfiles/cw-tags.log /usr/bin/doxygen doc/doxyfiles/cw-tags &>doc/doxyfiles/cw-tags.log Generating Documentation: Executing /usr/bin/time -v -a -o doc/doxyfiles/doxyfile.par.log /usr/bin/parallel -v <doc/doxyfiles/parallel-commands-gendocs.txt &>doc/doxyfiles/doxyfile.par.log: /usr/bin/time -v -a -o doc/doxyfiles/calloba.log /usr/bin/doxygen doc/doxyfiles/calloba &>doc/doxyfiles/calloba.log ... /usr/bin/time -v -a -o doc/doxyfiles/cw.log /usr/bin/doxygen doc/doxyfiles/cw &>doc/doxyfiles/cw.log
If you have a temperature guage on your desktop you will see it jump up. If you are using a notebook PC you will hear your system fans running at full speed. This will continue for a while before TAG files begin to appear in the doc/tagfiles folder.
At some point during Phase II Doxygen will start generating documentation into the doc/html folder. You can point your browser to one of the output folders to see the documentation that has been generated. If you have a reasonably speedy workstation the Gendocs script should finish in a couple of hours or so and the links within the documentation will all be working.
Later, if you add some notes to the source code, you will want to run the above command again to update the line numbers in the tag files and generate documentation that includes your notes.
However, if you edit only the template files, you may want to just update the documentation without updating the tag files. In this case you can speed things up a bit by asking for only the docs to be generated:
$ ./doc/gendocs docs ------------------------- gendocs ------------------------- Sanity check...done. Generating Documentation: Executing /usr/bin/time -v -a -o doc/doxyfiles/doxyfile.par.log /usr/bin/parallel -v <doc/doxyfiles/parallel-commands-gendocs.txt &>doc/doxyfiles/doxyfile.par.log:
For those of you who are running in a Linux environment the results should be fine as long as the tools are properly installed in the system and the various references are correct (ie: paths & folder names in the Gendocs script and the Doxygen configuration files.) It's a good idea to check these details first.
Note that the folder paths are all relative to the environment folder. In other words the paths are written with the assumption that the Gendocs script is being run from the environment folder (ie: the DV910FA folder in my case.)
Next you will want to review the contents of the many log files that are generated as the various commands execute. The log files are in doc/doxyfiles along with the many template files. If you see any error messages in the logs you will want to identify the associated problems and correct those.
Finally you may want to check the output options in the Doxygen configuration files (ie: tags-template and main-template.) Some combination of options may result in documentation that doesn't look right.
At one point I had a problem with the links in the documentation - they didn't work. It turned out that I didn't write the TAGFILES= option correctly. If you made changes to the Gendocs script you, too, might generate useful documentation but find that the links don't work. In this case you will want to check the Doxygen documentation for that option. Each TAG file that you list in the TAGFILES= setting should include a relative URL reference to the location of the HTML files for that subfolder, ie:
TAGFILES = tagfile1=htmldoc1 tagfile2=htmldoc2 [...]
See also the generate_doxygen_doctemplates( subfolder ) function in the original Gendocs script.
The resulting list of tagfiles looks something like this:
TAGFILES = doc/tagfiles/caec.tag=../caec doc/tagfiles/callbsfn.tag=../callbsfn [...]
Remember that there are 48 subfolders in the source folder for JDE v9.10 Update 2 - so your main-template for a given subfolder will generate documentation for that subfolder and must list the 47 referenced subfolders as TAG files in the TAGFILES= option setting.
Some thoughts that come to mind include:
- Making the documentation accessible through a convenient web
server allows everybody in the office to access the documentation
from any workstation or mobile device.
If you are using Apache you can put an index.html (or a .htaccess file) into the html folder. In there you can use a redirect to send browsers to the index page of the CALLBSFN folder. (html/callbsfn/index.html seems to be the start page for the documentation.)
- You will notice that there's a button, defined in the
DoxygenLayout.xml file, called JDE Doc, on the Doxygen documentation
toolbar. It links to html/PrintQueue/bsfnindx.htm.
If you think you might want to use the JDE BSFN documentation you can unzip the PrintQueue folder into the Doxygen HTML folder and use the JDE Doc button to get to it. If you are using a Linux-based web server you will find that there are some file names with capital letters while the files on in the PrintQueue folder all have lowercase letters. You can resolve these problems in your .htaccess file or you can rename the offending files (it seems that they are all index files.)
If you don't want that button you can delete it easily from the .xml file. Look for JDE Docs and delete the <tab> tag in which it is defined.
- In my case I've been generating documentation for JDE v9.10 update
2. I do realize that update 4 is available but I don't need it just
now. Obviously, if I learn something about the code and edit it to
include my notes, I will want to merge my notes into the code provided
with update 4.
Most of the people who will want this type of documentation will probably have some experience with versioning systems like Git. As you can imagine any future code merge operation, for update 4 or update 44, will be much easier if, before you edit the code, you commit it into a local repository and branch it.
- It would be really nice to be able to share User-Contributed Notes somehow. One way that comes to mind would be to share patch files. It's not legal for people to distribute the Oracle source code but it would be legal to distribute patch files. As long as they work with a currently downloadable version of the source code it would be possible for everybody to merge each-other's notes together. If you would be kind enough to do that I would really appreciate it.
- Another item on my wish-list: I'm thinking it would be nice to convert this documentation into epub or mobi format.
While writing this article I was tinkering with the template files. Below is a screenshot of the output as it looks now. Note the JDE Docs button:
Please use the comment form (in the About Menu) to send me your questions and comments. As always I look forward to hearing from you.