Most of this documentation applies to the JPRT administrator, or person setting up a JPRT system.
For JPRT users, try the How-To guide at: how-to.html.
Check List for initial setup (also see README.txt at the top of the source tree):
Once setup (with SSH), the entire system can be stopped with jprt_stop, and restarted with jprt_start. Updates to the tests and jdks can be done by updating the master copies and running the jprt_setup* tools.
The basic concept is a driver machine (ideally a Solaris machine but doesn't have to be) that has access to lots of disk space, and a batch of client machines of various OS and hardware flavors called "clients". (Some people objected to the term "clients", well, too late now). The driver and clients communicate via a socket connection and only that socket connection. The user of the system communicates with the driver through a set of command line tools, and the user never directly contacts or uses the clients. Results are sent back to the user via email and the files are available over NFS or the network file system.
What follows is mostly for JPRT Administrators, but may serve to educate others on how the system works.
Each client machine needs to be setup per the requirements to do JDK builds, with locally installed compilers (or NFS access to compilers, although local disk copies are preferred), local disk copies of the jdks needed (both BOOTDIR and IMPORT jdk images), and a local copy of the JPRT testbase. The scripts 'jprt_setupdriver' and 'jprt_setupclient' can be used to setup machines, which is normally done once and incrementally run again as things like the jdks change (uses rsync for fast updates). These setup scripts are CYGWIN only on the Windows machines, only CYGWIN seemed to have an rsync that worked well for me.
Each client machine needs to be reasonably fast, have at least 1GB RAM and about 10Gb of local disk space at a minimum. Ideally these client machines should be isolated and only used for JPRT, use of shared machines can be destabilizing for both the JPRT system and the other users sharing the machine. But slower machines work too, and slower machines could be fine for testing only clients, the above statements are more the 'ideals'.
The driver machine won't be doing builds itself, but will be moving around large files and providing access to the source control system. In the current situation, it is the only machine that needs the Teamware or SCM (Source Code Management) software utilities, and it also needs to be able to use 'sudo' so that actions can be done on behalf of the user that submitted a job. See the "Driver Machine Setup" below.
The standard and simple way to startup and shutdown all of the JPRT system requires the use of ssh or "secure shell" in a no-password-prompt mode. This ssh setup is not easy, but is only used to ease the startup and shutdown in the tools "jprt_start" and "jprt_stop". If you manually login to a driver or client machine, you can always run jprt_startdriver or jprt_startclient and start up a system, avoiding 'ssh' entirely. So SSH is only used as a quick and easy way to completely start up the JPRT system (jprt_start) and also shut it down (jprt_stop). Once the JPRT system is up and running, SSH is not needed.
If you choose to use SSH, each USER using jprt_start/jprt_stop, and possibly the user 'jprtadm' (if you created one) will need to be setup for use of ssh (and scp). You need to run ssh-keygen for each user, and setup the ~/.ssh directory, and also prime things by doing test uses of 'ssh' to all the machine short and long names (e.g. bonsai and bonsai.sfbay.sun.com). Once this is done, ssh should be able to securely login to any machine you have a password-less setup. On Windows, the CYGWIN secure shell service had to be setup and used, the MKS version did not work in a password-less mode for me.
Formal systems may benefit by having a separate "jprtadm" user account on each machine. Although not necessary, and somewhat of a pain in the patootee, it is kind of the right thing to do. But if jprtadm is not used, the 'sudo' command needs to work for the user who starts up the driver, and the Windows machine would need to use 'administrator' or have administrator privileges.
On Solaris and Linux (do the driver first, then the clients):
# mkdir /opt/jprt/jprtadm
# useradd -c "JPRT Administrator" -d /opt/jprt/jprtadm -g staff -u 667 jprtadm
# passwd jprtadm
New Password:
Re-enter new Password:
passwd: password successfully changed for jprtadm
# chown jprtadm:staff /opt/jprt/jprtadm
# ssh -l jprtadm `hostname`
# cd /opt/jprt/jprtadm ; chmod og-w /opt/jprt/jprtadm; chmod a+r /opt/jprt/jprtadm
# ssh-keygen -t rsa Empty answers
# cp Driver's: .ssh/*pub .ssh/authorized_keys IF a client
# cp .cshrc from driver to /opt/jprt/jprtadm (avoids Linux JAVA_HOME nightmare)
# exit
# exit
On Windows machines
Setting up SSH on Windows (CYGWIN OpenSSH):
1. run ssh-host-config as administrator
Answer yes to privilege separation and sshd local user.
Answer yes to installing sshd as a service and if it wants to create
an account sshd_server (windows 2003 only)
Use same password for sshd_server as jprt machine administrator.
Use ntsec as the name (CYGWIN)
2. mkdir -p /home/Administrator as administrator
3. Run ssh-keygen -t rsa as administrator
Use default path for .ssh directory, empty passphrase
4. Go into Control Panel, Services, Adminsitrative and shutdown and
disable the MKS Secure Shell service. Start the CYGWIN sshd service.
5. Make sure the firewall allows port 22 to be accessed.
# Check non-password access from driver to all clients
# ssh -l jprtadm driver
# foreach i ( all unix client machines )
# ssh -l jprtadm ${i} pwd
# ssh -l jprtadm ${i}.sfbay.sun.com pwd
# done
# foreach i ( all windows client machines )
# ssh -l administrator ${i} pwd
# ssh -l administrator ${i}.sfbay.sun.com pwd
# done
NOTE: There has been some historic problems with parts of the
JDK build not working when the Windows user isn't exactly 'administrator'
or the Solaris/Linux user isn't 'root' (building packages maybe?).
It was either the deploy or install parts of the control build.
In any case, we may decide to bag the use of the 'jprtadm' user on Windows
and use 'administrator' on Windows.
It is assumed there will be multiple "domains" where an instance of this system might exist. The JPRT domains or instance, is selected with the setting of the environment variable JPRT_DOMAIN, but may be automatically figured out in some cases, e.g. running in the SCA14 domain might automatically use JPRT_DOMAIN=sfbay. The JPRT_DOMAIN name IS NOT case sensitive. The test machines in the sfbay domain use JPRT_DOMAIN=testsfbay. So quite often I have two instances of JPRT running, usually using testsfbay to test out a newer version. The testsfbay domain uses shared machines from the Serviceability group, which demonstrates that JPRT can use arbitrary machines.
At runtime, any new machine can join the system. All the machines available can always be seen with a 'jprt_status -v' command.
See the files sbin/jprt_config* for the existing pre-configured JPRT_DOMAIN settings. They provide good examples. These configurations are mostly for the sake of jprt_start/jprt_stop, although some of these settings are needed by all the tools, like where the archive area is.
The machines can have any names you want, but a convention of jprt-web for the driver machine, and jprt-{solaris,linux,win}-{i586,sparc,x64}-* for the client machines is recommended. This way we can automate the startup in jprt_start/jprt_stop by finding all the machines with these names from the 'ypcat hosts' list (if we wanted to).
The driver machine (normally a Solaris machine, ideally a Solaris 10 machine with quick access to the archive area), should have:
chmod 4111 /opt/sfw/bin/sudo
/opt/sfw/sbin/visudo
Add the line:
jprtadm ALL=(ALL) NOPASSWD: ALL
The client machine setup procedures:
setprop ata-dma-enabled '0'
to
setprop ata-dma-enabled '1'
Job state changes are communicated via email to the user (submittor) and email aliases which can be configured with properties. The default system settings will try and send email to jprt-admin@sun.com when the system itself has problems, and send email of all major PRT pass and fail events to jprt-notify@sun.com.
The email aliases used are all NetAdmin email aliases: jprt-announce@sun.com, jprt-notify@sun.com, and jprt-admin@sun.com.
The system can be configured with properties, but not directory pathnames or machine names, see the sbin/jprt_config* files for machine/directory configuration. The basic properties are mostly driver/client/tool and platform independent and are in the file JPRT/jprt.properties. Domain specific or test configuration property files are also in JPRT called jprt.JPRT_DOMAIN.properties.
The distribution of the jprt software is in a 'dist' directory which is composed of a couple of jar files and a bin directory full of shell scripts. If javadocs were created, then they would also be in the dist directory along with all the documentation (including this file).
There is one primary jar file called jprt.jar, and one support file for xml called jdom.jar (which may be removed someday). Then there is a set of 'sh' shell scripts in the bin directory, one script per tool, and most have a matching Java main() class in the JPRT.tools package. These should work on all platforms, however on Windows thay may need to be run with a full path, e.g. C:/jprt/dist/bin/jprt_XXXX. Obviously job submits must be on a machine with teamware access or SCM utility access, so your ideal platform to submit jobs is usually Solaris, but not restricted to it
The set of tools provide the users ways to submit jobs to the driver which in turn will have the job done on the various clients, and the job data is saved away in a disk archive.
It must be run with JDK 5 or newer.
The system will allow user jobs to putback changes from your formal source tree to the parent formal source tree if requested, but all builds and tests must pass before this is allowed. A putback job is treated specially if it has conflicts with either older jobs in the queue or the parent source tree. If conflicts are detected at submit time, a "warning" email is sent to the submitter; no action is required at this time. Once the job reaches the head of the queue, a "notification" email is sent to the submitter. The submitter has 1 hour to resolve conflicts before the job is failed. After resolving conflicts the command 'jprt_continue jobID' must be used to notify JPRT that conflicts have been resolved. When the JPRT queue is performing a putback to a formal source tree, it needs to make sure that any other queues are not in the midsts of performing a putback on that same formal source tree, else we could end up with conflicts during putback or incomplete testing. In order to eliminate this possibility, the driver process will lock the parent source tree when it is working on a putback job by creating a file called jprt.lock (or prt.lock) in the Codemgr_ws directory of the parent source tree. It deletes the file when the job has completed. If, when attempting to lock the directory, the driver process determines that a lock file already exists, it will stall the job until it can create that file itself. While a job is running, the lock file's modification time is repeatedly updated by the running job, so that any waiting queues can detect that progress is being made. In the event that the running driver job unexpectedly exits or locks up, and does not delete the lock file, any other JPRT queue can detect that progress is not being made by examining the modification time of the lock file. If a queue detects that progress is not being made, and is waiting for that parent source tree to become available, then it will send out a "stuck job" email to the JPRT notify alias and the job's submitter. The JPRT administrator should verify that the source tree is not being actively used by a remote queue, and then should delete the lock file manually to "unstick" the waiting queue.
There is a test database for PRT that JPRT has just copied and used "as is" to start. This layout is mostly undocumented, but is rather simple and the current plan is to continue to use this exact same layout for now. The PRT test database contains Java applications in sub-directories, with a configuration file that creates unique testnames for various runs of this Java application. So each Java application could have any number of testnames that use it. It's these testnames that form part of the test target names (TestTargetID) and also the TestTargetStatus objects.
Each directory represents a Java application with everything needed to
run this application.
One file will identify all the testnames that use
this application.
This configuration file is called PRT-Test-Config.txt
and can contain blank lines and lines starting with '#' which will be
ignored.
The remaining lines will contain 4 comma separated items:
For example, for the jvm98 spec application:
jvm98,2400,,SpecApplication -s100 -M5 -m5 -g -d3000 -a _200_check _227_mtrt _202_jess _201_compress _209_db _222_mpegaudio _228_jack _213_javac
See jprt_setupclient for how a client machine gets it's copy of the test database, and it is important that each client has it's own copy.
Is pretty straight forward given the above information. However at this time it's assumed that the test is a simple and single Java application being run, and although it seems natural that this could extend to any kind of testing, that has not been proven.
Some of the issues are centered around the fact that having more than one PID to manage in the case of failures can be a problem. Core file creation on a hung test and killing a hung test would need to change. Also, on Windows, killing a hung test that involved multiple processes is problematic.
The JPRT User How To Guide: how-to.html.
For the current status on JPRT development and source changes see currentStatus.html. (Also search for the pattern "F I X M E:" without the spaces, in the source).
For information about the original hardware requirements, see resourceRequirements.html.
For information on the original JPRT plan, see originalPlan.html.
For information on using NetBeans, see usingNetBeans.html.
For information on the original concept of making the original PRT more generic, see genericIdea.html .