ARCHIVE: Java SE Automated Change Submission Process Plan

Updated 8/31/2005

We need some way to automate the on-demand need to build and do a qualification test of the JDK on all platforms. This document represents the plan for an automated Java SE change submission system.

The Problem

Currently there are 8 different platforms that the JDK product must be built on, 2 on Linux, 2 on Windows, and 4 on Solaris (SPARCV8, SPARCV9, i586, x64). And for each there are two types of builds that are done, a product build and a fastdebug build. Further, there are a multitude of ways to build the JDK, all the way from full RE control builds, control builds, partial control builds, j2se source tree only builds, to partial source tree or incremental builds. Each of these platforms has specific OS release, patch, service pack, or software installation requirements. Every JDK release can also have slightly different OS or compiler version requirements, further complicating things.

Besides the dozens of engineers having to setup these machines for themselves or their teams, everyone has to be able to use them to make sure changes build prior to putback. Technically, developers only making Java changes aren't required to build on all the platforms, but that loop-hole, along with various other build procedure loop-holes everyone use in an attempt to minimize the overhead and meet their schedules, has created an unpredictable build status for many integration areas. Some teams don't have a complete array of build machines, some have rather antiquated and slow hardware. So in general the overall situation, and end result is rather inefficient, haphazard and can be problematic. The diligence and dedication of the JDK engineering teams in avoiding any major disasters in the face of this, is in my view, quite amazing.

In general, solving this problem is probably the most effective way to improve the existing JDK developer productivity level. Here is at least a partial list of the specific reasons why implementing this plan makes sense:

Some Background

The Hotspot team has had imgr and PRT as source tree integration tools for quite some time, and if you ask any of these engineers what they would do if you took it away they would probably look at you in complete shock and assume you were nuts. Nobody is saying PRT is perfect, but it is for the most part a functioning system that is providing productivity benefits to their entire team. Other benefits include a history of built bits (saved by PRT), ability of testing and the quality teams to access the latest build of any given team, ability to investigate regressions (performance included) down to individual team member putbacks, ability to test build or dry run your changes without a putback, and a consistent set of putback comments for integration into higher level source trees.

PRT hasn't solved all the issues, but in my view it has solved the build issue very well, and could potentially solve more. Getting more use of PRT will likely get us on the road to making it better.

Choices

I'm obviously biased toward PRT, but I think it's justified. I won't completely rule out a different mechanism, but I have a hard time thinking that doing anything but extend PRT makes any sense at all. It IS my intention that whatever is done to PRT to make it do JDK builds/tests become part of the PRT sources, I do not want to copy PRT and create PRT-clone, we need to evolve the existing PRT, not create a new and different beast.

Plan for a JDK capable PRT system

PRT allows for separate instances, we have a PRT east and a PRT west, two different labs, two sets of machines, so the current thinking is that we would start with an experimental PRT-JDK west instance with it's own set of machines to start. Target j2se source tree putbacks to start, with minimal testing qualification to start.

As the project progresses we can determine at a later date if it makes sense to have a single PRT west system and/or what they would want to do with PRT east. If done right, we can allow for any PRT system to build hotspot or JDK, but the logisitics and needs of the VM teams may be a basis for keeping these systems separate, that remains to be seen. At a minimum the various PRT systems can serve as backups to each other, just as PRT-east and PRT-west do now. Eventually all changes to the PRT source based should be putback to the master PRT source base, depending on the level of PRT changes for Hotspot, this sync up must be done eventually, but we do not want to break the existing running PRT systems. PRT was initially designed to allow for a JDK build, but was never implemented.

I've been told that experiments with the PRT source base could be done with one machine doing everything, and limiting yourself to one platform. That's probably how the PRT source changes will be started.

Other concerns about using PRT:

What needs to be done?

Future Considerations

Items to consider once we have an operational system: