Java Source - last updated 08.13.06

Although going by Using Java compiler in your Web Start application my original ClassLoader approach should of worked for compiler tools. Figure that out some other time.

It also uses the AppBundler JavaAppLauncher executable so my own ClassicJavaAppLauncher is currently not included

(Updated 01.31.13) Project source, resources, documentation...
halfpipe.zip

openjdk java launcher XCode project
launcher.dmg

(Updated 05.13.12) Project Apple Java SE 6 application...
halfpipe.dmg

(Updated 01.28.13) Project openjdk Java SE 7 application...
halfpipe7.dmg

PDF documentation, if you just want that and not everything else in halfpipe.zip above Half-Pipe.pdf

extras related added 05.13.12, for now just the jar.
extras.dmg

The first part of this is my finally returning again to doing something along the lines of what I outlined here. Re: JDK 1.7 & listing MacOS X file metadata? [was Re: OpenJDK 1.7]

No update 08/26/12. The new neglected project. A little thought into how to get the kqueue version to figure out a OVERFLOW condition but no resulting code. I'll come up with something as soon as possible. Haven't lost interest in the project.

Am still tempted to try and get a new release of Election Games out before the election though. A lot of code could be worked on, not a lot of time to do it.

Updated 08/12/12

Not much of an update here. Started to frame out some of the fsevents code. A little time on the kqueue jdk LotsOfEvents. Got to where it legitimately errors on the right WatchService, thats it.

Mostly time this week went to the AppEdit code following this

Updated 06/24/12

Keeping the ongoing project version here. Leaving the home page version as the 'stable' one.

Versions are currently the same for the kqueue WatchService beta.

newtrz.zip

Home page version

Mac meta file attributes

Updated 05.10.13 - Fix for string value used for ClassPath in the Apple JVM plist instead of the expected array. What the types of some of these keys should be is difficult to determine from the documentation. VM options also accepted a single string besides the documented array of strings. For the application in question this resulted in a ClassCastException which crashed the app showing no error, the Application Support folder didn't display and nothing was logged into convert.log. A Throwable catch was added to avoid that. Also should now copy over a Contents/_CodeSignature file.

One problem that couldn't be addressed was resource and plugin directory jar files contained in the Apple JVM /Resources/Java directory. My code would have no standard location to put them in a openjdk app and no way of knowing how the application might be handling them to aid in placement. I thought about blogging on resource file handling in OS X java app's, or posting questions on the AppBundler or macosx port lists, on what might be any kind of a standard way to handle application resources that could be the same across apps, maybe easing development and tool support, but I have decided not to for now. This might not be an issue that is of concern to many others. If you do have any comments on this or any kind of application related feedback you can again send it to me at...
Feedback EMail: contact

Updated 12.11.12 - Version with an embedded JRE. Embedded is recommended and this should work if shared JRE isn't available I think.

Updated 12.03.12 - Plist parse bug fix, improved error and resource handling

Updated 09.14.12 - Multiple options for String value VMOptions

Updated 09.13.12 - $APP_PACKAGE and $APP_ROOT improved support

This attempts to replace any occurrences of $APP_PACKAGE with the appbundler $APP_ROOT. This includes post-copying the $APP_PACKAGE referenced resources to the new application bundle. It makes no changes for $JAVAROOT at this time. It now includes a simple test application as well to exercise some of the '$' macro support.

This may be the last update of this for a while unless someone suggests changes that seem to make sense. No suggestions, but a reported problem got the 12.03.12 update done. Finishing up on the reported problem adds the 12.11.12 update.
Feedback EMail: contact

The download

AppConverter.dmg

AppConverterJRE.dmg Embedded JRE version

extended.jar [OPTIONAL 07.17.05] is sort of the ClassLoader and search engine parts for standalone testing outside the application.

Combined.zip. The application itself.

Latest Update 12/3/6
Not a lot of progress. But uploading what there is.
Actually built on "ROME, the premier newsfeed parser library for the Java platform"*
- RSS And Atom In Action, David Johnson
At some point I am still planning on including search engine type support for personalization by allowing selection of content of interest. The intent for now is to base that on Lucene.
I also added in some code indicated on the OS/X java-dev list at The Rabbit Hole

Anyhow, its still interesting so I will probably put more time into it.

1.6 SHOCKER!
It appears that as of java version 1.6 the ClassLoader techniques I used to substitute for the standard java.io classes will not work. So TRZ is less aptly named as far as Transparent goes.
Now TRZ won't be based on a hack and will just have to stand on its own merits as a more powerful, flexible and enhanced API for dynamic zip file access
For my IDE code I am looking at using the new 1.6 provided javax.tools classes. This should still allow TRZ to be used with javac to directly compile to and update zip/jar archives. The other choice would be actually modifying the javac jdk code. This seems less a public API and less supported by Sun. The long term best solution I am hoping Sun will provide for me still seems to be officially supported pluggable filesystems. Then I just make TRZ one of those. See the new comments on JSR 203 farther down for where that seems to stand.

Older TRZ related reading
The following was pointed out on the MRJ list, somewhat of interest to me in relation to Transparent Random Zip. Details some of the file concerns for the upcoming Mac OSX. Some of it familiar to most MRJ developers already, from MRJ file handling. Some of it of seemed to parallel concerns I had with TRZ.
The Challenges of Integrating the Unix and Mac OS Environments.
Transparent Random Zip is an already working version of the "bundles" referred to as opaque directories in the document but with the opposite perspective. I prefer to think I am making the file transparent rather than making a directory opaque.
Another interesting page was pointed out on MRJ List [NOTE: That this is now part of JSR 203, see the following]
JSR-000051 new I/O APIs for the Java Platform.

JSR 203
TRZ would benefit from a more 'standard' way of 'plugging' in filesystem's. For java itself pluggable filesystems are now a part of JSR 203 which is slated for the 1.7 Dolphin release. My impression is that it is again the other parts of the API changes included along with it that are of broader interest and which are seen as complicating the implementation and have caused the series of delays. My impression could be wrong, but I've roughed out a pluggable zip filesystem and it wasn't that terribly difficult.
One way or the other, official pluggable filesystems won't be seen in core java until 1.7 at the earliest.

utilsrc.zip download.
utilgl4.zip gl4java related source.

Quaqua

Bouncy Castle

The full download includes plist parsing, double-clicked self extracting jars, stub launching with MRJ Adapter front-end and so on are included in appedit.jar

Application for verifying OS/X java application bundle correctness is AppEditVerify.dmg

Toolkit for doing other things with application bundles. This includes the start at a application to build more complex self-extracting (OS/X) jar files. Accum for now, will probably name it File Drop at some point. It supports the file drag and drop API to a JTree. There is a neon animated image tacked on that is also supposed to accept the file drops but doesn't yet. Trying DnD and doing some kind of neon l&f controls a couple things I've thought about trying in the past. All of this still very simple at this point. Definitely seems to work well enough for wrapping a singe .dmg, see DocumentMonster below as well.

Download
app_tk_SEA.jar

Temporarily trying to keep more current changes here until I get a better idea of cvs updating the SourceForge project

The source is here

Current work in generation of a Cocoa Java type interface - OS/X Terminal usage would be something like...

Update 09.25.05 I probably should mention here that my current JNIDirect project is trying to automatically generate as much of a interface to the Cocoa API's as I can manage. Last changes in brief, handles more non-'int' types better.

Successfully running some Cocoa things off the generated, JNIDirect and jnilib. Bundling it all up in a jar now jnicocoa.jar. That includes a README in the 'jni' folder so more information there now instead of here

The XSLT file ant.xsl that was just mentioned, as .txt.

It links to the ant documention for the tasks and file types. You can change the URL root to local disk in the ant.xsl here...

<!-- Location of ant documentation, change to local disk if you prefer -->
<xsl:variable name="root">
  <!-- <xsl:text>file:///???/apache-ant-1.6.5/docs/manual/</xsl:text> -->
  <xsl:text>http://ant.apache.org/manual/</xsl:text>
</xsl:variable>

If you have any ant build's that this doesn't display correctly or well, let me know and I'll see if I can improve this.

NSAppleScript execution and possibly other Cocoa Java classes should be run on the AppKit 'main' thread. OnMain.dmg provides a small JNI stub to do a callback to java on the main thread that should provide this.

10/28/2006 - More new functionality. Added Some JNI, generated of course, execution of the AppleScript related

10/22/2006 - Added some additional AppleScript testing and a couple execution option switches. More or less documented in the README file.

Download: ls.dmg

NEW - 10/15/2008
The JNIDirect SignatureToApp class has been one of my own favorites going back to when it supplied some enhanced Classical JDirect MRJFileUtil capability. The following is a straight JNI version showing how JNIDirect might still work as a JNI 'code generator'. I have trimmed out some of the unnecessary dependencies and have gotten the size of jnilib plus jar well below 300K. I think theres enough API for this to be reasonable overhead

There is nothing wrong with the OS X 'open' command. But it isn't extensible and it isn't open source. Such as it is, this is those things. It does add a couple bells and whistles already that 'open' doesn't do and more functionality could reasonably easily be added.

But for this one, for now, this is it. Enjoy if you wish.

Download: ls_jni.dmg

Viewer for Mail.app signatures with osascript Runtime exec of AppleScripts and the BouncyCastle API's for handling the digital signatures. I had added this to my Combined application but it probably makes more sense as a standalone demo.

Updates

Changed to use current Bouncy Castle crypto jars and now bundle them in the download.

Fixed incorrect 'return' character included in the AppleScript for mailboxes with multiple accounts.

Fixed name on certificate information display dialog

I Included a launching script with AppleEvent debug environment variables that I didn't get to work right.

Download is now a dmg

Mail.app still doesn't seem very good about showing this information itself. I filed a bug this time (ID 6460099). I also put in that Mail.app seemed to require two accounts to set up a signature. I found decent signature set up directions for Mail.app signing certificates here.

---

AKS and primes - 01/04/05

Previous of the coding blogs. Follow up curiosity to the MersenneTwist related - sort of.

First, since this is supposed to be a blog of sorts, maybe I should have something sort of anecdotal. Although blogs of the diary type, 'today I stubbed my toe' or whatever, usually seem pretty uninteresting to me.

The year is sometime in the very early 70's. I take the old fashioned telephone handset and stick it in the coupler that passed for a modem at the time. My high school math teacher sticks his head in the small closet sized room housing the dialup network equipment, the coupler and a teletype printer is about all of that, that I'm remembering, and asks what I'm doing. I'm not sure if this is simple curiosity or he suspects me of being the first hacker trying to break into the defense department or in some other way up to no good. I tell him just running my (Basic) prime number program.

So, this is one of my first memories of anything 'computer' - with prime numbers being about the first thing I remember that interested me related to computers. So my interest in prime number programs has a little history and some roots anyhow.

Ya know, I'm starting to sound a little like grandpa Simpson on the Simpsons.

Anyhow the main curiosity was could some real prime number crunching with the current state of the JIT be done as quickly as the same algorithm could native compiled. Alas, the answer so far is that compiled is 4x faster at least as far as my first pass programming skill goes. The code is number.jar if of interest.

A few of the related links I came across if any of those of interest

AKS Primes is in P

Primes

Mersenne Primes

For fast square roots

More on elementary functions

Wouldn't be complete without considering rational approximations

Some floating point considerations

Yet more functions

A floating point TechTip

Some of this from a more crypto viewpoint

A presentation of how algorithm efficiency, time, complexity, what have you is measured

The internet is in my opinion a tremendous resource. The truth is out there. Sometimes it takes some sifting, sometimes a fair of amount of slogging through, and sometimes you still aren't sure you found the absolute TRUTH. But there's a lot of good, free info and stuff. Really, in my opinion there should be courses just on researching the internet. As it is I'm remembering some interesting pages that are not included above. How does the expression go that the journey is more interesting than the destination. Tracking down these things is usually like that, you almost always find something at least equally interesting along the way to what you're looking for. Thats part of what these code blogs are supposed to demonstrate I guess.

A new coding blog. I became curious concerning questions of the speed of java.util.Random that came up as a topic on the BouncyCastle mailing list (Sign-up URL below with PDF stuff if interested). Sean Luke, who sometimes frequents the OS X javadev list, had done a java replacement for java.util.Random based on the MersenneTwist random number generator. For my coding I grabbed the version included with ECJ although it can be linked to more directly it seems here?.

Mersenne Twister is not cryptographically secure. (MT is based on a linear recursion. Any pseudorandom number sequence generated by a linear recursion is insecure, since from sufficiently long subsequence of the outputs, one can predict the rest of the outputs.)

To make it secure, you need to use some Secure Hashing Algorithm with MT. For example, you may gather every eight words of outputs, and compress them into one word (thus the length of the output sequence is 1/8 of the original one).

So that is how I proceeded. I decided to first see if native JNI provided any significant performance differences and if performance with a hash included would still be on a par with Random.

Sample numbers arrived at for Sean Lukes 100000000 generated random int's speed test were as follows...

Times in milliseconds, so 19440 is 19.440 seconds. Fast is the everything in JNI version. Cokus is a suggested optimized version from the Mersenne Twist page. Returned state means the 624 int array Mersenne Twist state is returned in one chunk instead of a int at a time. SHA1 w/ state means that first I hash the 624 int array already mentioned and then return the 390 element int array result as one chunk. I translated the BouncyCastle java SHA1Digest class to C for the Sha1 hash.

Something possibly to note would be that unless the state is returned the JNI versions use significantly more time for all those invocations. If state is returned - that is the 624 element int array for the MT state, or the 390 element array for the hashed result - accesses are then done against that until it's used up when another JNI invocation becomes necessary. 'Returned state' thus resulting in considerably fewer JNI invocations, where all that is usually being done is returning an already computed int value. If you get the state all at once you get considerably better time because you are calling JNI considerably fewer times.

The other thing maybe to notice is that if state is returned even with the SHA1 hash the times are faster than Random. This somewhat surprised me but I think may mean this wasn't a totally pointless exercise. Anyhow it isn't complete. I would like to appy some of the standard RNG tests to my hashed results to see if they still seem to be good RNG's. I'd also like to consider what is mentioned on John Savard's page concerning the Mersenne Twister and it's use as a keystream generator for a stream cipher. To throw in one last link on this rather link intensive blog, pLab - a RNG site or so I gather - mentioned AES makes a nice RNG which I find interesting and might look at.

I guess if you're still reading this you might be interested in mt.zip the code

BouncyCastle and iText PDF. Updated 03.01.04. Added PDFSign and related

The topic of signing/verifying PDF documents came up on the BouncyCastle mailing list. It seemed sort of interesting to me so I started seeing what I could do with it. Not as easy as it sounds - for me anyhow. Currently I can verify for the /adbe.pkcs7.detached signatures that are included in PRAcroForms. At least I assume what I have would successfully work for any such setups. Testing is limited so far to only a few signed PDF documents I have found. Left to do, I have verifying if not PRAcroForm included, signing support besides verifying, and support for the /adbe.x509.rsa_sha 'raw' signatures.

The first document I've been testing with is xsignarticle.pdf here. This document uses the PKCS7 detached and is included in the PDF PRAcroForm field. What I have should handle it.

The second document is Adobe provided from here. The income tax form. This I do not handle yet but hopefully will shortly. Assuming theis pae is somewhat current this may actually be assumed to be the current recommended method of handling signatures?

Anyhow, what I have currently is PDFVerify to verify

03.01.04 I have somewhat simplistic signing working. That is I can attach a signature to a document I am creating and PDFVerify will verify the signature as valid. AcrobatReader 6.0 will not. It will indicate the presence of the signature but doesn't show it as default certificate type as it shows it's own signed ones. And it doesn't enable validity checking itself of the signature as it does it's own.

Anyhow the stuff included in as far as I've got...

PDFSign.java.

PdfSignature.java modified version of the iText class.

PdfHexString.java an iText addition of my own for this. These seem to get handled elsewhere correctly without requiring this class but I haven't determined how and it seemed simple enough to add one.

What may also be of interest, located here for the moment until I figure out where to leave them.

I used this tweaked iText PdfLister.java for parsing practice

And this modifed BouncyCastle code, ASN1DumpLabeled.java to try to figure out what I was parsing

Thought there was more. But I guess that's it for now

astro.jar

download

...because 0 lengths crashed it. Might be files with variable bit rates mentioned as not handled somewhere. I put the following MyStation to the same, code base, directory as the sample jar file. I started up iTunes, did an "Open Stream" for http://localhost:8000 and there you go. Sounded as well as iTunes usually does.

There are probably more sophisticated options out there than this by now and it's probably sort of naive of me to think gosh thats kind of cool. But if anyone else has or does do any tinkering with this - I'd be interested in hearing about it.

stackWalker

Test Cases

Finish the year with a little code exercise inspired from this bouncy castle list thread.

Testing showed that it spent most of it's time in the BigInteger prime number constructor. So to improve things you need to improve BigInteger, I've messed with that, not easy. Or change the algorithm, but theres not much algorithm to change. Just the p = 2q + 1 equation - where q is found, and p is determined where both need to be prime. However, it occurred to me the equation could be applied twice. As done above, and then you can take your first prime as p instead of as q and solve in reverse for q = (p - 1) / 2.

Not real complicated math but it does give you another chance at a solution without having to just start over.

For my testing with 1000 trials at key sizes of 512, 768, 1024 and 1280, there were 9 successful p,q's found going forward [p = 2q+1] and 5 found going backward [q = (p-1)/2]. Those results can be seen at pgptester.txt. The test's PGPTester.java source

However, given Gauss's determination that the number of primes up to any given 'n' approaches...

Prime Number Theorem
π(n) = n/log(n)

which means as I understand it that the bigger the keys you try for the sparser the primes are going to be. So a method that increases the number of successes and actually does so by testing against smaller numbers might not be all that bad an idea. Of course the math for primes this size might just be ruling the whole approach out.

A couple URL's were mentioned on the bouncycastle forum that should improve determination of these "safe primes" as they are called.
Safe Prime Generation with a Combined Sieve
and Double-Speed Safe Prime Generation
My idea is mentioned in passing, so I guess I'll have to cancel that patent application. Someones been there and done that.

Epilogue
As it turns out, this idea doesn't work anyhow. The algorithm is designed to start out with the 'q' bit length one less than what the resulting 'p' length should be so that the final 'p' bit length value is correct after the 1 bit left shift that always happens from the multiplication by two in p = 2*q + 1.
Taking the original random value as 'p' for going backward [ q = (p-1)/2 ] means that it's length is one bit too short to start out. Which doesn't work for the exact length required by the ElGamal algorithm.
So not only no patent, but back to the drawing board.
A considerably faster version for bouncycastle may be appearing without the need for it

Updated 03.19.11: Uses the new API's if available OpenFilesHandler if not it falls back to the prior ApplicationListener interface.

As far as the open document handling itself goes to the best of my understanding the following is true

For more information on using a shell script instead of a JavaApplicationStub for launch, see Greg Guerin's prelude scripts

The test case that I wrote for this is DocumentMonster.dmg It includes some .dropper file extension files for testing by dragging to the application icon. It issues println messages to Console.app indicating success. If you look in the applications Contents/MacOS folder you will see the Prelude script that may be used in place of the JavaApplicationStub. You might want to look at Greg Guerin's information, from above, before getting much farther along with that.

The download
ae_jni.dmg

The download to just cut to the chase.
stas.dmg

If you are reading this then you most likely have an interest in either Java or AppleScript on OS X, hopefully both. You might then be aware that current versions of OS X Java support the JSR 223 scripting API's for AppleScript as indicated here...

Java Development Guide for Mac OS X: Mac OS X: Mac OS X Integration for Java

Please note it's mention of the API package documentation at...

javax.script (Java Platform SE 6)

An example of it's use, complete with parameters, was provided on the Apple java-dev mailing list...

Re: Java AppleScript Parameterization was Re: Create Date in Java - using AppleScript

My current version of that script, after incorporating in a couple improvements based on Mike Swingler's mail list post above, looks like this...

tell application "Mail"
    set msg to message $msgnum$ of mailbox "$mbox$" of account "$acct$"
    set who to sender of msg
    set mimever to the content of the header named "Mime-Version" of msg
    return { who, mimever }
end tell

In my response to the JSR 223 AppleScript parameterized example I claimed three potential advantages to using StringTemplate instead. They would be...

• StringTemplate allows the AppleScripts to be loaded from a separate text file. Given this, it can be modified and maintained independently of the Java source.
  You can change the AppleScript without touching the Java and is more easy to work with than when it is wrapped with Java source.
• StringTemplate allows the structuring of it's files into StringTemplateGroup's. This provides builtin convenience for organizing the AppleScripts.
• StringTemplate is designed for more sophisticated replacements and modifications of the template file than simple one-for-one parameters.

The first two advantages I think may not be that significant unless you are doing a substantial amount of scripting. This would be more of an advantage with a number of scripts or more complex scripts. It's functionality could possibly be duplicated with normal Java I/O without a major effort.

I had been less certain of the third advantage specific to AppleScript and wanted to look at that. For that I selected an ITunes script related to podcasts. Please consider the following script...

tell application "iTunes"
    set info to {}
    repeat with cast in (every track where podcast is true and unplayed is true)
       set castInfo to {album of cast, name of cast}
       copy castInfo to end of info
    end repeat
    return info
end tell

This script will return a list of podcasts which you haven't listened to yet. It is the 'unplayed' script in the bin directory. There are no paramters, it is specific to this task. It is difficult to imagine how you might parameterize it given the multiple search criteria of 'podcast' and 'unplayed'. Unless the AppleScript parameter bindings include some sort of varargs support. What if you want to get different information back? Do the bindings support a variable number of values returned? If it does allow this please let me know so I can reflect that fact here.

Can StringTemplate do that more generically and in some way that's more reusable? Consider this version of the script...

group tracks;

picker(criteriaList,returnList) ::= «

tell application "iTunes"
    set info to {}
    repeat with cast in (every track where ‹criteriaList;separator=" and "›)
       set castInfo to {‹returnList;separator=" of cast, "› of cast}
       copy castInfo to end of info
    end repeat
    return info
end tell

»

This script is run by the 'Stas' application in the iTunes tab. It uses ArrayList collections for both the tracks selection filtering and the return information. Either or both can be varied from the Java dynamically at runtime without modifying the AppleScript.

The returned properties of the tracks is simply an ArrayList of String's, the parameter list uses a customized class that seemed might be reusably useful for AppleScript as follows...

package org.stas;

public class STASPropertyCriterion {

    private String key = null, value = null;

    public STASPropertyCriterion(String key, String value) {
       this.key = key;
       this.value = value;
    }

    public String toString() {
       return new StringBuilder(key).append(" is ").append(value).toString();
    }
}

This demonstrates how StringTemplate will use the toString() method in customized instances to do it's replacements.

One thing that came up somewhat earlier in discussing building AppleScript was the question of injection...

In reviewing some of the StringTemplate introductory material I was reminded that a similar discussion concerning it's use came up there...

Let us briefly consider what AppleScript injection might be. So, submitted for your consideration",...

group injector;

inject(app,statementList) ::= «
tell application "‹app›"
    ‹statementList;separator="\n"›
end tell

»

Unlike the first iTunes script, where there was nothing in the script that could be modified, this one requires almost all it's contents to be injected. The 'helloworld' shell script in the bin directory puts it to fairly non-malevolent use. In either case, not parameterized or all parameterized, it really seems to make using StringTemplate sort of pointless. Where I would hope you might see some value is the more generic version of the iTunes track selector script where StringTemplate made for generalized and reusable AppleScript easily accessible from Java.

This project should facilitate the use of AppleScript from Java without necessarily requiring a lot of learning curve up front. Having a pre-built collection of sample scripts that might be seen of interest to OS X Java programmers could also encourage the popularity of the rather platform specific scripting language. I plan to continue support of this interface in that hope.