jpackage on OSX

Directions indicate that you should obtain a app-specific password for the app. Going in through my Apple Developer account didn't seem to show the option. I finally came across the link appleid.apple.com where the "App-Specific Passwords" option is right there.

To avoid having the password in scripts or entering it all the time you can add it to keychain with...

xcrun altool --store-password-in-keychain-item "AppName" -u "user@somewherecom" -p ????-????-????-????

The notarization process prefers to work with a zip you can get with something like...

/usr/bin/ditto -c -k --keepParent "outputdir/HalfPipe.app" "outputdir/HalfPipe.zip"

open -Wg MyApp-1.0.dmg
#cp -r /Volumes/MyApp/MyApp.app outputdir/MyApp.app
ditto /Volumes/MyApp/MyApp.app outputdir/MyApp.app

OK, now you're ready to notarize...

xcrun altool --notarize-app \
     --primary-bundle-id "whatever.id.you.want" \
     --username user@somewhere.com \
     --password "@keychain:MyApp" \
     --file outputdir/MyApp.zip

This uploads the request to a notarization server. If successful you should see something like...

No errors uploading 'outputdir/MyApp.zip'.
RequestUUID = ba966be7-e5ec-4380-8fa6-8bdb7e3c5fa1

xcrun altool --notarization-info ba966be7-e5ec-4380-8fa6-8bdb7e3c5fa1 -u user@somewhere.com -p "@keychain:MyApp"

    Status Code: 0
Status Message: Package Approved

Now you have to staple the result to your application. The difficulty here I realize is that I've built a read only dmg. How do I staple to it? Make it read/write.

First I rename the file so I can make it read only again back to the original name.

mv MyApp-1.0.dmg MyApp_tmp.dmg

hdiutil attach -owners on MyApp_tmp.dmg -shadow

expected    CRC32 $AF8EB328
/dev/disk3            GUID_partition_scheme
/dev/disk3s1         Apple_HFS               /Volumes/MyApp

xcrun stapler staple "/Volumes/MyApp/MyApp.app

hdiutil detach /dev/disk3
hdiutil convert -format UDZO -o MyApp-1.0.dmg MyApp.dmg -shadow

xcrun stapler validate "/Volumes/MyApp/MyApp.app"
Processing: /Volumes/MyApp/MyApp.app
The validate action worked!

spctl -a -t exec -vvv /Volumes/MyApp/MyApp.app
/Volumes/MyApp/MyApp.app: accepted
source=Notarized Developer ID
origin=Developer ID Application: My Signing Cert Info (XXXXXXXX)

Except this got kind of strange. The exact same attach command gave me usage errors. I finally found renaming the file again got around this. I later noticed .shadow files were being created. Having these already there and not cleaned before repeating might of contributed to the errors. The staple against the mounted dmg got errors. The application version in outputdir looked valid so I did a 'rm -r' against the read/write dmg and copied in the outputdir one. It verifies as shown so made it work anyhow.

Not currently a real simple process but it works. Good luck if you try it.

TL;DR

I decided to restore some of the AppleScript functionality to the HalfPipe application. As it turned out this exercises a few things for jpackage

NOTE: A current, 08/24/21, issue concerning jpackage and permissions/entitlements (/notarization?) is JDK-8272639.

JNI

The application has previously supported JNI. There is nothing currently new related to jpackage concerning this although the java.library.path issue is still in effect until JDK17.

There are potential issues for older JNI that makes use of Apple's old Java Native Foundation (JNF). That would include mine. Possibly I was able to continue using this for a while from java Frameworks on my old machine because I had it legacy. It now appears I have to get it through the XCode SDK. A 6 year old macosx-port, when java went from Apple to openjdk, email indicates this to be the case. A link to that email and a github project that still seems to provide JNF is mentioned in the compile here hp_compile.

I think I noticed the AWT group has eliminated their own use of JNF they inherited from Apple. It's future definitely seems uncertain.

The source is hp_jni_src.zip.

The additional JNI is AEPermit.m to invoke AEDeterminePermissionToAutomateTarget. Requesting user automation permission for the application. There were a fair number of Google hits concerning this method at the OSX Mojave release. After putting a fair amount of time into this I'm not that sure it's really necessary at OSX Catalina. More recently once everything is in place I have been testing with some changing of my signing certificate. This has resulted in the dialog requesting the permission whenever the code requires it. So, actually, more often than really seems necessary but without a need for any explicit developer requests. This then adds entries in System Preferences under Security & Privacy for Automation. Checked or unchecked depending on how the user responds to the permission dialog.

Informational, you can reset the permissions with the tccutil command. Either for everyone or a specific bundleid...

tccutil reset AppleEvents
tccutil reset AppleEvents us.hall.HalfPipe

You don't want to invoke AEPermit with your own bundleid or the app will hang. That was a misunderstanding I had for a while. I also saw somewhere that someone indicated you should not invoke it on the main thread as this would lock up the application waiting on the permission dialog. I did set it to invoke on the main thread after having problems, remembering back when Apple was working on a Cocoa-Java bridge and this was how you had to run AppleScript. Since the dialog is probably modal to what you are doing I don't think this is a problem, or hasn't been yet.

Anyhow, the code is in place to request permission if needed. For example the HalfPipe command...

us.hall.osx.AEPermit com.apple.systemevents true

Code Signing

To get the permissions for the AppleScript I am trying to use it requires the application to be code signed. My first required use for signing. I managed to mess it up by moving the signed application with "cp -r". From this though I did decide that verification of signed application would not be a bad idea. I just submitted a enhancement request for the jpackage builds to also include verifications. I have not received a jdk- identifier for yet. I have added verification to the jpackage script for my app jpackage.sh. Currently my application signature verifies indicating it is valid for my purposes. The spctl command and the Taccy application both still indicate errors related to not notarized. I do not know the current jpackage status for providing notarization. It is not currently required for my application. Again, using this for the spctl man.

Informational, stumbled upon, you can just provide your "name" in indicating the signing cert. Like...

--mac-signing-key-user-name "Michael Hall"

as long as you have the "Developer ID Application:" certificate in your login keychain. In this case jpackage appears to do this...

/usr/bin/security find-certificate -c Developer ID Application: -a

or you can fully qualify a different signing certificate like...

--mac-signing-key-user-name "3rd Party Mac Developer Application: Michael Hall (5X6BXQB3Q7)"

Both currently seem to get identical results for me. The "codesign -v" verifies the signature as valid, spctl gives notarization errors. Either case seems to work for me authorizing AppleScript for my own app for my own use.

But after noticing this I did add this to my jpackage script...

SIGNING_CERT_3RD_PARTY="3rd Party Mac Developer Application: Michael Hall (5X6BXQB3Q7)"
DEFAULT_SIGNING_CERT="Michael Hall"

SIGNING_CERT=$DEFAULT_SIGNING_CERT

And then have this on the invocation...

--mac-signing-key-user-name "$SIGNING_CERT"

For those looking at the script this might make it a little clearer what your choices are, maybe give a hint to what jpackage is doing and could make it a little easier to switch between certificates. (It is pretty clear what jpackage is doing from verbose output on OSX). Although, I'm not sure there would be much need for switching certs. I still have to understand the purposes and capabilities of the different certificates.

Info.plist changes and entitlements

Requesting permission for AppleEvents requires a Info.plist change...

<key>NSAppleEventsUsageDescription</key>
<string>Support automation</string>

mine is pretty simple and not real informative but meets the requirement. This does appear on the permission requesting dialogs.

It also requires an entitlement...

<key>com.apple.security.automation.apple-events</key>
<true/>

I had switched at one point to JDK17 to see if it resolved any of my signing concerns. When I thought I had things working I decided to go back to 16. At 17 jpackage allows you to invoke with --mac-entitlements , while at 16 an appropriately named file has to be placed in the --resource-dir. When I fell back I was thinking entitlements hadn't been a concern and didn't switch to --resource-dir. So, AppleScript again broke and it took me some time to re-discover that I had needed an entitlement at Catalina. I can somewhat understand the difficulties in providing for this where all of this needs to be implemented and what needs to be included changes from OS version to OS version. Sort of trying to ricochet off of a moving target and hit a different moving target.

I am glad entitlements are being made a invocation parameter. With a bit of a concern that for some time two different ways of providing them will be supported. I do not personally care for the use of --resource-dir. It reuses resource dir for something other than it's normal Mac application purpose. It is not self-documenting in the command --help. Currently this indicates the normal application usages and makes no mention of the Info.plist or entitlement usages. It requires generating the application at least once beforehand to get a Info.plist to modify. Then using XCode or a text editor to do the modifications before putting it back into the resource directory. It is not a continuous process.

An alternative could be to generate the application --app-image. Then apply whatever post-processing you want. Like, using PlistBuddy to make the necessary Info.plist changes. After that finish the build with a command like jpackage --use-app <post.processed.app> ...rest of parms...

Or a slightly more extreme change would be to have something like...

jpackage <action> <parms>
where action would be something like create | sign | package

Where you could do whatever pre- or post- processing you wanted at each step. I guess properly I should do a request for enhancement on this.

So Now You Have AppleScript

With this done AppleScript does work pretty much as I remember having it available from HalfPipe.

I need to call my old times, that used to be
Boz Scaggs & Duane Allman ~ Loan Me A Dime

At some point while I was looking at this and AppleScript was broke I forget a bundleid. I came across this and actually pasted it into a sticky thinking it would be good to include as a HalfPipe AppleScript command...

osascript -e 'id of app "HalfPipe"

when I went to add the command I found this in command.properties (currently jar'ed in my resources.jar)...

alias.bundleid=as "get id of application "${1}""

It's like deja-vu, all over again.
Yogi Berra

It seems like these days I have the joy more often of re-discovering things I've forgotten.

Trying this from the application...

bundleid Finder
com.apple.finder
bundleid HalfPipe
us.hall.HalfPipe

I haven't able to get it to successfully parse bundleid "System Events". Notice that HalfPipe does allow you to get a little clever in defining the commands. Also note that it uses the 'as' command and not osascript...

alias.as=org.cmdline.cmds.As

as jpackage does in preparing it's DMG installs. The 'as' command actually uses the old Apple AppleScript JSR 223 scripting engine...

engines
ScriptEngineFactory Info
     Factory class:class apple.applescript.AppleScriptEngineFactory
     Script Engine: AppleScriptEngine (1.1)
     Engine Alias: AppleScriptEngine
     Engine Alias: AppleScript
     Engine Alias: OSA
     Language: AppleScript (2.7)

This is no longer officially supported or even unofficially except that I still have it. I copied it during the OSX port project and put a copy on github AppleScriptEngine. Scripting has at times had some priority for the application and the engines command actually indicates 5 available scripting engines. That was a while ago and how well any of them work I don't know. Except as we see here in a jpackage prepared application Apple's old AppleScript one does still work for my purposes.

Download

As far as I know this should work downloaded for someone other than me. But I can't say I know that for sure. This is it if you'd like to give it a try...

HalfPipe-1.0.dmg

One bug report was JDK-8263154 : [macos]: OS X DMG builds have errors and end up incorrect

This bug has been indicated resolved at JDK17. In testing my application with jpackage I started out by just building the app to a --install-dir. I decided to go on from there to testing with DMG installs. However, I didn't remove the --install-dir parameter when I did this. jpackage was in some way trying to honor my request for a DMG install to the --install-dir, which resulted in the DMG with the missing alias.

The resolution was indicated to be issuing a warning if --install-dir is used with DMG builds and always setting the DMG alias to the Applications folder. I agree with this resolution and have of course already removed the --install-dir for my builds.

___________________

If you look at the picture, as above, of a opened jpackage install disk image or 'DMG', you will notice that it graphically indicates a drag and drop install of the application to the Applications folder. The layout of this window is done by running an AppleScript. This AppleScript is broken. Verbose jpackage output indicates there is an error while running it and the opened disk image picture above ends up missing the Applications folder icon. The script fails in trying to make an alias to the OS Applications Folder.

jpackage has java code that somehow must resolve parameters in the AppleScript for paths and names (MacDmgBundler class). I didn't figure out the exact mechanism. I was thinking about doing my own java version using StringTemplate to resolve these imbedded 'variables'. In looking at verbose runs of jpackage I realized it was all pretty much just java running commandline stuff. I thought you could fairly quickly throw together a shell script and your own version of the AppleScript to do the same functionality.

That is jpackage_DMG.dmg. I had to use a dmg for this right? The script is dmgsetup.sh. jpackage appears to use a temporary 'images' directory, I use one in the current directory. The Test application is put in the images directory and the script DEPLOY_VOLUME_NAME parameter needs to be set manually to 'Test' and then it just works.

So, it shows a workable version of the script, it also shows the steps jpackage takes on a DMG installer. I did omit signing but I believe again that is simply running a native command. It serves for these as a demonstration. With minor changes it could serve as a work-around while jpackage has this bug. It is possibly not legal for me to use or distribute some of this, I am not a lawyer.

Another bug report was JDK-8263157 : [macos]: java.library.path is being set incorrectly

This one is also indicated as a regression that is resolved at JDK17. I will have to try and remember to remove my java.library.path override when I get to 17. The application does include JNI placed in the app directory so the property does matter. It would continue to run though if I forget to remove setting the property.

This might remain an argument to allow developer use of $APPDIR though.
-Djava.library.path=$APPDIR

___________________

Although I usually just do DMG or application builds this might apply to any OSX jpackage build. I think the bug report is pretty straight forward there has been a regression between JDK 14 and 15 where java.library path is set wrong. I consider it wrong because it no longer includes any directories internal to the application so the developer can control loading their native code.

As an FYI. The test case for this is what I used as the installed application for the first DMG build bug above. If you try to run it, it appears to do nothing. It does a System.out.println of the java.library.path System property. Going back to Apple java println's are no longer output to Console log. You can use the trick of commandline launching the app like...
Test.app/Contents/MacOS/Test
where it will let you do good old fashioned println debugging.

Native NSLog's do go to Console log and I wrote some native to have a simple PrintStream using NSLog for messages you would like to go to log. It is in hp.dylib and the class is...
at us.hall.osx.LogOut.(LogOut.java:9)
which is where this bug report errors on HalfPipe.

Bug report JDK-8263156 : [macos]: OS X application signing concerns - a sealed resource is missing or invalid

It occurred to me that this might be more serious for applications that need signing for getting into the Mac App Store. After some discussion on core-libs-dev I again realized that it was something I was doing differently.

I have not in the past usually put my applications in the Applications directory. For my own testing and use I have put them in my own directory. This worked well with the --install-dir mentioned above. It doesn't work so will with DMG installs so I tried to force my DMG builds back to a --install-dir type. I would mount the DMG and copy the application back to the prior --install-dir location. It turned out that...

cp -r /Volumes/HalfPipe/HalfPipe.app outputdir/HalfPipe.app

is not a good idea for a signed application and breaks the signature.

I currently have changed this to...

ditto /Volumes/HalfPipe/HalfPipe.app outputdir/HalfPipe.app

although that is not exactly the command I posted to the list. ditto I remember being a Apple provided file copy command that I thought should be reliable and it still seems to be so. I am having trouble finding official Apple documentation for the ditto command but did find this man page for it.

This bug report was closed as incomplete and can remain so.

___________________

I haven't really done anything where the signatures matter yet.

Request For Enhancement - DMG again

RFE: JDK-8263155 : [macos]: Allow additional contents for DMG builds

Unchanged. Although somewhat oddly it seems to show me having a future comment in August?

___________________

Since I already have something sort of in place for the process maybe I'll see if I could add something that would integrate sort of consistently.

A couple were added after this...

RFE: JDK-8264261 : Allow class path specification in java-options

Wish list item that appbundler used to allow and would be useful for a current edge case type nio FileSystemProvider bug.

JDK-8264260 : [macos]: IOException while codesigning the application

"Not an Issue" but was an exception. After occurrence I couldn't recreate

Signed application verification RFE

And today (4/23/21) I submitted an RFE for signed application verification. No official number yet. Possibly if no incidents like the immediate prior there is no need for verification.