I am now requiring that you obtain your own api key to use these. You can enter these with something like...

us.hall.quartz.Finnhub -k <your.key.here>
     and/or
us.hall.quartz.YahooFinance -k <your.keiy.here>

us.hall.quartz.Finnhub -p <symbol>=<owned>
     like
us.hall.quartz.Finnhub -p TSLA=1000
     I wish, or list a quote with...
us.hall.quartz.Finnhub -q TSLA
Finnhub: symbol: TSLA
Previous Close: $1,070.34
Open: $1,073.44
Current: $1,056.78
High: $1,082.00
Low: $1,054.59
Change: -$13.56
Percent: -1.267%

     The Yahoo Finance indices looking something like...
Yahoo Finance: Indices
Index Change Change% Price Volume Previous Close
Dow Jones Industrial -59.801 -0.164% $36,338.30 218,213,126 $36,398.10
S&P 500 -12.55 -0.263% $4,766.18 1,340,241,000 $4,778.73
NASDAQ Composite -96.629 -0.614% $15,644.97 3,032,680,000 $15,741.60
Yahoo Finance: CBOE Voltility Indices
Voltility Index -0.11 -0.635% $17.22 $17.33
NASDAQ 100 Voltility -0.18 -0.842% $21.20 $21.38
     Looks nicer in the app

For nd4jcli for now I will only provide the current download and link to the current thought for numpy comparison html. Hopefully, stay tuned.

Download: nd4jcli.dmg

So I may have a little momentum going into the new year. Maybe haphazard momentum, but momentum. See if it lasts.

12/26/21 Some new functionality to the matrix cli. Like rank, transpose, array to create, + to do arithmetic on array. Assuming at some point adding in the functionality will speed up, as how to do more things is figured out. But it is not that simple an interface.

So, now can do things like...

a = array([[2,3],[4,5]])
a + 1
[[ 3.0000, 4.0000],
[ 5.0000, 6.0000]]
a
[[ 2.0000, 3.0000],
[ 4.0000, 5.0000]]
b = array([[6,7],[5,3]])
a + b
[[ 8.0000, 10.0000],
[ 9.0000, 8.0000]]
a
[[ 2.0000, 3.0000],
[ 4.0000, 5.0000]]
c = a + b
c
[[ 8.0000, 10.0000],
[ 9.0000, 8.0000]]
c + 5
[[ 13.0000, 15.0000],
[ 14.0000, 13.0000]]

Download: nd4jcli.dmg

12/19/21 The original thought was that the inactive matrix library used by the Weka application could use some updating. It shows some errors for my HalfPipe application on current JDK releases for one thing. ND4J seemed a possibility. I built it on OS/X, the download is the start up work in progress folder where everything is currently tossed in. It also includes some JavaFX related since I was thinking this might be a good place to switch to that from Swing. A fairly simple mostly text based GUI But, I haven't been looking at that real diligently either. The CLI is supposed to be numpy like since I have seen it indicated that ND4J is the java numpy. It seemed like this could make a good model for a CLI interface.

So, with some simple boilerplate in place and a couple functions implemented you can...

./nd4jcli.sh
a = arange(15).reshape(5,3)
log4j:WARN No appenders could be found for logger (org.nd4j.linalg.factory.Nd4jBackend).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
a
[[ 0, 1.0000, 2.0000],
[ 3.0000, 4.0000, 5.0000],
[ 6.0000, 7.0000, 8.0000],
[ 9.0000, 10.0000, 11.0000],
[ 12.0000, 13.0000, 14.0000]]
a.shape
[5, 3]
a.shape()
[5, 3]
s = a.shape()
s
[5, 3]
quit

A thought today was a verbose setting that indicated what the underlying java is...

verbose
verbose=false
verbose=true
v=true
arange(15).reshape(3,5).shape
    --> arange STATIC class org.nd4j.linalg.factory.Nd4j :: MethodHandle(double)INDArray
log4j:WARN No appenders could be found for logger (org.nd4j.linalg.factory.Nd4jBackend).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
    --> reshape VIRTUAL class org.nd4j.linalg.cpu.nativecpu.NDArray :: MethodHandle(INDArray,long,long)INDArray
    --> shape VIRTUAL class org.nd4j.linalg.cpu.nativecpu.NDArray :: MethodHandle(INDArray)long[]
[3, 5]

So, 'verbose' by itself shows the current setting. Otherwise, set it true or false. I need to get rid of the v=true debug message. For STATIC the class is in fact the class. For VIRTUAL it is the class of the invoked instance. Once the java to do the numpy-like is figured out, you can use the numpy-like to see the java used. It might make learning that a little easier.

About it for this one for now. It's a mess, but...
Download: nd4jcli.dmg

11/22/21 The latest JDK18 early access release of jpackage includes the --mac-dmg-content option to add additional files to the dmg. It is reasonably straight forward. You can have more than one file, or a directory. I included the HalfPipe source and seriously out of date pdf documentation. You need to include --type "dmg" or it objects to using this parameter with the default type. Although for me dmg is the default so this seems like a bug or something they would want to change going forward. The appearance when you open the dmg I think could use some improvement. This is controlled by an AppleScript I looked at sometime back related to something else I thought was a bug that wasn't. I am tempted to try changing this AppleScript myself.

Anyhow, this OS/X HalfPipe update should be available as usual at at HalfPipe downloads.

10/17/21 The prefs command has gotten a little more complicated than HalfPipe's usually tend to be. Tried to update usage, shown if no parms, to reflect this.

prefs
PrefsCmd: Usage: (all preferences are user not system)

     prefs action [filepath]

     import(f) [path]      - import preferences from file. forced importf if existing preferences
     export(f) [path]      - export preferences to file. forced exportf if existing file
     list                          - list user preferences
     tstamp node           - time stamp mark node as portable. can also do this from code
     remove node          - or delete named - node -. Preferences removeNode() not remove(String key)
     clear node              - leave empty node
     port [path]              - write timestamped nodes to xml file
     update [path]          - update nodes with older or missing timestamps

      default import/export [path] /Users/mjh/Documents/hpfiles/userprefs.xml
      default port [path] /Users/mjh/Documents/hpfiles/userprefs.port.xml

      NOTE: Only single node preference paths supported for port/update.

It's been a while since I've updated source. So..
src.zip

10/06/21

at weka.core.WekaPackageManager.<clinit>(WekaPackageManager.java:255)

So for a more complete Weka demo...

address jshell
/env -class-path ${app.path}/weka.jar
jshell adding path /Users/mjh/HalfPipe/HalfPipe_jpkg/outputdir/HalfPipe.app/Contents/app/weka.jar
import weka.core.*
import weka.core.converters.ConverterUtils.DataSource;
DataSource source = new DataSource("/Applications/weka-3-9-5/data/iris.arff");
weka.core.converters.ConverterUtils$DataSource@25bbe1b6
Instances data = source.getDataSet();
@relation iris

@attribute sepallength numeric
@attribute sepalwidth numeric
@attribute petallength numeric
data.setClassIndex(data.numAttributes() - 1);
import weka.classifiers.trees.J48;
String[] options = new String[1];
String[1] { null }
options[0] = "-U";
"-U"
J48 tree = new J48();
No classifier built
tree.setOptions(options);
tree.buildClassifier(data);

[At this point we get the error in class initialization so we just try it again]
java.lang.reflect.InaccessibleObjectException: Unable to make protected final java.lang.Class java.lang.ClassLoader.defineClass(java.lang.String,byte[],int,int,java.security.ProtectionDomain) throws java.lang.ClassFormatError accessible: module java.base does not "opens java.lang" to unnamed module @5e57643e

tree.buildClassifier(data);
import weka.classifiers.Evaluation;
import java.util.Random;
Evaluation eval = new Evaluation(data);
weka.classifiers.Evaluation@641147d0
eval.crossValidateModel(tree, data, 10, new Random(1));
System.out.println(eval.toSummaryString("\nResults\n======\n", false));

Results
======

Correctly Classified Instances 144 96 %
Incorrectly Classified Instances 6 4 %
Kappa statistic 0.94
Mean absolute error 0.035
Root mean squared error 0.1586
Relative absolute error 7.8705 %
Root relative squared error 33.6353 %
Total Number of Instances 150

10/03/21

Related links if you don't want to read all the following...

Started to look at Linux version of HalfPipe for Rexx support. Didn't get too far, although did make ~/.HalfPipe the USER directory for linux. I added the Quartz scheduler initialization files, so the scheduler console puts out more messages. Mostly noise but it more closely matches the OS/X output. I think after it's been up a while it starts putting out log4j error messages. If I remember right I need to port a config file for that as well.

One thing that always bothers me when I run the Linux version is that the scheduled command to show cryptocurrency values and my holdings. It reflects the amount I held when I ported OS/X (java.util) preferences and not the amount I currently hold after my last sell off. It reminds me every time it would be worth considerably more now if I hadn't sold.

So how to sync. I thought of some networked realtime update with something like RMI. From there into looking at more elaborate distributed peer to peer network solutions. Some of them pretty interesting but overkill for my purposes.

prefs tstamp preferneces_node

Speaking of XML. I was thinking about a logging FileSystemProvider. Actually I got the idea from the FAQ for Philipe Marschall's memory FileSystemProvider. It rated two items in the FAQ so might be something people are interested in.

I came across Java Method Logging with AOP and Annotations and thought, wow, this is perfect for this. It included some maven related for building so I'm trying to learn a little of that. I am barely familiar. The reference seems like more detail than I need to get going on this so I am looking at some tutorials. Anyhow in trying to do a build I am getting the following errors...

[ERROR] JSR-303 validator thrown during initialization: java.lang.NoClassDefFoundError: javax/xml/bind/JAXBException

I saw where it was indicated that JAXB was eliminated from the JDK at JDK11. I found various recommendations to use this or that repository to get it and use this or that version of whatever. No success. This is my current pom.xml if anyone wants to take a look. If you know how this should be successfully built to eliminate the above error feel free to contact me by email

So anyhow, thinking about Rexx and it's possible use as an REPL, I tried the JShell REPL implementation to refresh myself on how that goes. This had issues with OS/X HalfPipe's use of it's own DefaultFileSystemProvider that required changes to the provider to workaround.

I made a few changes to the JShell REPL so by way of demonstrating those and as a refresher of how the REPL support is supposed to work. It is roughly based on the REXX address command. For REXX it is used to manage execution environments for HalfPipe to manage REPL's.

address no args returns the current default REPL
address an_REPL sets the default REPL
address an_REPL with_statement executes the statement with the indicated REPL without changing the default REPL

You can execute normal HalfPipe commands. Only if the input is not recognized as a HalfPipe command is it passed to the default REPL. The REPL state should be persisted for the length of the session, whatever mix of REPL or command input are entered.

So...

address groovy
foo = "bar"
bar
foo
bar
println foo
bar
address jshell
address
jshell
String s = "Hello, World!"
"Hello, World!"
null
s.length()
13
show where
***** Current Directories *****
User: (user.dir) /
Home: (user.home) /Users/mjh
Java Home: (java.home) /Users/mjh/HalfPipe/HalfPipe_jpkg/outputdir/HalfPipe.app/Contents/runtime/Contents/Home
*****
address groovy foo
bar
address
jshell

I was going to use a Weka (Java Machine Learning application) simple classification as a somewhat non-trivial example but it eventually runs into the jdk17 encapsulation crackdown.

address jshell
address
jshell
/env -class-path ${app.path}/weka.jar
jshell adding pathh /Users/mjh/HalfPipe/HalfPipe_jpkg/outputdir/HalfPipe.app/Contents/app/weka.jar
import weka.core.*
null
null
import weka.core.converters.ConverterUtils.DataSource;
null
null
DataSource source = new DataSource("/Applications/weka-3-8-5/data/iris.arff");
weka.core.converters.ConverterUtils$DataSource@6dde5c8c
null
Instances data = source.getDataSet();
@relation iris

@attribute sepallength numeric
@attribute sepalwidth numeric
@attribute petallength numeric
@attribute petalwidth numeric
@attribute class {Iris-setosa,Iris-versicolor,Iris-virginica}

@data
5.1,3.5,1.4,0.2,Iris-setosa
...[I don't truncate this like Terminal JShell does]...
...
data.setClassIndex(data.numAttributes() - 1);
Instances newData = data

[Well you get the drift, also note]

/imports
weka.classifiers.trees.J48
weka.classifiers.Evaluation
java.util.Random
weka.core.*
weka.core.converters.ConverterUtils.DataSource

As mentioned it does end up with an error. Due to some relection classloader related. I raised this as an issue on the Weka mailing list. Weka is still running 1.8 so I'm not sure how much priority a JDK17 issue will get. But it does seem it does this only to support a pretty inactive 3rd party library that seems like it should be replaced anyhow.

I think I am going to drop the Coursera Cryptography I class I'm taking. It is pretty much beyond me. It seems more difficult than the first time I took it. Maybe updated. It was going to be a review before taking Cryptography II but I guess I'll save these for some other time. I plan to continue the freebie Udemy Kali class I'm taking off a Reddit post.

A little more information on whats new than usual. I have to clean up some of these web pages sometime. Another to-do list item.

09/26/21 Informed of a new Rexx release I added partial support for OS/X only so far. It is a little different from prior JSR scripting languages added. It needs files located at runtime. I include those in a jar file that I extract to the users Application Support HalfPipe directory. In theory anyhow, it works for me. I run the interface with ProcessBuilder and set a REXX_PATH environment variable internal to that.

This also required similar changes to the 'engines' command. It is now ProcessBuilder with REXX_PATH set if available. Some testing not done yet if not available for some reason or other unexpected conditions. I left the old one 'org.cmdline.cmds.engines' because I think it will continue to work as expected if BSF4ooRexx is installed on the machine. I may try to come up with an availability check for that.

Basically for now a interactive shell is in place. This is the 'rexxtry' command. Slightly modified from what the BSF4ooRexx implementation provides as rexxtry.rex

rexxtry
log4j:WARN No appenders could be found for logger (org.apache.bsf.BSFManager).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
REXXout>REXX-ooRexx_5.0.0(MT)_64-bit 6.05 18 Jul 2021
REXXout> HalfPipe rexxtry lets you interactively try REXX statements.
REXXout> Each string is executed when you hit Enter.
REXXout> Enter 'call tell' for a description of the features.
REXXout> Go on - try a few... Enter 'exit' to end.
REXXin?>say 'Hello, World!'
REXXout>Hello, World!
REXXout> ......................................... HalfPipe rexxtry on DARWIN
REXXin?>

I thought I had the reference JSR 223 interface in place but that doesn't currently seem to be working. I think the REPL version will pretty much be the ProcessBuilder rexxtry left to run daemon.

I hope to get the cross-platform Linux and Windows versions working as well. This would be an opportunity to fill in some of the dataset handling blanks on those versions now that VirtualBox and jpackage allow me to support versions on these platforms.

I remember when Rexx pretty much replaced CLIST as the primary mainframe scripting language. I worked a datacenter automation job with a product using Rexx. They had their own relational SQL to go with it. An interesting product. I remember, although I'd like to forget, messing up a program and corrupting files on a job. A quick and dirty Rexx script fixed them. I liked Rexx.

05/09/21 A separate version of the application that includes a custom DefaultFileSystemProvider with OS/X specific file attributes.

It is a separate version because of an issue where the only way I know how to make it work is with an exploded jar, or plain old directory with class files, in the classpath. This issue is NPE when creating default file system when default file system provider is packaged as JAR file on class path". This means that the application requires some manual tweaks to include the needed directory and to add it to classpath in the jpackage .cfg file. This in turn means the application can not be signed. (I believe, actually untested). The latest OS/X AppleScript version as indicated here, does require signing and entitlements.

Due to a separate issue. JDK-8266345 : Custom DefaultFileSystemProvider security related loops it also has to run against JDK 12, the last release I have that does not have the issue. This was managed in the original jpackage invocation with this...

--runtime-image `/usr/libexec/java_home -v 12`

It pays off sometimes to keep these old JDK's around. This didn't work the last time that I tried for using a GraalVM JDK. There was just some mention of Nashorn related concerns on the JDK lists. In searching I didn't immediately find a download of Nashorn available. Some pages did mention switching to the GraalVM JavaScript. If application use of GraalVM JavaScript is anticipated then it working with jpackage might be more of a concern. I believe I have at least mentioned this on JDK lists and IIRC I posted an issue to the GraalVM project.

FWIW, HalfPipe still uses a JSR 223 implementation for Rhino that I did. It functions as needed for the application.

The current jpackage invocation is jpkgnio.sh. To avoid issues with reusing application setup the application name and bundleid are changed. My main concern here is that one version is signed and the other isn't. bundleid for applications seems to sort of function as creator/type did for Mac Classic.

The application itself sometimes needs to know what features are available. So this is shown at application startup...

HalfPipe:Initializer - AppleScript available is false
HalfPipe:Initializer - MacFileSystemProvider available is true

for the HalfPipeNIO application. The reverse for the signed AppleScript HalfPipe app.

The can be checked with...

set available.*
available.AppleScript=false
available.MacFSP=true

I was going to post the actual initialization checking code I find somewhat interesting. But in the interest of something like brevity I won't. It does use the us.hall.osx.eio.FileHelper class that I wrote related to this. I did a replacement for the Apple FileManager class, a lot of it still the actual Apple code. Some of it using the overridden DefaultFileSystemProvider. Also, I added the FileHelper class to demonstrate some of the nio provider uses and as well as to show that there were native things that could be done with OS/X files beyond what FileManager had provided.

Most of that was a while ago. The code had been dormant. So there are things about it that I forgot. What I've tried to add...

I added a 'attr' command. No args lists the additionals views and their attributes...

attr
View: mac_ls
    creator
    type
    plain
...
View: mac_finder
    creator
    type
    invisible
...
View: mac_cocoa
    NSFileType
    NSFileTypeDirectory
    
...

mac_xattr is the newest and currently has no listed attributes. Some of these may or may not be present for a given file. Given the elapsed time I make no guarantees on completeness or accuracy for any of the views.

With two args ...

attr mac_ls:application /Users/mjh/HalfPipe/halfpipe_jpkg/outputnio/HalfPipeNIO.app
mac_ls:application = true

Three args is the same except view and attribute are split.

Try to wrap this up somewhat quickly.

One thing I noticed and had I think forgotten from the years back is that the attribute isn't always in the expected human readable output format. Like posix permissions are decimal and not octal. My current thought on that is something like...

// Builtin converter functions
public static Function<Integer,String> toOctal = Integer::toOctalString;

My first ever attempt at Java 8.

The creatively named FileHelperCommand is intended to provide CLI to the FileHelper class. I'll come up with a better command alias sometime.

Given the seeming current importance of bundleid I previously had...

bundleid /Users/mjh/HalfPipe/halfpipe_jpkg/outputnio/HalfPipeNIO.app
us.hall.nio.HalfPipe

That can now be round tripped with...

id2app us.hall.nio.HalfPipe
/Users/mjh/HalfPipe/HalfPipe_jpkg/outputnio/HalfPipeNIO.app

The task list off of the Resources tab now updates each time selected. I realized it was actually only being updated once very early in application initialization. Before actual initialization was setting AppleScript available. The JMX java related doesn't work for either verion of the application. So for now it just sets the colors of the pids for those to green and doesn't have them links. It now runs on a SwingWorker thread, the scripting managers and related initialized each invocation. There is java, AppleScript, JavaScript and native involved. The result is a lag before display. This is actually where most of the time I managed to put into this the last couple weeks went.

jconsole also doesn't seem to work for my app. I thought it had broke at some point and eventually would be fixed but it now seems to work for other apps. Just not mine. I haven't looked at this in any detail though.

It for now...

DownLoads
HalfPipe-1.0.dmg Signed AppleScript version
HalfPipeNIO.dmg Mac file attributes FileSystemProvider

03/04/21 HalfPipe has had an occasional tweak to the mac version. I noticed running it on VirtualBox Windows that it was somewhat out of date. This basically is to make all versions current again...

Download the OS X application .dmg
Download the Windows exe installer
Download the Unix deb installer
src.zip currently available separately

Most of what I remember about the changes concern scheduled REST API's. AlphaVantage stock quotes seemed pretty much broken and I turned it off. I need to find a good replacement one of these days. It was one of the first things I thought of to use the scheduling for. The random RSS feed headline selection also broke at some point. This was a bad feed returning something that wasn't handled if I remember right. My crypto-currencies balances show differently on each version I think. Unfortunately the one showing the least amount is correct. I sold at the wrong time. The application still needs a good way to sync preferences across platforms. It uses the prefs API for lightweight persistence. For things like the amount of crypto owned or amounts of stock owned. Commands having saved state/preferences I still think could be useful in more situations, and is not a general feature of most shells. Where I guess that information would be in separate files. Good places to save files, especially across platform aren't always easy to know.

The REST API's recently about totally broke. This might be because I am running my laptop as a client to a openvpn server I installed on my newer Raspberry PI. This might of been causing timeouts for my network availability checks? Anyhow I now do three different checks, the last one I found on StackOverflow is sort of slow but seems reliable. There is no API that needs really immediate real time output.

About what I'm remembering right now

03/29/20
Most of my time for this update went into the FastR implementation on GraalVM on OS X. Trying to use it with the HalfPipe application. I had embedded the graal jre into the application with fastr.sh as shown in the previous update. That, however, crashes the application on simple API calls to the FastR related due to some codesigning issue. I opened a more or less informational bug on the project . new (jdk14 preview) command jpackage created OS X app crashes #2269 that remains unassigned. I tried an idea that goes back to Greg Guerin on the old Apple java lists. Use a shell script to do the application launching. This gets around the codesigning problem and the API calls work.

I came up with two variations on the launcher. One for the app embedded JRE...
FastRGraalHPS
and one for the OS JavaVirtualMachines JRE...
FastRGraalHPSOS
Some situations still seem to require using the OS installed. This will be described a bit later.

Shell launched works fine for most purposes. But when I looked at this when Greg Guerin first suggested it, it doesn't handle AppleEvents. This would mainly mean no doubling clicking or drag and dropping application associated files. A proper application should support this but I don't make a lot of current use of these features myself.

I had been looking at some financial R code. Portfolio analysis and balancing. Not that my personal portfolio is really all that much and since the COVID-19 outbreak even less so. But I was a bit curious if I could learn anything as to how the portfolio could be weighted better, whether I could include my cryptocurrency holdings in the portfolio analysis, what happens if I buy a little bit of a bond ETF. Things like that, really with pretty much a toy portfolio for what I have. Since Bitcoin helped pay off the house, thank you Bitcoin, my crytocurrency holdings are even less than my stock ones.

So, I wanted to install R packages like quantmod. A lot of this is still guessing and learning on my part. I think FastR may about always build installed packages from source. Compiling them to get them into a LLVM representation I'm assuming/guessing. That ran into Fortran related problems in the TTR dependency. I opened another issue on this...

FastR quantmod/TTR package install errors - fortran on OS X #2286 With answers provided on a site related Slack forum that includes what was a workaround for the TTR package. Basically, how to handle the homebrew installed gcc/gfortran.

Some packages have a lot of varied dependencies and there have been a number of different issues related to installing them. I think it was the 'curl' package that had install issues. I finally noticed that it was getting "Operation not permitted" errors on a bin it was trying to run. Searching indicated that you could use the OS X 'xattr' command to clear whatever was ever causing the problem. How it got set I don't know.

The 'jsonlite' package was getting errors like...
/Users/mjh/HalfPipe/HalfPipe_jpkg/outFastR/FastRGraalHPS.app/Contents/runtime/Contents/Home/languages/llvm/native/bin/ar rcs yajl/libstatyajl.a yajl/yajl.o yajl/yajl_alloc.o yajl/yajl_buf.o yajl/yajl_encode.o yajl/yajl_gen.o yajl/yajl_lex.o yajl/yajl_parser.o yajl/yajl_tree.o
Exception in thread "main" java.lang.IllegalStateException: Could not find GraalVM home
   at com.oracle.truffle.llvm.toolchain.launchers.common.Driver.getRuntimeDir(Driver.java:90)
   at com.oracle.truffle.llvm.toolchain.launchers.common.Driver.getLLVMBinDir(Driver.java:110)
   at com.oracle.truffle.llvm.toolchain.launchers.BinUtil.main(BinUtil.java:54)
make: *** [yajl/libstatyajl.a] Error 1
ERROR: compilation failed for package ‘jsonlite’
* removing ‘/Users/mjh/Documents/R/fastr/jsonlite’

This is a case where it appears to work fine when run against to OS installed JRE but errors on the embedded. I spent a lot of time looking at it but couldn't figure out what might be the difference or even where the BinUtil call back into java happens. I finally just built the package with the OS JRE.

Anyhow still dealing with things like that on different installs. git2r wouldn't install which seemed to be fixed after doing a 'brew install libgit2'. Currently for the PerformanceAnalytics package I think it is, quadprog doesn't build...
** installing *source* package ‘quadprog’ ...
** package ‘quadprog’ successfully unpacked and MD5 sums checked
** using staged installation
** libs
"/Library/Java/JavaVirtualMachines/graalvm-ce-java11-20.0.0/Contents/Home/languages/R/bin/f2c-wrapper" "/usr/local/bin/gfortran" -fno-optimize-sibling-calls -fPIC -O2 -c aind.f -o aind.o
Running f2c transformation, the log will be in ./aind.f2c.log
"/Library/Java/JavaVirtualMachines/graalvm-ce-java11-20.0.0/Contents/Home/languages/llvm/native/bin/graalvm-native-clang" -I"/Library/Java/JavaVirtualMachines/graalvm-ce-java11-20.0.0/Contents/Home/languages/R/include" -DNDEBUG -O2 -DFASTR -DNO_GNUR -fPIC -O2 -DFASTR -DNO_GNUR -c init.c -o init.o
"/Library/Java/JavaVirtualMachines/graalvm-ce-java11-20.0.0/Contents/Home/languages/R/bin/f2c-wrapper" "/usr/local/bin/gfortran" -fPIC -O2 -msse2 -mfpmath=sse -c -o solve.QP.compact.o solve.QP.compact.f
Running f2c transformation, the log will be in ./solve.f2c.log
clang-9: error: no such file or directory: './solve.c'
clang-9: error: no input files
rm: ./solve.c: No such file or directory
make: *** [solve.QP.compact.o] Error 1
ERROR: compilation failed for package ‘quadprog’
* removing ‘/Users/mjh/Documents/R/fastr/quadprog’
This doesn't seem to work for either the OS or embedded JRE. It doesn't even seem to build from source with RStudio but a different error.

So looking at that sort of thing, a learning experience that I'm sure, or that I certainly hope, will be worth it.

03/15/20
I was going to do something with R. I thought as long as I'm putting some effort into the REPL related for HalfPipe I would try it there. HalfPipe R hasn't seemed to be working real well lately. Weka is included, and that has extensive R support through it's RPlugin. So I thought about basing something on that a while back. But it wasn't real clear how to do that and it may rely on some class loading that has potential issues. So, I thought first I would look again at the usual java interfaces. Maybe fixing up what I already have. However, searches turned up a couple things new to me that were related to this. Renjin and FastR

Plugins

The first notable thing about renjin was that it included a fair number of jars. If I were to try it as a R alternative I would rather not just dump all of those into the HalfPipe classpath. which brought me back to thinking about plugins. I have considered them for the app before. Finding something that provides them or coming up with some scheme of my own.

A number of possible solutions came up in a search. One that turned up frequently being OSGi. Still it seemed like the solution should consider JPMS, the java module stuff. Actually, it seemed like there would be discussion of JPMS for plugin architectures somewhere, but not many really turned up in my searches. There was Eclipse Equinox with Java Modules All the Way Down. Which appears to have finally ended up as Atomos. It appears to provide what I'd like to see at the moment; dynamic, discovery, JPMS. Some learning curve definitely seems to be needed and hopefully it doesn't turn out to be too heavyweight a plugin solution for the HalfPipe app. Yet to be seen.

renjin standalone from Terminal appears to work fine...
Renjin 3.5-beta76
Copyright (C) 2019 The R Foundation for Statistical Computing
Copyright (C) 2019 BeDataDriven
Using system BLAS libraries.

R on GraalVM

This involves running with it's own JDK, the initial sort of interesting thing to me was if jpackage could create a working application for it. Could it be as simple as just providing the 3rd party --runtime-image? That would look like fastr.sh. The -add-modules for the jpackage incubator module itself had to be removed. Which for some reason wasn't taking so I changed the included cfg file manually. I don't think even then the script ran through completely cleanly but the following dmg was produced...
FastRGraalHP-1.0.dmg
Which runs...
versions
System.in:3:java.version=11.0.6
System.in:5:java.version.date=2020-01-14
System.in:16:java.class.version=55.0
System.in:18:java.vm.specification.version=11
System.in:21:java.vm.version=11.0.6+9-jvmci-20.0-b02
System.in:24:java.runtime.version=11.0.6+9-jvmci-20.0-b02
System.in:26:java.vendor.version=GraalVM CE 20.0.0
System.in:28:java.specification.version=11
Including for some R...
set RScript = ${java.home}/languages/R/bin/RScript
exec $RScript test.r
[1] 1 2 3
[1] 3
The file is the simplistic test.r
Not how I would expect you to use it on a long term basis but for demonstration purposes. The thing for this week maybe being that while jpackage may be limited in what versions it will officially support for jdk installations, there may be no reason it won't work fine for quite different others.

It may also be interesting to see if this does provide a more performance efficient R interface for java. The existing did have limitations especially in how much data could be efficiently bridged across java and R.

03/03/20
Considering some updates to the HalfPipe application wrapping the jpackage/jlink java commands. These would be kind of along the lines of what I did for the preference command and use the preference API's.
So thinking of a command something like...
jpkg [project name] [action] [other action specific parameters]
There would be possible defautls for about everything including a 'default' project. All required parameters for the action would need to end up being present or will be flagged as missing.
Just starting that.

jpackage has been accepted for jdk14 but remains in a incubator/preview status. I first had problems trying to use it from the application with Runtime exec. The halfpipe 'exec' command didn't seem to find it. It turned out that Runtime was going after the currently installed JRE and not the application embedded one.

This is a concern for some of the newer commands on OS X because the necessary command links have not yet been provided. e.g.
which java
/usr/bin/java

ls -l /usr/bin/java
lrwxr-xr-x 1 root wheel 74 Mar 19 2019 /usr/bin/java -> /System/Library/Frameworks/JavaVM.framework/Versions/Current/Commands/java

I think it's been indicated that the java people can't provide these links because they don't own /usr/bin and there is no ongoing process to have Apple provide them. My possibly incorrect understanding. But at least jlink and jpackage currently don't have /usr/bin entries and seem to be missing if you just enter the commands from Terminal.

To work around this for Terminal I added the following to my .bash_profile file.
jdkhome=`/usr/libexec/java_home`
alias jpackage=${jdkhome}"/bin/jpackage"
alias jlink=${jdkhome}"/bin/jlink"
However, these are not the commands you might want to use from an application. You would want the commands exactly corresponding to the JRE that you have embedded. It should be noted that by default no commands are included with an embedded JRE. This seems to be due to it's inclusion of the --strip-native-commands parameter when it invokes jlink. As indicated in the jpackage --help. Somewhat oddly I didn't notice this option when I did a jlink --help. Just --strip-debug.

In discussion of this on the java core-libs-dev mailing list that there is an enhancement request to change this. JDK-8219536. Where you would set the parameters for the jpackage jlink yourself. For the wrapper I am considering writing all of this again could be persistently set on a project basis. Including the defaults.

To get commands included with the JRE you need to do the jlink separately and omit the --strip-native-commands parameter. Then include that JRE when doing jpackage with --runtime-image. I also looked at the ToolProvider interface for programmatic use of jpackage. It should be additionally noted that for that purpose and possibly the Runtime exec version as well that you need to not only add jdk.incubator.jpackage to the jlink --add-modules. It was further indicated that because jdk.incubator.jpackage is not default included in the application module graph --add-modules has to also be added to the java runtime invocation parameters.
--java-options '--add-modules jdk.incubator.jpackage ...

With that the ToolProvider interface worked successfully. It may of also been necessary for Runtime exec to work, I didn't verify.

Again in discussion on the java core-libs-list it was indicated that tools that needed to locate the actual executable location for the current JRE could use the java.home System property. I noticed that the JShell command needed the java executable. I got a missing message indicating the correct location before I did have it there. It was unclear to me whether JShell was correctly building this path or ProcessBuilder was. My thought was it would be better if ProcessBuilder or Runtime is doing it so that the application developers don't need to be aware they have to construct this path themselves. It should just work.

For my application I did this to fix my Runtime exec using 'exec' command. Simply adding this actually based on sample code from a Oracle web page.
Path jbinary = Paths.get(System.getProperty("java.home","NONE"),"bin",id);
boolean isRegularExecutableFile = Files.isRegularFile(jbinary) &
   Files.isReadable(jbinary) & Files.isExecutable(jbinary);
if (isRegularExecutableFile) {
   processid = jbinary.toString();
}
else {
   processid = id;
}
Don't ask me why I called this processid.

With that the HalfPipe exec command can go something like...
exec jlink --help | grep --strip
System.in:40:-G, --strip-debug Strip debug information
exec jpackage --help | grep --strip
System.in:88:--strip-debug, --no-header-files, --no-man-pages, and
System.in:89:--strip-native-commands.
showing that I don't have to determine the path myself to issue the java binary command. Also showing that for some reason the jlink help doesn't mention --strip-native-commands while the jpackage help does.

About it on that. I did put a little time into the REPL related. JShell now can do things like...
address jshell
void greet() { System.out.println("Hello, World!"); }
null
greet()
Hello, World!
Although the REPL support in general is still not really robust.

The Weka application I include some support for is now using the Zulu JRE. I downloaded the current Weka version to verify there didn't seem to be any compatibily issues in my use of weka.jar. So the weka jar version is now...
weka.version
3.9.4

The OS X only development download is...
halfpipe.dmg

The Weka support could also be more extensive and could probably make use of the groovy REPL. What I'm remembering that might be notable for this update.

12/04/19
The HalfPipe application makes some limited use of the java preferences API's. Some commands use these for command specific lightweight persistence. CoinbasePrices is scheduled to periodically show the Coinbase exchange current cryptocurrency prices and the worth of the holdings for any that I own. The amounts that I own are saved as preferences. The command has parameters to change these saved amounts or preferences.

It seemed useful for an existing application with preference settings to be able to copy these along with the application to a new machine. To do this I made use of the Preferences class provided XML import/export. I wrote the 'prefs' command something along the lines of...

prefs action [path]

Where the actions are 'list', 'import', 'importf', 'export', and 'exportf'. importf is a force option if the application has existing user preferences. Then import alone will not work, the importf will completely replace the current preferences with those from the file. The export force will write over an existing target file. export alone will not overwrite. 'path' is optional, the default is to put the xml file in the users Document folder at path hpfiles/userprefs.xml.

There appears to somehow be persistence in just porting the application. Windows appeared to have all the original OS X nodes but no key/value preference pairs. Ubuntu also had the nodes I added but not the nodes from apparent attempts to run tightvnc and jshell that came from the application. So use of the 'f' force options was even initially required to populate the user preferences.

The downloads are as before...
OS X
HalfPipe-1.0.dmg
Windows
HalfPipe-1.0.exe
Linux
halfpipe_1.0-1_amd64.deb

12/01/19
Following the same approach as on Windows with Linux - Ubuntu 19.04 on VirtualBox - It was fairly straight forward to get a deb installer.
halfpipe_1.0-1_amd64.deb

The command there...

jpackage @jpkg.txt @linux_jpkg.txt

With linux_jpkg.txt being the platform specific.

11/28/19
After some discussion on the java core-libs-dev list I have a jpackage Windows installer for the HalfPipe application.
HalfPipe-1.0.exe

The command I use ended up being...

jpackage @jpkg.txt @win_jpkg.txt

With jpkg.txt being the generic options and win_jpkg.txt being the platform specific ones. The current thought is that the same approach might be reused on a linux version. Using the jpackage filename options avoids my lack of knowledge concerning Windows scripting or jpackage path expectations.

The OS X related HalfPipe jpackage files also saw some updates following the mailing list discussions. It was pointed out that I was using conflicting options for the runtime. Including a full one option and then including --add-modules to jlink a application specific one. The full runtime was used and jlink was not done.
The current application
HalfPipe-1.0.dmg

The current build script being...
jpackage.sh

Having a correct modular runtime reduced the size from about 110MB to about 70.8MB.

So all good - and getting better.

11/22/19

REXX First off after I noticed , somewhat surprisingly, some mention of Object Oriented REXX on the openjfx-dev forum. I liked REXX as a language beside it having helped pay the bills for some period of time. I have looked at it in the past on the Mac but have yet to really do much with it. It was mentioned in the context of scripting which is an area where I still think my HalfPipe application has the potential to be useful. The 'address' command tied into HalfPipe's REPL attempts is REXX in origin.

I thought maybe it is time to take a look at REXX again. I had some difficulties in installing. It appeared to be missing the configure file for building. It wasn't clear where or what the Object Oriented REXX Mac DMG package installer did. I'm not remembering if I tried Homebrew or other options.

What I think finally got the oorexx installer version, was adding the following to my .bash_profile...
export PATH=/opt/ooRexx/bin:$PATH
export DYLD_LIBRARY_PATH=/opt/ooRexx/lib/ooRexx

The DYLD change I got from #37 [OSX, MacPorts] Logic error: Failure loading required base l

More on this follows in trying to figure out how I might embed it into the HalfPipe application.

After installing the Weka distributedWekaSpark package I started getting Weka version warning messages so I upgraded the HalfPipe included jar to...
weka.version
3.9.3

I still haven't done anything with Weka Spark besides review the MOOC and some Googled discussion.

In considering scripting I tried entering the HalfPipe 'engines' command which should give some summary information on installed JSR 223 implementations. It broke on Groovy. With the error discussed in Gradle 5.5.1 fails with the latest JDK 14 EA build #10248. I updated the Groovy included jars to...
ScriptEngineFactory Info
     Factory class:class org.codehaus.groovy.jsr223.GroovyScriptEngineFactory
     Script Engine: Groovy Scripting Engine (2.0)
     Engine Alias: groovy
     Engine Alias: Groovy
     Language: Groovy (3.0.0-rc-1)

The 3.0 release candidate ones which as shown appears to work fine with the 'engines' command. Groovy includes many jars and it isn't easy to know which ones should be included for an embedded application. I added groovy-3.0.0-rc-1.jar and groovy-jsr223-3.0.0-rc-1.jar. Maybe I should be more up to speed with jdeps. Maybe I should be putting Groovy to more use. It is supposed to be one of the available REPL's - with ... address "groovy" ... It is a common scripting choice for the Weka mailing list.

More time in general should go into the scripting and REPL's.

That is about it for what has changed to HalfPipe for jpackage and the jdk-14 build it uses. So far.

Briefly back to REXX it is also not real clear how to embed that into the application. BSF4ooRexx seems to be the be the way to go but I haven't made much headway yet in figuring out how to incorporate that.

The only other thing I've managed so far is a somewhat working version of the application on Windows. This is courtesy of VirtulBox. I had actually managed a jpackage built version for the platform on my old computer but ended up deleting that. Not a good idea, never mind.

The following just runs the application but seems to do it reasonably well so far.

Running the windows version of the application halfpipe.zip

Run the bat file next to the Mac application bundle. jpackage build yet to come.

Wrapping this up fairly quickly. It should probably be proof read but hopefully clear enough for now.

11/01/18

HalfPipe - A lot of stuff thrown together. Maybe not totally complete but roughed out and sort of working. I was thinking something to display stock option chains would be nice. R has the quantmod package with getOptionChain. Running this with my normal R interfaces crashes the HalfPipe application. Sometime I should take a closer look at how Weka interfaces to R, it is probably more robust. Another thing I had been thinking about was why not command line R? So I set up the class
us.hall.script.RscriptEval.
Not a command for a change, instead it has the static method
public static ArrayList eval(String arg)
So another command class can build a script and pass it to this to run and get back the output. To return the output it uses a org.cmdline.psuedoGestalt.CommandShell invocation with a ArrayCmdOut used for System.out.

So, then a command like...
us.hall.quartz.OptionChainsR
can build a script and use RscriptEval to run it. Something like...
us.hall.quartz.OptionChainsR TSLA
Loading required package: xts
Loading required package: zoo
Attaching package: 'zoo'
The following objects are masked from 'package:base':
as.Date, as.Date.numeric
Loading required package: TTR
Loading required package: methods
Version 0.4-0 included new data defaults. See ?getSymbols.
Warning messages:
1: package 'quantmod' was built under R version 3.4.4
2: package 'xts' was built under R version 3.4.4

Finally getting to the results...

$calls
Strike Last Chg Bid Ask Vol OI
TSLA181116C00050000 50.0 240.65 0 281.40 283.45 4 4
TSLA181116C00160000 160.0 150.65 0 171.60 173.65 10 10
TSLA181116C00242500 242.5 57.50 0 90.15 92.20 6 3
...

So some additional parsing is obviously needed to get rid of noisy non-data output. I noticed in looking at the getOptionChain function documentation that it currently only uses yahoo data. Both Google and Yahoo have seemed a little erratic in how they support publicly available financial data. At one point I gave up on Yahoo finance for financial quotes (class YahooQuotes) switching instead to AlphaVantage (class AlphaVantage).

But looking at what Yahoo has for options it really is rather nice...
us.hall.quartz.OptionChainsYQL TSLA
...
50.0 TSLA181116C00050000 Thu Nov 15 18:00:00 CST 2018 4
160.0 TSLA181116C00160000 Thu Nov 15 18:00:00 CST 2018 10
242.5 TSLA181116C00242500 Thu Nov 15 18:00:00 CST 2018 3
247.5 TSLA181116C00247500 Thu Nov 15 18:00:00 CST 2018 5
This one is still a work in progress, I have skipped the debug messages you currently see using it, and the output is obviously incomplete. But of note is that it includes a number of additional goodies along with the option chains. Like implied volatility, averages, what seems to be potentially interesting data.

In looking at this I noticed this page page which indicates that the YQL console is no longer supported. It didn't seem like it would be that difficult to support basic console type functionality with a HalfPipe REPL interface, so...
address yql
select centroid.latitude, centroid.longitude from geo.places where text="hudson, wi"
http://query.yahooapis.com/v1/public/yql?q=select+centroid.latitude%2C+centroid.longitude+from+geo.places+where+text%3D%22hudson%2C+wi%22&format=json
{"query":{"created":"2018-11-01T04:59:14Z","count":1,"lang":"en-US","results":{"place":{"centroid":{"latitude":"44.981911","longitude":"-92.757187"}}}}}

I have started setting up a preferences manipulating command...
org.cmdline.cmds.YQL -f xml
select centroid.latitude, centroid.longitude from geo.places where text="hudson, wi"

http://query.yahooapis.com/v1/public/yql?q=select+centroid.latitude%2C+centroid.longitude+from+geo.places+where+text%3D%22hudson%2C+wi%22
<?xml version="1.0" encoding="UTF-8"?>
<query xmlns:yahoo="http://www.yahooapis.com/v1/base.rng" yahoo:count="1" yahoo:created="2018-11-01T05:07:49Z" yahoo:lang="en-US"><results><place xmlns="http://where.yahooapis.com/v1/schema.rng"><centroid><latitude>44.981911</latitude><longitude>-92.757187</longitude></centroid></place></results></query><!-- total: 6 --><br/>

Remember that HalfPipe command and REPL input can be mixed. HalfPipe checks to see if it recognizes the input as a command before passing it to the REPL. There can be conflicts.

Consider...
show tables
Unknown argument tables
***** Options *****
show [ vols classpath env properties security where singleton_system_property(s)]
switches [[-]-c [-]-p [-]-e [-]-w]
*****
'show' is one of the application's oldest commands, which I never use anymore, except 'where' once in a while to see if the app is still accepting commands and responding.
address yql "show tables" http://query.yahooapis.com/v1/public/yql?q=show+tables&format=json {"query":{"created":"2018-11-01T19:57:44Z","count":102,"lang":"en-US","results":{"table":[{...and so on...

Shows that you can avoid the conflict by explicitly directing the input to the REPL.

The more usual HalfPipe REPL support definitely needs work. But, this seems sufficient to try out YQL queries and see the results as you used to be able to do with the Yahoo online console.

Looking at the default public Yahoo tables, they were also sort of interesting. There was a geolocation table and a weather table. I have thought for some time that local Weather would be a natural addition to the personalized, scheduled functionality, of HalfPipe. This would have to be two part. Something for location to access local information and then a feed to provide the desired local information. The default Yahoo tables provide both of these.

org.cmdline.cmds.Location "hudson, wi"
Name: Hudson
woeid: 2425270
lat: 44.981911
long: -92.757187
The location tables seem very good for the US, including after all Hudson WI, not the largest city. But the tables seem to be be somewhat hit and miss for other countries. Missing some major cities there.

org.cmdline.cmds.Weather
Yahoo! Weather for Hudson, WI, US Sunrise: 7:49 am Sunset: 5:59 pm Humidity: 93

Thu - 01 Nov 2018
High: 48
Low: 31
Partly Cloudy
... (Currently five days are forecast) ...
Notice that the command can take of advantage of the Location command setting a preferred location with a '-s' for set parameter. It can override this by specifying the location like org.cmdline.cmds.Weather "Nome, AK". Current data for some reason is wrapped in html and I haven't written anything to extract it yet, or maybe I'll pop it up in a temporary floating window. Unfortunately, it also seems Yahoo is going to drop support for the public weather tables. But, they work for now. Having the location I can probably find somewhere else to get this information.

Anyhow, as you may have noticed I don't have command alias's for any of these commands yet. I don't have them scheduled, or color highlighting styled. Some aren't even debugged or complete. But, I didn't want to get on a perpetual development cycle, never providing updates, so this is how it currently is.

Download

halfpipe.dmg

10/14/18
There is now a "RSS Feeds" tab for a simple RSS feed reader GUI. Seeing some of the headlines rolling across I was interested in reading the articles. The GUI is still pretty glitchy but I usually can manage to read an article if I ignore the gyrating scroll bar that I get. I might work on that. However, there is still no way to maintain the feed list or download headlines as data or anything like that and I might not get to those for a while.

Since as far as I know I'm about the only one using this I'm adding more things related to what I'm interested in. I had tried to avoid too much of that if it didn't seem to be something there might be a general interest in, but for now it's my app and I'm the only user I know of.

So, I added a command to track my Kaggle competitions. You need to have the Kaggle API installed. Kaggle API. I added it to the scheduler with highlighting so that competitions with plenty of time show green, with only two weeks to go show as yellow, and in the last week show as red. The 'salt' competition has been red for a while. I probably won't get back to that which is too bad because I had hoped to learn a little more about neural networks and imaging. Too many things going again.

I added the command for this in two parts. It seemed like the location of the native command itself could vary, so I gave it it's own alias.
set alias.kaggle.cmd
alias.kaggle.cmd=/anaconda3/bin/kaggle
Then the HalfPipe Kaggle command is...
set alias.kaggle
alias.kaggle=exec ${alias.kaggle.cmd}

The same for 'nmap'. nmap. I use a command to see if either of my Raspberry Pi's are booted and networked. Not heavy Matrix like hacking. I meant to do more with the Pi's. Again, too many things going. It also seemed possible the command might be different, other platforms or whatever.
set alias.nmap.*
alias.nmap.cmd=/usr/local/bin/nmap
Then the nmap command is...
set alias.nmap
alias.nmap=exec ${alias.nmap.cmd}

So, if the native commands are different for you, you can change the command properties file or command line change the cmd alias. Then not have to worry about the path after that.

I also did some tweaks trying to get rid of some of the messy looking exceptions that can get thrown when I take a nap and the scheduled network commands stop working. Trying to style these exceptions can get more problems. Probably other tweaks I'm forgetting about as well. Too many things going right? But, they're all interesting.

10/01/18

This adds the scheduled us.hall.quartz.RSS command. This saves off the headlines from a number of RSS feeds and is scheduled to show one every minute. This list will be updated every time you start the app or every four hours if the app stays up that long. There is nothing yet to modify the list of feeds or read any of the articles. Maybe yet to come.

It was partly inspired by a Kaggle competition that is attempting to predict market movement based on headlines. So it may also include an option to save these off as data.

This returns back to plain HalfPipe and not HalfPipe7 or HalfPipe9 or anything related to the java version. The JRE currently embedded in the application is still Java 9.

I'm also looking a little at the Scala language and thinking about maybe getting back to the HalfPipe REPL related.

Download

halfpipe.dmg

12/10/17
A break from HalfPipe. I looked at javapackager a little more. I applied this fix. fix to javapackager, the jar rt_packager.jar, and it does appear to work. This to use javapackager with the jar - packager.sh. My javapackager HalfPipe build had all it's jar files in the correct place. The app still didn't launch. There are probably a few reasons why this would be true, but one thing I thought might be is that it is missing it's native jnilib/dylib's. This appears to be a TODO in MacAppImageBuilder. I used my javapackager TestApp to see what it sets for java.library.path. It showed that javapackager applications set java.library.path to both Contents/MacOS and Contents/Java.

TestApp has been useful a couple of times, For one other, I used it to test whether I had the JShell api modularized and accessible. I put the basic parts in a dmg. TestApp.dmg. You can change TestApp.java to display whatever you want in a JFrame title. Lastly, as mentioned, the setting for java.library.path. After compiling I think you need to do this...
javapackager -createjar -appclass us.hall.TestApp -srcdir Classes -outdir . -outfile testapp.jar -v
assuming you compiled '-d Classes'. Then manually copy the jar to TestApp.app/Contents/Java. And test.

For java.library.path since Contents/Java is included I think you could just add them the same as classpath jars. However, more properly they should probably end up in Contents/MacOS? So I am looking at that. Doing it correctly doesn't look easy. Localized resource bundles and sort of involved parameter passing. Adding a platform specific argument might be another reason it hasn't been done yet. But I may see if I can hack something together for this and maybe github it or something.

12/03/17
Before attempting JavaScript as an REPL I thought I would try to get back the JMX attach functionality that I had added to my own Rhino JSR 223 implementation, working now with the current Java 9 application. The original code was from a JDK sample for Notebook or Notepad or something. The attach, I think, was my addition.

I was able to do this with some assistance on modular issues on the jigsaw list. So it should again work JSR 223. Like...
js
load('Scripts/mm.js')
load('Scripts/jtop.js')
jmxAttach('3035')
jtop(print)
[CTRL-D] entered
time - state - name
5.43348168E11 - RUNNABLE - main
4.338665E9 - TIMED_WAITING - org.eclipse.jdt.internal.ui.text.JavaReconciler
...

It should also work with the interactive Rhino Shell interface. Like...
jijs
js> hp('exec jps')
3035,4301 LoaderLaunchStub,4319 Jps
js> load('Scripts/mm.js')
js> load('Scripts/jtop.js')
js> jmxAttach('3035')
js> jtop(print)
time - state - name
5.54970123E11 - RUNNABLE - main
8.960147E9 - TIMED_WAITING - Worker-31
4.42682E9 - TIMED_WAITING - org.eclipse.jdt.internal.ui.text.JavaReconciler
...

Pretty much the same story. But notice that I gave the shell a 'hp' function to do callbacks out to the HalfPipe application shell, which can of course itself can Runtime exec native commands like 'jps'. I had done the same for at least JRuby but I have no idea how functioning that is at this point and I have no immediate plans to get back into it.
And last, but hopefully not least, it works for the HalfPipe application itself. If you select the Resources tabbed pane, then 'Active tasks' you can then select a blue underscore highlighted pid, meaning a java task. Other than HalfPipe itself which doesn't seem to work. But for some other java task you will see information filled in using Rhino JavaScript JMX scripting.

Maybe the Rhino REPL next, or maybe see how much of this can also be done with Nashorn?

Other more miscellaneous. Yet more tweaks to AlphaVantage, I'm afraid not quite flawless yet. A module display command, us.hall.cmdline.cmds.ModuleInfo. I was given some commands for this. But, working on these things for HalfPipe can help in learning a little about how the API is put together. I have also been looking at javapackager a little again.

11/26/17
I was thinking HalfPipe REPL's again. I corrected the application R_HOME environment variable setting which was bad. So now the JSR 223 type interface works again, like...
us.hall.scripting.RS
x <- c(1,2,3)
print(sum(x*t(x)))
[CTRL-D] shows...
[1] 14
14.0

I added a holiday table to the AlphaVantage code so that should work flawlessly, for the next couple years anyhow, in theory. I still want to add something for how scheduled network based code works when the computer sleeps. This basically doesn't work and throws exceptions or sometimes even hangs up trying to put up unavailable dialogs. I think the java 9 awt Desktop code has some computer 'sleep' related methods now that I might be able to use somehow. Maybe not to still get results, though, if the network related code does stop working when the computer 'sleeps'.

The quickly put together R REPL version doesn't work so well. It crashes the app. It does work when the app is shell script launched. It was the same when the app R_HOME was wrong, which is how I finally figured that out. But now it should be right for that and in this case I don't know why one works and the other doesn't. I'll included the shell launch if anyone is curious about that.

This was a simple implementation based on the JRI java/R code rtest1 example I think it was. I think a better version might be based on or actually be using the Weka RPlugin code. But I haven't really looked at that yet.

Some other things not working that I also looked at are not worth mentioning at this point.

Although one was curious. I was thinking maybe something wasn't working because of java 9 modular. I noticed jdeps showed that the app needed the java.corba module. So, I added it. The Quartz scheduler code then stopped working - getting classloading errors on javax.transaction.UserTransaction, IIRC. My JRTLister code didn't seem to show it in any jvm provided module and checking it appears to come from jta.jar. Why does adding the corba module interfere with this classloading? A bit of a modular mystery. I thought something to display the meta information associated with a module might be nice but the only java command I saw, so far, that seemed to provide this information was jar? So a command to display is on the app to-do list. I found what looks to be good related example code but it is heavily lambda-ized. So there is some learning curve to understanding it. But I am sort of trying to learn that as well, so all the better, right?

11/20/17
Updated the AlphaVantage stock quote code to use daily data instead of just giving you numbers for the last few minutes. Added percentage change as well. Should probably have a markets closed holiday table. It will still probably break on and immediately following those days. It should probably have a feature to enter a date/time to test with. Make sure it flips back a day before the market opens, that sort of thing.

I added source, which I haven't provided for a while, and what I have for the current PDF documentation. Should have javadoc sometime I suppose.

Download

halfpipe9.dmg

11/13/17
It seems Yahoo has discontinued it's stock quotes API Yahoo Finance API disabled. I was considering using the Thomson Reuters API which was some fairly involved code. Maybe, I could use this to implement some sort of plugin facility into HalfPipe. But, I finally decided this was too involved an effort to replace what had been a nice, simple, almost demo command. When I myself have no major need for realtime stock quotes. I did one last search and found Alpha Vantage. Which again seems a fairly simple REST API including this data. Except that it seems to be missing percentage change which I'd still like to add. As an example of using scheduling and java Preferences for lightweight command specific persistence it seems a command worth having. Usage is the same as YahooQuotes. I'm not really seeing anywhere at the moment where the persistence part is documented. It goes like...
The portfolio option if followed by a comma separated list of stock symbols persistently sets that as the portfolio to use on scheduled runs.
e.g.
us.hall.quartz.AlphaVantage -p "MSFT,AAPL,ORCL"
-p alone should give you the Dow Jones, Standard & Poor, and Nasdaq indices.
Again, the persisted list is what is used on scheduled runs which should be done with -s.
e.g.
us.hall.quartz.AlphaVantage -s
-c can be used to 'clear' the current persisted settings.
e.g.
us.hall.quartz.AlphaVantage -c

Download

halfpipe9.dmg

10/30/17
I decided to try and make sure the HalfPipe Groovy REPL implementation could handle the examples from the EdX Functional Programming course's first lab. I'm taking that to try and understand better what Java lambda expressions are all about. At some point I updated the Groovy jars to the 2.6 versions, so you could try out the latest Groovy version through HalfPipe.

To get a JRE version that would work with the Groovy dependencies I used 'jlink' this time. The jlink tool seemed to be missing for me so I filed a bug report. I'm losing track of what I've filed bug reports for. Most of it probably not bugs that will actually be addressed but for now I think at least some of this merits having one filed against it. It isn't good to even seem to be missing command line tools. It was actually included in the install JRE bin directory but no symbolic link was present in /usr/bin. I added my own alias in my .bash_profile. The jlink command I ended up with that resoved the Groovy dependencies was...

jlink --module-path `/usr/libexec/java_home`/jmods --add-modules jdk.jshell,java.xml.bind,java.scripting,java.rmi,java.xml,java.sql --output jlink

where --output jlink is a directory to be created. Then I would paste that into the correct place in the application. If a decent ant task were written for this I think it would be easy enough for appbundler to support Java 9 modules. One thing though since doing this when I use the JShell REPL from the application I get messages like...
error: No file system provider is available to handle this file: /Users/mjh/HalfPipe/HalfPipe7.app/Contents/Java/antlr-2.7.7.jar
For each jar I have in classpath. I don't know of anything I've changed other than the JRE but haven't definitely narrowed down why I am getting these.

Meanwhile back at the REPL's. They are getting a little more interesting to play with...
For example with JShell...
address jshell
String h = "Hello"
"Hello"
String w = "World"
"World"
h + " " + w
"Hello World"

For Groovy a closure example from the earlier mentioned course lab...
address groovy
add = { a, b -> a + b }
add(1,2)
3
add('Ta','Da')
TaDa
About it that I'm remembering for this week.

Download

halfpipe9.dmg

10/15/17
A version of HalfPipe with an embedded JRE that includes the JShell module. I'm still trying to figure out a good way to work with applications in 9. For this one javapackager created the embedded JRE with JShell included by -add-modules. I haven't figured out yet how to generate a complete HalfPipe application using javapackager though, all 23 jars and resources. I'm not even sure yet if that's a bug or me not understanding how it expects the parms to be set up. I have, I think, something in place so I can do some debugging/testing against the javapackager code. Whichever, the problem is. So, for now, I took the JRE from javapackager and embedded it back into an appbundler application. That was getting errors. It turns out the appbundler that came out of the macosx port project won't work with embedded java 9 JRE's. In searching on the error that I was getting I found a fix in the infinitekind fork of appbundler. So, I used one of theirs for the JavaAppLauncher application executable and it worked.

Example of current rudimentary JShell support...

About it for now. The download is a separate one for Java 9 for now so old one could I think continue to work java 8. But I haven't tried it. I'm thinking most of my on-going efforts for the application will be java 9.

Download

halfpipe9.dmg

10/01/17
Well the long awaited modular Java 9 has arrived. It has gone GA. Updated HalfPipe to launch with it. I tried adding JShell as an REPL option. Not accessible. I had added the commandline property launch switch to use my TRZ MacFileSystemProvider as the default provider. That is mostly Mac file api's exposed as nio.2 attributes. Not sure I remember why I wanted to do this. It did no harm Java 8 but failed to launch Java 9. I tried a bit to get it working Java 9 but no success. Obviously a ways to go but the app does launch. I think for now it should still launch Java 8 but haven't verified that.

A recent nuisance bug where it would throw out a lot of errors on GUI updates when the machine was asleep, or output was somehow mis-styled, sometimes even locking up. Mainly I think related to output from the scheduler output display pane. Anyhow, this seems pretty much fixed, maybe by a recent Java 8 update. If so, thanks to whoever fixed it. I hadn't figured it out yet. It has that going for it now.

Hopefully, more to come

Download

halfpipe.dmg

<03/02/16

HalfPipe update - I have still been making updates to HalfPipe. I've been trying to document them in the app pdf which is now always included in the OS X dmg download. Not for the latest update though. This is mainly tweaks to try and avoid application lockups when the computer goes to sleep. Ongoing, probably scheduled, activity either related to network stuff, or gui stuff, or both seems to freeze the app. Or it tosses a lot exceptions. There are still some early app startup gui exceptions getting thrown that I haven't figured out how to address.

I had added some finance related scheduled commands. Showing stock prices or bitcoin prices. I was next thinking some periodic network related. One being to try and alert me when I lose DNS settings. That is a occasional problem that I have to fix by going into the admin interface on my router and re-saving settings.

02/05/17

HalfPipe update - Gets output for the Quartz cron command - to schedule HalfPipe commands, outputting System.out correctly to the scheduler console panel. I left the example/test code in an initial command file embedded in the application. The one current cron command is a echo of 'This is scheduled'. I also eliminated some log4j debug level noise going to that console. Trying to keep meaningless messages of no interest rolling things off of the screen.

The thought might be that a file embedded in the application is a little awkward - what if I want to add my own? My long term intent there is something like the R or Weka package functionality. Someplace with directories that hold all the content for separate projects/domains. Including, jars with the java commands, scripts, schedules, etc.

Sometime.

Download

halfpipe.dmg

01/29/17

HalfPipe update - Corrects weka.pkgmgr.check command reporting mismatches on packages with multiple versions based on base Weka version. Changes this from using WekaPackageManager with command line invocation to instead use it as an API. This code is getting more complex to support multiple versions, new ClassLoader code, and extended support for packages with package dependencies. My code will probably need to be more complex as well because of this.

Corrected the R_HOME environmental variable setting for HalfPipe, which was incorrect, in debugging this. I might have more success with an R interface from HalfPipe at some point now.

Download

halfpipe.dmg

01/15/17

HalfPipe update - This is still pretty much OS X only until I get some quality time on a Windows, or I guess, other Unix machine.

Split the main console display in half. One for command output, one for scheduler. This should keep scheduler output from mixing in with normal command output or rolling it off of the screen.

Some other fairly minor cleanup while I was at this. This might of broken some functionality I never really use.

It is also compiled Java 8 and includes the Weka 3.8.1 version jar. Both for more current Weka compatibility. Log4j has also been updated to version 2.

This also eliminates the need for a separate launcher.jar. To do this it breaks the feature where code was loaded from tools.jar that allowed Rhino javascript to pid connect to different Java processes. My understanding is that this situation will change anyhow in Java 9. This simplifies the application build to a point where maybe it isn't so tied to multiple Eclipse projects and could possibly be moved to github without a major effort

The download dmg for this update includes the source and PDF documentation. Here...

halfpipe.dmg

12/19/15

Not quite so quick an update to HalfPipe. Using jdep and some Eclipse built-in refactoring and a little manual changing I eliminated the need for the separate utillib package. This was a project to itself in Eclipse. So now there is only one project to halfpipe.jar. There are still two projects for the launcher.jar file. More cleanup is needed there eventually, especially since Java 9 will eliminate the need for special classloading to access tools.jar. Some of the app is older than OS X, some of it is things I was interested in for a time span and then moved on. So, a lot that is there is more or less deadwood that will not see frequent or possibly any future use. I started an 'extra' project to start making some of this more optional. I moved just a little bit there as part of the refactoring done today. More along the lines could also be done in the future. Java 9 modules could figure into that as well.

If the application is worth further work for things of current interest it is worth general maintenance.

halfpipe.dmg

Due for making current source available, especially with the considerable refactoring of this update.
hp_src.zip

12/18/15

Quick update to HalfPipe. Now includes the 'address' alias in command.properties. Also should print simple not found message instead of throwing Groovy exception if groovy doesn't handle the text entered. At the least for the Groovy missing property exception I've seen up to now.

Also managed to get back to showing a Java 8 release instead of a java 9 ea. Either I hadn't reverted or hadn't reverted correctly the 9 ea out. For this you have to delete the 9 ea or drag it to the trash and then you can install the earlier 8 release. Just installing the 8 won't show a error but will leave the 9 version still there. The file to remove is...
/Library/Internet\ Plug-Ins/JavaAppletPlugin.plugin
You can use the Finder menu option 'Go to Folder' to get to the file to remove it.

NOTE This applies to the JRE install of java. NOT the JDK. OS X applications without an embedded runtime use the 'shared'/'browser' JRE install. My understanding of that still seems correct.

I am removing the earlier 9 ea comments.

12/13/15

An actual update to 'classic' HalfPipe this week, not just messing with it to experiment with java 9.

I had been looking a little at java 9's JShell but do not think I can currently use that as the basis of an REPL for HalfPipe. It gives an error on a simple assignment test. My post to the dev list asking about this didn't seem to show up although I thought I received notification that it was accepted. I did see that they appear to have a basic test pretty much exactly like what I was trying to do. After more looking I found a JShellTool, besides the JShell class, that also seems unusable as an 'internal' class with a fair amount of package private access. I didn't see anything clearly different that it was doing to ensure it success where mine fails. Although it does a lot more. But, it seems, no good to me right now.

So I decided to consider the alternative of having Groovy as another REPL implementation for HalfPipe. As I think I've said elsewhere, the basic idea is that anything HalfPipe doesn't know what to do with it will try passing to the active REPL. So I added in Groovy. A little differently from how I usually add in a scripting language. I did add a JSR 223 based class. us.hall.scripting.Groovy. (I have not added a command alias to the command.properties file yet). That makes for a lot of scripting engines in the app. Try an 'engines' command to see. I didn't set up a 'interactive' version yet. For Groovy itself that seems to be the 'Console' but that is written in Groovy and has it's own Swing gui. Not real easily merged into HalfPipe. An interactive command also seems slightly less necessary if an REPL interface is available. Although, there might yet be conflicts between the HalfPipe shell and the REPL that a standalone interactive command could avoid. Groovy has seemed popular on the Weka mailing list, and appears to actually be a pretty good choice as an REPL itself.

This brought me back to the 'address' command I've been thinking about since the application started having all these scripting interfaces. The 'address' command goes back to my mainframe REXX days. It allows you to have a default execution environment that commands are 'addressed to', or to 'address' a single command to a different execution environment. So something like...
set alias.address org.cmdline.psuedoGestalt.Address
address
hp
address groovy x = 5
5
x
x not found
x = 6
address groovy
x
5
address hp
x
6

So 'hp' for HalfPipe is the default execution environment(REPL). This is some REPL-like functionality I added at some point. Not very complete but being sort of the built-in it seemed the appropriate default. The doc says actually the function address() tells you the current default but I haven't tried yet to add function support into HalfPipe. The indicated 'no args' use of toggling environments I don't remember ever using. Of course it's been a couple decades more or less. You change the default with something like 'address groovy'. You direct a single command to a different execution environment with something like address groovy x = 5. This might be another way around any conflicts that might come up between the shell and the REPL. REPL's maintaining some persistent state is different from the usual REXX execution environments. Of course mixing a shell and REPL(s) is maybe also a little different. But I did have the 'dojava' command before in HalfPipe and this seems potentially a whole lot more powerful than that. REPL's in general having of course become somewhat interesting to me from Coursera classes with Python and R.

For now all address does is manipulate the 'repl' System property. Like 'set repl groovy'. Although it could do more involved execution environment configuration and initialization at some point.

ps If anyone is wondering what's the deal with psuedoGestalt? The Gestalt api's were classic Mac OS. They told you other api versions and indicated general capabilities of the machine your program was running on. I came up with my java psuedoGestalt package in the early days of my java app with sort of the same ideas in mind, but not exactly, so 'psuedo'. I just found an empty skeleton of the Address class sitting in this package I must of thought of starting some time ago, but not quite that far back, I decided to go with just starting to fill it in for now. What was old is new again.

So anyhow, there are still plenty of very interesting things for me to look at.

If of interest to anyone else...
halfpipe.dmg

11/22/15

I did come up with reinstall9.sh. For somethings it just works better swapping the 9 build back in. Aliasing commands in .bash_profile just didn't work right. Quitting out of Terminal and then signing back in does seem necessary to get the changes to take.

11/18/15

I had wanted to continue running java 8 on my machine while using an embedded java 9 jdk 9 (my app I think needs jdk) in a test HalfPipe version to test out any Java 9 related. I had built Java 9 from source which seemed to symlink commands in /usr/local/bin. I tried to reinstall the latest generally available Java 8. The commands still seemed to point to the java 9 ones. It seems the usual install symlink's it's commands in /usr/bin. For me and my PATH /usr/local/bin would always be first. Shell scripts are not a skill I really have to any extent but I did a quick one uninstall9.sh that just unlinks the java 9 commands in /usr/local/bin. It seemed to work for everything but java itself last night. The java command seemed to be saying it still wanted something from /usr/bin/local. I left it that, booted up this morning and now that seems to be ok? Strange. Anyhow, that seemed to be the situation if you want to try something similar. Use the uninstall9.sh script at your own risk, it seems so far to have worked for me. A version could be done easily enough to relink the java 9 commands to 'reinstall'.

11/15/15

JDK 9 activity sort of major escalated with the release of the first fully modular supporting jvm. I should of done something with this by now. Getting back to it now anyhow. I didn't even remember how to build. I didn't 'make all' but just 'make', ended up with nothing useful in the images dir. I had to ask, found out 'make images' also works. To update the build a 'make clean' should be done. Mine failed on permissions so did a 'sudo make clean'. The build suggested a 'make reconfigure' which I did. Anyhow, I spliced it into the HalfPipe9 app. Deleted some stuff and ended up with a smaller upload anyhow. Still not a really stripped down modularized upload. I'm still confused on some things like why are there still separate jdk and jre builds? I thought these were going to be merged somehow? Anyhow, currently intend to try and put some time into all the radically new things now in place.

jshell too now it seems. Java itself now has a working REPL interface which HalfPipe still doesn't have.

08/01/15
Simple update that actually uses the JRT package directory as intended. A package name lookup for a 'modules' entry. Using the example from [6] below...

java -cp . JRTLister -p java.sql.Array
/modules/java.sql/java/sql/Array.class

The updated source is still at [7].

07/26/15

Yet another 'How to build JDK 9 on OS X

Well, I came across a few 'How to builds', not really all that many. None that mentioned exactly the things I ran into while building it.

Some things came up that I was sort of curious to look at in the latest b74 build. This was indicated to be one build past the available early access b73 build[1]. So I decided to try and build it.

The directions I followed were mostly from [2]. Which worked fine except for a couple issues I'll go over.

• If you have installed an early access release of Java 9 you can have a problem with the build. It requires a boot JDK installation which on OS X it looks for in /Library/Java/JavaVirtualMachines/. If it finds a java 9 early access as the latest JDK there your build will fail. I removed mine which left a Java 8 build as the latest which I believe is the recommended boot JDK.
• You need to have XQuartz installed. Which I did. But it seems there is a problem with having it installed and then installing Yosemite which I found on the archives for some openjdk list. Something like this one probably although I'm not sure it was exactly this particular one.[3]. You need to re-install XQuartz after Yosemite is installed. The openjdk forum indicates a problem with XQuartz 2.7.7 although I seemed to work fine in using that. FYI XQuartz no longer is distributed by Apple, you can get it at [4]. FWIW I had a bug report on the XQuartz dependency sometime during the macosx port project. I'm just saying.
• For the time being you need to do the ./configure a little differently as...
  ./configure --disable-warnings-as-errors
  This isn't mentioned at [2]. I remembered mention of the error on the openjdk awt-dev list [5] which mentioned the configure change. So there is a fix where this won't be needed anymore that just hasn't made it into the build yet.
• The other issues were user error, I missed after the ./configure you should do a 'make all', I also misread the [2] directions a little and was trying to do the make from the wrong directory.

So, thats about it, you can build. Have I done anything with it yet? Following what is detailed at [6] I added a -p option to JRTLister [7]. /packages now provides a sort of package name lookup index into the new module location of the classes.

[1] JDK™ 9 Early Access Releases
[2] How to build Open JDK 9 on Mac OSX Yosemite
[3] Yosemite problems with FreeType
[4] A version of the X.Org X Window System that runs on OS X
[5] AWT dev fix
[6] JDK-8066492
[7] JRTLister.java

05/03/15
OK, I have a HalfPipe7 limited, simple, gui in place to update Weka packages that are out of sync. It could probably both work and look better. It still is putting out debug messages, etc. I might suggest canceling out of any actual updates at this point or be sure to back up your wekafiles/packages first. To use it you have to set gui like...
set weka.pkgmgr.gui true
then as usual...
weka.pkgmgr.check
It had deadlock issues. I had to get away from using the commandline invocation of WekaPackageManager and instead directly call the install method. Like I said at this point, limited debugging, just got it seeming to work now, use at your own risk.

04/26/15
I have been looking at a number of things lately. Probably too many. So, not so much of an update this week. I'll just put together something on a few of the rather simple things I've done with Weka lately.

I am at this time sort of going with an alias'ed naming convention for the Weka related.
So we have...

weka.version Version of weka.jar that HalfPipe is using. I was thinking I could maybe do something with classloader's to have two different versions available that you could switch between. This would be sort of consistent with Weka's current on-going support of the two development branches 3.6.XX and 3.7.XX.

weka.pkgmgr Simple alias for Weka's package manager command line interface.

weka.pkgmgr.check This checks to see if any of your Weka packages appear to have a more current version available.

It is based on code I wrote for the Rhino Javascript and JRuby interfaces to allow them to call back into the HalfPipe command shell and get the System.out lines back as an array type. A simple way to to handle the hookup and get the output back in a parseable way. It is my intention at some point to come up a simple gui for this command where it will allow you to select the packages you want to refresh. I think the command line package manager interface for Weka will handle all of this, if and when it gets done.

weka.log I entered this alias one-liner as a easy way to list out the contents of the Weka log from HalfPipe. Basically, it just does a 'cat' command of the file. However, I noticed that for some reason HalfPipe output is ending up getting written to this Weka log file. This may be a bug in the i/o redirection used for the package manager interface. A bug somewhere that needs to be tracked down anyhow.

weka.cmds Last for the moment, this just lists out everything starting with alias.weka.

04/19/15
I noticed my install of the Java 9 early access didn't update the jdeps command which is supposed to be becoming more important. 'which jdeps' pointed into the the bin directory for a Java 8 JDK. I indicated this on the jigsaw-dev list and was told this was the other way some commands work.

That, I guess, is for someone else to sort out but suppose that you did want to use the current Java 9 jdeps from the HalfPipe9 application? Here is one occasion where it is fortunate that javapackager applications are different from appbundler. Originally, Apple JVM applications set user.dir to the parent directory of the java application. I objected to appbundler changing that to the User home directory. This was apparently in anticipation of sandboxing and restricted application file system access. I felt a little out of touch not realizing the sense that this made with the upcoming file system access crackdown. javapackager now seems to set user.dir to the Java directory inside the application bundle itself. I don't know how this change makes me out of touch, but I guess I have no big problems with it. It is convenient for accessing a command within the embedded JDK...
alias.jdeps9=exec ${user.dir}/../Plugins/Java/Contents/Home/bin/jdeps
which can be used like...
jdeps9 java.lang.String
java.lang (java.base)
-> java.io
-> java.nio.charset
-> java.util
-> java.util.regex
-> java.util.stream
(with a little better spacing than I'm trying to get in html)

I also got a working version of the application set up for Windows. I't be a while, working with Windows seemed a bit like wading through mud. But anyhow...
HalfPipe.zip
Contains a bat file that just has the java command using the contents of the two companion directories, Java and JavaApp. bat files were state of the art the last time I did much with Windows.

04/12/15
Following up on last week I did come up with a OS X version of the HalfPipe application with an embedded Java 9 JDK. There still seem to be some things that don't work but that is probably the current early access state of the JDK.

The download: HalfPipe9.dmg (WARNING: This is a 120M+ download)

I'm not quite sure why the size is so big, whether that's my not knowing how to compress the dmg as well as javapackager does or just the jdk currently being that much bigger I don't know.

The application includes the Java 9 commands for finding things from the 03/07 update. It could provide a convenient testing platform for code of your own using Java 9 without having to commit your installed JDK or JRE to Java 9. You could have a simple command to launch GUI code, it doesn't have to be 'just' command line. You would just have to take the jar and add it into the application at HalfPipe9.app/Contents/Java. Then update the Info.plist property JVMAppClasspath to include it. Remember ':' is the OS X path separator. I always thought sometime, someone would tell me they had added their own code into HalfPipe or it's predecessors and tell me it worked nicely for testing this, or doing that. This hasn't happened yet.

As mentioned in passing I used javapackager to start out the application build. I tried it understanding that it was sort of the 'official' successor to appbundler. This embedded a default JDK 8 runtime. I then modified this to get the Java 9 app. It is a promising tool for deploying java code on OS, handling application signing automatically, creating package installers, drag and drop application dmg's, even nicer default icons than the old generic application ones.

The two commands representing my present mastery of javapackager are...
javapackager -createjar -appclass us.hall.hp.common.LoaderLaunchStub -srcdir . -srcfiles launcher.jar -outdir . -outfile launcher9.jar -v

javapackager -deploy -srcdir . -srcfiles launcher9.jar -outdir . -outfile HalfPipe9 -appclass us.hall.hp.common.LoaderLaunchStub -native -name HalfPipe9

It took some googling to figure that out. There is apparently some prep work that needs to be done on the jar initially, as shown in the first command. First impression is that it is not really that appbundler like. Maybe I'll learn to make it more so or get used to the idea that it isn't. It is more full featured than I expected.

04/04/15
Since some functionality doesn't seem to be there yet in Java 9 I decided to try to keep a test version of 9 available but fallback to Java 8 for normal use.

Setting up a separate version of the application with an embedded Java 9 seemed a good way to go. However, it doesn't look like this works yet. The application doesn't launch and an error appears in the console. So I set up a simple script halfpipe9.sh to do the launch. That should be in the same directory as the HalfPipe7.app as it makes relative file references into it. I have given it's own launcher9.jar that skips locating the tools.jar. This file should be in the same directory as launcher9.jar. For Java 9 my understanding is that tools.jar is a thing of a past. You will either have a JDK with what it used to include or you will have one without. Your choice. The script launched seems to have problems with my own javascript JSR 223 interface to Rhino. That may prove to be unneeded as well. But anyhow shouldn't be critical to any initial Java 9 testing.

03/29/15
Added simple command to show what version of the Weka jar is included.
us.hall.weka.WekaVersion
3.7.11
Different package name from what I was using before but "A foolish consistency is the hobgoblin of little minds". I'm just sort of leaning more to the 'us.hall' packaging for newer stuff.

I had installed the Java 9 early access JDK and have started noticing a few things not working. dojava being one of them. My command for doing a quick in-line compile of simple java.

I decided I needed to start figuring out where things were in Java 9 so first expanded on a little code I found in the jigsaw-dev mail list archives. Which would do pretty much...
us.hall.cmdline.cmds.JRTLister
/java.base
/java.compiler
/java.datatransfer
/java.desktop
...
My changes add this to narrow that down...
us.hall.cmdline.cmds.JRTLister /jdk.jvmstat /jdk.jvmstat/sun
/jdk.jvmstat/sun/jvmstat
/jdk.jvmstat/sun/jvmstat/monitor
You can also indicate where to search and something to find like...
us.hall.cmdline.cmds.JRTLister * java/lang/String.class
/java.base/java/lang/String.class
where * is a wild card character. You have to quote it in OS X Terminal or you get strange results.
Or to check if it's in a specific module...
us.hall.cmdline.cmds.JRTLister /java.base java/lang/String.class
/java.base/java/lang/String.class
So listing or searching. After trying this I think maybe more useful for listing.
org.cmdline.cmds.Locator java/lang/String.class
jrt:/java.base/java/lang/String.class
org.cmdline.cmds.Locator weka/core/Version.class
jar:file:/Users/mjh/HalfPipe/HalfPipe7.app/Contents/Java/weka.jar!/weka/core/Version.class
Which is a 2 line ClassLoader bit of code I've used from time to time for a long while. Never made it into HalfPipe for some reason. Much shorter and probably more general than the jrt FileSystem command. Notice the 'classic' command naming. Consistency, you know.

Still haven't figured out why 'dojava' isn't working. I think I understand a little better why the 'whereis' command isn't working. Definitely stuff to do, just a small matter of doing it.

Not remembering at the moment how I was providing source before either. But for now you can get it here if interested halfpipe_src.zip

09/14/14
More HalfPipe assignment related for 'variables'.
In application terms...
x = `exec jps`
x
257
3091 LoaderLaunchStub
3109 Jps
y = x
y
257
3091 LoaderLaunchStub
3109 Jps
x = `set java.vm.v*`
x
java.vm.version=24.65-b04
java.vm.vendor=Oracle Corporation
y
257
3091 LoaderLaunchStub
3109 Jps
z = y[2]
z
3109 Jps
Should give you the gist.

I think I'm about ready to start with actual 'object' support now. Not just for assignment but try to start doing things with these variables. The current main two candidates for this are Weka related objects and OS X application objects. Weka was the original plan, so getting to that finally. For OS X current thinking is mainly codesign and related for now. I'm still thinking about MAS (is it? Trying to sound like I'm hip to the acronyms) submissions. So this would multi-task in improving halfpipe while I learn more about application signing.

09/01/14

In Brief
Updated the HalfPipe assignment code a little bit.
Now besides...
text = "Now is the time"
or...
x = 3
You can do...
javaprocs = `exec jps`
or
javaprops = `set java.*.*`
You can array reference these with something like javaprops[1]
You still can't really do much with them as far applying operators to them or using them in function calls
Not even x = javaprocs[1]
To give them a little more functionality, for now, you can use them as 'environment variables' like HalfPipe currently supports for System properties. I'm not sure this is a good idea yet, but for now it gives you a little more you can do with them like...
greeting = "Hello, World!"
echo $greeting
I think I also removed most of the debug messages for these.

List of courses taken

Machine Learning Weka and Kaggle and anything else Machine Learning related

HalfPipe

Update 04/20/14

Also haven't been putting as much time into this as I'd meant to. I do have some simple assignmnet code in place.

Like s = "Hello, World!" or x = 10. Next would be like java_tasks = `exec jps`. I'm figuring the Unix like back ticks right now. I've done similar for the Ruby and Javascript interfaces using ArrayCmdOut to receive the output on calls back into the shell.

Sort of wondering how to handle the arrays right now. More java like or more R like. Have a R vector like idea?

Then I still need to handle objects and interface to Weka/R. Anyhow putting up the current application with assignment as it is. Might spew debug messages yet as well for those.

HalfPipe

Current Weka interface information can be found on the Machine Learning page added above. Information on old related changes can now be found at the Past Updates page.

AppleScriptEngine

Besides that, for a change. I do have a new Mac. Thank you. Yes, very nice. I didn't get around to trying my HalfPipe application for a while. When I did, it wouldn't launch. The app has a launch dependency on the JSR 223 AppleScript interface updating running task information. This should probably be lazy loaded when needed and not at launch but that's another bug. The interface didn't seem to work, the app crashed NPE. Something in my new configuration seems to have uncovered a miss in the openjdk configuration for this. Possibly something disappeared in the Apple java stuff, anyhow they don't have to config right and it started crashing my app.

This stackoverlow link got me on the right track. Although it indicated the config is the jre/jdk rt.jar file. When I changed this the AppleScript interface did start turning up in test but didn't with the 'jrunscript -q' command. This post on the macosx-port list indicated the correct configuration file is in resources.jar. Changing that also got the jrunscript command showing AppleScript. I filed a bug report that was moved to the openjdk JIRA.

Now, some this legacy(?) Apple stuff doesn't seem entirely at home in the openjdk effort. I have already moved some of the file related to github already along with my trz project. There I had even added a FileHelper class to go with the Apple derived FileManager class to show that we, the Mac Java community, might be able to even extend this stuff if we open source support it.

At least for my application, it seemed to make sense to have this interface available dylib/jar instead of expecting any users to make changes to the JDK/JRE configuration. So I have also added that out to github as AppleScriptEngine. It pretty just includes the pertinent parts, the built jar/dylib and a test class. The test class lists available engines, should inlude AppleScriptEngine, and test runs a AppleScript. I used my pick card test. Although AppleScript is mostly supposed to be about inter-process communication I usually try to avoid test cases that involve any other applications. My other usual choice is using the Zarvox voice to say Hello, World! If anyone would prefer that, let me know.

R

While I was on the JSR 223 thing I decided to go ahead and finish adding the one for R I had looked at before. That is now included. in HalfPipe. There can only be one unique instance of R running so that interface keeps a static reference.

That makes use something like...
us.hall.scripting.RS
t = 3
class(t)
[CTRL+D to terminate System.in input]
numeric
echo this
this
[So we are clearly no longer inputing to R now right?]
us.hall.scripting.RS
class(t)
numeric

The basic idea is that the t variable is still set to the same value. One persistent instance.

Give me back my debug println's

Engines

So, that now brings HalfPipe to supporting five JSR 223 interfaces. I thought a little bit about that would be an idea. You can, of course, see what they are with the engines command. The output of that is shown here.

Briefly, what they are and what they are good for...

Mozilla Rhino (1.7 release 3 PRERELEASE) The builtin javascript. Supported by openjdk. Basically Rhino I think.

AppleScriptEngine (1.1) The Apple donated default. My application still has dependencies including launch for one.

JSR 223 JRuby Engine (1.7.4) JRuby. I did a rather basic FFI interface to the ObjectiveC runtime using this. I was thinking this would be the one implementation of that I would do and anything else would interface to it for Cocoa native access. It also supported callback command execution into the HalfPipe shell so HalfPipe could be scripted in JRuby.

Rhino JavaScript Script Engine (1.0.0) Before openjdk shipped it's Rhino engine I added this one of my own. That allowed me for one thing to add some code to JMX pid attach to any local java task. I modified a lot of the openjdk provided sample javascript to do this for JConsole to my app where, again, it could be used on any OS X java task with a pid. It also supports the same sort of callback back into the HalfPipe shell. So HalfPipe could be scripted in javascript.

REngine (0.7) And now R. With a current interest in machine learning this one is almost a must. I also added the jar for Weka (latest 3.7) to classpath. Now I just have to something with them.

The Weka Kaggle Titanic info being updated should be a to-do list item. I had some better success although still have been unable to crack the 80% accuracy mark. I may yet make a few more attempts with ensemble stacking, as detailed here Weka Tutorial 13: Stacking Multiple Classifiers I also added a couple more arff files with competition related data that could be used by those trying out Weka with the Titanic competition. Data changes alone did not seem to get me significant improvements.

01/22/14 It occurred to me that a little information concerning Weka on OS X might be useful to someone. I put that here

I am trying to find time to back off some of the education related and do a little tinkering again. Besides the R interface added to the HalfPipe application mentioned below, items currently sort of on the to-do list would include...

• I had the Weka libsvm support vector machine working at one point, I thought, now it never seems selectable, I have started doing a little browsing through the Weka source to understand that a little better. The builtin SMO implementation has done fairly well a couple times I used it. [I just opened up the classic iris data set and it selected and worked fine for the LIbSVM classifier. Worked pretty well too, 145 correct instances(95.66%), edging out the builtin SMO by 1 more correct. Anyhow, I'm going to consider this one done for now. Didn't figure out from the source though why the Titanic data wouldn't LibSVM select. Maybe it requires all nominal attributes or something? Iris is all numeric, so changed sepallength to nominal. LibSVM still worked, but not as well. Still sort of a mystery why it is unavailable for my Titanic data, but still considering this DONE 1/21/14]
• Other Kaggle competitions I am sort of interested in are the active Galaxy Zoo one. Although I'm not that into image processing. Also the competition closed last year related to political investigative reporting, "Follow the Money". I had a similar idea where you would see how industry campaign contributions seemed to correlate with elected officials voting. This would not necessarily be an indication of corruption. Republicans supposedly tend to vote pro-business as a general principle. Supporting local industry might just be a representative looking after his local economy. Actually detecting possible corruption might be like 'anomaly' detection as the Coursera Stanford Machine Learning class got into towards the end.
• There are the usual to-do's like figure out why my kqueue java WatchService locks up.
• See if gmp mutable math speeds up BouncyCastle based Elliptic Curve crypto.
• Probably others if I thought about it. Not sure yet if I'll find the time or what I want to commit it to. Hopefully will get going a little bit on something.

I did have one idea to start applying some of what I am learning to the HalfPipe application. First, add a JSR 223 implementation of the R language. I found one, but haven't got it working yet. Then, use that in conjunction with Weka for a REPL machine learning/data mining interface.

Not REPL in the sense that HalfPipe is REPL. Yes, it reads, evaluates, prints if needed and loops. But, normally it provides little in the way of 'scope'. When I implemented Rhino and JRuby for HalfPipe I added two parts for each. For each, a JSR 223 command was included that reads a script either from System.in command line or from a indicated file and runs it. The second part was where a'scope'd environment figured in. For Rhino I also included the release provided javascript shell. For JRuby I found source for the java implementation of the IRB shell. These provide a command line scope where you can assign things and then manipulate them, etc. This is an extra dimension I haven't figured out how to do with the JSR 223 implementations yet. This could be my incomplete understanding of that api. However, it is a dimension that I feel is very important to an implementation of a command line language or scripting api.

Anyhow, the idea is to put together a scoped environment for data mining. Use weka for the machine learning parts, R for data manipulation and maybe data representation, whatever else for whatever else. But, interactive and scoped.

Thats the theory anyhow, so far in the time I've put into it I haven't even gotten the 'R' JSR 223 interface working.

I would, of course, also still like to figure out my trz kqueue deadlock when there are lots of events, and have one or two other things I'd like to put time into as well. If and when I find that time.

Also note, I did my own JSR 223 interface for Rhino before openjdk on OS X was including any javascript JSR 223 interface at all. I added additional functionality to the ScriptableObject to include a couple of the useful Rhino shell functions, also a function to JMX pid attach to any other java application, and finally a function to do callback HalfPipe itself shell commands. This gave me functionality I don't think Nashorn or any other shipping JSR 223 interface would give me. Looking at Nashorn now that it is included with JDK releases remains on my to-do list.

The JMX pid attach allowed any java application to do JMX based javascript scripting of any other java application on the local machine. This was an extension of the jdk demo javascript scripting of the JConsole application, I think it was. I modified and extended these demo scripts for my own use.