A couple of people wrote to me about porting Qi II to Clojure. What
are the specific gains/losses in using ABCL versus Clojure? We've had
several goes with ABCL without success so far. I have no axes to
grind on this matter btw.

Mark

On May 5, 8:03 pm, Mark Tarver <drmtar> wrote:
> A couple of people wrote to me about porting Qi II to Clojure.  What
> are the specific gains/losses in using ABCL versus Clojure?  We've had
> several goes with ABCL without success so far.  I have no axes to
> grind on this matter btw.
>
> Mark

I've never used Clojure; I have been using almost only ABCL from the
past few months and I have even contributed some code to it. So take
what I say with a grain of salt, as I am quite certainly biased.

Language:
ABCL is a full CL (with a few bugs left wrt ANSI standard compliance,
and some de-facto standards - most notably MOP - missing or only
partially implemented). Clojure is a young language defined by its
implementation, so it's more subject to change from release to
release. Clojure in general is more functional than CL (for example it
uses immutable data structures by default), and more oriented towards
concurrency, but it misses some of CL's best features like CLOS and
the condition system. Not sure about macros in Clojure. Also CL code
is more regular (it's based completely on lists) while AFAIK Clojure
mixes lists with vectors/tuples. Both are dynamically typed, though I
ignore the details of Clojure's type system.

Maturity:
I guess both implementations have a similar level of maturity. ABCL
has been developed mostly by a single person for a while, and now has
a bunch of regular contributors; it's not far from being able to run
big CL projects like Maxima and AP5, though there are still a few
bugs. Clojure AFAIK is developed by a single individual but has many
third-party contributions. Clojure's community is probably bigger now,
and it's receiving much more hype, but I have heard ABCL has been used
in some production-level projects without problems - not sure about
Clojure.

Performance:
ABCL's performance is not bad (except CLOS is quite slow). It has a
compiler that produces Java bytecode. I don't know anything about
Clojure's performance, except that its immutable data structures are
very well designed and so have little performance impact (for being
immutable). Clojure's greater use of functional programming might
suffer more from the lack of TCO on the JVM, but I'm freely
speculating here (Clojure offers an operator, "recur", that implements
some kind of manual TCO, but I don't know the details).

Java Integration:
I have the feeling (correct me if I'm wrong) that Clojure makes a much
stronger attempt at integrating Lisp with Java, to the point of
porting some of Java's (mis)features to Lisp, e.g. source files are
required to adhere to a fixed filesystem layout based on packages
(namespaces). ABCL gives you the basic building blocks to cooperate
with Java and leaves up to you writing a higher-level abstraction over
them. There's one ("invoke.lisp") floating on the net; I have written
a simpler one myself with little effort. By "basic building blocks" I
mean things like creating new Java objects, invoking methods on them,
and creating "proxies" (implementations of Java interfaces created at
runtime) implemented in Lisp (useful for Java->Lisp callbacks). Also I
have contributed a little patch to ABCL that integrates a bit the Java
type system with CLOS: you get a JAVA-CLASS metaclass that mirrors the
Java inheritance hierarchy, and that can be used to specialize CLOS
methods (using a non-standard (jclass "class") specializer). Ok, I
know Qi doesn't use CLOS :) - I'm just trying to make a general
comparison.

That's all I can think of. I won't say what I think is "better"
between ABCL and Clojure, because you have to judge it by yourself,
and anyway "better" only makes sense relatively to the problem you
have to solve.

hth,
Alessio

On May 5, 2:03 pm, Mark Tarver <drmtar> wrote:
> A couple of people wrote to me about porting Qi II to Clojure.  What
> are the specific gains/losses in using ABCL versus Clojure?  We've had
> several goes with ABCL without success so far.  I have no axes to
> grind on this matter btw.


I would go with clojure because, if you arrange your compiler
appropriately, you will be able to generate .class files that will be
callable by any jvm language (just as clojure does), whereas going the
abcl route will require that anyone using Qi II -> abcl will end up
with something that will only run under Qi-abcl.

On May 5, 1:13 pm, Raffael Cavallaro <raffaelcavall>
wrote:
> On May 5, 2:03 pm, Mark Tarver <drmtar> wrote:
>
> > A couple of people wrote to me about porting Qi II to Clojure.  What
> > are the specific gains/losses in using ABCL versus Clojure?  We've had
> > several goes with ABCL without success so far.  I have no axes to
> > grind on this matter btw.
>
> I would go with clojure because, if you arrange your compiler
> appropriately, you will be able to generate .class files that will be
> callable by any jvm language (just as clojure does), whereas going the
> abcl route will require that anyone using Qi II -> abcl will end up
> with something that will only run under Qi-abcl.

That's a very good point. Btw, what in the architecture of ABCL stops
generating .class files? (Although my question might sounds like -
what stops us from having flying cars anywhere)...

On May 5, 10:13 pm, Raffael Cavallaro <raffaelcavall>
wrote:
> I would go with clojure because, if you arrange your compiler
> appropriately, you will be able to generate .class files that will be
> callable by any jvm language (just as clojure does), whereas going the
> abcl route will require that anyone using Qi II -> abcl will end up
> with something that will only run under Qi-abcl.

Well, I think things are a bit more complex here. Sure, Clojure
generates .class files while ABCL does not; however the .class files
will still require Clojure in the classpath, and so you'll still get
something that runs only under Qi-Clojure - only Qi (class files) and
Clojure (jar?) will be hidden to the user of the code, who will see a
"normal" compiled java class, if I understood it correctly.

With ABCL you can do e.g.

Function f = Interpreter.eval("#'my-compiled-function");
f.execute(...args...);

and hide that behind a tiny layer so the user of the code is not aware
of ABCL or Qi. Yes, this requires writing some glue Java code and
making sure the Lisp system is correctly initialized (ABCL
unfortunately lacks a save-image functionality, so you'll have to load
your files manually - for example using asdf - at startup).

Note also that ABCL supports the JSR-223 specification which allows
easier (and standardized) Java->Lisp interoperation, for example:

MyInterface obj = abclScriptEngine.getInterface(MyInterface.class);
obj.myMethodImplementedInLisp(); //Provided a function my-method-
implemented-in-lisp exists

Fixnum f = (Fixnum) abclScriptEngine.invokeFunction("+", 40, 2);
int i = f.value;

In short, with ABCL it's probably not as nice and clean as with
Clojure, but still very usable IMHO.

Alessio

On May 5, 10:25 pm, "Dimiter \"malkia\" Stanev" <mal>
wrote:
> On May 5, 1:13 pm, Raffael Cavallaro <raffaelcavall>
> wrote:
>>
>
> That's a very good point. Btw, what in the architecture of ABCL stops
> generating .class files? (Although my question might sounds like -
> what stops us from having flying cars anywhere)...

As far as I know ABCL is theoretically capable of generating
proper .class files. This functionality however is not exposed to the
user: the compiler writes a .class (actually .cls) file for every
compiled function, and autogenerates the class (and file) name. I know
little of ABCL's compiler, but I suspect it should not be terribly
complicated to pass to it the name and package of the class to
generate and thus obtain a .class file directly loadable by the Java
classloader. The one-class-per-function limitation is less likely to
be avoidable. Again this might be completely wrong as I'm not at all
an expert on ABCL's internals.

Ale

On May 5, 2:00 pm, Alessio Stalla <alessiosta> wrote:
[..]
> user: the compiler writes a .class (actually .cls) file for every
> compiled function, and autogenerates the class (and file) name. I know
> little of ABCL's compiler, but I suspect it should not be terribly
> complicated to pass to it the name and package of the class to
> generate and thus obtain a .class file directly loadable by the Java
> classloader. The one-class-per-function limitation is less likely to
> be avoidable. Again this might be completely wrong as I'm not at all
> an expert on ABCL's internals.
>
> Ale

Actually you are right. And also ABCL is more productive when comes to
disassemble on a function - it calls externally JAD (the famous
javabyte code disassembler) and displays the result.

I wish such feature was ready for clojure (probably people are doing
it other ways - there are lot of tools for introspecting the running
JavaVM - I've basically started typing "j" then TAB on my bash in Mac
OS X - and started discovering all such kinds of tools).

On May 5, 1:53 pm, Alessio Stalla <alessiosta> wrote:
[..]
> obj.myMethodImplementedInLisp(); //Provided a function my-method-
> implemented-in-lisp exists
>
> Fixnum f = (Fixnum) abclScriptEngine.invokeFunction("+", 40, 2);
> int i = f.value;
>
> In short, with ABCL it's probably not as nice and clean as with
> Clojure, but still very usable IMHO.
>
> Alessio

Thanks for the clarification!

I think I saw that Clojure also supports this JSR, if you are on
1.6.0.

On May 5, 11:18 pm, "Dimiter \"malkia\" Stanev" <mal>
wrote:
[..]
>> Actually you are right. And also ABCL is more productive when comes to
> disassemble on a function - it calls externally JAD (the famous
> javabyte code disassembler) and displays the result.
>
> I wish such feature was ready for clojure (probably people are doing
> it other ways - there are lot of tools for introspecting the running
> JavaVM - I've basically started typing "j" then TAB on my bash in Mac
> OS X - and started discovering all such kinds of tools).

Yes, JAD is very nice, although I had some problems finding a version
to run on my Linux box - it seems not to be developed anymore and the
site that distributed it is no longer available.
The same tool, or other decompilers, can probably be used with Clojure
with little or no effort, since you have access to its .class files (I
believe there's a plugin for Eclipse that decompiles a class file on
the fly when you open it in the editor).
You can also launch the JVM in debug mode and connect to it with a
graphical debugger (again, Eclipse supports this) - I've done it with
ABCL, but for things like stepping line-by-line it requires that you
have the Java source code available, which isn't possible with
bytecode generated from Lisp. In any case I think you should still be
able to see method calls, parameters, return values, and evaluate
watch expressions.

Returning on the previous subject (generating class files), I also
know abcl can (*) generate Java classes at runtime, with methods
implemented in Lisp, and maybe those could be dumped to file somehow.

(*) I have seen the code which does this, but never tried it, I don't
know if it is complete, or has issues. Probably the abcl maintainers
know better.

Ale

Mark Tarver schrieb:
> A couple of people wrote to me about porting Qi II to Clojure. What
> are the specific gains/losses in using ABCL versus Clojure? We've had
> several goes with ABCL without success so far. I have no axes to
> grind on this matter btw.

As I understood it one of the basic problems for you was to get ABCL
working. Provided that it works it should be easier for you to port
Qi - most of your code should just run.
That's what I see as the biggest advantage of using ABCL.
You won't need to rewrite Qi *again*. Modifying.. probably. But not
the whole thing from scratch.

If you want your port to the JVM be written in Clojure, then you will
need a rewrite. That means you would first have to learn Clojure itself
and then adobt your algorithms.

Also you should check how much of Qi is written in CLOS. Clojure does
not have that (officially and mature). If Qi is mostly written in a
functional style, then there may be benefits in rewriting it in Clojure.
It's minor, but some people prefer Lisp-1 for functional style.

One thing however could be difficult for you to decide if you decide to
rewrite Qi in Clojure for the JVM port: if you want to use Clojures
data structures. If you do it, this would mean that people will need
Clojure for Qi. There is no problem with the license, and in the end
what you will offer will be just one qi.jar file. But that would be
1,3mb bigger with Clojure. Also: if you don't plan to make Qi concurrent
itself, then you will notice a performance hit. Clojures datastructures
can't compete performance-wise in single thread applications against
the native JVM arrays or hashtables.
Of course, you could exclusively use those in Clojure programs as well.
But back to the "if you do it": the advantage would be, that Qi itself
and programs you write in it could be very easily parallelized for
practical matters. In concurrent situations Clojure vectors, sets and
maps perform very good. Rich Hickey markets Clojure as one or even the
fastest dynamic language on the JVM.

Even if you decide in the end to rewrite Qi in Clojure.. I would still
consider to first get it running under ABCL.
Btw, how many LOC is Qi?


André

On May 5, 11:19 pm, "Dimiter \"malkia\" Stanev" <mal>
wrote:
> On May 5, 1:53 pm, Alessio Stalla <alessiosta> wrote:
>>
>>
>>
>>
>> Thanks for the clarification!
>
> I think I saw that Clojure also supports this JSR, if you are on
> 1.6.0.

Good to know, since I'm the author of the JSR-223 implementation in
ABCL (so now you know who to blame when it'll fail :D), I should take
a look at how the Clojure folks have done it.

PS abcl supports it also on 1.5, provided that it finds javax.script
classes in the classpath when you build it and when you run it: there
are implementations of the JSR you can use on 1.5.

A.

On May 5, 11:38 pm, André Thieme <address.good.until.
2009.may...@justmail.de> wrote:
[..]
> not have that (officially and mature). If Qi is mostly written in a
> functional style, then there may be benefits in rewriting it in Clojure.
> It's minor, but some people prefer Lisp-1 for functional style.
>
> One thing however could be difficult for you to decide if you decide to
> rewrite Qi in Clojure for the JVM port: if you want to use Clojures
> data structures. If you do it, this would mean that people will need
> Clojure for Qi. There is no problem with the license, and in the end
> what you will offer will be just one qi.jar file. But that would be
> 1,3mb bigger with Clojure.

So Clojure can produce .class files that don't depend on it, if you
don't use its immutable data structures? Then what I said earlier
about .class files is not completely true, and Clojure really has an
advantage here (ABCL always generates code that depends on the ABCL
runtime, as far as I know). This probably also means Clojure is much
more integrated with Java than I thought (again, by comparison, ABCL
does not reuse Java's data types such as strings or numbers directly,
but wraps them in its own classes: every object managed by ABCL
extends a common base class, LispObject).
[..]

On 5 May, 22:38, André Thieme <address.good.until.
2009.may...@justmail.de> wrote:
[..]
> maps perform very good. Rich Hickey markets Clojure as one or even the
> fastest dynamic language on the JVM.
>
> Even if you decide in the end to rewrite Qi in Clojure.. I would still
> consider to first get it running under ABCL.
> Btw, how many LOC is Qi?
>
> André
> --
> Lisp is not dead. It’s just the URL that has changed:[..]

Qi II in Qi is only 2,000 lines. In CL its about 8,500. But you're
right. I don't want to write Qi again. This was the last and the
definitive rewrite. I can generate Clojure no problem (like
Python).

I've filed a bug report on ABCL and its been given as fixed but I find
no code to download. Somebody on Qilang did manage to install ABCL
under Linux but again hit a bug whereby the readtable reverted to the
default during loading Qi. See Qilang for details. This is our
second joint attempt to port Qi to ABCL.

My impression is that ABCL has quite a few bugs but from all the
hammering and banging coming from SourceForge, it will eventually
make it as a platform. I'm happy to cooperate with ABCL people in
getting my work to run on top of their platform.

Btw does ABCL have threads?

Mark

On May 5, 11:03 am, Mark Tarver <drmtar> wrote:
> A couple of people wrote to me about porting Qi II to Clojure.  What
> are the specific gains/losses in using ABCL versus Clojure?  We've had
> several goes with ABCL without success so far.  I have no axes to
> grind on this matter btw.
>
> Mark

Hi Mark,

I have one benchmark, based on the code from this document:
http://openmap.bbn.com/~kanderso/per...ipt/in-poly.ps

The times are taken from my MacBook Pro, and I won't post the results
here, but you can check them out in the links:

ABCL: http://paste.lisp.org/display/79729
Clojure: http://paste.lisp.org/display/79726

The ABCL is (off course) Common Lisp, so it can be compiled (and I've
did) under Allegro, Lispworks, SBCL, CMUCL and others.

I don't think this benchmark represents anything significant, it's too
fixed, but my give you some starting point.

Dimiter "malkia" Stanev schrieb:
> On May 5, 2:00 pm, Alessio Stalla <alessiosta> wrote:
>
> Actually you are right. And also ABCL is more productive when comes to
> disassemble on a function - it calls externally JAD (the famous
> javabyte code disassembler) and displays the result.
>
> I wish such feature was ready for clojure (probably people are doing
> it other ways - there are lot of tools for introspecting the running
> JavaVM - I've basically started typing "j" then TAB on my bash in Mac
> OS X - and started discovering all such kinds of tools).

But this feature *is* ready for Clojure since several months.
Clojure generates .class files. Run jad on them.

You want to run a very nice profiler that can paint you colorful images?
I am not sure if it was Java 6 Update 8 or later from which on VisualVM
gets shipped with the JDK. Look in the bin folder of the JDK for
jvisualvm.exe (under Windows, Linux probably has a similar name).
Check https://visualvm.dev.java.net/ for screencasts and screenshots.
Or better: run Clojure, start VisualVM and connect into your Clojure
image. You can start memory or cpu profiling and see it at work.
Also code obfuscators and all other kinds of tools work on the .class
files generated by Clojure.


André