spiralofhope

Mythryl >

taken from the mailing list

From: Cynbe ru Taren <cynbe at mythryl.org> 
To: Hue White <hue.white at gmail.com> 
Cc: mythryl at mythryl.org
Subject: Re: [Mythryl] shebang
Date: Tue, 22 Oct 2013 12:36:50 -0500


[ Please forgive my cc:-ing the list;
  we're wandering into territory of more general interest. ]

Hue White <hue.white at gmail.com> writes: 

> And to finalize the definition, the pit was covered with a tent or a makeshift 
> lean-to.? Camp Lawton was in the middle of rural South where heat easily 
> approached 100 degrees and humidity was even worse.? It was open only a few 
> months before Sherman's approach forced its closure.? Yet, just to make things 
> more miserable, it was subjected to ... a snowstorm.   

Ah, wars -- how we love them, godz help us!

Too bad nukes will spoil the fun.  :-/


> Honestly, I've never attached a term to that particular bit of syntax.? And 
> heavens knows I've written too many scripts (usually ksh or bash - should have 
> switched to PERL a long time ago, but i don't care for the look of such 
> scripts) in my time. 
> 
> The closest I've come to attaching a name to these sorts of things is picking 
> up "shriek" for "!" in vi, and that from a detested acquaintance.   

Ah!  In mud culture circa 1990 that was always 'bang'.  Idiots were "bang clue".


> It always struck me that pretty printing required the front end of a compiler 
> in order to really understand how to format the code and bring out its 
> essence.  

Programs like 'cb' ("C beautifier") may use a real parser(?);  emacs and kin
just do kludgey sorta-parsing, in the general unix spirit of "hack something
up that works 90% of the time and ship it".

I've always had, and continue to have, terribly mixed feelings about that.

On the one hand, quick-and-dirty ad hoc solutions are a very effective form
of "first with the worst" which can be understood as riding an exponential
effectively by keeping the cycle time as short as possible.  The first cut
may suck but it gets you through the first cycle fast, and with the first-cut
solution in hand you're in a better position to do a better second cut than the
poor slob taking ten times as long to Do It Right.  By the time he ships, the
quick-and-dirty crowd are on their tenth generation and have attracted a critical
mass of users.

Also, in this specific case, quick-and-dirty means that one can process code
which is incomplete or incorrect, which is a real asset in a programming editor
where most of the code to be displayed is in fact incomplete or incorrect.

On the other hand, this sort of ad-hoc craftsmanship is why for thousands of
years our bridges fell down on a regular basis.  Unix programs have always
coredumped copiously when fed random input. (One peer-reviewed study did
literally that and found that virtually all unix programs crashed on randomly
generated input.)  That makes the entire system a straitjacket for conformity:
stray but a little from the path and all the tools turn and bite you.  That
doesn't encourage or facilitate brilliant innovative hacking taking things to
the next level.  At some point it would be really nice to take things to the
next level where software by and large Just Works, at least at a qualitative
level.  (I think it is inevitable that software will always be most efficient
on the most common patterns of use.  You cannot optimize for everything, so
you optimize for the most common cases.)

emacs is culturally sort of a transitional form.  It doesn't have the sort of
underlying analysis that you see in https://web.archive.org/web/20160407095249/http://www.cs.cmu.edu/~rwh/plbook/book.pdf
-- emacs is fundamentally old-style craftsmanship rather than modern engineering
in implementation style -- but Stallman does insist strongly that tools should
Just Work, and consequently emacs is the only common editor (species) that can
routinely edit binary executables and such.

Formatting programming text is a confusing middle case where the quick-and-dirty
approach has in practice more of a Just Works feel because it works reasonably
well on even broken source code, whereas a parser-based approach is likely to
just Not Work on any code which isn't 100% valid.



> Since I never really had access to compiler guts, I think I only 
> tried once.? (Evidently I've expunged the effort from my memory.)? The 
> prettyprinter I wrote to format and index the Citadel manuals doesn't count, of 
> course. 
> 
> I wonder, do you see this as a foundation for your projected Mythryl IDE?   

I definitely see this as one major component.  I want to have some sort of
Visible Compiler mode where you show the complete compile for a given source
file (say) in something like outline form, and you can expand any given pass
and see the intermediate code form for that compiler pass running on that
sourcefile in nicely prettyprinted form, along with tooltips or clickable
identifiers or whatever giving instant access to the relevant symbol table
entries etc.

However I continue to be of the opinion that text presentation is mostly for
programming-in-the-small and that programming-in-the-large is most naturally
done with a graphical (boxes-and-arrows) presentation.  Ask someone how
quicksort works and they'll write pseudocode;  ask them how gcc works and
they'll draw boxes and arrows showing the paths.  Even a very small graph
becomes humanly incomprehensible when rendered in text form as lists of
nodes and edges (which is what much OOP startup code in essence is -- lists
of objects and their connections in baffling text presentation).

The classic AVS scientific visualization system let you drag nodes onto a
blank canvas and connect them with dataflow edges and run the result as a
program.  The graph source nodes were things like video cameras and files,
the sinks were things like OpenGL display windows, and the interior nodes
were things like edge detection processes.  It was very effective and has
been much imitated, with good reason.

I envision the heart of a Mythryl IDE as being a GraphView-inspired knobs-and-tubes
programming canvas on which one can lay out communicating microthreads by
drag-drop-and-link, and watch them run with flashing colors showing activity,
dropping into emacs-style text editing mainly when dealing with the guts of
a particular microthread or such.

More generally, one crosses the boundary between essentially toyworld
programming to realworld programming when one crosses over from
dealing with tree structures like compiler intermediate code and
starts dealing with graph structures and matrices.  Graph structures
are essentially boolean matrices, or conversely matrices are
essentially graphs with weighted edges.  Serious real-world
programming is going to be intensively about visualizing and
processing graphs and matrices.  (And tensors?  I've never grokked
them.  But they are ubiquitous in real-world contexts as simple as
elastic solids.)

So, anyhow, one reason I'm pushing so hard to get usable, evolvable X
support into Mythryl is that one can't really assemble even a first cut
at that sort of development environment until one has both microthreads
and graphical displays to present and edit them.


> Yes, the Generics are *quite* useful - one of the first things I think about 
> when writing Mythryl is to wonder if I can write whatever it is as a generic 
> for later reuse.? I find I use them in the same way as I do for abstract 
> classes in OOP land - hope I'm not missing any tricks.? I do tend to overuse 
> them, sometimes - my first effort at a statistical library started with 
> generics, but I discarded that approach fairly quickly as anonymous types was 
> the better - and more obvious to most folks, I suspect - choice.   

Generics are SML's (hence Mythryl's) greatest advantage over other languages,
and I'll be astonished if it doesn't turn out that we are all barely scratching
the surface of their potential.  They are a compile-time package language, and
we've never had anything like that before.  It took us until quite recently
just to start getting a clue about what we can do with phantom types;  generics
are a much deeper pool, so how can we possibly have plumbed its depths if we're
just beginning to understand phantom types?  Mythryl inherits recursive generics
from SML/NJ;  I haven't heard or seen anyone even beginning to apply them. :-)

Anyhow, I'm far short of even understanding generics at the current state of
the art level, much less the full potential level.



> Say - just to punch you in the side - keep thinking about that customization 
> point for handling auto-generation of code (I can't recall how you phrased 
> this).? I'd love to see dimensional math become feasible - the package I wrote 
> does what it does nicely, but it doesn't "scale" (if I misuse the word 
> slightly) very well to real world problems.? What you described sounded just 
> about perfect for resolving problems of that sort.? ("that sort" makes me think 
> the problem should be better defined.)? Not that I have an aching need for such 
> things, it just sounds "open source sexy".? The current problem (on hold while 
> I do other things) is a logistic regression implementation.? I had to retrieve 
> my book on linear algebra as the only almost-comprehensible paper I've found so 
> far was using matrix math.? I'm afraid my brain just isn't really built to 
> understand mathematics lingo....   

As I see it, the whole compiler world has to date focussed almost exclusively on
space and time efficiency, which at this point is largely mined out and in any
event not really interesting -- languages like Python which are 100X slower than
compiled languages are flourishing precisely because execution efficiency doesn't
matter at all for 90% of programs and matters in only 1% of the code in the remaining
10% of programs.  Low-level constant-factor space and time efficiency are in essence
an obscure niche issue in programming best left to the specialists who write lapack etc.

The future of compiling and languages and software platforms is not about efficiency
of execution but robustness of product and efficiency of development and maintenance,
with the focus squarely on type systems in particular and automated code analysis and
transformation in general.

The SML generic system is the best-developed glimpse we currently have of that future;
a general compile-time algebra of packages and operators which transform input packages
into output packages.  The SML generic system is light years ahead of other languages,
even Java, but I strongly suspect will look quite antique, primitive and naive in 20 years.

As sort of a complement to the generic system, which does some simple principled things
very well, I want to introduce a plug-in system (even gcc is doing the plug-in dance :-)
which will let one define a package foo which does a transform of raw syntax tree to raw
syntax tree (say) and then write

    foo package bar: Bar {
        ...
    };

to indicate that the compiler should run the raw syntax tree for 'bar' through 'foo'
before doing actual typechecking and compile.

This will allow a wide range of code generation automation in a very natural and effective style,
while keeping the compiler proper simple and focussed.

For example the first-cut OOP-support logic (actually two of them) which I currently have
can be moved out of the compiler into a library.  If the relevant package is called 'class'
then we can write

    class package bar: Bar {
        ...
    };

to write conventional oop-flavored code in Mythryl with "compiler" support for the
dreary vtable type stuff that nobody wants to code up by hand.

You may or may not believe that conventional oop has much of a place in Mythryl, but
very similar issues arrise (I've discovered) when writing concurrent code in Mythryl.

I refer to the nodes in the concurrent-microthread-graph as 'imps' (daemons are
server processes; imps are server microthreads).  To participate effectively in the
concurrent-coding ecology, imps need to be much more than just microthreads; they
need to obey established start-up, shut-down and communication protocols in order to
interoperate smoothly, just as a daemon has to be much more than just a random process
in order to work effectively in the ecology of a running unix -- a daemon has to work
with the standard /etc/init.d/* or whatever startup-up/shut-down protocol for example.

So Mythryl imps wind up having a fair amount of boilerplate type code that has to be
cranked out, which is too type-sensitive to be canned in library routines and too
application-sensitive to be canned in a conventional generic.  The above facility
plus an imp.pkg compiler plug-in will let us write

    imp package lapack: Lapack {
        ...
    };

to define concisely a drag-and-drop impgraph component which applies LAPACK transforms
to a stream of matrices, say.

Much the same plug-in framework could provide for interventions in the typechecking
system (vs raw-syntax-tree processing) to patch little bits of behavior like that
needed for proper dimensioned-variables handling;  that could be a whole little essay
by itself. :-)

Lots of fun stuff like that pending, pushing the state of the art in programming
environments.

But first I/we have to get the fundamental stuff like prettyprinting and basic X
windows support working...


Life is Good!
 -- Cynbe
 

_______________________________________________
Mythryl mailing list
Mythryl at mythryl.org
https://web.archive.org/web/20160312032828/http://mythryl.org/mailman/listinfo/mythryl