451 lines
14 KiB
ReStructuredText
451 lines
14 KiB
ReStructuredText
Joypy
|
||
=====
|
||
|
||
Joy in Python
|
||
-------------
|
||
|
||
This implementation is meant as a tool for exploring the programming
|
||
model and method of Joy. Python seems like a great implementation
|
||
language for Joy for several reasons.
|
||
|
||
We can lean on the Python immutable types for our basic semantics and
|
||
types: ints, floats, strings, and tuples, which enforces functional
|
||
purity. We get garbage collection for free. Compilation via Cython. Glue
|
||
language with loads of libraries.
|
||
|
||
`Read-Eval-Print Loop (REPL) <https://en.wikipedia.org/wiki/Read%E2%80%93eval%E2%80%93print_loop>`__
|
||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||
|
||
The main way to interact with the Joy interpreter is through a simple
|
||
`REPL <https://en.wikipedia.org/wiki/Read%E2%80%93eval%E2%80%93print_loop>`__
|
||
that you start by running the package:
|
||
|
||
::
|
||
|
||
$ python -m joy
|
||
Joypy - Copyright © 2017 Simon Forman
|
||
This program comes with ABSOLUTELY NO WARRANTY; for details type "warranty".
|
||
This is free software, and you are welcome to redistribute it
|
||
under certain conditions; type "sharing" for details.
|
||
Type "words" to see a list of all words, and "[<name>] help" to print the
|
||
docs for a word.
|
||
|
||
|
||
<-top
|
||
|
||
joy? _
|
||
|
||
The ``<-top`` marker points to the top of the (initially empty) stack.
|
||
You can enter Joy notation at the prompt and a `trace of
|
||
evaluation <#The-TracePrinter.>`__ will be printed followed by the stack
|
||
and prompt again:
|
||
|
||
::
|
||
|
||
joy? 23 sqr 18 +
|
||
. 23 sqr 18 +
|
||
23 . sqr 18 +
|
||
23 . dup mul 18 +
|
||
23 23 . mul 18 +
|
||
529 . 18 +
|
||
529 18 . +
|
||
547 .
|
||
|
||
547 <-top
|
||
|
||
joy?
|
||
|
||
Stacks (aka list, quote, sequence, etc.)
|
||
========================================
|
||
|
||
In Joy, in addition to the types Boolean, integer, float, and string,
|
||
there is a single sequence type represented by enclosing a sequence of
|
||
terms in brackets ``[...]``. This sequence type is used to represent
|
||
both the stack and the expression. It is a `cons
|
||
list <https://en.wikipedia.org/wiki/Cons#Lists>`__ made from Python
|
||
tuples.
|
||
|
||
.. code:: ipython2
|
||
|
||
import inspect
|
||
import joy.utils.stack
|
||
|
||
|
||
print inspect.getdoc(joy.utils.stack)
|
||
|
||
The utility functions maintain order.
|
||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||
|
||
The 0th item in the list will be on the top of the stack and *vise
|
||
versa*.
|
||
|
||
.. code:: ipython2
|
||
|
||
joy.utils.stack.list_to_stack([1, 2, 3])
|
||
|
||
.. code:: ipython2
|
||
|
||
list(joy.utils.stack.iter_stack((1, (2, (3, ())))))
|
||
|
||
This requires reversing the sequence (or iterating backwards) otherwise:
|
||
|
||
.. code:: ipython2
|
||
|
||
stack = ()
|
||
|
||
for n in [1, 2, 3]:
|
||
stack = n, stack
|
||
|
||
print stack
|
||
print list(joy.utils.stack.iter_stack(stack))
|
||
|
||
Purely Functional Datastructures.
|
||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||
|
||
Because Joy lists are made out of Python tuples they are immutable, so
|
||
all Joy datastructures are `purely
|
||
functional <https://en.wikipedia.org/wiki/Purely_functional_data_structure>`__.
|
||
|
||
The ``joy()`` function.
|
||
=======================
|
||
|
||
An Interpreter
|
||
--------------
|
||
|
||
The ``joy()`` function is extrememly simple. It accepts a stack, an
|
||
expression, and a dictionary, and it iterates through the expression
|
||
putting values onto the stack and delegating execution to functions it
|
||
looks up in the dictionary.
|
||
|
||
Each function is passed the stack, expression, and dictionary and
|
||
returns them. Whatever the function returns becomes the new stack,
|
||
expression, and dictionary. (The dictionary is passed to enable
|
||
e.g. writing words that let you enter new words into the dictionary at
|
||
runtime, which nothing does yet and may be a bad idea, and the ``help``
|
||
command.)
|
||
|
||
.. code:: ipython2
|
||
|
||
import joy.joy
|
||
|
||
print inspect.getsource(joy.joy.joy)
|
||
|
||
View function
|
||
~~~~~~~~~~~~~
|
||
|
||
The ``joy()`` function accepts a “viewer” function which it calls on
|
||
each iteration passing the current stack and expression just before
|
||
evaluation. This can be used for tracing, breakpoints, retrying after
|
||
exceptions, or interrupting an evaluation and saving to disk or sending
|
||
over the network to resume later. The stack and expression together
|
||
contain all the state of the computation at each step.
|
||
|
||
The ``TracePrinter``.
|
||
~~~~~~~~~~~~~~~~~~~~~
|
||
|
||
A ``viewer`` records each step of the evaluation of a Joy program. The
|
||
``TracePrinter`` has a facility for printing out a trace of the
|
||
evaluation, one line per step. Each step is aligned to the current
|
||
interpreter position, signified by a period separating the stack on the
|
||
left from the pending expression (“continuation”) on the right.
|
||
|
||
`Continuation-Passing Style <https://en.wikipedia.org/wiki/Continuation-passing_style>`__
|
||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||
|
||
One day I thought, What happens if you rewrite Joy to use
|
||
`CSP <https://en.wikipedia.org/wiki/Continuation-passing_style>`__? I
|
||
made all the functions accept and return the expression as well as the
|
||
stack and found that all the combinators could be rewritten to work by
|
||
modifying the expression rather than making recursive calls to the
|
||
``joy()`` function.
|
||
|
||
Parser
|
||
======
|
||
|
||
.. code:: ipython2
|
||
|
||
import joy.parser
|
||
|
||
print inspect.getdoc(joy.parser)
|
||
|
||
|
||
.. parsed-literal::
|
||
|
||
This module exports a single function for converting text to a joy
|
||
expression as well as a single Symbol class and a single Exception type.
|
||
|
||
The Symbol string class is used by the interpreter to recognize literals
|
||
by the fact that they are not Symbol objects.
|
||
|
||
A crude grammar::
|
||
|
||
joy = term*
|
||
term = int | float | string | '[' joy ']' | symbol
|
||
|
||
A Joy expression is a sequence of zero or more terms. A term is a
|
||
literal value (integer, float, string, or Joy expression) or a function
|
||
symbol. Function symbols are unquoted strings and cannot contain square
|
||
brackets. Terms must be separated by blanks, which can be omitted
|
||
around square brackets.
|
||
|
||
|
||
The parser is extremely simple, the undocumented ``re.Scanner`` class
|
||
does most of the tokenizing work and then you just build the tuple
|
||
structure out of the tokens. There’s no Abstract Syntax Tree or anything
|
||
like that.
|
||
|
||
.. code:: ipython2
|
||
|
||
print inspect.getsource(joy.parser._parse)
|
||
|
||
|
||
.. parsed-literal::
|
||
|
||
def _parse(tokens):
|
||
'''
|
||
Return a stack/list expression of the tokens.
|
||
'''
|
||
frame = []
|
||
stack = []
|
||
for tok in tokens:
|
||
if tok == '[':
|
||
stack.append(frame)
|
||
frame = []
|
||
stack[-1].append(frame)
|
||
elif tok == ']':
|
||
try:
|
||
frame = stack.pop()
|
||
except IndexError:
|
||
raise ParseError('Extra closing bracket.')
|
||
frame[-1] = list_to_stack(frame[-1])
|
||
else:
|
||
frame.append(tok)
|
||
if stack:
|
||
raise ParseError('Unclosed bracket.')
|
||
return list_to_stack(frame)
|
||
|
||
|
||
|
||
That’s pretty much all there is to it.
|
||
|
||
.. code:: ipython2
|
||
|
||
joy.parser.text_to_expression('1 2 3 4 5') # A simple sequence.
|
||
|
||
|
||
|
||
|
||
.. parsed-literal::
|
||
|
||
(1, (2, (3, (4, (5, ())))))
|
||
|
||
|
||
|
||
.. code:: ipython2
|
||
|
||
joy.parser.text_to_expression('[1 2 3] 4 5') # Three items, the first is a list with three items
|
||
|
||
|
||
|
||
|
||
.. parsed-literal::
|
||
|
||
((1, (2, (3, ()))), (4, (5, ())))
|
||
|
||
|
||
|
||
.. code:: ipython2
|
||
|
||
joy.parser.text_to_expression('1 23 ["four" [-5.0] cons] 8888') # A mixed bag. cons is
|
||
# a Symbol, no lookup at
|
||
# parse-time. Haiku docs.
|
||
|
||
|
||
|
||
|
||
.. parsed-literal::
|
||
|
||
(1, (23, (('four', ((-5.0, ()), (cons, ()))), (8888, ()))))
|
||
|
||
|
||
|
||
.. code:: ipython2
|
||
|
||
joy.parser.text_to_expression('[][][][][]') # Five empty lists.
|
||
|
||
|
||
|
||
|
||
.. parsed-literal::
|
||
|
||
((), ((), ((), ((), ((), ())))))
|
||
|
||
|
||
|
||
.. code:: ipython2
|
||
|
||
joy.parser.text_to_expression('[[[[[]]]]]') # Five nested lists.
|
||
|
||
|
||
|
||
|
||
.. parsed-literal::
|
||
|
||
((((((), ()), ()), ()), ()), ())
|
||
|
||
|
||
|
||
Library
|
||
=======
|
||
|
||
The Joy library of functions (aka commands, or “words” after Forth
|
||
usage) encapsulates all the actual functionality (no pun intended) of
|
||
the Joy system. There are simple functions such as addition ``add`` (or
|
||
``+``, the library module supports aliases), and combinators which
|
||
provide control-flow and higher-order operations.
|
||
|
||
.. code:: ipython2
|
||
|
||
import joy.library
|
||
|
||
print ' '.join(sorted(joy.library.initialize()))
|
||
|
||
|
||
.. parsed-literal::
|
||
|
||
!= % & * *fraction *fraction0 + ++ - -- / // /floor < << <= <> = > >= >> ? ^ _Tree_add_Ee _Tree_delete_R0 _Tree_delete_clear_stuff _Tree_get_E abs add anamorphism and app1 app2 app3 at average b binary bool branch ccons choice clear cleave cmp codireco concat cond cons dinfrirst dip dipd dipdd disenstacken divmod down_to_zero drop dup dupd dupdd dupdip dupdipd enstacken eq first first_two flatten floor floordiv fork fourth gcd ge genrec getitem gt help i id ifte ii infer infra inscribe le least_fraction loop lshift lt make_generator map max min mod modulus mul ne neg not nullary of or over pam parse pick pm pop popd popdd popop popopd popopdd pow pred primrec product quoted range range_to_zero rem remainder remove rest reverse roll< roll> rolldown rollup round rrest rshift run second select sharing shunt size sort sqr sqrt stack step step_zero stuncons stununcons sub succ sum swaack swap swoncat swons take ternary third times truediv truthy tuck unary uncons unique unit unquoted unstack unswons void warranty while words x xor zip •
|
||
|
||
|
||
Many of the functions are defined in Python, like ``dip``:
|
||
|
||
.. code:: ipython2
|
||
|
||
print inspect.getsource(joy.library.dip)
|
||
|
||
|
||
.. parsed-literal::
|
||
|
||
@inscribe
|
||
@combinator_effect(_COMB_NUMS(), a1, s1)
|
||
@FunctionWrapper
|
||
def dip(stack, expression, dictionary):
|
||
'''
|
||
The dip combinator expects a quoted program on the stack and below it
|
||
some item, it hoists the item into the expression and runs the program
|
||
on the rest of the stack.
|
||
::
|
||
|
||
... x [Q] dip
|
||
-------------------
|
||
... Q x
|
||
|
||
'''
|
||
(quote, (x, stack)) = stack
|
||
expression = (x, expression)
|
||
return stack, concat(quote, expression), dictionary
|
||
|
||
|
||
|
||
Some functions are defined in equations in terms of other functions.
|
||
When the interpreter executes a definition function that function just
|
||
pushes its body expression onto the pending expression (the
|
||
continuation) and returns control to the interpreter.
|
||
|
||
.. code:: ipython2
|
||
|
||
print joy.library.definitions
|
||
|
||
|
||
.. parsed-literal::
|
||
|
||
? == dup truthy
|
||
*fraction == [uncons] dip uncons [swap] dip concat [*] infra [*] dip cons
|
||
*fraction0 == concat [[swap] dip * [*] dip] infra
|
||
anamorphism == [pop []] swap [dip swons] genrec
|
||
average == [sum 1.0 *] [size] cleave /
|
||
binary == nullary [popop] dip
|
||
cleave == fork [popd] dip
|
||
codireco == cons dip rest cons
|
||
dinfrirst == dip infra first
|
||
unstack == ? [uncons ?] loop pop
|
||
down_to_zero == [0 >] [dup --] while
|
||
dupdipd == dup dipd
|
||
enstacken == stack [clear] dip
|
||
flatten == [] swap [concat] step
|
||
fork == [i] app2
|
||
gcd == 1 [tuck modulus dup 0 >] loop pop
|
||
ifte == [nullary not] dipd branch
|
||
ii == [dip] dupdip i
|
||
least_fraction == dup [gcd] infra [div] concat map
|
||
make_generator == [codireco] ccons
|
||
nullary == [stack] dinfrirst
|
||
of == swap at
|
||
pam == [i] map
|
||
primrec == [i] genrec
|
||
product == 1 swap [*] step
|
||
quoted == [unit] dip
|
||
range == [0 <=] [1 - dup] anamorphism
|
||
range_to_zero == unit [down_to_zero] infra
|
||
run == [] swap infra
|
||
size == 0 swap [pop ++] step
|
||
sqr == dup mul
|
||
step_zero == 0 roll> step
|
||
swoncat == swap concat
|
||
ternary == unary [popop] dip
|
||
unary == nullary popd
|
||
unquoted == [i] dip
|
||
while == swap [nullary] cons dup dipd concat loop
|
||
|
||
|
||
|
||
Currently, there’s no function to add new definitions to the dictionary
|
||
from “within” Joy code itself. Adding new definitions remains a
|
||
meta-interpreter action. You have to do it yourself, in Python, and wash
|
||
your hands afterward.
|
||
|
||
It would be simple enough to define one, but it would open the door to
|
||
*name binding* and break the idea that all state is captured in the
|
||
stack and expression. There’s an implicit *standard dictionary* that
|
||
defines the actual semantics of the syntactic stack and expression
|
||
datastructures (which only contain symbols, not the actual functions.
|
||
Pickle some and see for yourself.)
|
||
|
||
“There should be only one.”
|
||
^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||
|
||
Which brings me to talking about one of my hopes and dreams for this
|
||
notation: “There should be only one.” What I mean is that there should
|
||
be one universal standard dictionary of commands, and all bespoke work
|
||
done in a UI for purposes takes place by direct interaction and macros.
|
||
There would be a *Grand Refactoring* biannually (two years, not six
|
||
months, that’s semi-annually) where any new definitions factored out of
|
||
the usage and macros of the previous time, along with new algorithms and
|
||
such, were entered into the dictionary and posted to e.g. IPFS.
|
||
|
||
Code should not burgeon wildly, as it does today. The variety of code
|
||
should map more-or-less to the well-factored variety of human
|
||
computably-solvable problems. There shouldn’t be dozens of chat apps, JS
|
||
frameworks, programming languages. It’s a waste of time, a `fractal
|
||
“thundering herd”
|
||
attack <https://en.wikipedia.org/wiki/Thundering_herd_problem>`__ on
|
||
human mentality.
|
||
|
||
Literary Code Library
|
||
^^^^^^^^^^^^^^^^^^^^^
|
||
|
||
If you read over the other notebooks you’ll see that developing code in
|
||
Joy is a lot like doing simple mathematics, and the descriptions of the
|
||
code resemble math papers. The code also works the first time, no bugs.
|
||
If you have any experience programming at all, you are probably
|
||
skeptical, as I was, but it seems to work: deriving code mathematically
|
||
seems to lead to fewer errors.
|
||
|
||
But my point now is that this great ratio of textual explanation to wind
|
||
up with code that consists of a few equations and could fit on an index
|
||
card is highly desirable. Less code has fewer errors. The structure of
|
||
Joy engenders a kind of thinking that seems to be very effective for
|
||
developing structured processes.
|
||
|
||
There seems to be an elegance and power to the notation.
|
||
|