#8

tehn that’s very exciting to hear!

#9

just discovered this


for these
https://en.wikibooks.org/wiki/Mizar32
…and remembered this thread
can’t be much work to get picolisp running on aleph/eurorack - think they’re targetting the same cpu (just did a build for AT32UC3A0512 that’s the one right?).

Bees refactoring is getting a bit brutal and I’m somewhat worried:

A. that my attempt to refactor will become a complete rewrite (how will I ever get time to just get everything back to where it was after ‘fixing’ the things that seemed wrong to me)
B. that I will want more and more repl-style support for bees to make patching more like coding
C. that my attempts at smarter memory management will come unstuck

it’s all starting to smell a bit like greenspuns 10th rule.

Going to down tools for a while from bees refactor to fiddle around with @zebra’s forth build and picolisp. Curiosity killed the cat but think I need to look around at some other ways to skin this particular kitty. At worst I can go back to improving BEES having silenced the voice saying ‘there has to be a better way’…

#10

It’s no fun when each improvement you make results in another 5 changes needing to be made. I’ve had refactorings like that before. It’s especially depressing when the smallest atomic change you can make is larger in scope than you can comfortably fit in your brain.

I will say that even if it doesn’t come of, you’ll have learned an awful lot about how bees works, and hopefully that knowledge will be very useful.

#11

hmm I’m more and more curious about picolisp - just finished another bit of horrendous refactoring at work (common lisp code to address a truly awful spaghetti-problem in a horribly mangled 30-year-old mudball). The relative ease of doing this has made my mind up - I have to at least try and get a lisp REPL on the aleph - my brain seems to work much better slinging parentheses & there’s a long weekend coming up…

So @zebra, @tehn what is the danger of flashing some random .hex file I have built from the hempl repository? (using avr32-objcopy to convert from the .elf). Remember reading somewhere that you guys went to great pains to ensure that it wasn’t possible to overwrite the bootloader with dodgy firmware? Does that come with the caveat that the hex is built against avr32lib? Only question to my mind is whether any difference the way usb UART is hooked up on the mizar32 boards compared to aleph. Well picolisp is like 5 .c files so maybe the easiest way is try to figure out how to hook it straight into avr32_lib…

Well I guess worst that happens is I have to get the jtagice programmer to get things back to working (which I should get anyway if I decide to finish the job on bees refactor).

#12

You can’t overwrite the bootloader without unlocking the relevant flash region over JTAG. So, should be safe enough

but, the .hex has to be compiled to start at the right location.
this is a hardcoded value (0x80008000) in the bootloader main:
( https://github.com/catfact/aleph/blob/master/utils/avr32_boot/src/main.c#L240 )

and in the linker script for each app:
( https://github.com/catfact/aleph/blob/master/apps/bees/aleph-bees.lds#L65 )

also, it is important that the compiler not add additional trampoline/startup code (we include this in the bootloader instead): ( https://github.com/catfact/aleph/blob/master/avr32_lib/avr32_lib_config.mk#L54 )

when flashing using the jtag ICE, first flash the app, then the bootloader.

and, yeah… this is not a great system…

1 Like
#14

re: picolisp,
sounds awesome, i would be very curious to know about runtime performance. i’m a little skeptical about that aspect. peter blasser’s shlisp is very cool because it compiles down to ASM opcodes…

(similar concerns about eLua - i’m sure we could get it running but not sure there would be headroom left to do many useful things.)

definitely gonna get the forth interpreter running on aleph and TT, one of these days (actually soon, i hope.) this isn’t exactly a ‘high-level’ environment but it is interpreted, mutable, and reflective. it would be a good tool for rolling your own scripting languages.

i totally understand not feeling like fixing bees because a more expressive coding environment seems more useful. the point of something like bees is to have an expressive environment for people who don’t want to code but do want to define their own instrument logic. of course there are other ways to acheive this but it would be a shame to totally lose sight of that goal.

#15

http://picolisp.com/wiki/?NeedForSpeed
hopefully fast enough to be useful for glue code, menus etc…

haven’t given up on bees but need to beat a tactical retreat on the big changes - I’ve got
stale on the refactoring it needs attacking from a fresh running start. The build is fixed for beekeep & bees after rewriting most of net.c based around a central data structure that doesn’t repeat itself. That cleans things up a bit but fallout from that change is huge:

  • only four operators currently in the build - still need to run elisp refactor script over the rest and cross fingers
  • I don’t understand much about preset system so a bunch of that stuff is just commented on refactor branch
  • haven’t even started writing the memory management scheme we discussed on github
  • binary scene format is in pieces on the floor
  • op_dsp still an empty shell
  • none of the menus are updated for operator-wise indexing and op_dsp

kind of daunting! Hope I can get somewhere with picolisp on aleph this weekend, then assess whether the technology could even be used to rewrite bees UI, moving net.c & operators into platform-independent ‘control-block’. As far as I know lisp and forth are both high/low-level languages combining speed and power but I already know quite a bit of lisp. interpreted language running on the device circumvents the flash time bottleneck/fear-factor!

If I do eventually opt for a straight C approach, catching all the UI bugs without debugger and with the long flash cycle will be gruelling - need to get a jtagice dongle and hone my gdb chops to fix multiple segfaults in a single build cycle before attempting…

#16

kind of topic-drift, but i was thinking it would be cool to add a flash-by-serial mode to the bootloader…

anyways. have you seen this project yet? looks like it has a target for atmel EVK1100, so pretty close to aleph/TT.
( https://github.com/simplemachines-italy/hempl )

#17

@rick_monster much respect to you for tackling these ambitious projects!

beyond cool, it would facilitate development in a great way

#18

So discovering the existence of hempl was the thing that triggered my interest in picolisp, though reading more about it’s design philosophy made some sense to me as someone already heavily invested in (common) lisp.

last night I attempted to replace hempl’s ld script with the one from aleph & ‘fixed’ all the resultant linker errors by blindly adding lines back in. They already have the -nostartfiles in their linker stage. Successfully compiled something with the bastardised linker script before dozing off but didn’t get as far as inspecting the top of the .hex file to compare it with bees/mix build.

I was blissfully unaware of linker scripts until last night and guess I will have a bit of study ahead to figure out what it will really take to add an aleph build target to hempl. Comparing this line of the hempl linker script to aleph I thought maybe they use a different type of RAM to us or something (sram != sdram) dunno if that is significant for linking purposes…

Also there is the possibility of trying to make a miniPicolisp avr32lib app like the pforth one, but not obvious to me how to cast a repl that blocks on read into avr32lib event-loop. Or maybe I’m mis-understanding what this is doing…

Looked to me like pforth only got as far as compiling it into an aleph app, not actually hooking up a forth REPL to the serial port?

#19

bahaha loving picolisp - I was able to read the source to miniPicoLisp enough to totally hack the implementation of main loop, then read, eval, print lisp expressions from in-memory buffer, rather than stdin. So that opens up a possible way to drive miniPicolisp repl through avr32lib event loop hurrah! (something’s weird though still some way to go before trying to jimmy the same code into an aleph app)

#20

Looked to me like pforth only got as far as compiling it into an aleph
app, not actually hooking up a forth REPL to the serial port?

yeah that’s right i’ve only cleaned out the project and added pforth csrcs.

but seems easy to finish, just hooking up getch() and so on.

unfortunately i don’t have the advantage of being comfortable in lisp or forth to begin with! either one would be good for me

#21

therein lies the problem hooking up picolisp. I’ve managed to get something will work as long as you don’t send any ill-formed s-expression to the interpreter. Trouble is if you send it any junk it blows up completely. It’s to do with the fact that a repl would normally block awaiting further input till EOF. If you’re sending the chars over an event loop the read function might get called on incomplete input then that needs to be handled correctly.

It’s a puzzler…

#22

ok so read up on linker scripts & tried to compile hempl for aleph last night, outside of avr32lib/aleph repo:

https://github.com/rick-monster/hempl
with:
scons cpu=AT32UC3A0512

no response from aleph on the serial port. Notice that power button works correctly with the new firmware flashed but other than that totally dark. Didn’t brick the aleph though - was able to reflash bees and back to normal… Any idea what might be going wrong or how I might be able to debug?

  • is it possible the usb UART is set up differently on the mizar32 board to aleph?
  • there was a bunch of stuff I didn’t understand in the aleph linker script. Is my revised hempl linker script missing vital stuff?
  • EVK1100 build target seems to be broken in the hempl master branch - maybe next step would be try to figure out what’s up with that?
  • Or somehow import config headers for the aleph board to hempl?

EDIT:

ok! on linux at least we can use fmemopen magic to spoof stdin/stdout! Fingers crossed it also works with avr32-gcc…

EDIT: aiieeeeeee! miniPicoLisp also uses a bunch of weird junk in a .d file (is that assembly!?) - tried hacking the picolisp from hempl into a clone of new aleph repo - somehow got it compiling but blows up on the device when you actually try to load simple (+ 1 1) expr using in-memory stream stdin. Trying again now to add aleph build target for hempl…

#23

It’s working it’s working!!! …

we have picolisp on aleph now! After many blind alleys, wrong turns & red herrings I finally figured out how to make picolisp accept input from a buffer. So we can run the usual avr32lib event loop, but there can now be a type of event to send a lisp expression for evaluation in the repl - just need to hook up some interesting lisp commmands like trigger dsp params, write to screen etc… Kind of curious how long will be the garbage collection pauses - will this be actually useable in the OS for a music computer??? Exciting stuff and there’s also @zebra’s pforth interpreter to try out I should try to hook that up tomorrow so we can compare lisp vs forth on this platform…

NOTE: umm yeah so although I can add numbers, define simple variables & functions etc it still blows up for some reason when lisp throws into debugger - but pretty sure now I will be able to catch & fix that. Most probably calling some undefined method on non-existant/hacked i/o streams. 4am now yeah better try and sleep…

EDIT: recovering gracefully from lisp errors now!

Because I just started hacking without planning properly or knowing for certain it would eventually work, code is all sitting in a mangled version of @zebra’s aleph blank app. So things need cleaning up a bit - but next question is how easy/hard is it to hook directly into avr32lib api calls (so is glue code necessary or can all the coding for that get done in lisp?). Time to hit the picolisp docs…

3 Likes
#24

that is so cool! now i just need a time machine so i can actually get a few hours to play with it.

FYI, pforth app isn’t in any kind of state. i just tossed the sources in there. i did look at it a bit since and got everything compiling. but need to make the same decision about how to restructure main and the event loop around the interpreter loop… i will have a gander at your solution…

#25

some points for anyone reading that code:

  • in order to arrive at the solution I eventually managed to refactor minipicolisp’s reader to not use stdio. What I ended up with is pretty much lisp-implementation-as-a-library. This minipicolisp was the starting point for my current version.

  • Host triggers the lisp by sending lisp expression as a char*, then print is via a host-defined callback function

  • That callback is currently defined here, but should be passed as an arg to init here

  • currently lisp doesn’t notify host program of lisp errors - shoud fix that really…

  • Hempl project includes a lightweight stdio implementation for avr32 providing proper stream abstraction for filesystem/uart, which is better, but I eventually gave up getting this to work inside avr32lib.

  • Would be nice to figure this out, even though I went round the stdio problem instead!

1 Like
#26

chucked the picolisp code into the old aleph repository & suddenly the thing initialises cleanly. Here’s some code https://github.com/rick-monster/aleph/tree/picolisp. Attempts to initialise the lisp image in the right place within @zebra’s new aleph repo framework were unsuccessful. With new aleph framework it needed some horrendous hack where the repl checks every time if it’s the first time - heinous!

I don’t have the patience to figure out why without a debugger - happy to offer pointers to anyone that wants to compile this for their teletype if it proves valuable on aleph…

Looking at mix app I was able to pilfer some code to hook up encoders to lisp function. So far I have the following functionality

  • mostly working lisp repl over serial (now with space-age backspace functionality)
    -… garbage collection works - the heap was specified as a whole megabyte by default which I guess is all the memory on aleph. with heap specified at 256kB gc seems totally instantaneous - see how that changes with an actual running program on there. gc explodes if you ask for much bigger heap than that…
  • lisp errors drop you into a debugger, as opposed to seventh circle of unresponsive segfault hell
  • encoders and front panel switches trigger lisp functions - you define what those things do by typing a function definition at the repl
  • lisp functions for playing etchasketch
  • decent callbacks from C into lisp

Here are the features required to turn this from a cool hack into an actual useful thing, then start putting together some performance scenes in lisp. In no particular order:

  • hook up lisp function (in C) to load a bfin prog from sdcard (think it’s quite easy - pulled in some code from bees to load waves on boot)
  • hook up lisp functions (in C) to query and tweak bfin params
  • lisp function to query the adc
  • steal the framed serial comms from aleph so you can send different messages over serial like we have in bees
  • an emacs mode for interactive coding on aleph from the comfort of your laptop - poor man’s slime would be just awesome and not fundamentally too difficult…
  • handle grid events
  • lisp function to light up grid
  • lisp function to callback another lisp function after a timeout (returns immediately)
  • handle keyboard events just like serial events
  • display lisp repl on aleph screen
  • lisp function returning the list of all lisp function definitions in RAM
  • lisp function to save all lisp function definitions in RAM to onboard storage for session persistence without sdcard
4 Likes
#27

OMG, aleph-picolisp is really coming along! great work!

sorry i’ve continued to be out of the loop, demands on time have made it really so hard to do anything fun like playing with aleph/TT.

yeah, the new aleph repo linked against new libavr32 isn’t really ready, sorry about that. libavr32 needs a handful of significant changes to work with aleph code, and haven’t really decided how to do that without also requiring changes to all the modules. when i look at it i just want to clean things up with sweeping refactors that affect all the repos.

anyways we should definitely fix that, should not even be hard. for the moment of course developing against the working aleph framework is fine - we shouldn’t have to break compatbility with the app code, between the old framework and the new.

i don’t know how useful it would be to have more than one interpreted language available. maybe detrimental actually, since ideally all scripts would be written in the same language and swapped on the fly? (or maybe they each have their areas of strength? what do you think?) so although i have a somewhat unreasonable desire to build things in FORTH right now, i should just start learning picolisp. [ tangential question: how different is it, on a practical level, from, say, common lisp? ]

in any case, your list of features reads like a good list for any interpreter to support. maybe could go on GH in case others want to contribute to small tasks? (i would love to but just can’t make promises right now.)

beyond cool, it would facilitate development in a great way

i will put a bootloader rewrite top priortiy of my task list since i don’t think it should be a great big task.

but yeah, anyone who wants to take advantage will need the ICE.

Hempl project includes a lightweight stdio implementation for avr321 providing proper stream abstraction for filesystem/uart, which is better, but I eventually gave up getting this to work inside avr32lib.

i missed this. i guess it would make a good GH issue for libavr32 repo.

1 Like
#28

depends whether lisp gc pauses are noticeable with bigger program running! As I understand it forth has no gc, which could be a crucial advantage. Who knows…

I also have been lured toward forth by an honest-to-god forth cpu running on little icestick fpga using 100% open-source verilog flow. Exciting times for sure!

Very very different - common lisp has soooo much stuff e.g native arrays, hashes, macros, reader-macros, compile-time-vs-run-time, lexical and dynamic vars, streams, format (printf(); on acid), loop (for(;; ); on crack). picolisp is probably closer to elisp than CL or scheme but I don’t know enough elisp or picolisp to really say for certain. Best place to start is probably here:
http://software-lab.de/doc/tut.html
but watch out there is some stuff in there you won’t find in the miniPicolisp that runs on aleph.

which ice is best? any idea if this one will do the job and play nice on linux? I don’t want to wait a month for this thing to arrive in BC from china

no prob I went sneaking around your github anyway to find it. Was clearly untested/experimental but yeah I was lured into this trap trying to jump on the spring cleaning bandwagon. Anyway it got things going - we’ll figure out what’s up with it eventually and get things into neat little boxes or something…

1 Like