#21

https://gcc.gnu.org/wiki/LinkTimeOptimization

Needs the -flto flag, which doesn’t work…

$ avr32-gcc -O2 -flto main.c
cc1: error: unrecognized command line option "-flto"

Otherwise… if you use a static inline header function, effectively each .c file gets it’s own copy of the function, which gcc is free to inline if it wants to (the inline is really more for documentation isn’t it?). I’d say let gcc decide rather than force it.

Anyway, AVR32s have caches, so for all we know trying to save the function call by increasing the code size could make it slower!

(edit: actually I’m not sure if the UC3 has a cache)

As you’ve rightly called out the most important thing to do is use a function rather than raw arrays, then we can change our minds about how the code is generated as much as we like. I doubt the modules are particularly taxing the 60MHz MCU. I’d be more fused about optimising the interrupts myself.

1 Like
#22

@sam: thanks for the clarification and research, and i’m in complete agreement with your points. (it’s true, AVR32UC doesn’t implement cache, but this is a detail.) particularly agree that some of the modules have way too much going on in ISRs (if that’s what you mean.)

@rick_monster: right, thanks for the reminder.

to recap: as discussed at length elsewhere, we proposed refactoring all the monome-related operators in BEES so that there would be a single pair of grid/arc ops that actually talk to the “OS” (handling presses/turns and setting LEDs.) all other operators would be abstracted to accept x/y/z and emit x/y/z. rick has implemented this change on his grid-refactor and related branches, at least for op_life (looks like WW, MP, cascades, &c, have not been changed over yet.)

pros:

  • lots more patching flexibility. it becomes trivial, for example, to move an operator around on a grid by applying x/y offsets, or store/manipulate grid state in arbitrary ways. in rick’s example, a raw grid op is patched to itself through the new mem1d or mem2d ops, which subsequently store state.
  • makes these operators more usable without grid hardware.
  • makes it easier to share code between modules / apps / ops.
  • single point of friction in case we want to change monome driver interface.

cons:

  • lots more bees net traffic; each LED update requires a bang. but netbangs are actually really cheap.
  • as rick says, more rope to hang yourself with BEES patching, and more connections that need to be made. this really highlights the need for a patch editor that doesn’t suck.

i think on balance this is a good change, but interested in other reactions now that we’ve all had a chance to think it over…

2 Likes
#23

since starting down that path I also wrote a lisp monome-serial driver and a lisp-on-linux white-whale-a-like. Learnt a lot, actually started making some crazy music I could never make before, combining aleph-processed guitar & headless monome-sequenced drums!

A white-whale op using the x-y bees-bangs would be horribly clunky and flashy/glitchy imo. Much better to have raw monome x,y,value bangs as an equal citizen to dedicated monolithic modules like white whale. The killer (scene-breaking) white-whale feature would be pickling the whole operator state, allowing to recall percussion patterns after a reboot…

Life is kind of special because the X, Y values of button presses are conceptually X,Y coordinates, not special ‘mod keys’ or whatever. I can dimly conceive some wild patches where the bees net actually populates/repopulates the life grid generatively - wouldn’t make as much sense to drive a white-whale in this kind of automated way…

Another interesting grid-toy I thought of in the same vein could be pong - hold down two buttons to ‘launch’ a particle, then it has rules to determine whether particles die going off screen, bounce or wrap. emit bangs on bounce, refresh, wrap, particle-death or whatever.

My ‘raw’ monome op should do the focus-grabbing song & dance. This is going as a feature request on my fork issues page before PR. Does the focus-grabbing currently work well for existing ops? Have a dim memory it may be buggy!?

Focus-grabbing for raw-grid op will also allow to have different raw-grid-ops doing different things using x-y indexing in same scene. Way better!

hmmm, well until we have that, we’d be silly to remove ops that save 30 mins of knob-twiddling, hence I’m saying add new x-y indexing ops as well as the ‘classic’ versions, at expense of op-list bloat…

1 Like
#24

Does the focus-grabbing currently work well for existing ops?

i thought so, but i could certainly be wrong.

A white-whale op using the x-y bees-bangs would be horribly clunky and flashy/glitchy imo. Much better to have raw monome x,y,value bangs as an equal citizen to dedicated monolithic modules like white whale.

i beieve you and stand corrected.

so, some good news is that the refactor basically works!
currently have aleph-libavr32-wip branches on my aleph and libavr32 forks.

there are some kinks to work out…

20161029_194927.jpg2395×1835 2.03 MB

:smiley:
(aleph screen is mounted “upside down” compared to teletype… and looks like when the screen code was ported to TT, half the drawing functions were inverted, and half were left unchanged and unused.)

i’ll keep cleaning things up and then rebase to a new branch for PRs.

also, i found it unfeasible to avoid making a couple of (very minor) changes to the ASF. so i made those changes by forking diet-asf (sorry github!), editing the script, running it and copying the result to libavr32.

i don’t actually think there are any breaking changes to other modules. main points of impact:

  • one pin rename that affects the module-specific init routines in the lib.
  • for now, i just added aleph-specific event types to the same enum as all the others. i couldn’t see any real downsides to this but i guess it could be more elegant.
  • re-added arc accessor to monome driver, this shouldn’t actually impact anything.

one more note: once again i don’t actually have a grid device for testing. my series 64 was on loan to lauren bousfield when her house burned down… i have a 40h but it’s in storage on the other side of the country, and i have a logic board with no leds or buttons… so i might need help testing stuff.

4 Likes
#25

The modules all have something along the lines of:

CPPFLAGS = -D BOARD=USER_BOARD -D UHD_ENABLE

in their config.mk, is that going to cause a problem?

#26

nah, that’s understood. the flag gets checked in board.h; the ASF then pulls in user_board.h and expects that you do hardware defines right there, within the ASF source tree. the system doesn’t anticipate using the same tree with multiple different “user boards.”

for whatever dumb reason, i just couldn’t get some of the peripheral modules (notably sd_mmc, which only the aleph uses) to function with an out-of-tree conf header, except by just skipping the inclusion of user_board and including the conf header directly in board.h. i banged my head on this for a bit and decided that’s not how i want to spend my life… :grimacing:! but if anyone wants to search a different solution it is fine by me.

anyways i’ll test it more tomorrow. the only module i have is teletype… but it should be enough to catch any really obvious problems. i’ll certainly check that each module build is pulling in the correct conf headers.

oh, fixed the screen issue btw. (here.)

1 Like
#27

@zebra - when you get to a point where you think the libavr32 port is far enough along for others to contribute without causing management headaches i’d love to know.

the midi sending code in libavr32 is untested and most likely needs to be fixed up to work again. the modules only receive usb midi so the sending side hasn’t been exercised since the code was refactored… i’m more than happy to fix up the sending code (and maybe consider MPE stuff) when the time is right.

#28

ahoy,

well, aleph with libavr32 seems to be working as far as i can tell. it could use more testing on hardware.

i went ahead and opened (tracking) PRs on aleph, libavr32, teletype and diet_asf:

https://github.com/monome/aleph/pull/268
https://github.com/monome/libavr32/pull/18
https://github.com/monome/teletype/pull/53
https://github.com/monome/diet-asf/pull/2

there are also minor breaking changes for trilogy modules. i’ve made the necessary fixes but though it might be worthwhile to check in before bothering with additional PRs. (maybe those modules should just be frozen anyway?)

since ansible doesn’t reference pin/register defines directly, it doesn’t break. yay

a couple concerns and questions:

  • the travis checks on teletype break because of a sort of catch-22; the libavr32 submodule commit hash has been updated for e.g. teletype, but of course that commit hasn’t been merged yet, so travis can’t find it.

  • on the flip side, travis checks on trilogy modules pass with the new lib, even though they shouldnt:

../src/main.c:381: warning: passing argument 2 of 'spi_selectChip' makes integer from pointer without a cast../libavr32/asf/avr32/drivers/spi/spi.h:587: note: expected 'uint8_t' but argument is of type 'volatile struct avr32_spi_t *'

pretty sure that warning spells disaster… :slight_smile:

  • .gitmodules in aleph is pointed at my fork of libavr32, which needs to change, but couldn’t test it any other way.

  • the aleph changes are made against master. merging with my fork of dev was looking super hairy. dev has been left too long, @rick_monster and i might need to have a powwow about what features in there are worth bringing in and how best to do it.

  • clang-format hasn’t been applied to aleph… in general i kept all changes to the minimum to get everything compiling.

  • everything needs more testing on hardware. oh, i said that already

  • i still don’t really get how to set up travis properly. because i haven’t looked yet. will attempt self-education.

@ngwese awesome, yes, we should check that out. i can’t find my MIDI dongle right now, but it’s around somewhere… embarassingly (?) i don’t know what MPE stands for.

ok! more later…

1 Like
#29

That’s fine. I can always trigger a retest once the libavr32 changes are in.

:smiley: ah, the joy of C. I suppose we could try and get travis to use -Werror, but that seems to be a bit of overkill.

.gitmodules is only used for git clone --recursive and git submodule init, from then on it only cares about hashes matching up, not where they come from. The trick to getting around your problem is to add the submodule pointing at monome/libavr32, but then change the remote of the submodule to your fork.

e.g. (untested)

git submodule add https://github.com/monome/libavbr32
cd libavr32
git remote set-url origin git@github.com:catfact/libavr32.git
git remote add upstream git@github.com:monome/libavr32.git
git pull

Fundamentally, git really doesn’t care where stuff comes from, only that commit hashes match up. So if someone clones your repo, they will have to make sure that their libavr32 submodule has the required commit available, either by manually fetching it from a remote, or waiting for those changes to be merged in.

One really helpful snippet for everyones ~/.gitconfig is

[url "git@github.com:"]
    ;; e.g git remote add gh:samdoshi/my_repo
    ;;;    git clone gh:samdoshi/my_repo
    insteadOf = gh:

Thus my workflow for testing your changes is:

cd <path>/teletype
git remote add catfact gh:catfact/teletype
git fetch catfact
cd libavr32
git remote add catfact gh:catfact/libavr32
git fetch catfact
cd ..
git checkout catfact/aleph-libavr32
# ..... oops?

Oops, looks like libavr32 commit 885a64 hasn’t been pushed to GitHub yet…


https://github.com/catfact/libavr32/tree/885a64afe42cc89bc4f59d72ffd10d8f97dd72ce

Another handy thing for your ~/.gitconfig

[status]
    submoduleSummary = true

Gives a git status of:

HEAD detached at catfact/aleph-libavr32
Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)

	modified:   libavr32 (new commits)

Submodules changed but not updated:

* libavr32 885a64a...8f04e5b:
  Warn: libavr32 doesn't contain commit 885a64afe42cc89bc4f59d72ffd10d8f97dd72ce

no changes added to commit (use "git add" and/or "git commit -a")
#30

ok this all sounds awesome!

Ha! There was me thinking master was basically an abandoned branch… Need to have a look at that merge to get a sense of it. Hairy merges are probably inevitable in coming months, time to dust off those magit merge chops!

umm yea can we make this, like, the last job? Personally don’t care about code formatting - appreciate it may help readability for some. From my side, aleph is the only readable large codebase I ever worked on - everything else > 10000 lines I ever worked on was a horrific brain-damaged mudball top to bottom. Running clang-format on everything will majorly mess with any subsequent merges so let’s get all feature branches & testing out the way first. Other than that, happy to use the auto-formatting if it helps others!

umm yea - this! I’ll get the ball rolling tonight with a couple hours systematic testing of the net-use-malloc branch I pull-requested. Kind of feel like I’ve been guilty of bikeshedding BEES strategy last couple of weeks, haven’t touched any of the code :zipper_mouth:. Ease back into it gently…

#31

cool, that’s what i figured. but i didn’t think it through originally, so my aleph fork was pointing at my libavr32 fork from the beginning, with monome/libavr32 added as upstream. i wondered how to change it after the fact… seems like it works to just edit .gitmodules and then run git submodule sync …? (which i have since done and pushed.)

Oops, looks like libavr32 commit 885a64 hasn’t been pushed to GitHub yet

ach, sorry. that is some useless merge commit arising from so much mucking around between all these different repos. rolled teletype to point back at 93241e and pushed.

I suppose we could try and get travis to use -Werror, but that seems to be a bit of overkill.

agreed… i guess the moral is that it behooves us to pay attention to the output on travis, and not just whether it passed.

Ha! There was me thinking master was basically an abandoned branch…

:laughing: well it kinda was. i haven’t added anything useful to avr32 in a good long while; everything else i’ve added to dev is silly experiments that haven’t gone anywhere useful and involve totally different parts of the tree. i gotta look more carefully at what the current upstream differences even are between master/dev…

but first i actually want to test your DSP pull request, cause it sems actually useful.

#32

ok well I stand by that part - it has been tested and does work (as far as I know/understand!) - need to double check this still works:

https://github.com/rick-monster/aleph/blob/dev_rebased/dsp/filter_svf.c#L150-L157

What happens to dsyn though? Isn’t half the point of dsyn that the filters are always kind of glitching out & going crazy? Never tried to wrap my head round the code, only played with the shiny confusing dials on that module from user p.o.v. An evening of testing/playing through the aleph will probably be pretty fun! So I’ll report back testing on those two pull-requests…

#33

ha, yeah i do actually kind of enjoy the RQ wrapping in dsyn context.

the main thing is that at high Q, the SVF should be dang close to a sine oscillator.

#34

yup - see these sound samples demonstrating adsr VCA/VCF (that old chestnut):


code ‘sounds’ the same on aleph as linux. If you import those .wav files to ardour & look at the waveform, you can see the high Q filter ringing at the cuttoff frequency, as it sweeps down.

It certainly scared my cat! will add back in the old code as a feature on that PR…

I’d also advocate strongly for arbitrary op deletion/insertion. This is genuinely useful! But yea - lots of experimenting/learning on my dev branch(es) not everything was necessarily a good idea. Man, synth control is hard!

2 Likes
#35

briefly glancing at the commits, this looks fantastic. i can’t do any hardware testing until saturday, so if someone else judges everything to be in good order feel free to merge. otherwise i’m happy to run trials later on.

1 Like
#36

Actually merging into dev & getting everyone using/testing features which work > 95% of the time may be the only way we get enough alpha testing to figure out whether things work 100% of the time…

I suggest the approach of squashing down pull-requests into the minimum number of commits so we can always change it back. And the award for most insane/unhelpful gitlog goes to… errr my net-use-malloc PR!

#37

i am glad we’re having this conversation.

i realize basing the refactor off master is a bit “wrong.” my thinking was that it would just be easier to deal with fallout from the refactor against the most known possible version (what most people are running) and deal with merging new features as a separate process.

i’m totally down to do this a different way. at worst, it’s really not that hard to redo the refactor on top of dev, now that i know what’s needed.

our old strategy was that the tip of master should always be tested and never ahead of of the latest release tag. at this point, i have doubts about the efficacy of this, because it seems to result in the piling-up of an unmanageable number of untested features on the dev branch. (and yeah, this is totally my fault.)

so i guess i’m proposing that we aim for more frequent integration of the dev and master branches? maintain both a ‘stable’ and a ‘testing’ release, both downloadable? i dunno.

1 Like
#38

hummm, so I guess the version everyone’s using is latest master HEAD? Seems like the time is right for a new aleph release containing all the new goodies on dev branch. Doesn’t seem to me anything’s really gone wrong with the management of branches, we just need to get a release out, get it tested, then maybe merge to master once a reliable release makes it through testing. Not the end of the world - ‘unmanageable’ amount of new features means at least few cool new features, right?

We can just revert any features on dev that’re not so cool…

#39

agreed. actually i was looking at the wrong thing for a moment there - hadn’t pulled in upstream master changes to my fork in quite some time. so it’s not quite as rough as i thought.

there are 650-odd commits (more than github will display, apparently):
aleph-commits-master-dev-20161031.txt (63.3 KB)

so… it’s doable. these can be sorted into categories:

  • you did a bunch of cool stuff with the serial protocol
  • you added a bunch of DSP and some new modules
  • i added some unfinished block-processing modules
  • i added some abortive/useless c++ wrappers for bees, and made some changes to bees source to allow this
  • there are a couple of useful, discrete bugfixes scattered in there (e.g. fixed op_gate pickling.)

so… yeah this could use a big rebase, it’s very hard to sort through.

we should try to separate the stuff that impacts code in avr32_lib and code in bees (probably your serial features mostly)… that’s the stuff that will come in contact with the libavr32 migration.

the DSP additions are just that - additions, that shouldn’t really impact anything else, but will be great fun to play with.

to be honest, i’m not really sure of the next step. maybe we should trade rebases somehow…

#40

Not really catching your drift here? Do you mean squash my many commits on monome/dev to one commit per new feature? I’m getting pretty good at that workflow after squashing down my DSP PR - easy enough to save the original branch & diff to check the rebase HEAD is identical…

ok yea so for starters if I squash down the new serial features on monome/dev into one commit for BEES, one for avr32lib, would that help?

Also rember now it would be really useful & really not much work to implement a BEES -> OSC bridge in C, allowing other people to actually use the serial protocol…