I'm wondering if there is any documentation for the serial protocol? I'm talking with someone about a potential used purchase as I've been considering adding support for the soundplane to my eurorack module... (the nw2s::b)
I've recently added support for a couple of USB devices including the monome and a gamepad. They went well, and now that I've got the USB host code basically worked out, I'd really like to include this guy for some easy computer-less soundplane-modular goodness.
I assume most folk work at the OSC or MIDI level, so that the only people who needed the raw USB protocol were the original programmers.
I'm also curious if there are any version differences between the different runs. The monome has some significant hardware capability changes from version to version. This seems a bit more consistent, but I thought I'd ask.
Thanks!
s
got it... soundplane with touch tracking is now working on my BeagleBone Black.
and also Mac... a simple oversight on my own part...
now just got to pretty the data up a little bit to integrate with my new setup :)
Great news! I have a BBB on the shelf, can't wait to try it here.
soundplane - beaglebone black status update.
ok, so Ive now split out all the necessary files for the soundplane and touch tracker
Ive integrated changes made by scott for vector for neon, and also made similar changes for signal.
I've also taken a slightly different approach, rather than using #ifdef which complicates the code line. Ive removed the SSE code and moved to separate files
so, now I have things like
MLVector.h
source/sse/MLVector.h
source/neon/MLVector.h
source/nofpu/MLVector.h
note: the arch sub dir files only contain the fpu code (as much as is possible, without introducing inefficiencies), common code is in the original file (so above MLVector.h)
one question for scott...
in your port of vector you say there is no neon replacement for _mm_div_ps, however, SSE2NEON uses vdivq_f32 , is this not correct?
its not a big deal, as its only used in makeDefaultTemplate() , so not on the performance line... but obviously as TT changes, it may become more important if that operator us used elsewhere.
also is the X15 actually shipping, I can't seem to find a way to order one.
(preferably in Europe, but US would do too :) )
The version of SSE2NEON I've found (https://github.com/jratcliff63367/sse2neon/blob/master/SSE2NEON.h) doesn't have _mm_div_ps in it at all...
It does have _mm_mul_ps which it maps to vmulq_f32. The division version of that intrinsic is vdivq_f32 which is listed as only being supported on 64 bit ARM v8.
http://infocenter.arm.com/help/topic/com.arm.doc.ihi0073a/IHI0073A_arm_neon_intrinsics_ref.pdf
...but now that you mention it, that document is 2 years old and when I swap in the div operation, it compiles just fine. So maybe I was being too cautious there.
-s
I did find a beagleboard x15 in stock earlier this year, but don't see it now. Mouser's not getting any more until May!
The x15 didn't fit my form factor or IO requirements, so I'm using a dev board for the SOM I will be using in the final hardware. It's the Compulab CL-SOM-AM57x and is a nice compact module with a better IO complement and less extraneous connectors that would make the eurorack module a bit awkward.
Would love to see your code - especially the command line test. I'm making my way through the new tracker, but I see I still have a ways to go even once I get this stuff compiling, There's a lot there that isn't used if all you really want is the touch tracker output.
Anyway, bedtime!
as such i don't 'need' an x15, it appears the BBB is enough for the soundplane.
(or at least at this stage, I need to connect it up to a synth and play it to check for sure :) )
but perhaps the X15 would give me a bit more headroom, my only real requirement, is I want all this to run off a small battery unit, as I want it all to be portable. Im not after integrating with eurorack.
Im thinking something like this:
http://www.amazon.co.uk/Poweradd-Pro-Multi-Voltage-Flashlight-Smartphone/dp/B00FHVE1CU/ref=sr_1_15?ie=UTF8&qid=1458477876&sr=8-15&keywords=poweradd+power+bank
Compulab CL-SOM-AM57x , yeah, seems nice, but perhaps not viable for a one off order... and I guess the x15 will see better general support.
hey, but whilst I'm not into eurorack, the idea with the soundplane is enticing, and your products could well tempt me further ;)
(though, Id need eigenharp support too... but I have the code for that already, assuming I can rebuild your firmware ;) )
funny about the SSE2NEON, I thought that was where I got it from... add I only added one function to get the whole ML DSP compiling, and that wasn't it ;)
( we don't need the whole DSP lib for the TT though)
anyway, I'll use vdivq_f32
code... I'll see what I can do. perhaps a private repo might work, would allow us to get the basics worked out. Id like to restrict access at this time to people who can/will contribute.
Yeah, the x15 will be great for general use and will be a great deal for ~$300 when you can readily get them. Like you say, though, the BBB will probably do fine assuming you're not asking it to also run X... It's available, cheap, and can be run on battery. The x15 is a bit more of a power hog.
For the same reason you mention, I've been weighing how best to build the b2/dsp in a standalone form factor. It could probably be useful in a form factor somewhere on the continuum of aleph/anushri/elektron. We'll see... I'm on a shoestring budget, but have a few smaller modules I need to get out to help fund this crazy thing.
yeah, Im not expecting much from the BBB...
Im currently just using it to communicate with controllers, in particular my soundplane , eigenharps and a Push 2 :) Im doing this in C++, and then sending/receiving OSC, which I'm picking up in PureData (-nogui) to translate to MIDI (MPE) and then that gets fired off to a bunch of Axolotis for sound generation and sequencing duties.
Im using pure data as its quite flexible to allow me to do the key/surface mapping, if I find I'm running out of processing power, I'll either move the controller stuff to a pure data external (to avoid osc handling) or just move the midi mapping etc to C++.
the later of course is most efficient, but makes the solution a little less flexible.
so no the BBB wont be running X, or anything else too heavy
the only thing I dont like about the BBB is you can't bus power the soundplane and eigenharps from it. so what Im currently doing is using a powered hub, which then is hosted by the BBB but also powers the BBB.
anyway, Im pretty close to being able to test this out.
Ive being looking at performance....
its a little strange on the BBB, the cpu load is varying quite a bit, from 60-80% BUT oddly if you start doing other things the cpu load drops. (!??) I thought this was some kind of busy wait, but not found any evidence of this.
I did do a gprof thats quite interesting, showing some definite 'hot spots' that perhaps with some fpu could be optimised. (Im sure Randy already knows all this)
convolve3x3r is the big consumer of resources, nearly 25% of time spent in it.
whats a bit odd, is I see in processTouches, its called 4 times but 3 times with same params
// to make sum of touches a bit bigger
mSumOfTouches.scale(2.0f);
mSumOfTouches.convolve3x3r(kc, ke, kk);
mSumOfTouches.convolve3x3r(kc, ke, kk);
mSumOfTouches.convolve3x3r(kc, ke, kk);
I wonder if this could be done in a different way, it could make a big difference
setting the coefficients on the background filter is also taking a good amount of time.
apart from that, its a few key functions (like clamp/max) which are called a huge number of times, so any minor improvement in these would make very large improvements.
anyway, as I said, I need to run this 'in context' to see if it its going to work, but good to know there are some things to look into if we need to squeeze a bit more out :)
ok, I've set up a private repo on bitbucket, which has this stuff in it.
you should have an invite, let me know if you would prefer it on a different email address.
Thanks. I saw a similar thing with the clamping. It's an obvious point for optimization... Two things will make it faster - vectorization and getting rid of conditionals.
The neon vmin/vmax will do the operations without conditionals, but not every call of clamp() is something that can be vectorized... not that that's an issue, the one-off calls are more about making sure the input parameters are in a valid range.
The big one is likely MLSignal::sigClamp - and vectorizing that will be simple and probably reduce it's time by a couple of orders of magnitude.
The convolution may take a bit more consideration to get everything lined up properly to accelerate with vectorizable operations. Not sure there's an easy way to optimize the fact that it needs to be done 3 times - That's the equivalent of running the same signal through three identical filters... To emulate that with a single filter would simply require the filter to be three times as big - defeating the purpose.
But considering there's no optimization whatsoever, once it's optimized, the fact that it's happening three times will be less of an issue - and will just give you three times the benefit for each optimization you make.
...specifically for clamping:
float32x4_t vminq_f32 (float32x4_t, float32x4_t)
float32x4_t vmaxq_f32 (float32x4_t, float32x4_t)
I appreciate these comments. I stopped worrying about optimizing the touch code because the best use of my time is in making a completely different algorithm.
You could leave out the convolve3x3r calls above, and probably get something usable.
yeah, Im not going to spend to much time on this for the same reason, once you get the new tracker working (after virta?!) , I'll then port this newer code (and dependent code), only from there do I think optimisation is worth really looking into.
as I mentioned, as far as I can tell so far, its seems to be 'good enough' so far on the BBB. I only investigated, due to the oddity of seeing CPU load drop as the BBB got heavier loading - so was really looking for some kind of busy wait... but that did materialise, so still at a bit of mystery, but one that currently seems to play to my advantage.
of course, its also fair to say, that the code on the Mac has plenty of performance, its only going to be when your start using low powered microcomputers, like these, that more use of the FPU is really going to be noticeable.
anyway, lots more things to try to improve and tidy up, and get running, before optimisation :)
Thanks for the code. Got it building on my dev board. for the record, it iterates between 5% and 15% CPU, so similar behavior that you were seeing. Really do wonder what that is... But still - great performance - plenty of overhead for the rest of the code.
I'll pass it along more optimizations as I get them running - first I'm going to work on some python bindings so I can incorporate it into the base framework of the module. Woot! Beautiful code by the way - easy to read and find my way around. I tried your new tracker, but yes, it's not quite there, so I'm going to stick with the one that's running now. This will get me way down the road, and that's all I can ask.
-s
cool, glad its working for you..
interesting you see the same behaviour, it is a bit 'odd' - the cpu usage you've got is pretty good, unsurprisingly better than the BBB.
as you say, time to get it doing stuff thats useful, so we can see how useable it is in a more 'real world scenario'. I do expect to have to do a few changes yet, in particular to some input filtering, and also probably loading the soundplane json file to get some better calibration data, and carrier settings. otherwise I suspect the playability will be variable.
(of course I think this depends on how accurate you need the tracking to be, for me, Im using it as a playing surface, so it has to be pretty good, but if your were after more general x/y/z , simple might work ok)
anyway, Id obviously appreciate it, if anything you can improve is fed back...
bit more progress... on soundplane with beaglebone
Ive now got the thing properly calibrating and filtering on the BBB, data looks cleaner,
though Im sometimes getting lingering touches. I suspect this may be because sometimes Im not using the optimum carrier. (Ive seen this with SP on the mac before). its good to see the extra processing really didnt appear to hit the CPU too much.
now this is going, Im now considering pulling in SoundplaneModel and 'hacking' this back to a slightly more naked form. The reason behind this, is the model has a lot of code that really binds the soundplane driver and touch tracker together. with my current experimentation its clear, if you dont use the model, then you have to still do much of the processing thats in the model anyway.
I need to decide now, exactly where the hatcheting starts and stops...
thoughts so far...
things going : OSC and MIDI output connection, OSC services,kyma, any storage of signals 'for display purposes' (we just don't have the memory to spare), references to the likes of MLFileCollection, anything that refers to Juce.
parameters : 'under consideration' , but I think will stay
zones : 'under consideration', theoretically I dont want the mapping code here, but its tempting to keep the flexibility.
main things Im going to be keeping/modifying are loading soundplaneapp.txt for normalisation maps, general interaction between TT and Driver.it will probably then have some simple callback which can be overridden for output.
I dont think this will take too long, as I'm 'reasonably' familiar with SoundplaneModel already, main effort I think will be breaking dependancies that i dont want :)
Yeah, Soundplane model has a lot in it. For my purposes, I am really only interested in touches and I have a centralized "server" that is made to translate raw events from different types of devices to different outputs - effects, CV, USB MIDI, OSC, csound, puredata, etc.
Even zones would be covered by what is in effect a service bus - then zones are not much different than splits on a keyboard from my code's perspective.
So what you've done serves my purposes perfectly.
Fo you - it would probably make more sense to pare it down to what you need rather than introduce overhead in a hub-spoke type of architecture like I'm working on.
...but I should have my python bindings done for soundplanelite in a few more days if you want to use it as an easier way to tack on things like OSC support through python libraries.
s
Done with the 'python' binding. I attempted to do a native binding where the soundplanelib code was running within the context of python via cython, however there's just a little too much multithreading going on for it to make sense. cython/boost-python, et al, are generally mean to wrap math libraries and such - not servers.
So instead, I built a socket server that dumps touch data to a unix socket. The python code connects to the socket and tracks the touches from there.
For your reference:
https://github.com/nw2s/libsoundplane/blob/master/src/SoundplaneServer.cpp
https://github.com/nw2s/libsoundplane/blob/master/test/touchtrackertest.py
-s
yeah, I think keeping it light is probably going to be key... that and I dislike python ;)
if your using UDP sockets, why not go to OSC? not a lot of overhead and a bit more flexible.
( I assume your using UDP, TCP is not really required)
Im a little stalled, as Ive encountered an issue when testing, that the soundplane disconnects, and at the kernel USB level... not the libusb level. I can't find much information about the issue, its possible its application level, but actually looks unlikely, it could be something to do with the BBB debian distro I'm using/kernel version :(
also, Ive learnt I dislike Pd even more than Max... so have to decide if I ditch that idea, and move to doing all the mapping in C++, for that I need to investigate the ALSA api, so that I can output MPE.
so some progress, but the issues, did take the 'wind out of my sails' a bit.
Is there any specific case where the USB is disconnecting? What does it take to get it connected again? plug/unplug or just restarting the app? Any syslog messages? I haven'e experienced that yet on the AM57x board... and I've left my soundplane plugged in for multiple weeks as I write code and deploy it remotely onto the board via rsync.
The sockets I'm using are AF_LOCAL/AF_UNIX, so they are stream based but not IP-based. Everything stays in the kernel, so there's really as minimal overhead as possible for IPC.
OSC would require serializing to strings and deserializing which isn't necessary. Any OSC/MIDI/etc control type protocols will be implemented at a slightly higher level in my architecture since it's a bit more general purpose and is meant to be adaptable to a number of different control surfaces.
I'm moving on to getting the DSP cores running, and may leverage the soundplane to be some sort of granular/spectral manipulation control surface once I get that code migrated over from SHARC to C66x.
s
yeah I get this
[ 286.283473] usb 1-1-port7: disabled by hub (EMI?), re-enabling...
[ 286.290275] usb 1-1.7: USB disconnect, device number 4
[ 287.343570] usb 1-1-port7: Cannot enable. Maybe the USB cable is bad?
if I use the same hub/cables on the Mac its fine...
I get this from anything from running it for 30 seconds, or might be 4-5 minutes.
and libusb returns LIBUSB_ERROR_INTERRUPTED ... but of course I dont know if this is cause or effect.
tried a different hub, same issue..
Ive noticed sometimes, I get the libusb error, but no sysmsg and the soundplane 'recovers'
Libusb error! -10
Device state changed: 0
Available Bus Power: 230 mA
Device state changed: 1
Device state changed: 2
ok, some 'promising signs'....
it looks like it might be some posix event that is indeed interrupting the poll.
(Im not quite sure how to find what posix event it is ... need to research that, but i suspect something from the hardware)
but the good news is, Ive altered the soundplane libusb code to just poll again if its interrupted, and this seems to have cured the issue, been running for 15+ minutes without issue now.
I suspect the hub/sysmsg was caused by reinitialising the device, and since it now doesnt need to do that, all is well.
note: using libusb 1.0.19
Yeah, that's irritating. There are virtual root "hubs" so the hub it's referring to is probably not your hub, but the driver's root hub. Definitely a low level issue that software won't likely fix.
seems fixed now...
think my post came just before your ;)
I've tried to get GDB to tell me what signal it is thats causing the interrupt,but it seems unable to catch it. (handle all print, yielded nothing)
from reading up, re-polling is the normal thing to do, if you get an interrupt during the poll, so in that sense seems like the driver was 'wrong', its a pity I can't find out the cause though... would be nice to know exactly whats interrupting it.
anyway, good news, is it puts me 'back on track', so means I can get back to making it do something useful...
(when the issue cropped up, late at night of course!, I feared it might be the end of the road for the BBB... but no, the journey continues ;) )
btw, off topic... the X15, have you tried the stereo jacks... how much noise is there?
(I'm wondering, standalone synth on it could be fun, but only if the noise floor is acceptable when amplified)
I don't have an X15... but the X15 has a AIC3104 which seems on the order of something you'd get in an android device or iphone? Should be serviceable.
The SOM I'm using has a WM8731 which is used on a lot of euro modules, but the driver isn't ready for it yet, so I'm using a USB audio device for my testing... that's the good thing about having a full ALSA stack.
I'm still debating what the module's final DACs will be. The 'b uses latching dacs which are better for CV and basically unusable for audio. I'll switch to some audio dacs if I can get them to perform with enough CV accuracy to run pitch CV. Otherwise, I'll have to have a few of each.
As an aside, I got an email stating that the X15s are going through FCC certification, so they'll be shipping soon. The BeagleBoard site has a place you can sign up to get notified when they ship.
Ive done an initial port of the Soundplane model... need to do some testing, probably next week... but looking good.
hopefully now Virta is done, Randy will be back on to the new touch tracking for the soundplane, so I can integrate that once its complete.
X15, sorry , my mistake, misread your original post.
yes, I signed up for the X15 announcement so got the mail too... could be quite interesting.
I also backed bela.io , so Im quite looking forward to getting that, initially for this, it will just mean moving to Xenomai... but looking forward to the possibility of some low latency IO provided by bela.
(might even try some audio, if the soundplane software doesn't consume all the resources ;))
ok.. soundmodel is working and has highlighted a small issue.
I can see that we are getting periods where the driver appears to be sending incomplete data...
what I see is 'diff too large' errors, this means that the driver has determined there is too large a difference between consecutive frames.
we didn't see this on the test program because we did not implement:
virtual void handleDeviceError(int errorType, int di1, int di2, float df1, float df2) {}
virtual void handleDeviceDataDump(const float* pData, int size) {}
errorType = kDevDataDiffTooLarge , df1 contains the difference, and by default if this is greater than 8.0 it generates a fault.
Im pretty sure this accounts for the 'peaky' cpu I was noticing, as if the error is generated, of course the touch tracker is not invoked i.e. the code executed is significantly reduced, hence the drop in cpu.
could you possibly implement handleDevicError on your board, and see if your getting the same...
The reason I think its a usb error, is the difference are very suspicious when they are dumped on my system
Im seeing the second frame , for every carrier, somewhere across the board the data turns to zeros.....
e.g. for carrier zero only
first frame
[0] 0 0.16 0.16 0.16 0.17 0.16 0.18 0.17 0.19 0.17 0.18 0.17 0.16 0.15 0.15 0.14 0.19 0.19 0.19 0.18 0.16 0.16 0.16 0.17 0.19 0.19 0.19 0.18 0.19 0.19 0.18 0.18 0.17 0.17 0.19 0.18 0.18 0.19 0.19 0.18 0.17 0.16 0.15 0.15 0.18 0.18 0.19 0.19 0.14 0.14 0.15 0.16 0.17 0.18 0.17 0.19 0.17 0.18 0.17 0.17 0.16 0.17 0.17 0
next frame
[0] 0 0.16 0.16 0.16 0.17 0.16 0.18 0.17 0.19 0.18 0.18 0.17 0.16 0.15 0.15 0.14 0.19 0.19 0.19 0.18 0.16 0.16 0.16 0.17 0.19 0.19 0.19 0.18 0.19 0.19 0.18 0.18 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
see how about half way across the data turns to zeros, hence causing the large difference
Its a bit deja vue, Ive seen something like this before with the eigenharps and libusb.... something to do with the data not being fully complete.
I will also test on a my Linux VM on my Mac, to just checks its that I'm not polling quickly enough, but i dont think thats the case, as that usually looks a bit different.
Im also assuming no-one has really used the LibusbSoundplane driver in 'anger' so perhaps there is an issue there, or perhaps its just an issue under 'heavy load' conditions....
we will see :)
EDIT: ok, Im seeing it on Ubuntu 14.04 / 64bit (under vmware).
Note: the mac version of my code is not reporting the issue, the only difference being it doesn't use libusb. ( I guess, I could compile the mac version with the libusb code too?!)
Im going to take a look at the libusbsoundplane driver, see if I can find the issue.
EDIT: found the issue and a fix, will detail later... (nearly 3am here ;))
I haven't quite decided if its an oddity in the data sent back by the SP, or libusb , or the implementation using libusb... Ive a couple of things to check to determine which :)
Interesting. Good catch. I am guessing that this area of libusb gets nearly no exercise at all since aside from audio devices, isochronous devices are few and far between... and those rarely have a use for userspace usb drivers.
in fairness this is not a libusb issue, looks to either be an SP oddity, or kernel oddity.
but yeah, generally its true iso usb is not well supported for many devices, as I mentioned on the RaspPI is a source of unending issues, due to the chipset not having good support. I have to say its made be a bit wary, of iso usb on embedded devices... as even its suppose to be support, it may not work.
(as you say, because its main use if audio and video devices, which are that commonly supported on embedded devices)
Im going to test later, if the same issues exist on the Mac with the SP, to see if its a matter of the coding using libusb, or the SP sending data with some inconsistencies.
bare in mind Ive got the Eigenharp running using iso usb with libusb, with no similar issues... so I'm pretty confident, its either the implementation or the SP...
I actually suspect the SP, partly I went through the implementation last night, and it looked fine ... actually, it looked better than fine - its a really nice/tidy implementation
as i say though, no real issue, it can be worked 'around', its really just a matter of expectations.