Sketch out general architecture and first steps

[vine77]

Let's sketch out the very basics of a looper based on the Web MIDI API so we can get this project stasrted!

/cc @abstractmachines @luciusbono @obensource

[obensource]

@vine77 awesome! I'm off for Halloween fun atm/tonight but will get some thoughts in tomorrow! 🎉

[natevw]

Oooh, this looks fun! Basically a shared sequencer interface?

[obensource]

Hi @natevw! So glad to see you chime in! 🎉🎉🎉

Something like that!

We're definitely at the 'define what it is stage'! ⚡

[luciusbono]

Basic features:

• Load Loop
• Play Loop (trigger loop)
• Set start point
• Set end point
• Speed (pitch)
• Gain

[abstractmachines]

@luciusbono Good call. Adding a couple more here:

• Start record (same as set start point)
• Stop record (same as set end point)

Most loop pedals I've worked with have that intuitive start/stop interface, with either no capability for editing the start/stop points, or it's buried in menus somewhere.

[vine77]

@natevw, yeah I think a "shared sequencer interface" could definitely be an outcome of this work. I even ponder a "collaborative Ableton Live in the cloud" though that's currently outside the scope of this project.

As I mentioned in the thread @obensource referenced, there is always the issue of latency in networked music performance (with some citing 30ms as the bound of human perception), but if you can't guarantee a very low latency, it seems like a looping paradigm might present ways to sidestep the issue, e.g. by essentially quickly syncing multiple workspaces even if the transport/play positions may or may not be nearly real-time.

@obensource, do you have any sense for the latency in milliseconds you were getting with the WebRTC version of midisocket?

[vine77]

For this project, I think @abstractmachines framed a good goal:

Remote MIDI looping that 2 musicians in different locations can sync in on. The MIDI can then be piped to control synths at each musician's separate location.

I like that for a few reasons:

• Demonstrates the power of the web in enabling collaboration
• Limits scope by offloading a synthesizer/sampler to MIDI OUT

For v0.1, I'm assuming we'd also want to initially limit scope to a self-contained non-collaborative looper and then add the ability for multiple musicians to collaborate in an upcoming version? Thoughts? There really are a lot of things we could do with a project like this, but in the spirit of agile development, starting with the simplest usable product may be beneficial. What do we think is a good order of operations such that we can start from a minimal proof-of-concept and work up from there?

I've been thinking of the first iteration as basically a MIDI version of the traditional guitar stompbox looper:

• Record and store MIDI
• Loop MIDI clips
• Combine stream of live MIDI with looped MIDI
• Record/overdub/play/stop
• User interface to control the above (Design user interface for first iteration #2)

Future features:

• Assignable controls (MIDI map, key map)
• Import/export MIDI clips
• Visualization of MIDI notes in clips (piano roll)
• Multiple tracks (though single track can have unlimited overdubs)
• Effects: 2x, 1/2x speed
• Remote collaboration with other musicians

I'm just brainstorming. Would love to hear more about what other think about first steps.

Questions:

• Do we want to pull in a library like React for the interface or just keep it as simple as possible for now? The fewer dependencies the better at this early stage?
• Can we get away with vanilla JavaScript? Our target platform is basically limited to browsers that support the Web MIDI API, i.e. Google Chrome (although that might also include Android and Opera now, but others only through the WebMIDIAPIShim requiring the Jazz-Plugin download), so we could hypothetically go ahead use features like ES2015 Modules in Chrome 61+ without a build step.

[natevw]

@vine77 Yeah, sidestepping the latency issue seems a key win to something like this! That's what caught my eye (besides just liking loopers in general :-)

With the right design, jitter should be a complete non-issue, and you might even have time on the order of a whole bar to get everyone in sync.

Sounds like for now the goal is a "solo instrument" though, rather than something networked?

[vine77]

I think something networked would be awesome if y’all are up for it. Just trying to think about how to implement incrementally.

[obensource]

@vine77 @natevw definitely iterating towards a networked app is what makes this project rad to me. No question for me there. 😎

[natevw]

+1 for networking too, but that doesn't mean it has to be the first lines of code either.

My vote would be "React" — preact+redux+reselect compiled from ES2017 — but I'm not opposed to a "raw JS" approach either (maybe with the core of D3?) especially since the browsers with MIDI are the browsers with niceties like => and {…obj} and async/await anyway.

I think the main key is to keep the data layer rigorously separate from the DOM rendering. That will make a potential transition from any initial solo experiments to adding network features much easier too.

[Clarification: if the interface is just some control buttons, plain JS is probably the way to go. I was imagining more of a sequencer/piano-roll display, which probably isn't needed at first?]

[natevw]

What's the plan here, crew? I have a synth coming this week that this would be great for — so I'd love to take on something here, but I don't want to just take over either.

If there's no objections, I'd like to start a real simple prototype of the core MIDI stuff. It might not have much interface to speak of at all at first: I might just map two "unused" keys on the input device to toggle record and play/pause?

If there are objections, I guess I'll race to the first Pull Request then :-P

[vine77]

Yeah, I agree we need to get some of the core MIDI stuff out of the way. @natevw, feel free to start a prototype. I think as long as we're communicating about it, anyone should feel free to move the project forward!

• It seems like one of the first things we'll need is a recording function, i.e. the ability to translate an incoming MIDI stream into a data structure (e.g. that could then be serialized into a MIDI file).

[vine77]

Questions:

• Would it be valuable to include a very simple "test tone" synth? I think any real synth or Web Audio is currently out of scope of this MIDI Looper project, but I wonder if a simple test tone would simplify the dev environment by removing the requirement to hook up MIDI OUT into another synth. (IDK, I was just playing with the Xfer Cthulhu arpeggiator plugin, which had something like that.) Thoughts?
• Is there any prior art in this space we should be aware of?
  • I haven't looked into many of these in detail (github, npm), but here's an example of MIDI parsed to JSON: https://github.com/Tonejs/MidiConvert#format

[abstractmachines]

Hooking up MIDI OUT into another synth isn't necessarily a requirement, but that is the part that will help a remote musician collaborate. Not married to the idea or anything. The test tone may help development move more quickly. Sorry I haven't been involved too much yet, been busy!

oh yeah! " i.e. the ability to translate an incoming MIDI stream into a data structure..." my coworker just did that on a project very recently with WebMIDI. Pinging him now about it.