Tried my hand at a keyboard too... (Can't type on non-qwerty...)

Found that the horizontal precision is actually nice on this touch sensor, it's the vertical registration where I go wrong the most. So I made a keyboard with only three rows of keys, and I can actually type on it. Trick for me is to briefly but consciously look at a key before hitting it: as soon as I start relying on muscle memory my error rate goes way up. Also find that if correcting mistakes is easy enough and doesn't lead to additional mistakes, it's OK to have some misses. Try the experimental branch on my repo, the Touch / Keyboard example for a taste.

The help screen (which now just says "HELP"), will eventually explain that space is swipe right, backspace is swipe left and to get upper case, just drag a letter upwards. If you move across the typed text you move the caret unless text is wider than window, then you (smoothly..) drag the text. In that case drag down to bottom key row and then sideways to move caret. You can grab the text anywhere above the keys, even in the help/prompt bar.

Note that I used sprites both for the text entry field and also (briefly) for the keyboard change so that I prevent flicker. Also, there's now M5.Buttons.pushState() and M5.Buttons.popState() to save and restore buttons, gestures and eventhandlers so that one can switch context (such as into and out of keyboard mode) without setting up one's entire environment again.

Lots of emerging thoughts on how to structurally build out from here, but also just happily playing around building cool stuff and seeing how I would most like things to work for people to program with ease.

It's just a web frontend to the adafruit font converter thing. Might have a look once I'm done with a bunch of other Core2-related goodness. Feel free figure out WTF...

The forum software is funky somehow, sometimes I get timeouts as well here in Berlin. And the javascript hangs on to messages when you've left the page. Some of it is not very intuitive somehow...

Oh well...

That font is "Run Length Encoded", which I think is not supported by the display library. Maybe try my font conversion tool to make your own font files.

@vkichline Yes I have been making a few changes, just what I needed for M5Sound. Which did include pianoMode, which by necessity changes a bit of a voodoo state-machine. My examples all still work I think, no?

Just checked in the new improvements. There is now a polyphonic piano demo. Only two notes can be sustained – if you touch at different screen heights – because of the touch-sensor limitation, but keys that were hit before keep sounding for a bit. Shows M5Sound has no problem with 14 concurrent Synth instances.

To make this satisfying, I needed to be able to move my finger from key to key (glissando), which M5Button didn't let me do. Enter M5.Buttons.pianoMode = true. Now you can slip off one button and onto the next. The price to pay is that gesture recognition is switched off in pianoMode.

(The DTMF dialer is also cooler in pianoMode, so changed that too.)

Also there's now 1000_hzand C_major examples, just so that not all examples are somewhat advanced bits of code. But Piano, like the DTMF_dialer, is still pretty small and quite readable for a stylish polyphonic piano:

/*

Polyphonic synthesizer. Lingering notes will continue to sound as new notes
are played. If you place one finger high on the screen and another low, you
can even play two notes at the same time. As you can see you can have many
synths going at the same time.

*/

#include <M5Core2.h>

#define WKW         40
#define WKH        240
#define BKW         30
#define BKH        160
#define WHITE_KEY  {WHITE, NODRAW, BLACK}
#define BLACK_KEY  {BLACK, NODRAW, NODRAW}
#define NUM_KEYS    14

float notes[NUM_KEYS] = { NOTE_F4 , NOTE_G4 , NOTE_A4 , NOTE_B4 , NOTE_C5,
                          NOTE_D5 , NOTE_E5 , NOTE_F5 , NOTE_Gb4, NOTE_Ab4,
                          NOTE_Bb4, NOTE_Db5, NOTE_Eb5, NOTE_Gb5 };

// (waveform, freq, attack, decay, sustain, release)
Synth synth[NUM_KEYS] = Synth(TRIANGLE, 0, 50, 300, 0.7, 1000);

Button f_(   0, 0, WKW, WKH, false, "f" , WHITE_KEY);
Button g_(  40, 0, WKW, WKH, false, "g" , WHITE_KEY);
Button A_(  80, 0, WKW, WKH, false, "A" , WHITE_KEY);
Button B_( 120, 0, WKW, WKH, false, "B" , WHITE_KEY);
Button C_( 160, 0, WKW, WKH, false, "C" , WHITE_KEY);
Button D_( 200, 0, WKW, WKH, false, "D" , WHITE_KEY);
Button E_( 240, 0, WKW, WKH, false, "E" , WHITE_KEY);
Button F_( 280, 0, WKW, WKH, false, "F" , WHITE_KEY);
Button gb(  25, 0, BKW, BKH, false, "gb", BLACK_KEY);
Button Ab(  65, 0, BKW, BKH, false, "Ab", BLACK_KEY);
Button Bb( 105, 0, BKW, BKH, false, "Bb", BLACK_KEY);
Button Db( 185, 0, BKW, BKH, false, "Db", BLACK_KEY);
Button Eb( 225, 0, BKW, BKH, false, "Eb", BLACK_KEY);
Button Gb( 305, 0,  15, BKH, false, "Gb", BLACK_KEY);

void setup() {
  M5.begin();
  M5.Buttons.draw();

  // Prettier with top of keys as straight line
  M5.Lcd.fillRect(0, 0, 320, 5, BLACK);

  // Trick to make sure buttons do not draw over eachother anymore
  M5.Buttons.drawFn = nullptr;

  // So that you can swipe from one button to another
  M5.Buttons.pianoMode = true;

  // Set up syths with their notes
  for (uint8_t n = 0; n < 14; n++) {
    synth[n].freq = notes[n];
  }

  M5.Buttons.addHandler(pressKey  , E_TOUCH);
  M5.Buttons.addHandler(releaseKey, E_RELEASE);
}

void loop() {
  M5.update();
}

void pressKey(Event& e) {
  // instanceIndex() -4 because of background, BtnA, BtnB and BtnC.
  uint8_t key = e.button->instanceIndex() - 4;
  synth[key].start();
}


void releaseKey(Event& e) {
  uint8_t key = e.button->instanceIndex() - 4;
  synth[key].stop();
}

@vkichline Made checkRotation more generic:

bool checkRotation(uint16_t msec) {
  if (millis() - rotationLastChecked < msec) return false;
  rotationLastChecked = millis();
  const float threshold = 0.85;
  float ax, ay, az;
  M5.IMU.getAccelData(&ax, &ay, &az);
  uint8_t newRotation;
  if      (ay >  threshold) newRotation = 1;
  else if (ay < -threshold) newRotation = 3;
  else if (ax >  threshold) newRotation = 2;
  else if (ax < -threshold) newRotation = 0;
  else return false;
  if (M5.Lcd.rotation == newRotation) return false;
  columns = newRotation % 2 ? 4 : 3;
  M5.Lcd.clearDisplay();
  M5.Lcd.setRotation(newRotation);
  return true;
}

All it needs is a global uint32_t rotationLastChecked and for you to have ran M5.IMU.Init() (which seems to set power things on the MPU, so there's probably a reason it's not in M5.begin(), haven't read the datasheet.)

checkRotation returns a bool and takes the number of milliseconds between checks. It returns true if it has already rotated and cleared the display for you and needs you to set things up again. So my loop() now does if (checkRotation(1000)) doButtons()

I'm pretty sure it can do other tones as well. I chose blue for a reason... :)

@felmue

☑ Valid DTMF. (Didn't have an easy means of testing...)

I just wrote a simple polyphonic background synthesizer library for the Core2:

#include <M5Core2.h>

Synth a;

void setup() {
  M5.begin();
  a.freq = 1000;
  a.gain = 0.4;
}

void loop() {
  M5.update();
  if (M5.Buttons.event == E_TOUCH) a.start();
  if (M5.Buttons.event == E_RELEASE) a.stop();
}

You can have multiple Synth instances, their output is mixed. Every synth has attack, decay, sustain release envelope, and the waveform member can be set to SINE, SQUARE, SAWTOOTH, TRIANGLE or NOISE. Sound will continue as long as M5.update() is called. Default attack and release at 5 ms, so no ugly clicks.

Under 200 lines in the cpp file...

A few more convenience features coming, buffering to improve a bit still, but this is the core.

Do play with the DTMF_dialer example, and try holding it sideways...

It's all on the M5Sound branch of my fork.