Programming help (help a kid learn)

General
1

Hi folks,

I currently work with a kid who has developed differently than most of us. I’m trying to teach the kid to match, and have been for a while. We are working on matching a shape to itself (e.g., blue triangle = blue triangle), this may sound simple-- something we may take for granted, but it’s important to know that things are themselves even when they show up in different places.

The apparatus that we use is quite simple. A kid is handed a triangle (or square or circle), and is expected to place that shape on it’s match (the triangle). When the kid gets it correct, we provide things that the kid likes (a video on an iPad, a tiny snack, a toy). When then kid gets it incorrect, we let them try again (and provide help if needed). While this is an evidence based practice that has been shown to work, there are aspects of it that can be improved upon.

Problems:
-there is no feedback provided other than from a human (the ones working with the kid)-- therefore feedback is not instantaneous.
-the human (working with kid) may incorrectly provide feedback (human error)
-the human may accidental cue the kid as to what the correct answer is (Clever Hans)

Solution (what I need help with):
I want to make an iPad (preferably) app or windows program that displays basic shapes, receives input via midi (so I can make a controller where the kid presses a button shaped like a square, circle or triangle, depending on the shape on the screen), and plays a video clip (that I can change easily) when the kid gets it correct. I would also like it to be programmable (e.g., I can set up a certain number of opportunities to respond, I can vary what shapes are presented or make features of a shape stand out).
I’m hoping that making these changes will help the kid learn matching (we are really not sure what the kids current skill set is, as the kid cannot communicate). Once the kid learns matching an item to an item, it will hopefully be easier to teach them other things.

I have very limited experience with visual basic. I do not particularly have the time to learn more about programming (grad school). I figured this may be a relatively easy thing for someone with experience to do. If this works for this kid, I would love for it to be available for others. From what I can tell, there are not accessible learning applications for kids missing some foundational skills.

If you would be interested in helping with this (or if you know that this program already exists somewhere), please message me on here.

Many thanks!

(kid and they/them pronouns used for kids privacy)

8 Likes
2

I can’t program but wanted to chime in just to say this sounds like a very fun and important project. I don’t know of any, but it sounds weird if it really isn’t a dedicated learning platform like that in some form. Given that behavioural analysis is, as you’re saying, a very well known field of work. I’m sure you’ve done your research, but just to give my two cents I would look for stuff related to behavioural analysis and positive reinforcement. I work as a social educator/learning disability nurse myself, but not currently in this specific field. I can ask some of my past study partners if they know of a software for matching, shaping, reinforcement and so on.

3 Likes
3

Interesting project. I don’t have direct experience of this myself, but I’ll have a word with the right people at my school to see if they have used anything that might be a useful starting point.

1 Like
4

Thanks for the reply! I agree, it does seem very weird. I’m not very familiar with assistive technology-- and I plan to speak to some folks who specialize in that area, but so far I haven’t found anything I can use. I appreciate the suggestion to research behavior analysis (I’m actually delighted to see a stranger suggest it), as I am a behavior analyst. I was trying to not use behavior analytic language in the original post because it often seems to bog things down (a recognized problem in our field :slight_smile:).

There are likely programs that behavior analysts develop for research that would be close to suiting my needs, but none that I have found that are easily accessible (or can be used with a usb midi controller). I will also reach out to folks involved in that type of research as a possible resource.

1 Like
5

Thanks, that would be great!

6

How about p5.js (a javascript library based on processing) inside the ipad-browser? That’s the simplest way of doing very easy visual, interactive programming I can think of. It has a online editor for trying things out quickly as well as good tutorials and reference-pages and works reasonably well with touchscreens of mobile devices (e.g. check out this on a mobile device). It’s not necessarily the most professional approach, but it’s great for quick and easy prototyping.

2 Likes
7

I have some knowledge of ionic, a hybrid framework for easy app development (it’s basically web technology which can get encapsulated to an iOS or Android app).

I could make a simple questionnaire type app with both “question” and choices being pictures or vector graphics which you can put in a settings file, also a video on success (all media lying in a folder). this is done relatively quickly. the tricky part will be to make it really pretty/useful, like what kind of feedback comes if the kid gets it wrong? just showing static content seems boring.

I have a full time job though, so contributions (especially to the last part) would likely be slow… but in the open source spirit I might just begin, and worst case someone with more CSS Animation / frontend / styling skills could take over?

but wait, the midi input seems crucial? not sure if this will work. but I also wondered this in the past, so I might just do a proof of concept of this first…

1 Like
8

Thanks for the reply!
I guess a rough outline of what I’m hoping the finished program to look like would be helpful.

There would be 12 presentations of a single shape (triangle, circle, or square).
–Each shape would be correlated with a midi note (e.g., circle = C1).
During each presentation, the shape appearing on the screen would chosen randomly(with no more than 2 appearing in a row) (from a bank of 3 to start-- triangle, circle, or square).
–If the correct midi note is pressed (e.g., C1 is pressed following the presentation of circle), a video clip plays (this kid really digs certain videos).
–If the incorrect midi note is pressed (e.g., the notes correlated with triangle or square), the screen goes black for ~3 seconds (or until a human presses a 4th midi note) and the shape is presented again (this cycle can happen up to three times before the next shape is presented).

The midi input is definitely crucial.
–Some input that will work with a few big buttons could be an alternative, but honestly USB midi>iPad with camera connection kit (or bluetooth midi) currently seems easiest on my end.

–Ideally, some of these parameters will be set up so that we can change them somewhat easily to adapt to the kid’s learning style.

I see how playing a short video clip for correct answers and switching to a black screen briefly before re-presenting the same shape for incorrect answers seems boring, but this setup would be most similar to how these skills are typically taught (and taught effectively) in our situation. The kid is really motivated by specific video clips (she wants to work more than play sometimes :slight_smile: ), and we also provide other rewards are working on these skills.

9

Thanks for the outline! That helps a lot for the minimum requirements…

I already envisioned some fancy animation highlighting certain properties, that’s because I said boring when they would not be present. Your described workflow certainly makes it easier to implement, which is good :slight_smile:

So let me investigate the midi thing… Hope to have an answer to that before the weekend as I’m away then…

10

Thanks! Yeah I think if the kid does not progress much when the shapes are presented plainly, one thing we might want to try is to animate them (e.g., have them glow-pulsate (not flash as seizures may be a concern) or change size or something). Ideally, whatever animation used could be gradually faded back to the original (non-animated) shape.

I appreciate your help, and the help of everyone participating in this thread!

11

Hi! Just wanted to let you know that I had a look into this on my train ride and some things to try, but then noticed I forgot my usb-c adapter to actually test on device. Urgh.
I’ll come back to you next week…

On a different note, do you have a mac? Otherwise app sharing would need to go through the app store process iirc. Though the alternative involves you setting up as a dev basically (for free) and copy pasting some commands to a shell, then you could compile/build the open source code yourself to the ipad (mac only).

1 Like
12

Just a wee update. I haven’t been able to find any pre existing software that our teachers use, but I still have to speak with the head of our Support for Learning department, she may have ideas.

1 Like
13

Expo might be a good option here. You can write JavaScript which then runs as native code (not a native-wrapped web app) and it can be run within the Expo app so you don’t need to worry about distribution.

2 Likes
14

Don;t know if i suggested this before on this thread but
http://earsketch.gatech.edu/landing/#/

one may choose Javascript or Python and learn the basics of OOP by making beats

how great is that?
:slight_smile:

2 Likes
15

@sakul Cool! Thanks for the update :sunny:

I do not have a mac, but I can likely access one (although I’m concerned about my ability to use it competently, as I’ve never used a mac before). I’m glad you asked, because I have not yet thought about how to actually get the program onto an iPad (or how to export data about the session from the iPad-- we can continue using paper and pencil, but automated data collection would be sweet).