Printable Version of Topic

Posted by: sevenfx Oct 8 2011, 09:56 AM

Hey guys,

Since this is my favorite guitar community online, I thought I'd ask you guys for ideas concerning a project. I'm a computer science graduate student getting ready to start a big project, and I'm hoping to find a good topic having to with audio processing or visualization of some kind.

I was thinking about making a realtime visualization of pitch, so you can see which notes you are hitting, when you are hitting them, and how they fit in the scales. If possible, I would make this 3D, so you could get more information out of it, dynamics, attack, decay, etc. Also, I would be trying to do this so it would work from the browser and you wouldn't have to download anything to use it. (In the future, maybe android phones would be supported).

I think a program like this would be helpful for beginnings and experienced players alike. What do you guys think? Any other ideas?

Posted by: Ivan Milenkovic Oct 8 2011, 10:31 AM

That sounds interesting. What kind of visual representation did you have in mind? How should that look?

Posted by: thefireball Oct 8 2011, 11:11 AM

Wow - I wonder how that would work? Sounds interesting! And fun!

Posted by: Cosmin Lupu Oct 8 2011, 12:31 PM

What is your work process on this project? I'm curious to know more details!

Posted by: Daniel Realpe Oct 8 2011, 04:23 PM

Here's a very basic version of what you want to do if I'm not mistaken



definitely you will involve MIDI to trigger every note, I think once you ensemble the template for the 12 chromatic notes then all you have to do is let MIDI trigger each note...maybe I'm oversimplifying it, but 3D! that should great! and helpful even to teach music

Posted by: Gabriel Leopardi Oct 8 2011, 08:26 PM

It's a very good idea. I would like to know more about it!

Posted by: sevenfx Oct 8 2011, 10:41 PM

Alright, I'll try to give you guys a little more in depth explanation of what I'm thinking

Daniel - yes, that is similar to what I'm trying to achieve, but real time, and I'm not as focused on the midi output at this point. Basically, it would process the raw guitar input (as it comes in from a Line6 box, or microphone, or whatever). There is a program out there that does what I am trying to achieve already.. it's located here:

http://miracle.otago.ac.nz/tartini/

Like the video daniel posted, it shows the pitch that is being played while you are playing it, only its not broken into 12 notes. For example, you could see that you are bending a note from C up to D, and could analyze your vibrato. If you download that program you will see what I'm talking about. I find it very interesting to use while I practice.

Note for teachers: if you use this program, notice how it creates a "picture" of what you played. You could save this image, and students could strive to create a similar image to yours. Notes that were not bent to the correct pitch, or vibratos that were the wrong speed would be easy to see visually, and specific areas could be identified for practice.

As for the 3D aspect (my potential contribution to this project), think of Rockband or Guitar hero. You can see the sound as a time continuum, say moving from right to left. You will see the pitches as they move up and down (and possibly change color for different key parts of the scale, if you decide to select a specific key). In addition to pitches, you would be able to see the envelope in sound based on the amplitude of a frequency.

So if you're playing loud chords, you will see large notes appear, and playing softly would display a thin line. I would love to experiment with visualizing the attack, sustain, and decay of individual notes different ways.

When it comes to chords, and assuming it was possible, I'd love to be able to visualize the individual notes that make up the chord. Doing this based on the raw sound input is difficult, but could be a challenging enough problem to constitute my masters thesis .

In addition (for those of you still reading), I'd like to add in beat detection, and rhythm analysis. Still not sure exactly how this would tie in.

For an example of how I'd deploy this sort of project on a website look at:
http://www.tree-axis.com/Ess/_examples/inputFFT/

This is just showing frequency information of a microphone, but if you have java run-times installed on your computer, you should see it react to microphone data without having to install anything else.


Does any of this make sense? I'm kind of rushed for explanations right now, but if you have any more specific questions or ideas let me know!

-john

Posted by: Adrian Figallo Oct 9 2011, 04:15 AM

that looks great and all, but you know what is missing online?, a chromatic tuner, if you can somehow measure the pitch and dynamic of a note and put it into a graphical interface, you can have a tuner .

Posted by: sevenfx Oct 9 2011, 05:54 AM

maybe i'll do that this week as an experiment!

i always use the one that comes with line6 gearbox software, but you have a good point

Posted by: Ivan Milenkovic Oct 9 2011, 03:46 PM

It makes great sense man, keep us updated about the project!

Posted by: Sinisa Cekic Oct 9 2011, 09:47 PM

It looks promising man, just keep working

Posted by: Cosmin Lupu Oct 9 2011, 09:56 PM

QUOTE (sevenfx @ Oct 8 2011, 09:41 PM)

Alright, I'll try to give you guys a little more in depth explanation of what I'm thinking

Daniel - yes, that is similar to what I'm trying to achieve, but real time, and I'm not as focused on the midi output at this point. Basically, it would process the raw guitar input (as it comes in from a Line6 box, or microphone, or whatever). There is a program out there that does what I am trying to achieve already.. it's located here:

http://miracle.otago.ac.nz/tartini/

Like the video daniel posted, it shows the pitch that is being played while you are playing it, only its not broken into 12 notes. For example, you could see that you are bending a note from C up to D, and could analyze your vibrato. If you download that program you will see what I'm talking about. I find it very interesting to use while I practice.

Note for teachers: if you use this program, notice how it creates a "picture" of what you played. You could save this image, and students could strive to create a similar image to yours. Notes that were not bent to the correct pitch, or vibratos that were the wrong speed would be easy to see visually, and specific areas could be identified for practice.

As for the 3D aspect (my potential contribution to this project), think of Rockband or Guitar hero. You can see the sound as a time continuum, say moving from right to left. You will see the pitches as they move up and down (and possibly change color for different key parts of the scale, if you decide to select a specific key). In addition to pitches, you would be able to see the envelope in sound based on the amplitude of a frequency.

So if you're playing loud chords, you will see large notes appear, and playing softly would display a thin line. I would love to experiment with visualizing the attack, sustain, and decay of individual notes different ways.

When it comes to chords, and assuming it was possible, I'd love to be able to visualize the individual notes that make up the chord. Doing this based on the raw sound input is difficult, but could be a challenging enough problem to constitute my masters thesis .

In addition (for those of you still reading), I'd like to add in beat detection, and rhythm analysis. Still not sure exactly how this would tie in.

For an example of how I'd deploy this sort of project on a website look at:
http://www.tree-axis.com/Ess/_examples/inputFFT/

This is just showing frequency information of a microphone, but if you have java run-times installed on your computer, you should see it react to microphone data without having to install anything else.


Does any of this make sense? I'm kind of rushed for explanations right now, but if you have any more specific questions or ideas let me know!

-john

Posted by: sevenfx Oct 9 2011, 11:21 PM

QUOTE (Adrian Figallo @ Oct 9 2011, 03:15 AM)

that looks great and all, but you know what is missing online?, a chromatic tuner, if you can somehow measure the pitch and dynamic of a note and put it into a graphical interface, you can have a tuner .