Axtell's Notes: June 17

The BigGUI class has all but two features working! Those that aren't working are windowing (because I haven't gotten around to setting it up) and showing the FFT of a specific sample (haven't figured out how to do this yet.)

Changing buffer size, sample size, the text file to write to and the sound file to read from all work. SPLIT splits the given .wav into smaller .wavs (each of the length given in the sample length text field) and does all the math so that Show Peaks and Show Notes are fast. Show Peaks makes the graph which is the highest (or few highest) frequencies of each mini .wav (x-axis is time, y-aixs is frequency in MHz.) Show Notes makes the array (note by octave) the number in each cell represents how many times that note of that octave occurs in the given .wav.


So Tayloe's code is now completely integrated with mine in BigGUI. NoteFinder and PeakData are both graphed through GraphSplitter (a boolean determines which to graph). I thought I was having problems with the boolean because clicking on Show Notes did nothing and clicking on Show Peaks did what it should and what Show Notes should. Eventually I switched to two classes (GraphPeaks and GraphNotes) to avoid the boolean, but that didn't change anything. I finally thought to look at where I initialize the buttons and sure enough Peak and Note listeners were both controlled by the Show Peaks button. Luckily I had saved the GraphSplitter class and that all works as it should now.

I might have made a bit of progress on the finding decibels side of things. Audacity's formula to scale their amplitudes is value[i] = 20. * log10(sample[i]) according to a few forums I found about it. This scales the sound to a range of -1 to 1 dB. I'm not sure if this is accurate, but since we have access to Audacity's code, I'll look at that tomorrow and see if I can find something useful.

Tomorrow, besides looking at Audacity's code, I hope to get the windows and get FFT features of BigGUI working, clean up BigGUI and other new classes, and start looking at constant Q transforms.

6/17/10 Daily Journal of AT

Another fine day. I could get used to this.

Started out by getting all the programs to play nice together. All I had left from yesterday was the Note Finder, which now works a lot more efficiently. It also is able to count how many of each note are in a peak data file, and return it in a double array, to be printed out with our Big GUI. The notes are still a little off in the higher pitches, and I'll probably try and figure out a better splitting point tonight. The difference between notes starts out small and doubles each time through, so higher notes have a higher range of error than lower notes. Trouble is, I don't know a way to get a threshold other than getting the point halfway between two notes. Maybe research will help.

I also helped out Axtell with the Big GUI. We've got it to run the splitter and fft minifiles, then save the files to disk. We can change what file, sample size, filtering (though the non-null ones don't work yet), sample size, and save location. We can also print out a graph of the peaks over time, or a table of the total number of notes. So, once that's shined up and we get our new FFT function, we should have a working program ready for delivery. Let us know if you want any other features.

Daily Blog 3

Even though this is my first post in several days, I do not have much to report. I have been continuing research on FFTs in hopes to fully understand them so I can write a program to run it from scratch. I got a hold of a textbook on computer music, The Computer Music Tutorial by Curtis Roads, and started reading it. The appendix included a very detailed outline on how to program DFTs. After reading part of this textbook and re-reading some of the other sources, I believe I have a firm enough grasp on the subject to be able to write a FFT algorithm.

I started working on creating an Audio object and DFT object to store the information needed to perform the FFT. I believe my Audio class is finished and I am almost done with the DFT class. Today I perfected the code to do bit reversals on an array, which is the first step to running a FFT. I will hopefully be able to finish the code for running the FFT tomorrow and will be able to start combining my code with Axtell and Tayloe's GUIs, other interfaces, and file splitting programs.

Axtell's Notes: June 16

Today I updated the FFTGUI class and started work on a BigGUI class that will combine my FFTGUI and Tayloe's WaveSplitter, PeakData and NoteFinder.

Updates to the FFTGUI class:

Clicking on the comparison of the FFT cycles through which windowed FFT is on top. There is a drop-down menu of windows so that any single windowed FFT can be seen on its own. There is a clear button that sort of works (the FFT and Sine graphs are cleared by running a .wav of silence through them and the drop-down menus and text field don't change.) If the filename in the text field does not exist, it uses the silence again. A bunch of null pointer errors came up and were fixed. Most of them came from the program trying to do something on a mouse click or move over the window when there was no graph in the window.

I realised that FFT.java used all floats, GraphingData used all doubles and FFTGUI used a little of each. So everything is now using doubles. They are more precise which means they slow down the program, but seeing as we will be using it for very small files, it shouldn't be too noticeable.

The BigGUI is currently just buttons, text fields, and drop-down menus that don't do anything. Tomorrow I will be working on getting a few of BigGUIs buttons working.

I still need to figure out what units the raw sound bytes are in. I'm doing some research on this tonight and will report back tomorrow.

6/16/10 Daily Journal of AT

Two good days in a row! Something must be going wrong, but stuff works!

This morning, I was inspired. I finally figured out what sort of PeakFinder to use! The secret? More rigorous peak identification and standard deviations. To elaborate, the function, as it is now, first finds the peaks based on the surrounding points, fifteen to the left and right, and only saves them if the data is larger than all the surrounding points. (This may be changed to a smaller number when windowing is used, in order to save smaller peaks and peaks close to each other.) Then, of the remaining peaks, a standard deviation is taken, and all below the 64th percentile are discarded (the noisy peaks). It works super well!

I had to readjust the note finder, as the C notes generated were a step lower than they should have been. I also wrote in a bit that tells when a peak is inaudible (as in, lower than the 0th octave or higher than the 9th octave).

This afternoon, I worked on cleaning and commenting my code, and looked at Axtell's shiny new GUI. We discussed how we would implement our two programs, and figured out a new GUI that would take a file, split it, and display either hertz peaks over time or a list of the notes present in the piece. I've managed to re-do Wave Splitter and Peak Finder to work with it, although I may have to overhaul Note Finder more tomorrow. Anyway, it was a surprisingly productive day, and we should hopefully have a working GUI by Friday morning.

Also: Everything's better nested, like grids, loops, and baby birds.

To Do: Comment, clean, and get everyone to play nice with each other

Axtell's Notes: June 15

Lots of updates to the GUI today.

GUI Showing FFT and Curve of a440.wav

So, three separate windows open now. This is mostly so the two graphs will resize as their windows d0. The menu window has a text field that takes any .wav file, a combo box that changes the buffer size (how many samples are taken) from 512 (2^9) to 262,144 (2^18), three buttons (FFT, comparison, Sine curve), and a check box that controls whether or not the spill on the sine curve is shown. As the buffer size goes up, the program slows down. Gregor is looking at writing a recursive FFT method to see if that is fast than nested for-loops.

The peak of the file is printed in the corner (the peak does read exactly 440 Hz for the a440 file unless the buffer is 2^16 or greater. If the mouse is on the graph, the frequency in Hz at that point appears next to the mouse.

There was an interesting bug when adding in the buffer menu that changed the shape of the sine curve each time a different buffer size was chosen. I figured out that the GraphingSound class gets the 512 samples to plot from the array of raw data of size buffer that is rewritten each time the buffer size changes. The raw data is read from the file using audioInputStream.read(byte array). That method takes as many samples as will fit in the byte array, but it takes them from evenly spaced intervals across the file so that the FFT will be more accurate. To get the first 512 samples, I had to make a new method that read every sample there is, get the first 512 of those, and graph that. This slows down the program, but as we work with smaller and smaller files, it will be less and less of a problem.

I also did some much needed general clean up and commenting of my classes.

I still need to: set up cycling through the windows in comparison(I'm going to try setting up some states to do this), get the raw FFT data into decibels, add Tayloe's WaveSplitter and peak/note finder, clean up the GUI a bit more and research constant Q transforms.

I'm starting with the window cycling tomorrow.

6/15/10 Daily Journal of AT

So I figured out the problem from yesterday. I had created my files with a low amplitude (.1), in case I decided to add them together. However, at that low, the peaks are lost entirely in our FFT. At an amplitude of 1, they give a much different peak. (I should have guess when the waving occurred.) *sigh*

So I ran the numbers again, with a higher amplitude and shorter length (ten times as loud, but four percent of the length). Because it was so short, there was no data available for frequencies below 40 Hz. However, the ratio is now what was expected, half the sampling rate (44.1 kHz) over on fourth the buffer size (8192). Well, more or less. Annoyingly, this means at the very large buffer size of 32768, there are 2.691650390625 Hz per array. If I wanted to reduce it, get closer to (at least) a 1:1 ratio, we'd have to quadruple the buffer size (to 131072) which would drastically increase the time it takes to run the FFT. Potentially, we could merely double the buffer size, and have the ratio be at 1.3:1, which would allow us to at least tell the difference between lower notes (C1 and lower).

After all that, I made a function to figure out, based on frequency, what note was at the peak. It took a little figuring, but I managed to create it and then improve it. Initially, I had thought to use if/elif statements, but realized that would be a lot of code. Instead, I used a shortcut. I put the frequencies for the lowest scale (Ab0 to G0) into an array called basicNotes, with the corresponding letter value in literalNotes. I also put the distance between each element into another array (that was one longer) called basicRanges. The program takes an array of floats, presumably frequencies. For each one, it first figures out what scale it's in by testing to see if it's greater than the C of that scale, using a for loop. In a new for loop, it tests against each of the basic notes, multiplied by 2^scale (because notes are logarithmic), and returns the note that it's closest to (based on threshold between notes. Right now it's in half, which probably isn't exactly right, but it's close enough for jazz.) I had thought to store all this in one array, but Java requires them all to be the same, so I split them into floats and strings.

The rest of the day was spent testing different files to see how high their frequency peaks were, to try and get a ballpark range for the filtering; and testing the note finder. The latter is going better. Tomorrow will be cleaning up the files so that they play nice with Axtell's GUI, and trying to find a rule of thumb for getting the higher peaks.

P.S. For the test file we were given, the peaks are (approximately) at 292.0, 585.4, 522.2, and 259.7. In note terms, they are D4, C5, D5, and C4, respectively. As analyzed in thirds, the first two notes are D4 and C5, second two are D4 and D5, third two are C4 and C5.

Axtell's Notes: June 14

First job today was a program that would graph the .wav file as time/amplitude (as opposed to frequency/amplitude as in an FFT) It was simple enough to plot the float values from the raw data of the audio file, but there was a lot of spill.

a440 Sine Curve Points

Using a simple a440 file (one sustained note) a sine curve can be seen, but there is a lot of spill. I wrote a method that goes through each point and of the 5 points either way in the x-axis, it finds the closest and draws a line between those two points and only draws the lines and no points.

a440 Sine Curve Without Spill

a440 With A Partial At 880

There is still some odd spill, but not nearly as much and the curve is very clear. That option is now available from the menu in FFTGraph.

I figured out that the FFT graph prints a range of 0 to 22 MHz. With a little math, the FFTGraph window now prints the frequency with the highest amplitude in the corner of the screen.

Next I worked on turning everything I had into a GUI. Eventually if the mouse hovers over the graph the GUI will show the frequency at that point, and if the mouse clicks on the graph while comparing it will cycle through each graph.

Having never written my own GUI from scratch before, a lot of time was spent in working out how to use what I already had in a GUI. The GUI now works minimally. The FFTGraph class still provides a lot more useful information. Also, the GUI image will not scale when the window changes size.

Blank GUI Screen

GUI Showing an a440 FFT

GUI Showing an a440 FFT Comparison

There is a lot more to do with the GUI just to get it caught up with the FFTGraph class, then I'll look at adding Tayloe's WaveSplitter class so the files are of a more useful length, and then we'll move on to the constant Q transform.

I also still need to figure out how to get the raw data from the FFT into amplitude in decibels.

6/14/10 Daily Journal of AT

Another day where things go wrong. They seem to come in cycles.

I basically had two things I wanted to do today: figure out a better peak finder, and find what peaks correspond to what frequencies. Since the correspondence was easier, I decided to get that out of the way first. It started out fairly easily: I created a number of files of frequencies ranging from 10 to 200, going up by 10 each time, then wrote a shell program to run an FFT and a peak finder on each of them. (Since there was only one peak, I ran the halfing function six times on each, which got the major peak, or occasionally peaks.) I plotted the data and found the ratio. To be on the safe side, I did so for a variety of buffer sizes, from 2^15 to 2^10. I decided to use the largest, as it gave the most exact data. With 2^15, or 32768, the ratio of array placement to Hz was 28.6. So far so good.

The problem came when I decided to run some more frequencies through; namely, frequencies 50 through 2000, going up by 50. That's when the problems began. For 50, 100, 200, and 250, the peaks corresponded perfectly with their respective Hz. (150 had some problems with resonance, I think. I don't know why.) However, for 300 onward, there were no peaks in their respective frequencies. Every peak was less than 300. I thought it might be the multiplier, but changing it didn't help with the higher frequencies, and made the lower ones wrong as well. The only conclusion I could come to was that the FFT was cutting off data. Which made sense, given the data.

However, this flies in the face of the testing data done on one of our files, a440AndOnePartial.wav. When graphed, it clearly gave two peaks, the nearer one much larger. To try and figure out what was going on, I graphed the data from 50, 100, 150, 200, 250, and the a440 file. A little blurry, but I'll explain.
The five graphs on the left are 50 through 250, with the yellow boxes pointing out the peaks. 150 is hard to see, but looking at 140 and 160 (immediately to the right) it is clear where 150 should be. However, to the top right is the a440 and partial graph, which, fairly clearly, shows those peaks. In the completely wrong positions!

I haven't had much time to mess with the FFT. I tried taking out the mirroring function, which gives a few peaks close to accurate for 300 to 500. However, beyond that, the peaks do not exist. I'm going to do some reading tonight to try and figure out what's wrong, but unless this is fixed, the FFT function is useless for any frequencies above middle C.

6/11/10 Daily Journal of AT

Hooray! I feel like everything that went wrong yesterday has been put right. Well, some of it, anyway.

We scrapped the JASS FFT function and implemented the version from Sonogram, which works a lot better. Apparently, the underlying waves were still showing through in the JASS code, which was throwing everything off. Now, nearly everything is working as it should. The windows work, we're pretty sure correctly, although the redrawing problem still exists. The 440 FFT now works and gives the same single peak consistently, regardless of the length of the file. In addition, an older problem of the FFT not working on wave files smaller than a second has been repaired, almost certainly by changing the FFT. Now, an accurate FFT can be run on files of any length (within reason, as data on a few frames will be far from accurate, and the computer may not be able to handle files larger than 20 seconds. However, the FFT will likely only be run on files at a max of 1 second, so that's not really a problem.)

My main dilemma is with the Peak Data program that finds the peaks of a given set of FFT data. There are three options that, at one time or another today, I've implemented:
1) Use a pre-generated threshold to pick the peaks. This is rather dangerous and inaccurate, as the range of peaks generated in different data varies greatly based on the length and the occurrence of other peaks. This method will not be used.
2) Use a threshold based on the top percentage of the range of data. This would be similar to the above, but much more accurate on any given set of data. However, the issue becomes as to what percentage to take. With testing, I've determined that, in general, anything less than 10% will miss many smaller peaks, while anything larger than 20% will pick up too much noise. This method may be used.
3) Get the average of the data and delete anything smaller. Run it multiple times to leave the largest peaks. I had initially thought this would half my data each time; however, it has turned out to get rid of anywhere from half to nine tenths of the previous data, based on what was left. This seems much better for finding more accurate peaks; however, the problem of how many times to run it comes up again. From testing, it should be run between 4 and 7 times. This method may be used.
4) Some combination of the above. This is most likely what I will do, along with more accurate peak finding.

Alternatively, I should probably write a more accurate peak finder. Right now, it considers anything a peak if the data immediately before and after is less than the given value. Perhaps with checking more surrounding data and having more rigorous qualifications will allow the finder to do more of the work, and perhaps not need the cleaner at all.

The other thing I found was that the WaveSplitter is kind of slow. When the length of the resulting mini files is a second, or a largish fraction of a second (as low as .1 sec) it isn't noticeably slow. However, getting files that are 512 samples long (which is around .01 seconds) takes a lot longer. For example, on a two minute song, the program generates 9654 files, and takes about twenty minutes to generate and run an FFT on each. Adding in the analysis will probably bring the total to 30 minutes per song analyzed. Which isn't too bad, but if it can be faster, I'd like it to be.

TODO: Write function to give frequency (range) of a given peak. Write peaks to file. Sort file. Use file to print new graph of total peaks in a song to graph. Get WaveSplitter to go faster, possibly by not saving wave files (?) or by having larger sample size. Find peaks more accurately.

Axtell's Notes: June 11

Today has been massively productive. First off, the doFFT() method (from JASS) has been replaced with Sonogram's method because it does not write over the array that it is given, as the JASS method does.

It also returns a more accurate array of data without any negative values. Because there are no longer negative values, the graph has been reformatted to only show the positive x- and y-axises. On the graphing side of things, the FFT graph now displays the frequency in megahertz (MHz). I hope to set up a feature that will display the frequency of a peak in hertz when hovered over/clicked on.

I also discovered that the windowing functions have been working, but because the graphing class scales each set of data to fill the graph, the difference couldn't be seen. When called in comparison, the four FFTs (FFT and three different windowed FFTs of the same file) appear on top of each other in different colors and in the same scale.

Windowed FFT Comparison

(Click on the image for a larger version)

Red = unwindowed FFT
Blue = Bartlett window
Green = Hamming window
Yellow = Hanning window

I will be working on changing which window is in front through some buttons or clicking on the graph.

Another update to the graphs is that if multiple windows are open at once, closing one does not close all of them and does not end the program.

The next step will be to combine this now working code with Tayloe's wave splitting code to make FFTs (windowed or not) of much shorter sound files (512 samples so much shorter than one second).

Next week I will also continue update the GraphingData class, and work on adding a function that will plot the sound file given on a graph (x-axis as time, y-axis as amplitude). So if the sound file given were a single sustained note, it would graph a sine curve.

6/10/10 Daily Journal of AT

I really hope the professor comes in tomorrow, because I feel like I'm stuck. Most of the morning was debugging and discovery of problems. The afternoon was more testing, comparing graphs and functions, and discovery of more problems. To name a few that I found/helped with (and the solutions we tried/implemented/can't figure out):

• Data files add information, making graphs longer and longer without adding information. (Solution: Clear the data files.)
  
• Data files are cleared at the end of each program, making it impossible to tell where errors occur. (Solution: Clear data files at the beginning of program.)
• Data files are not passed any zeroes. (Solution: Let them pass zeroes.)
• Data files have a lot of zeroes at the end that mean nothing. (Solution: Reduce size of array.)
• Data is mirrored. (Solution: Reduce size of the array still more.)
• Data is overwritten each time FFT is run, making it impossible to do peak comparison on graphs. (Solution: Manually rename the files so they are not over written. Temporary solution.)
• Windowing doesn't work. (Solution: Run windowing algorithms on wave data, not on FFT data.)
  
• FFT data is wrong when multiple windows are used. (Solution: Re-write imaginary array of zeroes at the begining of FFT call, rather then at creation.)
  
• FFT data of rectangular, Bartlett, and Hamming functions is the same. (Solution: ???)
  
• Peak generator only gets highest points, regardless of their peak-quality. (Solution: Find the peaks, then clean the data, rather than the other way around. To go into more detail, the clean function gets the average of all the data, disregarding any zeroes or negatives, then sets any number below the average to zero. Run multiple times, it reduces the size of the data severalfold. The getPeaks function got the peaks by checking a given piece of data against its immediate neighbors. If it was greater than both, it was considered a peak. However, running clean, then getPeaks, resulted in the same data as only running clean. The reason for this was that, if its neighbors were set to zero, a given large number would always be considered a peak, regardless of if it actually was or not. Run the other way around, so that the peaks were generated first, then cleaned, led to a more accurate analysis.)
• When FFT is run on a single 440 wave, with the only difference being in how long the file is (1 sec, 3 sec, and 12 sec), different data is generated. (Solution: ??? I really don't know why this is happening, or even if it's a problem. The FFT data generated is put into an array, then printed from their. The size of the array holding the bytes does not change based on the length of the sample, but is set to the buffer limit. This could be limiting the data given to the FFT; however, even if this were the case, the data generated should still be identical, because it is exactly the same wave. If it were to vary based on the length of the file, it should only do so in terms of height of the peaks, not their placement. The only option I can think of is to scrap our FFT and start again, but we thought it was working! Also, it's what we've been doing all week, and I really hope we haven't wasted all that time.)
• FFT data is in an array, and graphs have no way of telling what peaks correspond to what frequencies. (Solution: Test single frequencies in the peak finder, find what array values corespond to what frequencies, find a function, and translate songs based on that. However, with the FFT on the fritz, any data generated by it cannot be trusted. In addition, when I tried this with single frequencies, it gave multiple peaks of similar heights. In fact, the file with a single base note (with more actual noise) generated less noise in the FFT! This is frustrating, especially as I did not write the code that does the FFT, and thus have trouble tweaking it to make it work. When zoomed in upon, the graphs the FFT generates are similar to graphs created by Audacity analyzing the same files, or at least, they were yesterday. Today, they show a lot of discrepancy between the pure frequencies.)
  

The only thing that seems to be working consistently is the WaveSplitter, probably because it's too simple to mess up much. I'm pretty sure getPeaks is working; however, until the FFT is fixed, it will be impossible to say for sure.

Axtell's Notes: June 10

Unproductivity is apparently the word of the week.

I did get some good work done today though. I figured out the flaw in the windowing methods. It was returning a list of zeros because each term was being multiplied by a null. Having fixed that, I realised that I was asking it to graph the curve of the window rather than the window applied to an FFT. That too was fixed and now each window method prints a graph. I believe the graphs are incorrect, but at least something is shown.

Each window function when called on their own (as opposed to together through the compare option) seems to be the same as the FFT without windowing. The Hanning window is a little different, but it doesn't get rid of the leakage. If anything, it adds more.

When called together, something is still off. The Hanning window looks the same, but the others lose several values towards the end of the frequency spectrum.

Comparison of FFT Graphs

(Click on the image to see a larger, more legible version)

In this process a few other things got accomplished. I got rid of the mirroring of graphs the was seen yesterday. I had been only plotting the first half of the FFT values, thinking that would get rid of the mirroring. It turns out that it did and the image was mirrored twice, so I now plot only the first quarter of the FFT. Not sure why this is true. I'll do some research into that at a later date.

I also discovered that there were two float arrays with the same data but different names floating around (no pun intended) After some searching I found that the doFFT() method rewrites the float array it is given with the FFT values and returning a second version of that array. That has been remedied.

Another big change in the code is that it no longer writes to Text files. That was getting confusing as it had to write over the same file each time, so the information is stored in a float array in the FFT class that is passed to the GraphingData class to be graphed. That cleaned up the code quite a bit.

The last change was to change the if-else loops that dealt with the menu choice and which window to use are now switch-case blocks. I just find that it's neater. I also added a try-catch block around the menu's switch-case blocks to keep it from crashing.

So maybe today wasn't that unproductive, quite a lot got updated if not fixed, but it felt it because the windows still do not work right. That's top of my list tomorrow. Again.

I'm going to end today with some research into what I should add to this next. Assuming that the windows are fixed and working tomorrow, I will need something else to do for next week.

6/9/10 Daily Journal of AT

Today, fortunately, was much more productive than yesterday. In the morning, I wrote and tested WaveSplitter, a program that splits one wave file into many smaller ones, then runs an FFT on them. (Note: Code used to run FFT and draw graphs was from yesterday, still have to implement changes made to them by partners.) After doing research on the AudioInputStream class, I managed to get my code to read the buffered file and write it out again in chunks (rather than using nested loops, as I had originally planned). Each wave file can be successfully FFT'ed upon. Only problem I discovered was that, if one tried to split the wave file into larger chunks (the default that works is one second) it will only give one second to be analyzed, just space them out more and have fewer mini-files. I'm not quite sure how to fix it, so am waiting until Friday. There was another problem but I can't think of it right now.

This afternoon, I worked on a function to get the peaks from the FFT data generated. This involved a bit of trial-and-error with reading the files in (and the hope that the FFT file generation hasn't changed much today). I wrote a simple loop that goes through the data and gets rid of the smallest half (because they are obviously not large enough peaks to bother with). Running this "cleaner function" multiple times gives the peaks of the data. Comparing the graphs to the numbers, it appears fairly accurate. The only trick will be to translate the location of the peak to a frequency, in order to make this useful for analysis, and be able to write and append this data to a single file, then use the file to generate a graph for the song as a whole (with higher accuracy than taking an FFT of the entire song, as in the latter, small peaks would be lost in the noise, while in the former, smaller peaks are preserved).

Also, as a special tangential issue, I researched some information about logarithmic thinking. I first heard about this on a podcast, RadioLab, done through NPR. They did a special on numbers, and their first story was about how babies and toddlers "think" of numbers.

Found at RadioLab's website.
Basically, there was a study done on babies, children from two to three months old. They were hooked up to an electrode net and shown a series of eight identical pictures, in one case, ducks. They were measured on their reactions when the pictures changed to, say, eight trucks. Then, they changed from eight ducks to sixteen ducks. The babies' brains still showed a change, but in a different area. The study went on to show that babies can tell the difference between large differences of numbers, like ten and twenty, but not smaller ones, like nine and ten.

The reason for this is that, initially, our brains think of numbers logarithmically. The show goes on to tell about studies done among non-counting people living in the Amazon basin, who, when asked what is between one and nine, reply "Three." To us, it makes no sense, as the number directly between one and nine is five (four more than one and four less than nine). But the way they think of it is, one times three is three. Three times three is nine. Therefore, three is exactly between one and nine.

This way of thinking is not unique to infants and non-counters, either. Say, for example, there's a sale on a pair of pants. Normally, the pants cost twenty dollars, but today, they cost ten. That's a really great deal, isn't it? Now say that you find a jacket that's also on sale. It used to be a hundred dollars, but now it's only ninety. Is it as good a deal?

Thinking linearly about it, it seems both these deals are the same. However, half off is far better than ten percent off, and our gut reaction is to say that the pants are a better deal than the jacket. As a third example, if you were thinking of buying a ten thousand dollar car, the salesman knocking ten dollars off the price will probably not sway you much.

This relates to our project with the fact that steps on the western scales are arranged logarithmically. For example, going from middle C to high C doubles the frequency (and halves the wavelength). Now, this has something to do with the waves that make up sounds, as waves that have peaks at similar times will reinforce each other, creating a stronger sound. If wave 1 has half the frequency of wave 2, they will reinforce each other every other wave, whereas if wave A has a wavelength of a hundred less than wave B, there's no telling when they will reinforce each other. However, our brains prefer the reinforced waves, which in turn lead to our appreciation for things "in key." I'm fairly sure it's related to our thinking logarithmically, or perhaps we think logarithmically and like chordes due to some other facet of human nature. Regardless, it's pretty interesting.

Axtell's Notes: June 9

Another not as productive day as I would have liked. The main discovery of the day is that I was giving the doFFT() method the wrong length of array. The array length should be a "buffer" that is always a power of two. The higher the power, the more accurate the result. I had been giving it a list as long as there were samples (Often in the hundred thousands for a sound clip of several seconds). It is now set to 8192 (2^13) and seems to be working better. It is displaying a much nicer image, but still has the mirroring effect. Should figure out how to fix that.

I did some basic clean-up of my classes so the code would only have to work as hard as it had to. I also found a new way of converting bytes to floats. Instead of the confusing method I had (borrowed from JASS) I found four lines in Sonogram's code that accomplished the same thing. It casts the bytes as Bytes, and the Byte class has a method floatValue().

The window that pops up with the FFT is neater now. The line graph now is scaled as the window is re-sized so that it always fills the x- and y-axises. Other enhancements to be added to the graph window at a later date include:
-If more than one window is opened, offset each additional window so all are visible
-One window should close without closing all windows and ending the program
-The compare option will print in the same window in a different color for each window

Windowing functions are still not working correctly. Bartlett and Hamming windows seem not to make any change on the FFT. The Hanning window still produces nothing. A little research into this showed that this is because the method returns all zeros. I tried replacing the equation with several other versions of the Hanning window I found, but they all returned zeros. Because of this, I decided that I must be implementing the window in my code incorrectly. Once again, windows are on the to do list tomorrow.

On the topics of windows, I discovered that the Hanning window is more accurately called the "Hann window" in honor of van Hann. Because of the existence of the Hamming window and the ensuing confusion it tends to be called "Hanning" nowadays.

Tomorrow's Goals:
-Get windows working
-Figure out why the function mirrors itself and fix
-Print the corresponding frequency's along the x-axis
-Continue to update the graph window

Daily Blog 2

Today I spent most of the day finishing reading the articles I started yesterday. Part II of An Introduction to the Mathematics of Digital Signal Processing focused on sampling,m transforms, and filtering. DFTs along with z-transforms were discussed. I skipped the section on z-transforms as we are focusing on DFTs right now.

After finishing the articles, I believe I have a much better understand on how DFTs and sound sampling works and how to deal with the phenomena that can occur when sampling a signal and applying a DFT. The rest of this post will focus on summarizing what I read (hopefully) in a manner that will make the main points clear to Axtell and Tayloe.

As mentioned yesterday, the first part of the article focused on explaining the algebraic and trigonometric topics that are relevant to the understanding of transforms and sampling. Some of these topics included a description of the different types of numbers in math; whole (Z), natural (N), integer (I), real (R), and complex (C). Each set is "included" the sets to the right. For example, the integer number set includes the sets of whole and natural numbers, etc. The set of complex numbers are the most important to the discussion of signal processing as all signals have both a real and an imaginary part and can be described as: x + jy where x,y are elements of R and j² = -1. j is used to represent the square root of 1 in signal processing as it is a branch of engineering and i is already used to represent current.

The article then goes on to explain polynomials, roots, exponents, logarithms, and e giving the rules and identities (if any) for each topic. He then explains sums and series. Summations are an important as a DFT is expressed mathematically as a summation.

After explaining the important topics in algebra, the topics of trigonometry are explained. Three measurements can be used in trig; radians, degrees, and grads. Pi/2 radians = 90^o = 100 grads. Pythagoras' theorem is discussed and the trig the 13 most common trig identities are given. sin(x) and cos(x) only differ in phase (when x = 0, sin(x) = 0 and cos(x) = 1).

I found the following explanations to be helpful:

• period = frequency = repetition rate of a periodic waveform
• amplitude = the difference between peaks and/or troughs = strength
• timbre = tone quality = general shape
• pitch = "tonal height" of a sound

Fourier's theorem states the following:

• any periodic waveform can be described as the sum of a number of sinusoidal variations with each one having a particular
  • frequency
  • amplitude
  • phase
• must be Dirichlet as well as periodic meaning it must satisfy the conditions for a real-valued, periodic function
• f(t) = f(t + T) where f is the periodic waveform, t is time, and T is the period

The last important piece of information from the first article was Euler's equation which is: e^jx = cosx + jsinx. In other words, the natural number e raised to the power of x multiplied by the imaginary number j can be broken up into two parts; the real part (cosx) and the imaginary part (jsinx).

The second article focused on sampling, transforms, and filtering. With sampling, a phenomenon known as "the wagon-wheel effect" can occur and happens when the sampling rate in relationship to the actual frequency is too small. Using the example given by its namesake, in films, as a wagon increases in speed, the wheel will appear to increase as well until half of the frequency of the sampling rate (frames per second). At this point, the wheel will appear to decelerate or even go backwards. At R/2 (where R equals sampling rate), the frequency of the wheel is indistinguishable from its negative counterpart. For example, in old Western films the frames per second was equal to 24 (R = 24 Hz). At F = 12 Hz (F = actual frequency of the wheel), the wheel appears to be rotating at its maximum velocity. At F = 18 Hz, the wheel appears to be rotating at -6 Hz because it is impossible to tell if the wheel is completely 3/4 of a rotation each frame or -1/4 of a rotation. The Wikipedia article provides a good animation of this effect. The sampling theorem states to avoid aliasing or foldover (the wagon-wheel effect), any simple harmonic variation (a sinusoid) occurring at a rate of F Hz must be sampled at least 2F times per second. However, to avoid ambiguity in the equation statement of this theorem, the sampling rate R can not exactly equal 2F. So the sampling rate must be at least two times greater than the highest frequency component of the original waveform.

Digital signals can be thought of as functions of discrete values of time (n) or sequences of numbers, with each number representing the instantaneous value of a continuous time function (the original analog signal). I came across a good analogy while reading: "Prism is to light as Fourier analysis is to sound." White noise is the sound equivalent to white light in optics.

The article then discusses the DFT. It is "used to calculate the spectrum of a waveform in terms of a set of harmonically related sinusoids, each with a particular amplitude and phase" and is most commonly implemented by the FFT. Some quick notes about the DFT sequence x(n):

• x(n) is modeled as one period with N samples (entire audio file is one period)
• FFT generally requires N to be a power of two
• more than two samples are required (N > 2)
• DFT[x(n)] = X(k) =

N - 1
-----
\
| x(n)e^-jwnk 0 <= k <= N - 1
/
-----
n = 0
where w(omega) = 2Pi/N and e^-jwnk = cos(wnk) - jsin(wnk)

• magnitude (amplitude) = |X(k)| = sqrt(a_k² + b_k²)
• harmonics of a sound = R/N

Hopefully much of this was a review and me pointing out the obvious. I hope this will help clear up any remaining issues with understanding sampling and DFTs. The full articles were written by F.R. Moore and were published in Computer Music Journal Vol. 2 Nos. 1-2. These articles can be accessed through the JSTOR database for further reading.

6/8/10 Daily Journal of AT

Unfortunately, most of today was spent debugging with little to show for it, except for how not to do things.

Started out with a copy of Axtell's code from yesterday, an FFT function and a graph-drawing function, uncommented. Initially, we both thought that, since the imaginary in an FFT starts as an empty (or all zeroed) array unless a reverse FFT was being done, we could do away with it. I did so in my version. Then I started work on splitting audio files into portions, then analyzing them. This led to the morning of endless debugging.

First, I reasoned that the file gives the information of the number of samples per second, or srate. So, if I wanted to take a one-second sample of the song, I would get srate bytes from a file. If I wanted half a second, I would get half that many. So, I ran a loop to get all the samples, and copy them a portion at a time into a temporary array, then run an FFT on the array and store the data. I realized early on that the song samples do not tend to end exactly on the second, so I added an exception to write the data as zeroes if the loop was trying to copy samples that didn't exist. However, I was still stuck, as every time I tried to run the program, I got a Null Pointer Exception, not within the copying loop, where I almost expected it, but in the FFT function.

After messing with the various variables and becoming very confunsed, I asked Axtell what normally caused this problem. Apparently, if the number of samples is fewer than the srate, the FFT function won't compute. I still don't know why, because as far as I can tell, the function never uses the srate, but I adjusted the loop so that it only used audio samples of srate length or longer. I finally ran the function and printed the graph of the data...to see a mess. I tried zooming in and out, adjusting the window, printing only one graph at a time, still, nothing useful. Then I put the imaginary numbers back in.

Apparently they do do something, as that fixed the problem for the first graph generated. Unfortunately, all subsequent graphs, rather than having peaks in a few places, showed striking similarities to a sin wave. I thought the problem might be the imaginary number file, which was a global variable and never got reset back to zero. Doing that helped, but I still got a wave rather than peaks. I'm still trying to figure that one out, but I have a feeling it has something to do with all the global variables in that function.

The plan for tomorrow is to bypass all this mess. Rather than writing a function to split the sound file within the FFT function, I'm going to write a separate class to split a sound file into half-second (or so) increments, then run each of the resulting files through the FFT. Problems I forsee include having to figure out Java's AudioReader class, which confused me greatly before, or using BufferedReader, which lead to a loss of aliasing, or something syntactically similar. However, Gregor is starting work on getting the wave-byte-wave function working, which would solve that problem. Either way, I should have at least a function to show by tomorrow for my work these past two days.

Related minor issues: Our FFT will not read any music file longer than ~15 seconds. This is a problem, as the function should ultimately analyze full-length songs, but may be solved with the function I'm trying to write. The Hanning window is not working, and no windows are working consistently (Hamming only works for Axtell, and Barlett only worked on half my files). Sample rates need to be really large, as in 16K, for the FFT, because it's only at that level that the minor waves become noticeable above the noise (discovered when creating test files in Audacity).

Interesting notes: In music, whole notes can last anywhere from a second to ten, depending on the tempo. While the written values do not change, the speed at which they're played does. This means that, depending on the song, a second could contain a flurry of pitches or just one, and it cannot necessarily be told in advance based on the sheet music. Ideally, I'd like to analyze each note, but due to not only the variance in notes but in their length from song to song, this would be impossible.

Axtell's Notes: June 8

Today I built a class that could compute the FFT and show the graph of any WAVE file it was given. It includes a simple menu with a few different options.

I checked the graphs it was making against the graphs Sonogram made for the same WAVE files to ensure that the FFT algorithm works properly.

Once it was written, the rest of the day went to debugging it. The plain FFT works and automatically scales it so it is nicely visible. The windowing and comparison options do not work as they should (instead of a smoother FFT it displays a straight line), so that's on my to do list for tomorrow.

I also need to add numbers to the x-axis saying what frequency is where. I was starting work on finding the frequencies today and hopefully will have that working tomorrow.

I did find out that my FFT method is doing something odd. I only use half the number the method makes because it mirrors itself. Ignoring the second half of the numbers should fix that, but I found out that I was too far zoomed in. If I zoomed out I could see that my function still mirrored itself. I was not able to find out how that was happening, but if I used only a 100th of the list or so, I got rid of the mirroring.

So I have the start of a useful program that is still full of bugs. Tomorrow I hope to work out most of the bugs and do some more research into what should be added to the program next.

Daily Blog Entry 1

Today was the first day of my second week of research. Last week I devoted much of the week to background research in order to understand the material needed tackle this research project. I mainly researched the mathematical meanings and derivations of Fourier transformations slowly working my way towards understanding discrete Fourier transformations (DFTs) and fast Fourier transformations (FFTs). I discovered an FFT is the one and the same with a DFT; the only difference between the two are the way they are derived. In computing, calculating a DFT is nearly impossible as it is a function of magnitude O(N²). FFTs, in comparison, are functions of magnitude O(NlogN).

I began this week by continuing where I left off last week with my research regarding the mathematical origins of digital signal processing (DSP). I started the day by looking at Spectral Tools, a music analysis program implementing Java and Max Runtime. I was unable to install Max Runtime on the lab computer but plan to run the program on my personal computer.

I also discovered a MIT Press publication entitled Computer Music Journal which can be accessed online through the MIT Press Journal website. While searching their database, I discovered a textbook I thought would be helpful, The Computer Music Tutorial by Curtis Roads, which I subsequently requested at the library.

Through my research on this site I discovered a two-part article, An Introduction to the Mathematics of Digital Signal Processing, by F. R. Moore which was published in Computer Music Journal which can be accessed through the JSTOR archive. These articles proved to be very enlightening regarding the mathematical aspects of DSP. The first article was a useful tool to recall the main themes of algebra and trigonometry that are imperative to the understanding of DSP. This article was written as an attempt to explain the math without explaining it through calculus, making it easier to understand.

The second article delves into sampling and transforms. Tomorrow I plan on finishing this article and will (hopefully) fully understand the math behind the transforms. I also plan to start programming a FFT from scratch.

Axtell's Notes From June 7

On Friday, I ended the day with a short piece of code that I had pieced together using code from JASS (Java Audio Synthesis System) and some modifications of my own. When ran, it printed a long list of floats, but I couldn't figure out what they represented.

So that's how I started today. I did some research into what would be returned by an FFT algorithm, but didn't find anything. I eventually decided that I would graph the points and see if I could see anything useful.

I got a simple graph building class on Java Forums written by hardwired. Again I modified it for my needs.

I ran an FFT on an a440 with on partial meaning an FFT should show 2 peaks (440 and 880)

I decided that the array of floats I had were the amplitudes, so I plotted the points as the y-coords (amplitude) and spreading them evenly across the x-axis (frequency) I got the following graph:

a440 FFT

Two peaks! I then used Tayloe's windowing methods on this same a440.wav file and got the following graphs:

a440 FFT with Bartlett window

a440 FFT with Hamming window

The Hanning window method left just a straight line, but I hopefully that will be fixed tomorrow.

Other than that breakthrough, I found two amusing explanations of FFTs:
A song about FFTs (Lyrics here)
A picture book explaining FFTs applied to visuals