09/29/24
High level functional block diagram:
This week we are receiving our ESP32 boards. I want to begin installing an RTOS and getting more conformable in the environment. I want to use use the breadboard microphones that we ordered and gather and test both I2S and analog audio samples. Hopefully we can achieve a live demonstration by the CDR on Friday.
To do:
- Implement RTOS on ESP32
- Gather audio samples via I2S and analog
- Test localization methods on audio samples
I am worried about CDR.
09/22/24
This week I got some more stereo microphone samples with Eric, and found a good way of processing them for testing. I have been using cross-correlation in Python to test these samples with some pretty solid results. The error I believe is coming from delays from hardware and software, which I want to make my next priority. I talked with Liam about implementing a real-time operating system, so I want to spend some time on that next week and see if its a good solution for us. I also met with Luc to get an update on hardware, as I think software has finally caught up to hardware on the project.
To do:
- Investigate a real-time operating system
- Order an ESP32 for collecting audio
- Convert cross-correlation function to C
I’m concerned we won’t have enough time to put together a real “finished product” this semester. I think we all have great ideas and have been working hard, but there just isn’t enough time in a semester for all the work we want to do.
09/13/24
This week, to start off, I created the Github for our project, which is still private but will be public by the end of the semester. I spent a lot of time looking for a more convenient development environment rather than a working on a dedicated Linux machine. I’ve begun using Windows Subsystem for Linux (WSL) which is super convenient because you can share the same filesystem as your Windows machine. I then was able to clone our empty Github repo from my WSL machine after some finagling with SSH keys.
Lastly, I worked with Eric to start getting sound samples to start looking for characteristics of the sounds signals that we can use for localization. Right now, we are able to take two simultaneous audio inputs, and we are looking for a software that we can use to analyze these signals.
To do:
- Find a suitable software to analyze sound characteristics
- Find characteristics in sound that we can use to localize a sound (amplitude/timing/frequency response)
- Get the rest of the software team able to clone Github repo
My biggest concern right now is that the simultaneous inputs are not precise enough to measure signals. We are attempting to mitigate this with a way of “calibrating” the two mics at the beginning of each sample, I just hope the delay between signals is consistent enough across a single sample.
Start of Fall 2024 Semester
04/21/24
This week I spent a lot of time preparing our report and preparing for my capstone presentation. We tested the board some more and tried simulating the circuit, but we still weren’t able to identify our issue. We need to figure that out so we can start getting audio from the boards, which I would like to be able to test over the summer.
Below is an image of the current frames with the boards inserted, although these are the boards without the actual components solder on.
To do list:
- Figure out issue with our PCBs
- Fix issue or order new boards to test over the summer
- Complete report
04/13/24
This week I spent most of time writing our report and going over peer reviews. Since I was the only one writing the report it took up a lot of time that I wanted to spend on testing the boards. I put 5V into the board but it doesn’t seem that the MEMS mics are getting the 2V that they’re supposed to be. I want to spend a lot more time on the boards this week.
To do list:
- Verify and test PCBs
- Lots of conductivity tests
- Hope to put full power into the board by the end of the week
- Print new iteration of frames to fit new PCBs
04/06/24
This week I created the second version of our PCBs. Instead of using square boards embedded into the frame, this time the board is the entire arm of the frame. We did this because we we’re approaching space constraints with all of new components, and it leaves us with some room for expansion. Also, using two mics on the same board will give us more precision in terms of the distance between mics as opposed to two separate boards. I also spent a lot of time with Luc in the Plexus lab, we took turns soldering on the components although he definitely did the majority.
To do list:
- Verify and test PCBs
- Lots of conductivity tests
- Hope to put full power into the board by the end of the week
- Print new iteration of frames to fit new PCBs
My biggest concern is that the boards just won’t function due to a soldering error or some other error. Hopefully that’s not the case!
03/23/24
This week we were able to get the PCB’s fabricated, but we had some issues attaching our components. We also realized we needed an amplifier circuit between our mics and our ADC. I was able to create and test an amplifier circuit, but we haven’t been able to test it with the MEMS mics yet. That will be the goal of this week, get our components attached properly and test our amplifier circuit, and get output into an arduino via I2C with the ADC. That way we can confirm things work before we go to test them with the Linux box.
To do list:
- Attach parts to PCBs
- Must use solder paste instead of hand soldering
- Test MEMS PCBs with amplifier circuit and oscilloscope
- Move to testing with ADC and arduino
03/15/2024
This week I was able to print the first iteration of the glasses. They fit pretty well, and the print also showed me what I should do differently for the next print. I added a small nose bridge and extended the arms to go down a little more to help with slipping. I also included the slots for the PCBs, but I want to wait to print these until we confirm the size of our PCBs. We weren’t able to fabricate the PCBs this week, but we’re hoping to have them complete this upcoming week so we can hook up the mics.
To do list:
- Fabricate PCBs
- Fill out form for Mark and Chuck
- Get more info on Plexus solder oven
- Receive rest of parts to go on PCBs
- Wire up PCBs, ADC, Regulator and I2C Bus
- See above
My biggest concern right now is that our PCB is too small for the capabilities of the printer in the EERC, and we’ll have to compromise on size, making the glasses themselves bulkier and look kind of weird.
03/09/2024
This week I was able to get Ubuntu on the industrial computer, and we ordered and received several parts for the hardware re-work. With Ubuntu installed, the computer instantly recognized the new I2C bus that we will be connection the MEMS microphones to. Now we just need PCBs to put the MEMS microphones on and we’ll be rocking and rolling. I also printed a first and second iteration of the frame for the glasses, so once the PCBs are finalized I’ll be able to make the prototype for the entire frame.
To do list:
- Complete first physical prototype
- Connect MEMS to ADC and ADC to I2C bus
- Begin audio analysis
02/24/2024
This week we had our Critical Design Review for our enterprise. We got a lot of good feedback that will give us direction for the upcoming weeks. We will be transitioning from working on the Pi to try to accomplish our goals on a beefier piece of hardware first. The hardware that we’ll be using is an industrial computer that I was able to acquire at no cost, shown below:
This hardware should accelerate progress on sound localization, since we won’t have to wrestle with hardware limitations as much.
To do list:
- Get Ubunutu running on industrial computer
- Order new hardware that is compatible with computer
My biggest concern now is timing. We are making a major switch halfway through the semester. Although I do think it will be worth it, otherwise we wouldn’t be doing it. And now that I don’t have statue anymore I’ll have much more time to put in.
02/17/2024
This week I tried Openai’s Whisper on the Rapsberry Pi, with Raspian and also Ubuntu. Whisper is an open-source speech-to-text program built on Python. Raspian proved very difficult to work with, even doing basic things like adding repositories and downloading packages. However, Whisper was able to work on Ubuntu, but the speed of the tests were disappointing. This week I will work on seeing if I can speed up the tests, I am hoping feedback from the CDR will assist in this.
To do list:
- Find ways to speed up Whisper on the Pi
- Find alternate speech-to-text programs that may be faster
- Receive feedback from the CDR about current issues
I still have several concerns, mainly being that we haven’t received our Digi-Key parts yet. We still have plenty of work to do with just the Pi, we it would be nice to test the quality of those parts. My other concern is that the Pi just may not have the processing power for what we want to do with it.
02/03/2024
I wasn’t able to accomplish much this week. We looked into potential software that will come in handy, and we’ve started to narrow down our options and the methods that we’re going to use. We have received the Pi and temperature and humidity sensor. Interfacing the two will be on the to-do list this week. Unfortunately I can’t knock any of the items off from last week since we don’t have all of the parts, most importantly the microphones.
To do list:
- Receive and inspect remaining parts
- Connect components to Pi, install drivers, etc
- Listen to an audio playback of MEMS microphone recordings, ear-test quality
My concern remains the same, I am hoping we get our parts ASAP so we can begin testing things out with them.
01/26/2024
Here is my sketch of what the state of the project will ideally look like at the end of this semester:
To do list:
- Receive and inspect parts
- Begin connecting components to Pi, install drivers etc.
- Make sure we have necessary cables
- Listen to an audio playback of MEMS microphone recordings, ear-test quality
Right now, my biggest concern is waiting for the parts to arrive. We had a bit of a delay on order, partially due to our budget expanding, but also because we needed to conduct a ton of research on each component we ordered, as the quality of them is integral to the success of the project.