We Won 1st Place Grand Prize at Cal Hacks
1st Place Grand Prize @ Cal Hacks 12.0, the world's largest collegiate hackathon ever.
I'm shocked. Cal Hacks 12.0 was the world's largest collegiate hackathon ever (3000+ hackers and 695 submitted projects)... and we won it all.
I remember the exact moment. I was at SFO on a FaceTime with my dad to say hi before I boarded my flight back to Santa Barbara. I told him we left early since we didn't get as many judges as the teams around us—which likely meant we didn't make it to the final judging rounds. My teammates were too tired, and even though I really wanted to stay, I didn't know anyone else so I just decided to leave with them.
So we left.
Then, while still on call with my dad, I get a text from our teammate's friend at the event.
I deadass stood up in disbelief: "U won best overall... 1st. Dumbass."
Holy shit! That's crazy.
Dylan Lu
ThePickleGawdFaceTimeOS: Mac-use AI Voice Agent
We built a computer-use agent that allows you to use your Mac from anywhere in the world, all through FaceTime audio + screenshare. If you need to do something on your computer, just call and it'll be done!
How It Works
Basically we just stitched together many cool parts through REST APIs. We have a virtual audio driver to capture FaceTime audio and pipe it into our speech understander/synthesis. Then, we take user queries and pass it to Agent S (the computer use agent), which will perform the task. The trajectory is analyzed by our Fetch.ai agent (which is total bs btw, ngl fetch is kinda useless) to generate the natural language speech response in parallel with the computer use actions. The UI is based on an open-source Cluely implementation, but we made it a lot cooler.
ThePickleGawd
What I Worked On
Since I'm doing research with the Professor and PhD student behind Simular's Agent S computer-use agent, I know a lot about how it works. I planned out the project by splitting it into three distinct tasks (computer-use, UI, backend) and described every REST interface needed between the services. This allowed us to work in our own folders in parallel.
Once we had a plan, we all successfully implemented our REST API interfaces and stitched all components together successfully. Everything worked better than expected!
Challenges We Ran Into
- No WiFi at hackathon. Crazy right? Luckily Calvin lives in SF so we took a Waymo and stayed the night there
- Computer use agent struggled with clicking. The solution is to create as many tools as possible to replace manual clicking, such as a tool to robustly FaceTime a user and share screen.
We almost gave up... It was 4 AM and my teammates went to sleep, saying that we tried our best and it's okay even if we don't finish. That's crazy bro, no way we spent 30 hours already just to call it quits. So I stayed up, fixed the rest of the bugs, recorded a demo video, and made our presentation pitch. Guess it was worth it lol, I finally finished the video in the Waymo back 30 mins before demos.
Fun Times at Cal Hacks
Honestly, I treated this event almost like a vacation right before midterms—something fun to get my mind off of problem sets and labs. We got to chat with a bunch sponsors and cool students, then spent the nighttime exploring SF and driving around in Waymos. And the best part, tons of free merch!
This was probably the worst I've ever eaten (health-wise).
- I started the event with some Popeyes fried chicken
- Then pizza for dinner at the cheapest place we could find
- Chocolate chip pancakes for breakfast at a vegan restaurant (bleh)
- Left over pizza and fried chicken for lunch
- Finally a good meal: sushi for dinner during my all-nighter last stand before the due date
Lowkey super fun. Food is so yummy!
Winning a Hackathon
Maybe it's survivorship bias, but I came up with a framework for how I approached the hackathon. I think this could be a useful reminder for when I eventually do TreeHacks in the winter.
1. Craft a Winning Idea
I think too many ideas are kinda the same. I recently judged for a hackathon at ASU and almost every project was just a chatbot clone. Super boring.
Key idea #1: Building something cooler than the last year's winner
I genuinely came into the hackathon thinking we'd win the whole thing (obviously I realized how naive we were once I saw how many people there actually were). But we mostly looked at the previous year's Devpost for TreeHacks and Cal Hacks and thought: "hey, I think our idea is cooler than all of these, so we totally have a chance".
One way to come up with ideas is to find recent survey papers on AI topics you're interested in. This can prompt ideas for new "AI agents" use-cases, which is really all anyone cares about. I think nowadays, any project that doesn't contain the keyword "AI agent" doesn't produce enough dopamine to excite anyone.
Survey papers are great since they summarize the latest advancements in AI and how people use them. Make sure you find one within the last 3 months to ensure it's still relevant. After skimming a bit, I realized that AI agents are really only used for three main tasks: 1) computer-use, 2) robotics, 3) chatbot-related.
#2 is not really feasible and #3 is boring. So let's do computer-use! This is the paper I skimmed a few weeks before the hackathon: https://arxiv.org/pdf/2501.16150
2. Balance Between a Novel Niche and General Applicability
Computer-use is not a novel idea. It's been around for many years (i.e. macros that can automate clicking actions or other interactions), though only recently has it gained widespread attention from the AI research community.
The important thing here is to find a balance between novel specialty and general applicability: on one dimension, it's easy to find unseen solutions as we move towards niche use-cases, yet we also want to show greater impact so the judges will actually care. I think FaceTimeOS has a good balance of being applicable to everyone (most people have iPhones and Macs) while having a unique twist (used for remote desktop controlled by voice only).
Key idea #2: Spend time thinking about what makes your idea unique and widely applicable
I'm not gonna lie, I'd previously written a whole section on how research abstracts provide good guidelines for unique pitches, but it was totally cringe so I just deleted it.
3. Make A Really Cool Demo with Great UI
This might be the most important part, and I've often overlooked this step in previous hackathons. Everyone wants to work on the "cool" stuff, like the AI agents or backend design, but not many people care about making things look pretty (centering divs and all that). But this is all the judges are going to see!
Key idea #3: Have a guy primarily focus on the UI and demo
I really think you need a guy to do this, and maybe everyone should have a good idea on what makes UI look good. Here's a book I read in high school recommended by the most cracked web dev I know (to this day!). At least for me, I think I have a general intuition of how to do padding, colors, and layouts, which makes my diagrams and UI sketches stand out against most beginners.
Best UI book I've read, written by TailwindCSS developers
In fact, I'd argue that you should focus everything on building a user story and demo, which is what the judges will see. That means you should always have a hardcoded fallback for when things stop working, and your hardcoded demo must look really impressive. In our case, the computer-use agent was not very robust in clicking buttons, so for the demo, I removed clicking functionality entirely, forcing the agent to operate using keyboard only. This made the product look higher quality in the video.
Hope some of this is useful/interesting for future me, (and maybe you?).
Conclusion
Overall, I had a great weekend. Looking forward to getting my Macbook Air (and giving it to my sister or little cousins), and I hope this can be a good conversation starter as I interview for internships.
Praying🙏: Nvidia Ignite or Amazon Annapurna Labs. Please!!!
Edit (12/5/2025): It's been 2 months and I haven't finished this post yet (whoops). I gave the Macbook to my little sister for her 18th birthday, and she returned it to Amazon 😭😭. I don't know what to do anymore...