Automating Software Engineering: Genie Tops SWE-Bench, w/ Alistair Pullen, from Latent.Space podcast

In this special crossover episode of The Cognitive Revolution, Nathan shares an insightful conversation from the Latent.Space podcast. Swyx and Alessio interview Alistair Pullen of Cosine, creators of Genie, showcasing the cutting edge of AI automation in software engineering. Learn how Cosine achieves state-of-the-art results on the SWE-bench benchmark by implementing advanced AI techniques. This episode complements Nathan's recent discussion on AI Automation, demonstrating how far these practices can be pushed in real-world applications. Don't miss this opportunity to explore the future of AI-driven software development and its implications for businesses across industries. Check out the Latent.Space podcast here: https://www.latent.space Apply to join over 400 Founders and Execs in the Turpentine Network: https://www.turpentinenetwork.co/ SPONSORS: WorkOS: Building an enterprise-ready SaaS app? WorkOS has got you covered with easy-to-integrate APIs for SAML, SCIM, and more. Join top startups like Vercel, Perplexity, Jasper & Webflow in powering your app with WorkOS. Enjoy a free tier for up to 1M users! Start now at https://bit.ly/WorkOS-Turpentine-Network Weights & Biases Weave: Weights & Biases Weave is a lightweight AI developer toolkit designed to simplify your LLM app development. With Weave, you can trace and debug input, metadata and output with just 2 lines of code. Make real progress on your LLM development and visit the following link to get started with Weave today: https://wandb.me/cr 80,000 Hours: 80,000 Hours offers free one-on-one career advising for Cognitive Revolution listeners aiming to tackle global challenges, especially in AI. They connect high-potential individuals with experts, opportunities, and personalized career plans to maximize positive impact. Apply for a free call at https://80000hours.org/cognitiverevolution to accelerate your career and contribute to solving pressing AI-related issues. Omneky: Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off https://www.omneky.com/ CHAPTERS: (00:00:00) About the Show (00:00:22) Sponsors: WorkOS (00:01:22) About the Episode (00:04:29) Alistair and Cosine intro (00:13:50) Building the Code Retrieval Tool (00:17:36) Sponsors: Weights & Biases Weave | 80,000 Hours (00:20:15) Developing Genie and Fine-tuning Process (00:27:41) Working with Customer Data (00:30:53) Code Retrieval Challenges and Solutions (00:36:39) Sponsors: Omneky (00:37:02) Planning and Reasoning in AI Models (00:45:55) Language Support and Generalization (00:49:46) Fine-tuning Experience with OpenAI (00:52:56) Synthetic Data and Self-improvement Loop (00:55:57) Benchmarking and SWE-bench Results (01:01:47) Future Plans for Genie (01:03:02) Industry Trends and Cursor's Success (01:05:23) Calls to Action and Ideal Customers (01:08:43) Outro