2025 - Week #28
A Framework for Understanding AI Companies
AI companies can be categorized into three distinct groups:
- Model companies focus on developing and training large language models and foundation models.
- Infrastructure companies build the tools, platforms, and services that enable AI development and deployment.
- Application companies create end-user products and services powered by AI.
This framework is helpful for understanding how different players in the AI ecosystem create value.
Sources:
- The Pragmatic Engineer podcast episode with Janvi Kalra
- “AI Engineering” by Chip Huyen
The Four Pillars of Product Discovery
Product discovery should systematically address four critical risks:
- Value risk: customers will actually buy or use what you’re building?
- Usability risk: users can figure out how to use your product effectively?
- Feasibility risk: your team can actually build the solution?
- Business viability risk: the solution makes sense for your business model and constraints?
Source: chapter 33 of “Inspired” by Marty Cagan, which provides a clear framework for de-risking product development before significant investment.
Understanding Temperature in Language Models
Language models work by predicting the next token in a sequence of text. They calculate probability distributions across their entire vocabulary to determine what comes next.
For example, after “Hi! How are you?” the model might predict “Hi” with 70% probability, “Hello” with 15%, “Fine” with 5%, “Why” with 0.4%, and so on.
The challenge is choosing which token to present to users—a process called sampling. Greedy sampling simply picks the highest probability token (“Hi” in our example). Probabilistic sampling follows the calculated distribution, presenting “Hi” 70% of the time, “Hello” 15% of the time, and so forth.
The temperature parameter adds flexibility to probabilistic sampling by redistributing these probabilities. Higher temperatures increase the likelihood of rarer tokens, producing more creative and varied responses. Lower temperatures favor the most probable tokens, creating more consistent and predictable outputs.
Source: Chapter 2 of “AI Engineering” by Chip Huyen
What I’m Listening To
What I’m Reading
- How to evaluate model, infrastructure, and product companies in AI by Janvi Kalra
- I Shipped a macOS App Built Entirely by Claude Code by Indragie Karunaratne
- AI Engineering by Chip Huyen. I started reading this book this week, and it appears to be an excellent introduction to the world of large language models, providing practical guidance for using these tools more effectively
- Inspired by Marty Cagan
- Deploy Empathy by Michele Hansen
- Holly by Stephen King. I finished reading it this week. Personally, I preferred Fairy Tale and 11/22/63, the two other books I’ve read by this author. While it’s not a bad book—it’s a page-turner like the others—it explores different compelling themes, particularly evil and aging.
What I’m Working On
I’m continuing the development of my macOS app that lets users chat with multiple LLM providers. Following Indragie Karunaratne’s inspiring post, I’m building it using also Claude Code.
Using AI agents has improved my productivity in ways that are hard to appreciate without direct experience. The quality of Claude Code often impresses me, though success requires following best practices to avoid getting stuck in unproductive cycles (for example: Claude Code - Best practices).