Since ChatGPT came onto the scene on 30th November, 2022, I've been doing a deep dive finding out what exactly LLMs and specifically ChatGPT are capable of, what their limitations are and how we can use them at Orbital Witness (the company I'm CTO at). This was obviously important since we're currently building software that 'thinks' like a lawyer using the latest natural language processing (NLP).
Given my current ongoing research on the topic, I'm going to use this blog post as a curated collection of the best content I've read across a variety of themes related to LLMs / ChatGPT / Dall-E 2 / etc. I'll update it regularly with links to new blog posts, podcasts, videos and anything else I find particularly compelling. The date in parentheses is when I added the link to this blog post and over time it will provide a change-log for easier reading.
Overview
- The Large Language Model Landscape by Cobus Greyling (Nov '22 - Pre-ChatGPT)
- The Near Future of AI is Action-Driven by John McDonnell (Nov '22 - Pre-ChatGPT)
- Why generative AI is getting VCs all excited by Tim Smith (Jan '23)
- "What Is ChatGPT Doing … and Why Does It Work?" by Stephen Wolfram (Feb '23)
- Stephen also has a 3hr+ Youtube video by the same name
- An Analysis of ChatGPT and OpenAI GPT-3: How to Use it For Your Business by Version 1 (Feb '23)
- Who Owns the Generative AI Platform? by Matt Bornstein, Guido Appenzeller, and Martin Casado of a16z (Feb '23)
- Impromptu: Amplifying Our Humanity Through AI by Reid Hoffman with GPT-4 (Mar '23)
- ChatGPT: Everything you need to know about the AI-powered chatbot by TechCrunch (Mar '23)
Prompt Engineering
- EMNLP: Prompt engineering is the new feature engineering by Larry Hardesty (Mar '23)
- The DALL·E 2 Prompt Book by Guy Parsons (Mar '23)
- a16z's podcast with Guy Parsons titled Unlocking Creativity with Prompt Engineering
- Learn Prompting Course (Feb '23)
- ChatGPT's API is So Good and Cheap, It Makes Most Text Generating AI Obsolete by Max Woolf (Mar '23)
- Using ChatGPT with Additive Prompting for Midjourney Image Creation by Nick St. Pierre (Mar '23)
- Examples of Bing Chat Prompt Engineering by Dan Berridge (Mar '23)
Context Windows
- LlamaIndex 🦙 (GPT Index) (Mar '23)
- Collection of content on the upcoming GPT4 by Life Architect (Mar '23)
- OpenAI released GPT4-API: everything you need to know! by Wassim Jouini (Mar '23)
- "Larger context means more complex applications: A 32k tokens context means about 25 000 words or 50 pages! This is huge! It means it can process full reports, short books or long chapters, taxation code, and so on, enabling scenarios that were impossible with ChatGPT’s 4k token limitation!"
Accuracy
- GPT Takes the Bar Exam by Michael Bommarito (Mar '23)
- TruthfulQA: Measuring How Models Mimic Human Falsehoods by Stephanie Lin, Jacob Hilton and Owain Evans (Mar '23)
- How we cut the rate of GPT hallucinations from 20%+ to less than 2% by Jason Fan (Mar '23)
- The results of OpenAI's ChatGPT ran against every (non-premium) Leetcode problem (Mar '23)
- GPT-4 Research Blog Post by OpenAI (Mar '23)
Productivity
- "Secret Cyborgs: The Present Disruption in Three Papers" by Ethan Mollick (Mar '23)
- Ethan's Twitter (he's a Professor at Wharton studying innovation & startups) has been one of the best resources to see what kinds of things are possible with ChatGPT
User Experience (UX)
- Say Hi To ChatSpot.ai: The All-In-One A.I. Powered Chat App For Growing Better by Dharmesh Shah, CTO at HubSpot (Mar '23)
- Here are 7 large companies that now incorporate ChatGPT’s model in their product (Hubspot, Brex, Slack, Snapchat, Shopify, Instacart, Quizlet) by Dan Berridge (Mar '23)
- ChatGPT Plugins by Greg Brockman (Mar '23)
Traction of LLMs
- ChatGPT on track to surpass 100 million users faster than TikTok or Instagram: UBS by Yahoo Finance (Feb '23)
Long-term Costs
- OpenAI's leaked Foundry pricing says a lot by Nathan Labenz (Feb '23)
- You can now run a GPT-3-level AI model on your laptop, phone, and Raspberry Pi by ArsTechnica (Mar '23)
- How much does GPT-4 cost? (gpt-4 & gpt-4-32k) by OpenAI (Mar '23)
How-to
- I built a Lenny chatbot using GPT-3. Here’s how to build your own by Dan Shipper
- Visual ChatGPT by Microsoft (Mar '23 prior to GPT-4's release)
- GPT-4 Developer Livestream (Recorded) by OpenAI (Mar '23)