<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:itunes="http://www.itunes.com/dtds/podcast-1.0.dtd" xmlns:googleplay="http://www.google.com/schemas/play-podcasts/1.0"><channel><title><![CDATA[Simcha's Newsletter]]></title><description><![CDATA[Simcha's Newsletter]]></description><link>https://newsletter.simcha.ai</link><image><url>https://substackcdn.com/image/fetch/$s_!1c-h!,w_256,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc8626802-8859-44ea-bcd3-fffaa6dac209_2048x2048.png</url><title>Simcha&apos;s Newsletter</title><link>https://newsletter.simcha.ai</link></image><generator>Substack</generator><lastBuildDate>Tue, 21 Apr 2026 11:47:25 GMT</lastBuildDate><atom:link href="https://newsletter.simcha.ai/feed" rel="self" type="application/rss+xml"/><copyright><![CDATA[Simcha AI]]></copyright><language><![CDATA[en]]></language><webMaster><![CDATA[simchaai@substack.com]]></webMaster><itunes:owner><itunes:email><![CDATA[simchaai@substack.com]]></itunes:email><itunes:name><![CDATA[Simcha AI]]></itunes:name></itunes:owner><itunes:author><![CDATA[Simcha AI]]></itunes:author><googleplay:owner><![CDATA[simchaai@substack.com]]></googleplay:owner><googleplay:email><![CDATA[simchaai@substack.com]]></googleplay:email><googleplay:author><![CDATA[Simcha AI]]></googleplay:author><itunes:block><![CDATA[Yes]]></itunes:block><item><title><![CDATA[Electricity In, Emotional Energy Out]]></title><description><![CDATA[With great power comes&#8230; a power bill (unless you can do it locally)]]></description><link>https://newsletter.simcha.ai/p/electricity-in-emotional-energy-out</link><guid isPermaLink="false">https://newsletter.simcha.ai/p/electricity-in-emotional-energy-out</guid><dc:creator><![CDATA[Simcha AI]]></dc:creator><pubDate>Mon, 22 Dec 2025 17:02:11 GMT</pubDate><enclosure url="https://substack-post-media.s3.amazonaws.com/public/images/57e3c608-8b06-45d2-abf1-dd09e372025f_1536x1024.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>The headlines are loud right now. &#8220;Data center demand will push power use to record highs.&#8221; (<a href="https://www.reuters.com/business/energy/data-center-demand-push-us-power-use-record-highs-2025-26-eia-says-2025-06-10/?utm_source=chatgpt.com">Reuters</a>) &#8220;Hundreds of groups want a pause on new data centers.&#8221; (<a href="https://www.theguardian.com/us-news/2025/dec/08/us-data-centers?utm_source=chatgpt.com">The Guardian</a>) &#8220;Artificial intelligence is draining water.&#8221; (<a href="https://www.wired.com/story/karen-hao-empire-of-ai-water-use-statistics?utm_source=chatgpt.com">WIRED</a>)</p><p>All fair questions. The part that usually goes missing is scale.</p><h3>The zoomed-out answer</h3><p>In 2024, data centers used about <strong>415 terawatt-hours</strong>, around <strong>1.5% of the world&#8217;s electricity</strong>. (<a href="https://www.iea.org/reports/energy-and-ai/executive-summary?utm_source=chatgpt.com">IEA</a>)<br>In the International Energy Agency Base Case, they reach about <strong>945 terawatt-hours by 2030</strong>, just under <strong>3%</strong>. (<a href="https://www.iea.org/reports/energy-and-ai/energy-demand-from-ai?utm_source=chatgpt.com">IEA</a>)</p><p>That is meaningful growth. It is not &#8220;everything.&#8221;</p><h3>Electricity, in perspective</h3><div class="captioned-image-container"><figure><a class="image-link image2" target="_blank" href="https://substackcdn.com/image/fetch/$s_!_EjZ!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F7d1b6fe2-4cf1-4ca4-82f7-0fd2a2d71daf_1580x356.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!_EjZ!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F7d1b6fe2-4cf1-4ca4-82f7-0fd2a2d71daf_1580x356.png 424w, https://substackcdn.com/image/fetch/$s_!_EjZ!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F7d1b6fe2-4cf1-4ca4-82f7-0fd2a2d71daf_1580x356.png 848w, https://substackcdn.com/image/fetch/$s_!_EjZ!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F7d1b6fe2-4cf1-4ca4-82f7-0fd2a2d71daf_1580x356.png 1272w, https://substackcdn.com/image/fetch/$s_!_EjZ!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F7d1b6fe2-4cf1-4ca4-82f7-0fd2a2d71daf_1580x356.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!_EjZ!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F7d1b6fe2-4cf1-4ca4-82f7-0fd2a2d71daf_1580x356.png" width="1456" height="328" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/7d1b6fe2-4cf1-4ca4-82f7-0fd2a2d71daf_1580x356.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:328,&quot;width&quot;:1456,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:63285,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:false,&quot;topImage&quot;:true,&quot;internalRedirect&quot;:&quot;https://newsletter.simcha.ai/i/181656680?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F7d1b6fe2-4cf1-4ca4-82f7-0fd2a2d71daf_1580x356.png&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!_EjZ!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F7d1b6fe2-4cf1-4ca4-82f7-0fd2a2d71daf_1580x356.png 424w, https://substackcdn.com/image/fetch/$s_!_EjZ!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F7d1b6fe2-4cf1-4ca4-82f7-0fd2a2d71daf_1580x356.png 848w, https://substackcdn.com/image/fetch/$s_!_EjZ!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F7d1b6fe2-4cf1-4ca4-82f7-0fd2a2d71daf_1580x356.png 1272w, https://substackcdn.com/image/fetch/$s_!_EjZ!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F7d1b6fe2-4cf1-4ca4-82f7-0fd2a2d71daf_1580x356.png 1456w" sizes="100vw" fetchpriority="high"></picture><div></div></div></a></figure></div><p>Sources for the table: data center electricity from the International Energy Agency. (<a href="https://www.iea.org/reports/energy-and-ai/executive-summary?utm_source=chatgpt.com">IEA</a>) Cooling share from the International Energy Agency. (<a href="https://www.iea.org/news/air-conditioning-use-emerges-as-one-of-the-key-drivers-of-global-electricity-demand-growth?utm_source=chatgpt.com">IEA</a>)</p><h3>Water, in perspective</h3><p>Globally, the International Energy Agency estimate that data centers consume about <strong>560 billion liters of water per year today</strong>, rising to about <strong>1,200 billion liters by 2030</strong> in the Base Case. (<a href="https://assets.publishing.service.gov.uk/media/688cb407dc6688ed50878367/Water_use_in_data_centre_and_AI_report.pdf?utm_source=chatgpt.com">GOV.UK</a>)<br>For context, global freshwater withdrawals were <strong>just under 4,000 cubic kilometers in 2021</strong>, and <strong>agriculture is 72%</strong> of that. (<a href="https://unesdoc.unesco.org/ark%3A/48223/pf0000393090?utm_source=chatgpt.com">UNESCO Documents</a>)</p><div class="captioned-image-container"><figure><a class="image-link image2" target="_blank" href="https://substackcdn.com/image/fetch/$s_!IxRR!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9a60a55d-73d6-4d14-bc7e-04875da04b82_1612x274.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!IxRR!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9a60a55d-73d6-4d14-bc7e-04875da04b82_1612x274.png 424w, https://substackcdn.com/image/fetch/$s_!IxRR!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9a60a55d-73d6-4d14-bc7e-04875da04b82_1612x274.png 848w, https://substackcdn.com/image/fetch/$s_!IxRR!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9a60a55d-73d6-4d14-bc7e-04875da04b82_1612x274.png 1272w, https://substackcdn.com/image/fetch/$s_!IxRR!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9a60a55d-73d6-4d14-bc7e-04875da04b82_1612x274.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!IxRR!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9a60a55d-73d6-4d14-bc7e-04875da04b82_1612x274.png" width="1456" height="247" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/9a60a55d-73d6-4d14-bc7e-04875da04b82_1612x274.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:247,&quot;width&quot;:1456,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:58246,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:&quot;https://newsletter.simcha.ai/i/181656680?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9a60a55d-73d6-4d14-bc7e-04875da04b82_1612x274.png&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!IxRR!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9a60a55d-73d6-4d14-bc7e-04875da04b82_1612x274.png 424w, https://substackcdn.com/image/fetch/$s_!IxRR!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9a60a55d-73d6-4d14-bc7e-04875da04b82_1612x274.png 848w, https://substackcdn.com/image/fetch/$s_!IxRR!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9a60a55d-73d6-4d14-bc7e-04875da04b82_1612x274.png 1272w, https://substackcdn.com/image/fetch/$s_!IxRR!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9a60a55d-73d6-4d14-bc7e-04875da04b82_1612x274.png 1456w" sizes="100vw" loading="lazy"></picture><div></div></div></a></figure></div><p></p><p>Sources for the table: data center water from the International Energy Agency estimate (summarized in a UK government report). (<a href="https://assets.publishing.service.gov.uk/media/688cb407dc6688ed50878367/Water_use_in_data_centre_and_AI_report.pdf?utm_source=chatgpt.com">GOV.UK</a>) Global withdrawals and agriculture share from the United Nations World Water Development Report 2025. (<a href="https://unesdoc.unesco.org/ark%3A/48223/pf0000393090?utm_source=chatgpt.com">UNESCO Documents</a>)</p><p>Also, &#8220;data center water&#8221; is not one single thing. Cooling designs vary, and the International Energy Agency notes that options like direct liquid cooling and immersion cooling can reduce direct water consumption. (<a href="https://iea.blob.core.windows.net/assets/601eaec9-ba91-4623-819b-4ded331ec9e8/EnergyandAI.pdf?utm_source=chatgpt.com">IEA Blob Storage</a>)</p><div><hr></div><h3>Power, Water, Heat, Panic</h3><p>When people talk about AI and the environment, the conversation usually sounds like this: <em>power, water, heat, panic.</em> Fair. Those inputs matter.</p><p>But the question we keep coming back to is simpler:</p><p>If we&#8217;re going to spend energy, <strong>what do we get for it?</strong></p><p>With great power comes&#8230; a <strong>power bill</strong>. Unless you can do more of the work locally.</p><p>That sentence is basically the quiet spine of this whole series.</p><p>In the beginning, we met the &#8220;muscle.&#8221; <strong>Cat, GPU, GPT</strong> was the moment you realize the modern era runs on a specific kind of hardware scale. GPUs made the leap possible. Scale is why electricity and water entered the chat.</p><p>Then we met the &#8220;habit.&#8221; <strong>The guessing game learned to listen</strong> wasn&#8217;t really about fancy magic. It was about the model getting less scattered. It stopped treating everything as equally important and started holding onto the thread.</p><p>And then we finally got the &#8220;skill.&#8221; <strong>Attention</strong> was the 2017 shift that made language work at scale: not more words, but <em>better focus</em> learning what to highlight in a messy paragraph. (Vaswani et al., 2017) ([9])</p><p>That&#8217;s the bridge from <strong>electricity</strong> to <strong>usefulness</strong>.</p><p>Because the question is not &#8220;Can the model generate more text?&#8221;<br>The question is: <strong>Does the compute turn into something that gives a human being energy back?</strong></p><p>Here&#8217;s an example of what we mean: a small cognitive support that helps a person see a pattern clearly enough to interrupt it and make a positive change.</p><h3>A Common Anxiety Moment, From Overwhelm to Clarity</h3><blockquote><p>&#8220;Sunday night hits and my chest tightens. I start thinking about work tomorrow. I replay everything I might mess up. I keep checking my calendar and messages. Then I feel exhausted and mad at myself.&#8221;</p></blockquote><p>For most people, the problem here isn&#8217;t a lack of insight. It&#8217;s that everything shows up at once.</p><p>In moments like this, more words don&#8217;t help. What helps is <strong>compression</strong>: turning a fog of experience into a recognizable pattern. When the swirl gets smaller, the nervous system can finally orient.</p><p>Seen clearly, this moment has a simple shape: a predictable trigger, an immediate body response, a loop of worry and reassurance seeking, and a narrow window where interruption is possible. Nothing is &#8220;fixed.&#8221; But the fear becomes <strong>specific</strong> instead of <strong>everywhere</strong>.</p><p>That&#8217;s the &#8220;emotional energy&#8221; return. Not because a tool fixed them but because it reduced the experience to a size their nervous system could hold.</p><p>From &#8220;I&#8217;m trapped in it&#8221; &#8594; to &#8220;I see the loop, and I know where I can step in.&#8221;</p><p>And yes: doing that kind of sorting takes compute. (With great power comes&#8230; a power bill.)</p><p>This is only one example. Not a defense of all AI, and not a claim that technology is the answer to human suffering. But it is a concrete case of something that matters: energy spent turning into human capacity returned. When AI does that when compute becomes clarity, orientation, or relief it earns its place. The question isn&#8217;t whether AI uses resources. It&#8217;s whether those resources come back as something that genuinely helps people live better.</p><p>When this patterning can happen locally, it avoids unnecessary trips to data centers for every turn. That&#8217;s not about purity. It&#8217;s about treating compute as a shared resource and using it carefully when less is enough.</p><p>We are designing Simcha to be cost-efficient and resource-aware. Not because energy use is bad, but because context matters. In cases like this, local models simply make sense.</p><h3>Next article</h3><p>Next: how to carry that &#8220;small loop, small step&#8221; energy between sessions so the plan doesn&#8217;t evaporate by Tuesday, and progress shows up in the week, not just in the room.</p><div><hr></div><h2>References</h2><ul><li><p>International Energy Agency. (2025). <em>Executive summary &#8211; Energy and AI</em>. <a href="https://www.iea.org/reports/energy-and-ai/executive-summary">https://www.iea.org/reports/energy-and-ai/executive-summary</a> (<a href="https://www.iea.org/reports/energy-and-ai/executive-summary?utm_source=chatgpt.com">IEA</a>)</p></li><li><p>International Energy Agency. (2025). <em>Energy demand from AI</em>. <a href="https://www.iea.org/reports/energy-and-ai/energy-demand-from-ai">https://www.iea.org/reports/energy-and-ai/energy-demand-from-ai</a> (<a href="https://www.iea.org/reports/energy-and-ai/energy-demand-from-ai?utm_source=chatgpt.com">IEA</a>)</p></li><li><p>International Energy Agency. (2018). <em>Air conditioning use emerges as one of the key drivers of global electricity demand growth</em>. <a href="https://www.iea.org/news/air-conditioning-use-emerges-as-one-of-the-key-drivers-of-global-electricity-demand-growth">https://www.iea.org/news/air-conditioning-use-emerges-as-one-of-the-key-drivers-of-global-electricity-demand-growth</a> (<a href="https://www.iea.org/news/air-conditioning-use-emerges-as-one-of-the-key-drivers-of-global-electricity-demand-growth?utm_source=chatgpt.com">IEA</a>)</p></li><li><p>UNESCO World Water Assessment Programme. (2025). <em>The United Nations World Water Development Report 2025: Mountains and glaciers: Water towers</em>. <a href="https://unesdoc.unesco.org/ark%3A/48223/pf0000393090">https://unesdoc.unesco.org/ark%3A/48223/pf0000393090</a> (<a href="https://unesdoc.unesco.org/ark%3A/48223/pf0000393090?utm_source=chatgpt.com">UNESCO Documents</a>)</p></li><li><p>UK Government. (2025). <em>Water use in data centre and AI report (executive summary)</em>. <a href="https://assets.publishing.service.gov.uk/media/688cb407dc6688ed50878367/Water_use_in_data_centre_and_AI_report.pdf">https://assets.publishing.service.gov.uk/media/688cb407dc6688ed50878367/Water_use_in_data_centre_and_AI_report.pdf</a> (<a href="https://assets.publishing.service.gov.uk/media/688cb407dc6688ed50878367/Water_use_in_data_centre_and_AI_report.pdf?utm_source=chatgpt.com">GOV.UK</a>)</p></li><li><p>Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, &#321;., &amp; Polosukhin, I. (2017). <em>Attention Is All You Need</em>. <em>Advances in Neural Information Processing Systems</em>, 30. <a href="https://arxiv.org/abs/1706.03762">https://arxiv.org/abs/1706.03762</a> (<a href="https://iea.blob.core.windows.net/assets/601eaec9-ba91-4623-819b-4ded331ec9e8/EnergyandAI.pdf?utm_source=chatgpt.com">IEA Blob Storage</a>)</p></li><li><p>Reuters. (2025). <em>Data center demand to push US power use to record highs in 2025-26, EIA says</em>. (<a href="https://www.reuters.com/business/energy/data-center-demand-push-us-power-use-record-highs-2025-26-eia-says-2025-06-10/?utm_source=chatgpt.com">Reuters</a>)</p></li><li><p>WIRED. (2025). <em>You&#8217;re Thinking About AI and Water All Wrong</em>. (<a href="https://www.wired.com/story/karen-hao-empire-of-ai-water-use-statistics?utm_source=chatgpt.com">WIRED</a>)</p></li><li><p>The Guardian. (2025). <em>More than 200 environmental groups demand halt to new US datacenters</em>. (<a href="https://www.theguardian.com/us-news/2025/dec/08/us-data-centers?utm_source=chatgpt.com">The Guardian</a>)</p></li></ul>]]></content:encoded></item><item><title><![CDATA[How AI Learned to Pay Attention]]></title><description><![CDATA[The little &#8220;highlighter&#8221; trick behind GPT]]></description><link>https://newsletter.simcha.ai/p/how-ai-learned-to-pay-attention</link><guid isPermaLink="false">https://newsletter.simcha.ai/p/how-ai-learned-to-pay-attention</guid><dc:creator><![CDATA[Simcha AI]]></dc:creator><pubDate>Mon, 15 Dec 2025 15:02:58 GMT</pubDate><enclosure url="https://substack-post-media.s3.amazonaws.com/public/images/3b221961-962a-4093-89fd-baba18e5a76c_1536x1024.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>So far in this series:</p><ul><li><p>We saw how early models played a <strong>guess-the-next-word</strong> game.</p></li><li><p>We met <strong>LSTMs</strong>, which made that game less forgetful.</p></li><li><p>We detoured through <strong>cats + GPUs</strong>, where computers learned to see patterns in images.</p></li></ul><p>This article is about one last ingredient:</p><blockquote><p>How modern AI learned to <strong>focus</strong> on the important parts of what someone says or writes.</p></blockquote><p>That ingredient is called <strong>attention</strong>.</p><div><hr></div><h2>Quick reminder: where LSTMs left off</h2><p>In the earlier article, LSTMs were the &#8220;better listener&#8221;:</p><ul><li><p>Old models read a sentence and forgot almost everything by the end.</p></li><li><p>LSTMs were better at <strong>holding onto important bits</strong> and <strong>dropping noise</strong> as they moved word by word.</p></li></ul><p>But they still had one big limitation:</p><blockquote><p>They read like someone walking down a hallway with blinders on&#8212;<br>always moving forward, only really seeing what&#8217;s right in front of them.</p></blockquote><p>If something crucial was said at the start, it had to survive being passed along through every step of the sentence or paragraph.</p><p>Helpful, but not how humans usually read or listen.</p><div><hr></div><h2>How people actually read notes</h2><p>Take a line like this:</p><blockquote><p>&#8220;Feeling more on edge in group settings, skipping team lunches, and noticing this has gotten worse over the past month since the promotion; denies full-blown panic attacks.&#8221;</p></blockquote><p>A person doesn&#8217;t treat that as a long string of equal words.</p><p>The mind quietly does things like:</p><ul><li><p>Connect <strong>&#8220;on edge&#8221;</strong> with <strong>&#8220;group settings&#8221;</strong> and <strong>&#8220;skipping team lunches.&#8221;</strong></p></li><li><p>Link <strong>&#8220;worse over the past month&#8221;</strong> with <strong>&#8220;since the promotion.&#8221;</strong></p></li><li><p>Notice <strong>&#8220;denies full-blown panic attacks&#8221;</strong> as a limit on how severe this is.</p></li></ul><p>You skim, jump back and forth, and <strong>mentally highlight</strong> what matters:</p><ul><li><p>What changed?</p></li><li><p>When did it change?</p></li><li><p>How bad is it?</p></li><li><p>What&#8217;s context and what&#8217;s noise?</p></li></ul><p>That mental highlighting is the everyday version of <strong>attention</strong>.</p><div><hr></div><h2>The core idea: give AI a highlighter</h2><p>Attention is basically this question, asked over and over inside the model:</p><blockquote><p>&#8220;For what I&#8217;m doing <strong>right now</strong>,<br>which parts of this text should I pay the most attention to?&#8221;</p></blockquote><p>Instead of carrying one fragile memory from left to right, the model can:</p><ol><li><p>Look at the <strong>whole</strong> sentence or paragraph.</p></li><li><p>Decide which words are most related to each other.</p></li><li><p>Give the important parts a <strong>stronger &#8220;glow&#8221;</strong> inside its math.</p></li><li><p>Use that focused view to decide what to say next.</p></li></ol><p>Back to the example:</p><ul><li><p>Thinking about &#8220;on edge&#8221; &#8594; it leans on &#8220;group settings&#8221; and &#8220;skipping team lunches.&#8221;</p></li><li><p>Thinking about &#8220;worse&#8221; &#8594; it leans on &#8220;past month&#8221; and &#8220;since the promotion.&#8221;</p></li><li><p>Seeing &#8220;denies&#8221; &#8594; it leans on &#8220;full-blown panic attacks.&#8221;</p></li></ul><p>Everything is still visible.<br>But the model stops treating everything as equally important.</p><p>That&#8217;s all &#8220;attention&#8221; really is here:</p><blockquote><p>Noticing what matters, and quietly turning up the volume on those parts.</p></blockquote><div><hr></div><h2>How this is different from LSTMs</h2><p>If you like visual metaphors:</p><ul><li><p><strong>LSTM:</strong> one long page of notes, and you keep writing at the bottom. You <em>try</em> to remember what mattered at the top.</p></li><li><p><strong>Attention:</strong> the same page of notes, but now you can instantly scan the whole page and <strong>highlight</strong> whatever is relevant at this moment.</p></li></ul><p>LSTMs helped with <strong>remembering</strong>.<br>Attention helps with <strong>finding and focusing</strong>.</p><p>Modern models use both ideas, but attention is what lets them handle much longer, messier text without getting lost.</p><div><hr></div><h2>The Famous Paper: &#8220;Attention Is All You Need&#8221;</h2><p>In 2017, a group of researchers turned this highlighter idea into a full model design, called a <strong>Transformer</strong>.<br>The paper&#8217;s title was:</p><blockquote><p><strong>&#8220;Attention Is All You Need&#8221;</strong></p></blockquote><p>You can think of a Transformer as a stack of &#8220;focus blocks&#8221;:</p><ol><li><p>Each block looks over the text and decides what to highlight.</p></li><li><p>Then it thinks a bit about what it just focused on.</p></li><li><p>Then it passes that forward to the next block, which does the same thing at a deeper level.</p></li></ol><p>This design turned out to be:</p><ul><li><p>Very good at language tasks (like translation, summarizing, etc.)</p></li><li><p>Very friendly to GPUs, so it could scale up to big models</p></li></ul><p>That&#8217;s why almost every modern language model is built on some version of this idea.</p><div><hr></div><h2>From attention to GPT: the giant guessing game</h2><p>Once this &#8220;highlighter&#8221; model existed, another group of researchers took it and said:</p><blockquote><p>&#8220;Let&#8217;s give it a <strong>lot</strong> of text<br>and let it play the next-word guessing game for a very long time.&#8221;</p></blockquote><p>They called this <strong>generative pre-training</strong>:</p><ol><li><p>Show the model real text.</p></li><li><p>Hide the next word.</p></li><li><p>Ask it to guess.</p></li><li><p>Tell it if it was wrong.</p></li><li><p>Repeat this billions of times.</p></li></ol><p>The result is a <strong>Generative Pre-trained Transformer</strong>&#8212;or <strong>GPT</strong>:</p><ul><li><p><em>Transformer</em> &#8594; the attention-based &#8220;focus block&#8221; design</p></li><li><p><em>Pre-trained</em> &#8594; it learned from a mountain of text before you ever typed anything</p></li><li><p><em>Generative</em> &#8594; it can now write new text, not just label things</p></li></ul><p>When you ask it to summarize, rewrite, or brainstorm, it&#8217;s basically doing:</p><blockquote><p>&#8220;Given everything I&#8217;ve read in this conversation and everything I practiced on,<br>what&#8217;s a reasonable next sentence or phrase?&#8221;</p></blockquote><p>Attention is the part that lets it look back over what you gave it and actually <strong>use</strong> the important bits, not just the last one or two lines.</p><div><hr></div><h2>Why this matters in day-to-day use</h2><p>Because of attention, models can now:</p><ul><li><p><strong>Handle longer text</strong><br>They can keep track of things said at the beginning, not just whatever is most recent.</p></li><li><p><strong>Pick out the key themes</strong><br>Repeated patterns (like &#8220;feeling like a burden,&#8221; or &#8220;since the promotion&#8221;) can get highlighted across multiple paragraphs.</p></li><li><p><strong>Ignore a lot of boilerplate</strong><br>Template sections and repeated phrases can be downplayed so the unique parts stand out.</p></li><li><p><strong>Summarize without losing the point</strong><br>Because the model is literally focusing on the important pieces, the &#8220;short version&#8221; has a better chance of capturing what matters.</p></li></ul><p>LSTMs took us from &#8220;goldfish memory&#8221; to &#8220;slightly better memory.&#8221;<br>Attention takes us from &#8220;linear note-taker&#8221; to &#8220;someone who can flip through the whole notebook and quickly spot what&#8217;s important.&#8221;</p><div><hr></div><blockquote><p>LSTMs taught AI to <strong>remember</strong> more of what it heard.<br>Attention taught it to <strong>notice and highlight</strong> the most important parts&#8212;<br>and that little focusing trick is what made today&#8217;s GPT-style models feel much more present and useful. &#9889;</p></blockquote><div><hr></div><h3>References</h3><ul><li><p>Hochreiter, S., &amp; Schmidhuber, J. (1997). Long short-term memory. <em>Neural Computation, 9</em>(8), 1735&#8211;1780.<br>PDF: <a href="https://www.bioinf.jku.at/publications/older/2604.pdf">https://www.bioinf.jku.at/publications/older/2604.pdf</a> (<a href="https://www.bioinf.jku.at/publications/older/2604.pdf?utm_source=chatgpt.com">bioinf.jku.at</a>)</p></li><li><p>Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, &#321;., &amp; Polosukhin, I. (2017). Attention is all you need. In <em>Advances in Neural Information Processing Systems, 30 (NeurIPS 2017)</em>.<br>arXiv: <a href="https://arxiv.org/abs/1706.03762">https://arxiv.org/abs/1706.03762</a><br>NeurIPS version: <a href="https://papers.nips.cc/paper/7181-attention-is-all-you-need">https://papers.nips.cc/paper/7181-attention-is-all-you-need</a> (<a href="https://arxiv.org/abs/1706.03762?utm_source=chatgpt.com">arXiv</a>)</p></li><li><p>Radford, A., Narasimhan, K., Salimans, T., &amp; Sutskever, I. (2018). Improving language understanding by generative pre-training. <em>OpenAI Technical Report.</em><br>PDF: <a href="https://cdn.openai.com/research-covers/language-unsupervised/language_understanding_paper.pdf">https://cdn.openai.com/research-covers/language-unsupervised/language_understanding_paper.pdf</a> (<a href="https://cdn.openai.com/research-covers/language-unsupervised/language_understanding_paper.pdf?utm_source=chatgpt.com">OpenAI</a>)</p></li><li><p>Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., et al. (2020). Language models are few-shot learners. In <em>Advances in Neural Information Processing Systems, 33 (NeurIPS 2020)</em>.<br>arXiv: <a href="https://arxiv.org/abs/2005.14165">https://arxiv.org/abs/2005.14165</a><br>NeurIPS PDF: <a href="https://proceedings.neurips.cc/paper/2020/file/1457c0d6bfcb4967418bfb8ac142f64a-Paper.pdf">https://proceedings.neurips.cc/paper/2020/file/1457c0d6bfcb4967418bfb8ac142f64a-Paper.pdf</a> (<a href="https://arxiv.org/abs/2005.14165?utm_source=chatgpt.com">arXiv</a>)</p></li></ul>]]></content:encoded></item><item><title><![CDATA[CAT + GPU = GPT]]></title><description><![CDATA[How Cat Photos Sent a Graphics Card to the Gym]]></description><link>https://newsletter.simcha.ai/p/cat-gpu-gpt</link><guid isPermaLink="false">https://newsletter.simcha.ai/p/cat-gpu-gpt</guid><dc:creator><![CDATA[Simcha AI]]></dc:creator><pubDate>Sun, 07 Dec 2025 21:39:32 GMT</pubDate><enclosure url="https://substack-post-media.s3.amazonaws.com/public/images/8d62228e-069a-4433-98a3-7c53d46538f5_1024x1024.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p><em>Last time, we talked about AI as a giant guessing game for words.</em><br>Before that game ever existed, there was an older cousin: a system that spent years looking at pictures&#8212;especially cats&#8212;and quietly changed what computers could do.</p><p>This article is about that cousin.</p><p>It&#8217;s a story about:</p><ul><li><p>A special computer part (a <strong>graphics card</strong>)</p></li><li><p>A huge pile of internet photos</p></li><li><p>A model called <strong>AlexNet</strong></p></li><li><p>And how &#8220;cat recognition&#8221; accidentally paved the way for the tools we use now</p></li></ul><p>We&#8217;ll end with a pointer back to words and notes, but we&#8217;re not going there yet.<br>For now, it&#8217;s just us, GPUs, and cats.</p><div><hr></div><h2>The Late-Night Problem: Too Many Images, Not Enough Muscle</h2><p>Imagine you at 9:47 p.m.:</p><ul><li><p>Last note of the day</p></li><li><p>Brain gently fried</p></li><li><p>EHR cursor blinking like it has all the time in the world</p></li></ul><p>Now translate that feeling to early AI researchers.</p><p>Back in the early 2010s, they had:</p><ul><li><p><strong>Millions of labeled photos</strong> (of animals, objects, outdoor scenes&#8212;everything)</p></li><li><p><strong>Promising ideas</strong> for teaching computers to recognize what was in those photos</p></li><li><p><strong>Regular computer chips</strong> that were&#8230; tired</p></li></ul><p>Asking those chips to learn from all those photos was like asking you to write all your notes for the week in one sitting, by hand, after a full day of sessions.</p><p>Technically possible. Emotionally: no.</p><p>So they did the thing all good tinkerers do.<br>They looked around the room and asked, &#8220;What else could help?&#8221;</p><div><hr></div><h2>Meet the GPU: The Graphics Card That Got a Promotion</h2><p>If you&#8217;ve ever seen a gaming laptop or a big desktop, you&#8217;ve met a GPU&#8212;even if you didn&#8217;t know its name.</p><p>A <strong>GPU</strong> (graphics card) is:</p><ul><li><p>A separate piece inside the computer</p></li><li><p>Originally designed to make games look smooth and pretty</p></li><li><p>Very good at doing lots of tiny calculations at the same time</p></li></ul><p>Games need that because every frame&#8212;every shadow, every bit of movement&#8212;is made of thousands of little dots that have to be drawn quickly.</p><p>It turns out, <strong>teaching a computer to see</strong> is&#8230; kind of similar:</p><ul><li><p>You take an image</p></li><li><p>You turn every tiny piece of it into numbers</p></li><li><p>You do a lot of small calculations again and again</p></li><li><p>You repeat that for millions of images until patterns start to emerge</p></li></ul><p>Researchers realized:</p><blockquote><p>&#8220;This graphics card is already great at moving lots of dots around.<br>What if we ask it to move numbers for learning instead of just for games?&#8221;</p></blockquote><p>That&#8217;s the heart of the story:<br><strong>a game part quietly got promoted to a learning part.</strong></p><div><hr></div><h2>AlexNet: The System That Looked at a Lot of Pictures (Including Cats)</h2><p>In 2012, three researchers&#8212;<strong>Alex Krizhevsky, Ilya Sutskever, and Geoffrey Hinton</strong>&#8212;entered a big image competition called ImageNet.</p><p>Very roughly, the challenge was:</p><blockquote><p>&#8220;Here are over a million labeled photos.<br>Can your program tell us what&#8217;s in them?&#8221;</p></blockquote><p>Their system, later known as <strong>AlexNet</strong>, did a few important things:</p><ul><li><p>It used <strong>two gaming-style graphics cards</strong> as its main engine.</p></li><li><p>It stacked many simple steps so the computer could slowly notice what made, say, a &#8220;cat&#8221; different from &#8220;not a cat.&#8221;</p></li><li><p>It trained on a huge variety of images&#8212;dogs, birds, furniture, landscapes&#8230; and plenty of cats.</p></li></ul><p>When the results came out, AlexNet:</p><ul><li><p>Did <strong>far better</strong> than other approaches at recognizing what was in the pictures</p></li><li><p>Convinced a lot of skeptical people that this &#8220;let&#8217;s give it lots of data and a strong graphics card&#8221; approach really worked</p></li><li><p>Helped start a wave of work that eventually led Geoffrey Hinton, Yann LeCun, and Yoshua Bengio to receive the <strong>Turing Award</strong> (often described as the Nobel Prize of computing)</p></li></ul><p>People summarize it as:</p><blockquote><p>&#8220;That&#8217;s when AI finally got good at cat photos.&#8221;</p></blockquote><p>What actually happened was more like:</p><blockquote><p>&#8220;We learned how to give computers enough practice,<br>on the right hardware,<br>so they could notice patterns we couldn&#8217;t hard-code ourselves.&#8221;</p></blockquote><p>The cats were the cute front end of a much deeper shift.</p><div><hr></div><h2>Why We Care About This (Besides the Cat Memes)</h2><p>From our side, working on Simcha, this story matters for a few reasons:</p><ul><li><p>It shows that <strong>big changes can come from reusing something ordinary</strong> (a game card) in a new way.</p></li><li><p>It reminds us that <strong>constraints are helpful</strong>&#8212;AlexNet ran on just two graphics cards by today&#8217;s standards, so its creators had to be careful and thoughtful.</p></li><li><p>It&#8217;s a clear moment where we went from &#8220;computers kind of see&#8221; to &#8220;computers can actually pick patterns out of messy, real-world data.&#8221;</p></li></ul><p>And now, more than a decade later:</p><ul><li><p>Your phone and laptop have their own small &#8220;pattern helpers&#8221; built in.</p></li><li><p>Smaller, more efficient versions of these ideas can run on devices you already own.</p></li><li><p>The kind of power that once needed a loud desktop in a lab is inching closer to fitting quietly on your desk or in your pocket.</p></li></ul><p>We&#8217;re not saying your phone is AlexNet.<br>We are saying the <strong>same style of thinking</strong>&#8212;let the machine practice a lot, on the right hardware&#8212;is now part of everyday technology.</p><div><hr></div><h2>Where This Series Goes Next: From Pictures to Words</h2><p>In the last article, we talked about AI as a <strong>guessing game for words</strong>&#8212;something like a supercharged autocomplete that has seen a lot of sentences.</p><p>This article is the &#8220;prequel&#8221;:</p><ul><li><p>Before words, there were pictures</p></li><li><p>Before note helpers, there were cat recognizers</p></li><li><p>Before language models, there were graphics cards doing image practice at scale</p></li></ul><p>We&#8217;re stopping the story here on purpose.</p><blockquote><p>A small team, two graphics cards, and a pile of everyday photos<br>showed how far practice plus the right hardware could go.</p></blockquote><p>In the <strong>next</strong> article, we&#8217;ll walk across the hallway&#8212;from the &#8220;cat cousin&#8221; to the &#8220;word cousin&#8221;:</p><ul><li><p>From images to sentences</p></li><li><p>From &#8220;Is this a cat?&#8221; to &#8220;Does this look like part of a useful note?&#8221;</p></li><li><p>From GPUs staring at pictures to models that help us shape words</p></li></ul><p>For now, we&#8217;re staying with this simple truth:</p><p>Cats helped teach computers how to see. &#128049;&#9889;<br>Next up: how the same lineage learned to write&#8212;and what that might eventually mean for the way we build notes.</p>]]></content:encoded></item><item><title><![CDATA[How the Guessing Game Started Listening Better]]></title><description><![CDATA[From Quick Scribbles to Meaningful Recall &#8212; Just Like We Do in Real Conversations]]></description><link>https://newsletter.simcha.ai/p/how-the-guessing-game-started-listening</link><guid isPermaLink="false">https://newsletter.simcha.ai/p/how-the-guessing-game-started-listening</guid><dc:creator><![CDATA[Simcha AI]]></dc:creator><pubDate>Sun, 30 Nov 2025 23:01:15 GMT</pubDate><enclosure url="https://substack-post-media.s3.amazonaws.com/public/images/bf44f45a-986f-4fa0-83a4-075c4a084cc6_1024x1536.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>In the last chapter, our little Markov Chain friend could only remember <strong>one word</strong> at a time.</p><p>Great for simple patterns&#8230;<br>not great for following a <em>real</em> story or a meaningful conversation.</p><p>To make AI genuinely helpful, we needed it to develop something humans rely on all the time:</p><blockquote><h2>The ability to remember what came before &#8212; and notice what matters.</h2></blockquote><p>Not perfect memory.<br>Not total recall.<br>Just the kind of mental note-taking that helps people stay present, understand themes, and connect ideas over time.</p><p><strong>Here&#8217;s how machines learned that skill &#8212; small tech detour ahead, but we promise to keep it human, clear, and nowhere near the scary kind of nerdy.</strong></p><div><hr></div><h2><strong>Step 1 -  N-grams:  &#8220;Quick Scribbles&#8221; </strong></h2><p>N-grams let AI remember a <em>few</em> words instead of just one &#8212;<br>like glancing at a small sticky note mid-conversation.</p><p>It helps with short, obvious conversational patterns like:</p><ul><li><p>&#8220;I&#8217;m feeling really ___&#8221;</p></li><li><p>&#8220;This week has been kind of ___&#8221;</p></li><li><p>&#8220;I want to work on ___&#8221;</p></li></ul><p>These quick-fill moments are easy because the clues are right next to the blank.</p><p>But the moment a conversation stretches beyond that tiny window, those little mental scribbles stop being enough.</p><div><hr></div><h3><strong>Human parallel </strong></h3><p>Someone who remembers the last sentence you said&#8230;<br>but not the feeling or intention behind it.<br>They&#8217;re present, but only in the moment-to-moment surface of the conversation.</p><p>Good for basic coordination.<br>Not great for deeper work or insight.</p><p>As the conversation gets longer, those little scribbles can&#8217;t hold the bigger story.<br>So AI needed a better way to track meaning.</p><div><hr></div><h2><strong>Step 2 - RNNs:  &#8220;I&#8217;m Following the Thread&#8221; </strong></h2><p>Recurrent Neural Networks (RNNs) were the first time AI could keep a <strong>running sense</strong> of what was happening.</p><p>Instead of short scribbles, it started building a flowing mental model:</p><ul><li><p>word after word</p></li><li><p>idea after idea</p></li><li><p>carrying the earlier parts into the later ones</p></li></ul><p>AI wasn&#8217;t just reacting anymore.<br>It was <strong>following the arc</strong>.</p><h3><strong>Human parallel</strong></h3><p>Someone who stays with you through an extended conversation &#8212;<br>noting your shifts, your hesitations, your tone &#8212;<br>even if some details from the beginning soften around the edges.</p><p>They get the general thread, even if the nuances blur.</p><p>That was RNNs.<br>Capable of holding the conversation&#8230;<br>but prone to losing the early, important moments.</p><p>To go deeper, AI needed a way to <strong>protect what matters most.</strong></p><div><hr></div><h2><strong>Step 3 - LSTMs:  &#8220;Keep the Meaningful Notes&#8221; </strong></h2><p>LSTMs (Long Short-Term Memory networks) added a powerful idea:<br><strong>selective memory.</strong></p><p>They didn&#8217;t treat every detail equally.<br>They learned to:</p><ul><li><p>notice important signals</p></li><li><p>hold onto them</p></li><li><p>let go of the noise</p></li></ul><p>This turned AI from &#8220;following the thread&#8221; into &#8220;tracking themes.&#8221;</p><h3><strong>Human parallel</strong></h3><p>Someone who unconsciously catches the deeper patterns in what you&#8217;re saying &#8212;<br>the repeated phrases, the concerns that keep resurfacing,<br>the emotional moments that matter more than the words themselves.</p><p>They don&#8217;t memorize everything.<br>They remember the things that <em>shape</em> the conversation.</p><p>That&#8217;s what LSTMs gave AI:<br>a sense of what&#8217;s meaningful, not just what&#8217;s recent.</p><p>This made AI far better at:</p><ul><li><p>connecting ideas</p></li><li><p>noticing patterns</p></li><li><p>remembering the heart of the message, not just the wording</p></li></ul><p>For years, this was the most &#8220;thoughtful&#8221; memory AI had.</p><div><hr></div><h2><strong>From Scribbles to Sense-Making</strong></h2><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!xRdX!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F04116603-4dc6-4f58-b03d-d292ca49f127_1566x462.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!xRdX!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F04116603-4dc6-4f58-b03d-d292ca49f127_1566x462.png 424w, https://substackcdn.com/image/fetch/$s_!xRdX!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F04116603-4dc6-4f58-b03d-d292ca49f127_1566x462.png 848w, https://substackcdn.com/image/fetch/$s_!xRdX!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F04116603-4dc6-4f58-b03d-d292ca49f127_1566x462.png 1272w, https://substackcdn.com/image/fetch/$s_!xRdX!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F04116603-4dc6-4f58-b03d-d292ca49f127_1566x462.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!xRdX!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F04116603-4dc6-4f58-b03d-d292ca49f127_1566x462.png" width="1456" height="430" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/04116603-4dc6-4f58-b03d-d292ca49f127_1566x462.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:430,&quot;width&quot;:1456,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:70219,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:&quot;https://newsletter.simcha.ai/i/179869668?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F04116603-4dc6-4f58-b03d-d292ca49f127_1566x462.png&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!xRdX!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F04116603-4dc6-4f58-b03d-d292ca49f127_1566x462.png 424w, https://substackcdn.com/image/fetch/$s_!xRdX!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F04116603-4dc6-4f58-b03d-d292ca49f127_1566x462.png 848w, https://substackcdn.com/image/fetch/$s_!xRdX!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F04116603-4dc6-4f58-b03d-d292ca49f127_1566x462.png 1272w, https://substackcdn.com/image/fetch/$s_!xRdX!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F04116603-4dc6-4f58-b03d-d292ca49f127_1566x462.png 1456w" sizes="100vw" loading="lazy"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image"><svg role="img" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg"><g><title></title><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p>Each step brought AI closer to how humans naturally track a meaningful conversation &#8212;<br>not by remembering <em>everything</em>, but by remembering <strong>what matters</strong>.</p><p>But even at their best, these models read in a strictly linear way:</p><p>Left to right.<br>One word after the next.<br>No jumping back.<br>No zooming out.</p><p>Humans don&#8217;t think like that.<br>We revisit earlier ideas, shift perspective, and connect distant moments quickly.</p><p>For AI to make the next leap, it needed a new superpower.</p><p>To understand how that happened, we need to step away from language for a moment&#8230; and visit a surprising branch of AI&#8217;s family tree.</p><div><hr></div><h2><strong>Next Up: The Buff Cousin Who Hit the GPU Gym</strong></h2><p>Before GPT shows up, we take a detour to meet the cousin who spent YEARS staring at cat pictures, lifting math weights, and leveling up like a video game character.</p><p>Turns out: cats gave us supercomputers. &#128049;&#9889;</p>]]></content:encoded></item><item><title><![CDATA[How a Clever Guessing Game Became the Brains Behind AI]]></title><description><![CDATA[Imagine you&#8217;re playing a word-guessing game with a friend.]]></description><link>https://newsletter.simcha.ai/p/how-a-clever-guessing-game-became</link><guid isPermaLink="false">https://newsletter.simcha.ai/p/how-a-clever-guessing-game-became</guid><dc:creator><![CDATA[Simcha AI]]></dc:creator><pubDate>Mon, 24 Nov 2025 04:09:20 GMT</pubDate><enclosure url="https://substack-post-media.s3.amazonaws.com/public/images/a3095eaa-666b-4d3d-87f0-42a95ce3d842_1536x1024.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<blockquote><p>Before we talk about how AI affects our planet&#8217;s energy and our own emotional energy, we will spend a moment understanding the basics. When we know how these systems actually work, we gain clarity and strength. We become more intentional, more creative and more confident in how we use technology.</p><p>This article is our starting point.</p></blockquote><div><hr></div><p>Imagine you&#8217;re playing a word-guessing game with a friend. You say the first word: <strong>&#8220;Once&#8230;&#8221;</strong><br>Your friend guesses the next word: <strong>&#8220;upon.&#8221;</strong><br>Then you say, &#8220;a&#8230;&#8221;<br>They say, &#8220;time.&#8221;</p><p>It feels almost magical, right? But underneath the magic is something simple: your friend is predicting what <em>usually</em> comes next.</p><div class="subscription-widget-wrap-editor" data-attrs="{&quot;url&quot;:&quot;https://newsletter.simcha.ai/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe&quot;,&quot;language&quot;:&quot;en&quot;}" data-component-name="SubscribeWidgetToDOM"><div class="subscription-widget show-subscribe"><div class="preamble"><p class="cta-caption">Thanks for reading Simcha's Newsletter! Subscribe for free to receive new posts and support my work.</p></div><form class="subscription-widget-subscribe"><input type="email" class="email-input" name="email" placeholder="Type your email&#8230;" tabindex="-1"><input type="submit" class="button primary" value="Subscribe"><div class="fake-input-wrapper"><div class="fake-input"></div><div class="fake-button"></div></div></form></div></div><p>Believe it or not, that small, almost boring idea is the ancestor of today&#8217;s giant AI systems systems that can write essays, answer questions, and even help you brainstorm your science project. The family tree begins with something called a <strong>Markov chain</strong>, and grows into something called a <strong>large language model</strong>, or <strong>LLM</strong>.</p><p>Let&#8217;s walk through how the guessing game grew up.</p><div><hr></div><h2><strong>Where It All Started: The Markov Chain</strong></h2><p>A Markov chain is basically a rule:</p><blockquote><p><strong>&#8220;Choose the next word based only on the word right before it.&#8221;</strong></p></blockquote><p>That&#8217;s it.<br>It doesn&#8217;t remember whole sentences. It doesn&#8217;t understand meaning. It&#8217;s like a goldfish with math powers.</p><p>If you fed a Markov chain a million fairy tales, it might learn that after the word <strong>&#8220;Once&#8221;</strong>, the most common next word is <strong>&#8220;upon.&#8221;</strong><br>After <strong>&#8220;upon&#8221;</strong>, the most common next word is <strong>&#8220;a.&#8221;</strong><br>After <strong>&#8220;a&#8221;</strong>, you get <strong>&#8220;time.&#8221;</strong></p><p>So it can write things like:</p><blockquote><p><em>&#8220;Once upon a time a dragon lived in a&#8230;&#8221;</em></p></blockquote><p>It&#8217;s not thinking. It&#8217;s just following probability trails.<br>Still, for its time, it was a big deal. Computers could <em>predict text</em>.</p><p>But it had a problem.</p><div><hr></div><h2><strong>The Big Limitation (and Why It Matters)</strong></h2><p>A Markov chain doesn&#8217;t remember what happened five or ten words ago.<br>It lives moment to moment.</p><p>So if you were writing a story about a dragon who meets a robot who meets a princess, the Markov chain would forget the dragon by the next paragraph. It can&#8217;t keep track of big ideas, themes, or even who a sentence is <em>about</em>.</p><p>It can predict the <em>next</em> word.<br>But not the <em>meaning</em> of what it&#8217;s writing.</p><p>That&#8217;s where people started asking a bigger question:</p><blockquote><p><strong>What if a computer could look at not just the last word, but the whole sentence or even the whole story?</strong></p></blockquote><p>And that&#8217;s how we start climbing toward modern AI.</p><div><hr></div><h2><strong>The Jump to Neural Networks: When the Machine Begins to &#8220;Pay Attention&#8221;</strong></h2><p>Engineers began building systems that could look at more than one word at a time.<br>Instead of remembering only the last step, the computer started learning patterns across whole sentences.</p><p>Then came a huge breakthrough: <strong>the transformer model</strong>, which uses something called <strong>attention</strong>. It lets the computer look at all the words at once and figure out which ones matter most.</p><p>It&#8217;s like the difference between:</p><ul><li><p>reading a story one word at a time with your nose touching the page<br><strong>vs.</strong></p></li><li><p>stepping back and seeing the whole page, all at once</p></li></ul><p>Suddenly, the computer could learn real patterns: how ideas connect, how questions work, how stories unfold.</p><p>This is the moment the guessing game grew up.</p><div><hr></div><h2><strong>LLMs: Markov Chains on Superpowers</strong></h2><p>Modern large language models, like ChatGPT, still guess the next word just like a Markov chain.</p><p>But they do it with:</p><ul><li><p><strong>massive memory of patterns</strong></p></li><li><p><strong>millions of examples</strong></p></li><li><p><strong>the ability to see long-range connections</strong></p></li><li><p><strong>deep layers that learn meaning, tone, and structure</strong></p></li></ul><p>A Markov chain is like predicting one note of a song only from the previous note.<br>An LLM is like predicting the next note after listening to the whole song and every song ever made.</p><p>Same basic idea.<br>Radically different power.</p><div><hr></div><h2><strong>Why This Matters for You</strong></h2><p>Here&#8217;s the quiet truth:<br>Every big invention even the ones that feel futuristic begins with a tiny, almost silly idea.</p><p>A simple rule.<br>A basic guess.<br>A tiny step.</p><p>If you ever feel like your ideas are too small, remember: today&#8217;s smartest AIs are built on the world&#8217;s simplest game.</p><p><strong>Guess what comes next.</strong></p><div><hr></div><h2>Here&#8217;s a version reframed for therapist&#8211;coaches, counselors, mentors, and advisors, in the same style profile:</h2><div><hr></div><h3><strong>A Quick Experiment You Can Try</strong></h3><p>Tonight, try this gentle exercise:</p><ol><li><p>Write the first <strong>two sentences</strong> of a client scenario&#8212;something simple and familiar.</p></li><li><p>Pause and notice the <strong>response your mind predicts next</strong>.</p></li><li><p>Then ask an AI to generate its next step in the same scenario.</p></li><li><p>Compare the two&#8212;not for accuracy, but for <strong>perspective</strong>.</p></li></ol><p>Notice how both you and the model are doing the same fundamental thing:<br>responding from learned patterns&#8212;just shaped by very different histories.</p><p>And maybe ask yourself:</p><blockquote><p><strong>What new possibilities open up when I can see my own predictive patterns alongside an external one?<br>What insight might grow from simply expanding the space of responses I consider?</strong></p></blockquote><div class="subscription-widget-wrap-editor" data-attrs="{&quot;url&quot;:&quot;https://newsletter.simcha.ai/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe&quot;,&quot;language&quot;:&quot;en&quot;}" data-component-name="SubscribeWidgetToDOM"><div class="subscription-widget show-subscribe"><div class="preamble"><p class="cta-caption">Thanks for reading Simcha's Newsletter! Subscribe for free to receive new posts and support my work.</p></div><form class="subscription-widget-subscribe"><input type="email" class="email-input" name="email" placeholder="Type your email&#8230;" tabindex="-1"><input type="submit" class="button primary" value="Subscribe"><div class="fake-input-wrapper"><div class="fake-input"></div><div class="fake-button"></div></div></form></div></div>]]></content:encoded></item><item><title><![CDATA[A Big Idea We Can’t Let Go Of]]></title><description><![CDATA[Why Joy Still Matters in an AI World That&#8217;s Trying to Save the Planet, and How It Gives Us the Energy To Be Better Humans To Each Other]]></description><link>https://newsletter.simcha.ai/p/a-big-idea-we-cant-let-go-of</link><guid isPermaLink="false">https://newsletter.simcha.ai/p/a-big-idea-we-cant-let-go-of</guid><dc:creator><![CDATA[Simcha AI]]></dc:creator><pubDate>Mon, 17 Nov 2025 04:33:30 GMT</pubDate><enclosure url="https://substack-post-media.s3.amazonaws.com/public/images/8256bc86-631a-4824-81ba-93a191839178_1024x1536.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>Imagine this:</p><p>You wake up tomorrow, open your inbox, and there&#8217;s an article waiting for you. Not a generic broadcast, but something that feels tuned into what your nervous system has been trying to tell you. It knows you&#8217;ve been carrying too much. It knows your mind runs faster than your body can keep up. It knows that even your coffee is tired of being your emotional support beverage.</p><p>And instead of telling you to optimize your workflow or become a shinier version of yourself, it asks a quieter question:<br><strong>What would bring you joy today, not efficiency, not self-improvement, but actual energy-giving joy?</strong></p><p>That question is the spark behind this entire project.<br>It is also the big idea we can&#8217;t let go of.</p><p>Because the more we watch AI transform how we work, relate, and grow, the more convinced we become that the next real frontier isn&#8217;t intelligence. It&#8217;s energy.<br>The kind that makes us more alive.<br>The kind that makes us more ourselves.<br>The kind that actually helps us be better humans in our relationships, not just more productive ones at our desks.</p><p>But here&#8217;s the tension:<br>How do we use AI to support mental well-being, cultivate joy, and restore human-to-human energy without degrading the environment we depend on?</p><p>That&#8217;s the thread we want to pull on here.</p><div><hr></div><h2><strong>Joy Isn&#8217;t a Luxury. It&#8217;s an Energy Source.</strong></h2><p>Most people think of joy as a nice-to-have, like dessert after the &#8220;real&#8221; work of being functioning adults. But psychologically, joy is much closer to an energy technology. It regulates our systems, restores our capacity to connect, and refuels the emotional batteries we use in every relationship we care about.</p><p>Joy doesn&#8217;t just feel good.<br><strong>Joy is what makes us capable.</strong></p><p>Capable of listening instead of reacting.<br>Capable of reaching out instead of shutting down.<br>Capable of giving without collapsing.</p><p>If we&#8217;ve ever been burned out, we know the opposite. Everything becomes harder, thinking, relating, loving. Even the people we care about most can feel like &#8220;one more thing.&#8221; You know that moment when someone texts &#8220;can we talk?&#8221; and you briefly consider moving to another continent? Exactly.</p><p>This is why joy matters so much.<br>Joy replenishes the energy that lets us be good partners, good friends, good parents, good colleagues. Good humans.</p><p>And many of us have been running low.</p><p>This is where AI, surprisingly, can help.</p><div><hr></div><h2><strong>AI as an Energy-Regulation Tool</strong></h2><p>We rarely talk about AI as something that can help calm us down. Usually it&#8217;s portrayed as a productivity machine, a creativity amplifier, or occasionally a misunderstood superintelligence that just wants to tidy up the internet.</p><p>But AI also has the potential to become a kind of external nervous-system support structure.</p><p>Not in a clinical sense.<br>More like a companion that helps us:</p><ul><li><p>calm our systems when we&#8217;re overloaded,</p></li><li><p>name what we&#8217;re feeling when it&#8217;s a soup of emotions with no label,</p></li><li><p>notice patterns that drain our energy,</p></li><li><p>and remember the tiny rituals that restore us.</p></li></ul><p>Picture this:<br>It&#8217;s 10:30pm. We&#8217;re not melting down, but we&#8217;re also not okay. We&#8217;re emotionally out of fuel. This is the moment many of us start doom-scrolling, eating toast over the sink, or giving ourselves a TED Talk about how tomorrow will be different.</p><p>Now imagine an AI companion gently catching us before the spiral begins:</p><p>&#8220;You&#8217;re not failing. You&#8217;re tired. Your body needs grounding more than answers. Try two slow breaths. I&#8217;ll wait.&#8221;</p><p>Not replacing human relationships. Supporting them.<br>Not giving us more to do. Giving us more energy to show up as ourselves.</p><p>This kind of everyday nervous-system scaffolding could fundamentally change how we relate to each other. When our energy is steadier, our humanity is more available.</p><p>But this leads us to the uncomfortable part.</p><div><hr></div><h2><strong>The Environmental Tension</strong></h2><p>We can&#8217;t talk about AI and well-being without acknowledging the cost: compute, electricity, water, infrastructure. Tools that help our inner world can still strain the outer world we depend on.</p><p>It&#8217;s a paradox we can&#8217;t ignore.<br>We want technology that heals us.<br>But we can&#8217;t heal ourselves with tools that hurt the planet.</p><p>And yet the story isn&#8217;t predetermined.<br>AI doesn&#8217;t have to be extractive.</p><p>There is a quiet but growing movement toward:</p><ul><li><p>smaller, more efficient models,</p></li><li><p>renewable-energy data centers,</p></li><li><p>local on-device AI,</p></li><li><p>and architectures that sip resources instead of chugging them like a college student with a Red Bull.</p></li></ul><p>The real question becomes:</p><p><strong>Can we build AI that gives humans energy without draining the planet&#8217;s?</strong></p><p>We believe the answer is yes.<br>But it requires intention, both technological and human.</p><div><hr></div><h2><strong>Three Questions Guiding Simcha</strong></h2><p>These are the questions we&#8217;re exploring here, the questions we believe we need to wrestle with together:</p><h3><strong>1. How can AI help us generate more human energy, not less?</strong></h3><p>Not hustle energy. Not adrenaline energy.<br>The kind of internal spaciousness that lets us show up generously with the people we care about.</p><h3><strong>2. What does sustainable emotional technology look like?</strong></h3><p>Can we design tools that soothe our systems and also lighten our planetary footprint?<br>Can the tech that helps us heal also model sustainability in its own architecture?</p><h3><strong>3. How do we build a future where joy, not output, is the metric?</strong></h3><p>Joy fuels energy.<br>Energy fuels connection.<br>Connection fuels humanity.</p><p>This is the feedback loop worth architecting<br>and probably the only loop we&#8217;re truly excited to get stuck in.</p><div><hr></div><h2><strong>A Small Practice for This Week</strong></h2><p>Before the next post, try this:</p><p><strong>Once a day, ask yourself: &#8220;What tiny moment would give me energy right now?&#8221;</strong></p><p>Not a big transformation.<br>Not a lifestyle overhaul.<br>Just an energy spark.</p><ul><li><p>Step outside for 30 seconds</p></li><li><p>Text someone &#8220;thinking of you &#10084;&#65039;&#8221;</p></li><li><p>Drink cold water slowly</p></li><li><p>Look at the sky</p></li><li><p>Put your phone down for one deep breath</p></li><li><p>Or, if all else fails, pet a dog, borrow a dog, or look at a picture of a dog</p></li></ul><p>Joy and energy are often the same thing.<br>And restoring even a little makes us more present, more open, more human with the people we love.</p><div><hr></div><p>If this resonates, you&#8217;re in the right place.<br>Simcha is about joy as fuel, AI as support, and sustainability as the boundary that keeps both honest.</p><p>Simcha AI is about becoming more human, not in spite of technology, but with it.<br>And the story has only just begun.</p>]]></content:encoded></item></channel></rss>