Speaker 1: Hey Mason! July ninth, twenty twenty-six. We have a packed show for you today, starting with a quick radar briefing on the newest research we have been keeping an eye on, followed by a deep dive into geospatial artificial intelligence, and then we will wrap up with the biggest news in general artificial intelligence from this past week.
Speaker 2: I am especially interested in that geospatial piece, especially if it touches on how we map indigenous lands.
Speaker 1: That is exactly where we are headed. But first, let us get into the radar briefing to see what the latest papers are telling us.

Speaker 1: Mason, the research filter has been busy lately, particularly with papers that bridge the gap between high-resource reasoning and the messy reality of underrepresented languages.

Speaker 2: I noticed a lot of the recent activity seems to be circling around how we actually measure if these models are being truthful when they move away from English.

Speaker 1: Exactly, and that leads us to a study by Felix Feldman and several colleagues regarding retrieval-augmented generation for public health questions.

Speaker 2: That sounds like a direct parallel to our work with grounding Lakota models in community-approved texts, right?

Speaker 1: It is, because they provide a systematic evaluation of how retrieval quality and faithfulness actually impact the final answer.

Speaker 2: It makes sense that if the retrieval is shaky, the whole sovereignty of the information collapses.

Speaker 1: Precisely, and that connects to a broader theme of cultural preservation, like the work by Aparna Madva and her team on rethinking artificial intelligence for Indic languages through the lens of heritage.

Speaker 2: They are looking at how linguistic diversity and cultural preservation intersect, which is the core of what we are trying to do with community-governed data.

Speaker 1: It is a similar struggle to what Andrea Alfarano and others found when studying uncertainty in multi-lingual multiple-choice questions.

Speaker 2: Wait, didn't they find that reasoning in English actually improves performance in low-resource languages?

Speaker 1: They did, and that is a double-edged sword for us because it informs how we handle data-efficient adaptation.

Speaker 2: It's a bit of a catch-twenty-two if the model relies on English logic to understand a language it's supposed to be preserving.

Speaker 1: That is why the work on specialized scripts is so vital, like the study by Burte Bayarsaikhan and colleagues on cognitive pivot translation for Mongolian.

Speaker 2: They are dealing with digraphic languages and the ambiguity that comes with traditional scripts, which is a huge hurdle for optical character recognition.

Speaker 1: It's the same technical wall we hit when dealing with complex orthographies.

Speaker 2: Speaking of technical hurdles, I saw something about mathematical reasoning in the mix too.

Speaker 1: Yes, Daryna Dementieva and her team are working on PluraMath, which tries to extend mathematical reasoning evaluation beyond just high-resource languages.

Speaker 2: That's a massive gap, because if a model can't reason mathematically in a local language, it's not truly functional for that community.

Speaker 1: And finally, there is the work by Lukmal Ilyas and Nevidu Jayatilleke on cross-lingual transfer learning for speech recognition, moving from Sinhala to Dhivehi.

Speaker 2: That transfer learning approach is exactly what we need to make our speech models more efficient when we don't have massive datasets.

Speaker 1: It all points to the same conclusion: the frontier isn't just making models bigger, it is making them more reliable across the world's actual linguistic diversity.

Speaker 1: We are looking at a paper that essentially tries to solve the "how do we scale" problem for Earth observation foundation models, specifically focusing on the TESSERA version two project.

Speaker 2: It sounds like they are moving away from the standard way of training these models, Mason, because the usual metrics aren't actually working for satellite data.

Speaker 1: Exactly. The core problem they identify is that in Earth observation, the data is incredibly messy. You have clouds, shadows, and irregular satellite revisit times that make the signal very noisy. Because of this, the researchers found that the standard pretraining loss—the number the model tries to minimize during its initial training phase—is a terrible predictor of how well the model will actually perform on real tasks like land cover classification or change detection.

Speaker 2: Wait, so if the loss doesn't tell you if the model is getting better at the actual job, how do you know when to stop training or which model architecture to pick?

Speaker 1: That is the million-dollar question. They addressed this by running a massive study with three hundred and ninety-five different training runs. They found that if you try to pick your best model based on the lowest pretraining loss, you end up wasting about two hundred and fifty-four percent more compute than necessary to reach the same level of performance.

Speaker 2: That is a massive inefficiency. So, what did the data actually say about where that compute should go instead?

Speaker 1: They discovered a specific scaling rule. When you increase your total compute budget, you should increase the size of your encoder and the amount of training data, but you should keep the projector size—the part of the model that maps the features—essentially fixed. The projector is just a temporary scaffold; the real heavy lifting happens in the encoder.

Speaker 2: Okay, so the recipe is: build a massive, high-capacity encoder, feed it as much data as possible, and ignore the projector size once you hit a certain point. But how do you actually use a massive one-billion parameter model if you are a practitioner running things on local hardware?

Speaker 1: That is where the second half of the paper comes in. They use a "teacher-student" approach. They train this giant one-billion parameter "teacher" model to capture the best possible representations, and then they use a process called distillation to shrink that knowledge into much smaller "student" models. They even created a family of students ranging from forty-four million parameters down to just one million.

Speaker 2: I noticed they mentioned something called Matryoshka representations. Is that like those Russian nesting dolls?

Speaker 1: It is exactly like that. They designed the student models so that the embeddings are nested. This means a single embedding can be truncated. For example, you can take a one hundred and twenty-eight dimensional embedding and just use the first sixteen dimensions.

Speaker 2: Does that come with a huge hit to accuracy?

Speaker 1: Surprisingly, no. They found that using only sixteen dimensions retains about ninety-two percent of the performance of the full one hundred and twenty-eight dimensions, but it uses only one-eighth of the storage. It gives you a literal knob to turn between accuracy and storage costs.

Speaker 2: That sounds incredibly practical for someone building a geospatial tool. But let's look at the results. They claim to outperform both open and proprietary models. How much of an improvement are we talking about?

Speaker 1: On a twenty-nine task evaluation suite, their medium-sized student model achieved a composite score of zero point six one one. To put that in perspective, the previous version of TESSERA scored zero point five four one, and other major players like Alpha Earth were around zero point five seven four. It is a clear lead, especially considering the student models are orders of magnitude smaller than the teachers.

Speaker 2: I want to push back on the "analysis-ready" claim. They say these are embeddings-as-data, meaning you don't have to process raw Sentinel imagery anymore. But if I am a practitioner, I still have to deal with the sheer volume of these embeddings. Even at sixteen dimensions, if I am covering the whole planet, isn't the I/O still a nightmare?

Speaker 1: It is a trade-off, for sure. But the authors argue that the bottleneck in Earth observation is usually the preprocessing—the cloud masking, the radiometric calibration, and the sensor harmonization. By providing precomputed embeddings, they are moving that entire burden from your local machine to their cloud. You just download the vectors and train a tiny, lightweight head on top of them.

Speaker 2: What are the actual limitations here? It sounds almost too good to be true.

Speaker 1: There are a few. First, the model is still heavily tied to the Sentinel-one and Sentinel-two ecosystem. If you are working with high-resolution commercial imagery or different sensor types, these embeddings might not translate perfectly. Second, while the Matryoshka trick is great for storage, it is still a form of lossy compression. If you are doing extremely fine-grained tasks, like detecting tiny individual buildings, those sixteen dimensions might not be enough.

Speaker 2: And what about the "black box" aspect? If they are serving these as a product, do we lose the ability to see why a certain pixel was classified a certain way?

Speaker 1: That is a valid concern. You are essentially trusting their distillation process. If the teacher model had a bias or an artifact, the student will inherit it. However, they did note that version two is much more stable across years than version one, meaning it has shed many of the seasonal artifacts that used to plague the embeddings.

Speaker 2: So, if I am sitting at my desk right now, building a tool with Sentinel-two data, what should I actually take away from this?

Speaker 1: If you are currently spending all your time writing custom preprocessing pipelines for every new project, you should look into the "embeddings-as-data" paradigm. Specifically, look at how they use distillation to create small, fast models. If you have a specific task, don't try to train a massive backbone from scratch. Instead, try to find a way to distill a larger, pre-trained representation into a small student that fits your specific hardware constraints.

Speaker 2: And what should I ignore?

Speaker 1: Ignore the pretraining loss. If you are experimenting with your own self-supervised models, do not spend your time trying to optimize the loss function to get a lower number. It is a vanity metric in Earth observation. Instead, evaluate your models directly on the downstream tasks you actually care about, like your specific classification or regression targets. That is the only way to know if your scaling is actually working.

Speaker 2: It sounds like the era of "just make the model bigger and hope the loss goes down" is officially over for geospatial AI.

Speaker 1: Precisely. It is time to scale the data and the encoder, and then distill it down to something that actually fits in a real-world workflow.

Speaker 1: Moving from that specific technical breakdown, let's look at the broader landscape of what happened in the industry this week, Mason.

Speaker 2: It feels like a massive week for the big players, but I am wondering if any of this actually moves the needle for people running local hardware.

Speaker 1: It might, especially with the OpenAI release of the GPT five point six family, which includes Sol, Terra, and Luna.

Speaker 2: Sol sounds like a heavy hitter, but what about the smaller ones?

Speaker 1: Luna is the fast, cheap one, and Terra is a mid-tier model that aims for the performance of the previous version at half the cost.

Speaker 2: That is interesting for API users, but for someone building community-governed models, does this just increase the gap between proprietary giants and open weights?

Speaker 1: It likely does, but there is a side to this that matters for safety and evaluation. Anthropic released research about an internal workspace they call J-Space.

Speaker 2: J-Space? Is that a new architecture?

Speaker 1: Not exactly, it is a functional workspace they identified using a Jacobian-based method. They are saying it looks a lot like how humans access conscious information, though they are very careful to say it is not actual consciousness.

Speaker 2: If they can actually see how a model is reasoning through a Jacobian lens, does that help us catch when a model is hallucinating or, more importantly, when it is being steered?

Speaker 1: Exactly. It is being framed as a safety tool to detect hidden reasoning or even fabricated data. And that ties directly into what the Federal Trade Commission is doing right now.

Speaker 2: I saw something about the FTC. Are they looking at model accuracy?

Speaker 1: They are proposing a policy statement regarding the suppression of accuracy and ideological steering. They want to know if companies are pushing models toward undisclosed objectives, which could violate consumer protection laws.

Speaker 2: That is a huge deal for language sovereignty. If a model is being steered by a central authority, it is not really serving the community it is supposed to represent.

Speaker 1: It is a massive tension point. And speaking of tension, Anthropic just had to redeploy Claude Fable five globally after the United States lifted export controls.

Speaker 2: Wait, why were they controlled in the first place?

Speaker 1: Researchers at Amazon found that the safeguards could be bypassed to generate exploit code, so the government stepped in. Now that the controls are lifted, Anthropic is back online with new security reporting obligations.

Speaker 2: It feels like a constant tug of war between capability and control.

Speaker 1: It is. And the push for capability is accelerating. SpaceXAI, which has absorbed xAI, launched Grok four point five in partnership with the coding tool Cursor.

Speaker 2: Is that another massive model meant to compete with the flagship versions?

Speaker 1: It is aimed at coding, finance, and legal work. They are calling it Opus-class, but they are trying to make it faster and cheaper than the current leaders.

Speaker 2: It seems like every major release this week is about making these agents more autonomous. I noticed Anthropic also expanded Claude Cowork to web and mobile.

Speaker 1: Right, they are moving from just coding agents to general office agents that can run scheduled tasks in the cloud even when your device is offline.

Speaker 2: It is a lot of noise. For someone like us, focusing on small, high-quality, community-approved datasets, does any of this actually change the math?

Speaker 1: The math stays the same, but the stakes for evaluation are rising. If the giants are building these massive, steerable workspaces, our job of proving that a local, sovereign model is more accurate and less biased becomes even more critical.

Speaker 1: We have covered a lot of ground today, from the nuances of radar briefing to the massive scaling potential of the Tessera version two pixel-wise Earth foundation models.
Speaker 2: It is a lot to process, especially seeing how quickly geospatial intelligence is moving from simple imagery to these complex, multi-layered models.
Speaker 1: It certainly is, and it will be interesting to see how these foundation models change the way we monitor the planet in the coming months.
Speaker 2: I agree, there is a lot to keep an eye on.
Speaker 1: That is all for this week, Mason, and I will talk to you in the next episode.