This was sobering forecast for ๐๐ฎ๐ซ๐ซ๐ข๐๐๐ง๐ ๐๐๐ฅ๐ข๐ฌ๐ฌ๐ on Google DeepMind’s experimental Weather Lab. (Seen on Sunday 26th, 2025)
The AI model’s ensemble predicted with high confidence the devastating track: CAT 4 landfall in ๐๐๐ฆ๐๐ข๐๐ (around Tuesday, Oct 28th) [Turned out it was a CAT 5], followed by a CAT 3 impact on Cuba (Wednesday, Oct 29th).
Google DeepMind and UCL experts have released a ๐ณ๐ฟ๐ฒ๐ฒ, hands-on curriculum. It covers the fundamentals of building and fine-tuning language models, including data preparation, neural networks, and the transformer architecture.
Hereโs a look at what the courses cover:
๐๐๐ถ๐น๐ฑ ๐ฌ๐ผ๐๐ฟ ๐ข๐๐ป ๐ฆ๐บ๐ฎ๐น๐น ๐๐ฎ๐ป๐ด๐๐ฎ๐ด๐ฒ ๐ ๐ผ๐ฑ๐ฒ๐น: Learn the fundamentals of LMs, from traditional n-grams to modern transformers.
๐ฅ๐ฒ๐ฝ๐ฟ๐ฒ๐๐ฒ๐ป๐ ๐ฌ๐ผ๐๐ฟ ๐๐ฎ๐ป๐ด๐๐ฎ๐ด๐ฒ ๐๐ฎ๐๐ฎ: Dive deep into preparing text data with tokenization and embeddings.
๐๐ฒ๐๐ถ๐ด๐ป ๐๐ป๐ฑ ๐ง๐ฟ๐ฎ๐ถ๐ป ๐ก๐ฒ๐๐ฟ๐ฎ๐น ๐ก๐ฒ๐๐๐ผ๐ฟ๐ธ๐: Understand the training process, how to spot overfitting, and implement neural networks.
๐๐ถ๐๐ฐ๐ผ๐๐ฒ๐ฟ ๐ง๐ต๐ฒ ๐ง๐ฟ๐ฎ๐ป๐๐ณ๐ผ๐ฟ๐บ๐ฒ๐ฟ ๐๐ฟ๐ฐ๐ต๐ถ๐๐ฒ๐ฐ๐๐๐ฟ๐ฒ: Explore the mechanisms of transformers, including the all-important attention mechanism.
๐ง๐ฟ๐ฎ๐ถ๐ป ๐ฎ ๐ฆ๐บ๐ฎ๐น๐น ๐๐ฎ๐ป๐ด๐๐ฎ๐ด๐ฒ ๐ ๐ผ๐ฑ๐ฒ๐น (๐๐ต๐ฎ๐น๐น๐ฒ๐ป๐ด๐ฒ ๐๐ฎ๐ฏ): Apply everything you’ve learned in a final challenge to build a character-based model from scratch.
This is one of the most interesting (and hilarious) stories I’ve heard this year. A competitor in a major RNA folding competition lacked GPU access. This “GPU poor” competitor had to innovate, and they ended up beating everyone. (2400 participants) How? Pure engineering and ingenuity. Instead of tackling the problem with a very large AI model, they were forced to be smarter. They built a complex data pipeline that just… achieved better results. The focus was on data quality and better algorithms. The method used was a TBM data pipeline (1990s tech…). ๐ Now, the officially winning solution was a hybrid. But the real story is that a heavy, data-centric approach can still out-innovate a pure AI one. This was RNA folding (not protein folding), a problem with a much smaller dataset, and the “classic” method won. The author even mentions in the comments that the original pipeline had no AI at all and a better score. They technically won despite AI. ๐
There are so many lessons here, but the main ones are: AI is not always the solution. ๐๐๐๐๐ ๐ ๐๐ก๐ฆ ๐๐ ๐กโ๐ ๐๐๐กโ๐๐ ๐๐ ๐๐๐ฃ๐๐๐ก๐๐๐, as you may have heard. My main takeaway, though? If you are a researcher in a low-resource setting, know that you can compete. You can win by being more ๐๐๐ ๐๐ข๐๐๐๐๐ข๐. The solution and must read: Stanford- RNA 3D Folding competition solution write up
On the computational biology side, allow me to also plug in some important updates recently from Google:
This week, Google Research and partners (including UC Santa Cruz) released Deepsomatic, an AI tool that identifies cancer-related mutations in a tumor’s genetic sequence to help pinpoint what’s driving the cancer. Deepsomatic
The AlphaFold Database has been updated with new data and functionalities, continuing its partnership with Google DeepMind and EMBL-EBI. Alaphafold DB
Fun Sunday exercise, how much useful information I can squeeze in a tiny AI model. My goal was to take a general-purpose language model and turn it into an expert on a topic vital such as agriculture.
Here are my steps:
1. The Foundation: Google’s Gemma 270M
I started with this small but powerful but compact base model, gemma-3-270m-it. At just 550 MB, it’s a brilliant piece of engineering that can run on consumer-grade hardware.I am using my laptop.
2. The Technique: Parameter-Efficient Fine-Tuning (PEFT) with LoRA
Instead of retraining the entire model (which is slow and resource-intensive), I used a technique called LoRA. Think of it like adding a small, highly specialized “expert module” to the model’s existing brain. The original model’s knowledge remains, but we efficiently โteachโ it a new skill, in this case agricultural information.
3. The Curriculum: The Agriculture Q&A Dataset
I used the KisanVaani/agriculture-qa dataset to teach the model the nuances of farming, crops, pests, and soil.
After a 15m training session, the new “expert module” I created was only 45 MB! That’s right. For just 45 MB, I layered deep agricultural knowledge onto a powerful base model. This process has created a specialized AI assistant that is more accurate and relevant for agricultural queries than the original.
Model output:
— Loading Model and Tokenizer —
Model and tokenizer loaded successfully.
Dataset loaded successfully from ‘/home/abdoulaye/aiplayground/agriculture_qa_dataset’.
Device set to use cuda:0
— Testing Base Model Performance —
— Test Question 1 —
Question:
which maize disease survive well in warm and humid weather.
Original Answer:
Gray leaf spot
Generated Answer (Base Model):
The maize disease, also known as the maize blight, is a fungal disease that can affect maize plants, particularly in areas with high humidity and high temperatures. It’s a common problem in many parts of the world, and it can be difficult to control.
————————————–
— Test Question 2 —
Question:
how can overuse of pesticides destroy soil nutrients?
Original Answer:
Pesticides can kill beneficial soil microorganisms and reduce soil biodiversity, which can lead to nutrient depletion and reduced soil fertility.
Generated Answer (Base Model):
Overuse of pesticides can be a serious threat to soil nutrients, which are essential for plant growth, soil health, and overall ecosystem function. Here are some ways pesticides can negatively impact soil nutrients:
* **Reduced nutrient availability:** Pesticides can disrupt the natural nutrient cycle, leading to nutrient deficiencies and reduced plant growth.
* **Soil degradation:** Pesticides can cause soil erosion, compaction, and altered soil structure, weakening the soil’s ability to retain nutrients.
* **Reduced plant health:** Pesticides can suppress plant growth, leading to stunted development, reduced yields, and increased susceptibility to disease.
* **Soil contamination:** Pesticides can contaminate soil with harmful chemicals, which can harm soil microorganisms, leading to soil degradation and reduced nutrient availability.
* **Impact on plant physiology:** Pesticides can affect plant physiology, including nutrient uptake, metabolism, and stress tolerance.
* **Altered soil pH:** Pesticides can alter soil pH, which can affect the availability of essential nutrients.
This quick experiment shows small AI models can provide practical solutions. By using efficient models like Gemma and smart techniques like LoRA, we can build tools that understand various local contexts.
The power to build specialized AI is here, and I’m excited to see what people will build in my region.
For those interested in the technical details, I used the Hugging Face Transformers library to handle the model and the PEFT library’s implementation of LoRA for efficient training. You can learn more about them at the links below:
* For the Hugging Face `transformers` library: This is the main documentation, the central hub for everything related to the library.
* For LoRA and PEFT (Parameter-Efficient Fine-Tuning): This link goes directly to the Hugging Face documentation for the peft library, which is what you used to implement LoRA.
๐คฏ This incredible image showcases the stunning beauty and diversity of the African continent, generated using the new AlphaEarth Foundations dataset on Google Earth Engine. So what is this dataset all about?ย
Imagine being able to X-ray the entire Earth across multiple years, even seeing through clouds! Dealing with clouds in remote sensing is a huge challenge (something I know well from my Open Buildings research project). The AlphaEarth team has essentially created a “virtual satellite” capable of doing just that. To achieve this, the AlphaEarth team combined vast amounts of data from dozens of public sources, including optical satellite images, radar, 3D laser mapping, etc.. weaving it all into a seamless picture.
Even after just a few minutes of exploring the dataset, I’ve stumbled upon fascinating insights. For example, why have Central Mali or Lake Kalala in Zambia changed so much? There’s likely a clear explanation, though I don’t know it yet.
This open dataset release is a huge step forward, likely to help scientists and experts make more informed decisions on critical global issues like food security, deforestation, urban expansion, and water resources.
If you think you can leverage this dataset for your research on our changing world, consider applying for the Satellite Embedding Grant. (Link below)
For those in genomic research: Google DeepMind has released AlphaGenome, an AI model for predicting DNA sequences. The API is free for non-commercial research use.
Feel free to share this with anyone in the field who might be interested.
(Note: This post reflects my personal opinions and may not reflect those of my employer)
Example of the HeAR encoder that generates a machine learning representation (known as “embeddings”)
This image is a spectrogram representing my name, “Abdoulaye,” generated from my voice audio by HeAR (Health Acoustic Representations). HeAR is one of the recently released Health AI foundation models by Google. I’ve been captivated by these foundation models lately, spending time digging into them, playing with the demos and notebooks, reading ML papers about the models, and also learning more about embeddings in general and their usefulness in low-resource environments. All of this started after playing with a couple of the notebooks.
Embeddings are numerical representations of data. AI models learn to create these compact summaries (vectors) from various inputs like images, sounds, or text, capturing essential features. These information-rich numerical representations are useful because they can serve as a foundation for developing new, specialized AI models, potentially reducing the amount of task-specific data and development time required. This efficiency is especially crucial in settings where large, labeled medical datasets may be scarce.
If you would like to read further into what Embeddings are, Vicki Boykis’ essay is such a great free resource; this essay is also ideal to learn or dive into machine learning. I know many of my previous colleagues from the telco and engineering world will love this: https://vickiboykis.com/what_are_embeddings/
The HeAR model, which processed my voice audio, is trained on over 300 million audio clips (e.g., coughs, breathing, speech). Its application can extend to identifying acoustic biomarkers for conditions like TB or COVID-19. It utilizes a Vision Transformer (ViT) to analyze spectrograms. Below, you can see an example of sneezing being detected within an audio file, and later, throat clearing detected at the end.
This release also includes other open-weight foundation models, each designed to generate high-quality embeddings:
Derm Foundation (Skin Images) This model processes dermatology images to produce embeddings, aiming to make AI development for skin image analysis more efficient by reducing data and compute needs. It facilitates the development of tools for various tasks, such as classifying clinical conditions or assessing image quality.
Explore the Derm Foundation model site for more information and to download the model use this link.
CXR Foundation (Chest X-rays) The CXR Foundation model produces embeddings from chest X-ray images, which can then be used to train models for various chest X-ray related tasks. The models were trained on very large X-ray datasets. What got my attention, some models within the collection, like ELIXR-C, use an approach inspired by CLIP (contrastive language-image pre-training) to link images with text descriptions, enabling powerful zero-shot classification. This means the model might classify an X-ray for a condition it wasn’t specifically trained on, simply by understanding a text description of that condition which i find fascinating. The embeddings generated can also be used to train models that can detect diseases like tuberculosis without a large amount of data; for instance, โmodels trained on the embeddings derived from just 45 tuberculosis-positive images were able to achieve diagnostic performance non-inferior to radiologists.โ This data efficiency is particularly valuable in regions with limited access to large, labeled datasets. Read the paper for more details.
Path Foundation (Pathology Slides) Google’s Path Foundation model is trained on large-scale digital pathology datasets to produce embeddings from these complex microscopy images. Its primary purpose is to enable more efficient development of AI tools for pathology image analysis. This approach supports tasks like identifying tumor tissue or searching for similar image regions, using significantly less data and compute. See the impressive Path Foundation demos on HuggingFace.
Path foundation demos
Outlier Tissue Detector Demo
These models are provided as Open Weight with the goal of enabling developers and researchers to download and adapt them, fostering the creation of localized AI tools. In my opinion, this is particularly exciting for regions like Africa, where such tools could help address unique health challenges and bridge gaps in access to specialist diagnostic capabilities.
For those interested in the architectural and training methodologies, here are some of the pivotal papers and concepts relevant to these foundation models:
Vision Transformer (ViT): Applied in HeAR and Path Foundation. (An Image is Worth 16×16 Words: https://arxiv.org/abs/2010.11929)
Masked Autoencoders (MAE): A self-supervised learning technique used for HeAR. (Masked Autoencoders Are Scalable Vision Learners: https://arxiv.org/abs/2111.06377)
EfficientNet: The family of architectures related to the backbone for CXR Foundation models. (EfficientNet: https://arxiv.org/pdf/1905.11946)
How do you manage ML projects? ๐คย A question I hear often! Working in research over the years, I often got asked about the day-to-day of managing machine learning projects. That’s why I’m excited about Google’s new, FREE “Managing ML Projects” guide which I can now point to going forward. it’s only 90 minutes but a good start!
It can be useful for:
* Those entering the ML field ๐: Providing a clear, structured approach. * Professionals seeking to refine their ML project management skills. * Individuals preparing for ML-related interviews: Offering practical insights and frameworks.
This guide covers:
* ML project lifecycle management. * Applying established project management principles to ML. * Navigating traditional and generative AI projects. * Effective stakeholder collaboration.
If you’re curious about ML project management, or want to level up your skills, take a look!
Google DeepMind has released SigLIP 2, a family of Open-weight (Apache V2) vision-language encoders trained on data covering 109 languages, including Swahili. The released models are available in four sizes: ViT-B (86M), L (303M), So400m (400M), and g (1B).
Why is this important?
This release offers improved multilingual capabilities, covering 109 languages, which can contribute to more inclusive and accurate AI systems. It also features better image recognition and document understanding. The four model sizes offer flexibility and potentially increased accessibility for resource-constrained environments.
Credits: "SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Features" by Michael Tschannen, Alexey Gritsenko, Xiao Wang, Muhammad Ferjad Naeem, Ibrahim Alabdulmohsin, Nikhil Parthasarathy, Talfan Evans, Lucas Beyer, Ye Xia, Basil Mustafa, Olivier Hรฉnaff, Jeremiah Harmsen, Andreas Steiner, and Xiaohua Zhai (2025).
