ML irl

More or less specific resources for machine learning which I may or may not keep up to date. For general or theoretical machine learning resources, see:

Using machine learning libraries

Best known: PyTorch, TensorFlow

PyTorch

… can be tricky to install so here are some potential starting points with tips:

pip install light-the-torch
ltt install torch

Aggregated data on ML applications

REALM Dataset Dashboard

Model development: training, evaluation

Domain-specific evals: code

Multi-SWE-bench

  • TL;DR 2025.04.14: LLMs are garbage in languages that aren’t Python
    • mostly garbage at Java
    • complete trash at Go, Rust, C, C++, Javascript, Typescript

Synthetic data

Synthetic data for privacy-preserving clinical risk prediction | Scientific Reports

Inference: using pretrained base models

Fine tuning IRL

Fine-tuning Guide with a 4090

e-p-armstrong/augmentoolkit: Convert Compute And Books Into Instruct-Tuning Datasets! Makes: QA, RP, Classifiers.

[2312.05934] Fine-Tuning or Retrieval? Comparing Knowledge Injection in LLMs

Inference: using deployed models (agents, bots, apps)

Agents

Recent reasoning research: GRPO tweaks, base model RL, and data curation

DeepSeek's open-source week and why it's a big deal | PySpur - AI Agent Builder

Prompt engineering

Prompt Engineering Guide | Prompt Engineering Guide

(Twisted) RAG

Roaming RAG – RAG without the Vector Database - Arcturus Labs

Synthetic data

Synthetic data for privacy-preserving clinical risk prediction | Scientific Reports

[2407.01490] LLM See, LLM Do: Guiding Data Generation to Target Non-Differentiable Objectives

[2408.14960] Multilingual Arbitrage: Optimizing Data Pools to Accelerate Multilingual Progress

This is AI's brain on AI

Data: ETL applications of AI

The overlooked GenAI use case

‘Learnable’ tasks (aka can AI/ML help?)

[2210.17011] A picture of the space of typical learnable tasks

Practical risks

Slopsquatting

[2406.10279] We Have a Package for You! A Comprehensive Analysis of Package Hallucinations by Code Generating LLMs

Risky Bulletin: AI slopsquatting... it's coming!

The Rise of Slopsquatting: How AI Hallucinations Are Fueling...

Etc

Training an LLM from scratch for personal use

Not really something an individual would be expected to be able to do? (And even just fine tuning is hard!) Though, if you have the compute, it can be attempted, it takes a lot of time and effort, and it’s probably not going to be great, though you can try a light pretrain on domain specific data with fine tune on instructions to maybe get okay one shot performance.

And here’s a full screen recording of someone training a llama.cpp mini-ggml-model from scratch with the script to train.

If you have ~$500k to do the training, you can use the MosaicML platform to get a GPT-3 quality model.

Model distillation

The unreasonable effectiveness of reasoning distillation, Bespoke Labs

(1) jack morris on X: "it's a baffling fact about deep learning that model distillation works method 1 - train small model M1 on dataset D method 2 (distillation) - train large model L on D - train small model M2 to mimic output of L - M2 will outperform M1 no theory explains this; it's magic" / X

ML for science

The AI Scientist Generates its First Peer-Reviewed Scientific Publication

Severe deviation in protein fold prediction by advanced AI: a case study | Scientific Reports

[2412.21154] Aviary: training language agents on challenging scientific tasks

How we evaluated Elicit Systematic Review

[2503.20511] From reductionism to realism: Holistic mathematical modelling for complex biological systems

AI skepticism

Who and What comprise AI Skepticism? - by Benjamin Riley

Timelines

AI Timeline - The Road to AGI

AI timelines: What do experts in artificial intelligence expect for the future? - Our World in Data

The History of Artificial Intelligence: Complete AI Timeline

Timeline of artificial intelligence - Wikipedia

The History of AI: A Timeline of Artificial Intelligence | Coursera

My AI Timelines Have Sped Up (Again)

AI Timeline - The Road to AGI

AI Timelines - LessWrong

The Timeline of Artificial Intelligence - From the 1940s to the 2020s

Google AI - Our AI journey and milestones

AI Timeline

Literature Review of Transformative Artificial Intelligence Timelines | Epoch AI

AI Timeline: Key Events in Artificial Intelligence from 1950-2025

Timelines to Transformative AI: an investigation — EA Forum

Timeline of AI timelines - Timelines

AI Timeline Surveys – AI Impacts

The A.I. Timeline is Accelerating... - YouTube

Timelines Forecast — AI 2027

Rising Tide | Helen Toner | Substack

Evaluating Large Language Models | Center for Security and Emerging Technology