AI Daily · 2026-10-20

Two threads stand out: model capabilities reaching deeper into legacy systems and faster on-device inference. OpenAI reported strong gains in low-resource programming languages, with GPT-6 Astra averaging 87.4% on MultiLangBench and COBOL rising from 10.9% to 72.1%, while GPT-6.1 Sol neared saturation on COBOL migration. Hugging Face and the PyTorch community shipped MPS improvements in PyTorch 2.15, making MoE decoding 5–6.5× faster than the previous release, adding FP8 support, and fixing silent image corruption in Qwen-Image-2.1 VAE. Elsewhere, NVIDIA ran a record fault-tolerant quantum computing simulation on JUPITER, and AnyLanguageModel 1.0 unified local and hosted model access on Apple platforms. Evaluation practice also drew attention: Numan moved to scoring only what a single turn can evidence, while Hamel Husain argued decision-model evals need dataset-specific threshold tuning instead of the default 0.50.

North America · First-hand

OpenAI

⭐⭐⭐ [Research] Advancing coding in low‑resource programming languages

OpenAI published a research post describing stronger model capabilities for building and migrating software in low-resource programming languages such as COBOL, Fortran, and Visual Basic, which have far less code and documentation available than Python or JavaScript yet underpin critical infrastructure: COBOL supports over 40% of online banking systems, 80% of in-person credit card transactions and 95% of ATM transactions, while Fortran is used in weather forecasting and nuclear reactor safety analysis. To measure progress, OpenAI built two evaluations: MultiLangBench, which asks a model to build a SQLite-compatible database engine from scratch in 20 languages, and a COBOL migration evaluation that tests migrating existing programs to another language while preserving behavior. On MultiLangBench, GPT‑6 Astra reaches an average test pass rate of 87.4% versus 26.8% for GPT‑5.6 Sol, with gains in every language including COBOL (10.9% to 72.1%); GPT‑6 Luna reaches 40.3% at roughly 78% lower estimated API cost. On the COBOL migration evaluation, GPT‑6.1 Sol achieves 86.6% accuracy at low reasoning effort versus 5.4% for GPT‑5.6 Sol while costing 39% less, and reaches 97.9% at higher reasoning effort, nearly saturating the evaluation.

Ecosystem & Beyond (Products / Agents / Tools / Opinions)

Product Update

⭐⭐⭐ [Product Update] MPS in PyTorch 2.15

Hugging Face's PyTorch team, working with PyTorch maintainers and community contributors, detailed the MPS (Mac backend) changes in PyTorch 2.15. MoE models decode 5–6.5× faster than in 2.14: on an M3 Ultra with torch.compile, Qwen3.6-35B-A3B goes from 9.4 to 61.3 tokens/s and LFM2.5-8B-A1B from 23.6 to 123.6 tokens/s, attributed to a native grouped matmul with a dedicated decoding path, fewer GPU-CPU syncs and faster top-k routing. FP8 (float8_e4m3fn) arrives on MPS at half the memory of BF16 with scaled matrix multiplication, while dense models gain specialized matrix-vector kernels for single-row linear layers and an approximately 8× faster softmax over the vocabulary. Wan2.2's tiled VAE video decode drops from a 270 GiB driver-memory peak to about 8 GiB, and the same padding fix stops the Qwen-Image-2.1 VAE encoder from silently corrupting images, raising an encode-decode round trip to 29.95 dB PSNR versus CPU, up from 10.06 dB. The release also adds complex LU and triangular solves, faster QR decomposition, public MPS streams and more reliable event timing, requires macOS 15+ and Python 3.11+, and previews fully-Metal attention in 2.16.

⭐⭐ [Product Update] AnyLanguageModel 1.0: One Year of Local Models on Apple Platforms

AnyLanguageModel 1.0 is out, alongside swift-huggingface 1.0 and swift-xet 1.0. The library is a drop-in replacement for Apple's Foundation Models framework: changing the import from FoundationModels to AnyLanguageModel lets the same session code run against the system model, open-weight models via llama.cpp, MLX and Core ML, and a range of hosted providers. Features added over the past year include tool calling and guided generation for every provider (including nested @Generable types and arrays of them), image attachments using Foundation Models 27 syntax, and support on OS 27 for any model conforming to the LanguageModel protocol, including Core AI models. Hosted support now covers OpenAI's Chat Completions and Responses APIs, Open Responses, Anthropic's Messages API with adaptive thinking and effort options, Google's Gemini API, Ollama, Private Cloud Compute on OS 27, and OpenAI-compatible endpoints such as Hugging Face Inference Providers and OpenRouter via a custom base URL. Reasoning is now part of the transcript, API keys come from a closure so apps can fetch short-lived tokens, and the library also runs on Linux; the post also lists popular Hub models such as Gemma 4 E2B (about 3.4 GB at Q4_0), LFM2.5 2.6B (about 1.7 GB at Q4_K_M) and Qwen3.8 27B (about 16 GB at 4 bits).

Research

⭐⭐ [Research] NVIDIA Runs Record Simulation of Fault-Tolerant Quantum Computing at Jülich Supercomputing Centre

NVIDIA announced a record-breaking quantum computing simulation on JUPITER, Europe's first exascale supercomputer, located at the Jülich Supercomputing Centre. The simulation lets researchers fully explore and validate magic state cultivation, described as a leading approach to fault-tolerant quantum computing. NVIDIA says the approach requires 90% fewer qubits than previous methods.

⭐⭐ [Research] How To Use Decision Models for Evals

The post explains how to use decision models for automated evals, naming TypeSafe's Jev, Perplexity's Decider, Liquid's d1, Cloudflare's Clef and OpenAI's Decisions API as low-cost alternatives to LLM judges. The author argues that accepting the APIs' default 0.50 decision threshold without checking it against your own data can leave a lot of performance on the table, and that thresholds should be tuned by trading off precision against recall (measured here with F1, which weights both equally). The experiment used 100 traces from an apartment leasing agent to find traces containing errors, comparing Jev 1.13, Perplexity Decider 1.1 27B, Liquid d1 and d1-ultra, Cloudflare Clef Flash and OpenAI Decisions (GPT-6 Luna) against human pass/fail labels, with API cost computed from token usage and provider prices; latency was left aside because it depends on factors outside the experiment's control. The post closes by discussing how to interpret model scores and how calibration makes them more useful as confidence estimates.

Other

⭐⭐ [Other] How Numan Built an Eval-First AI Health Coach with LangGraph and LangSmith

Numan is a UK digital health company whose AI health coach Nu helps patients manage long-term conditions across weight, hormones and sexual health, while staying outside medical diagnosis, prescriptions and clinical decisions. The team started with static single-turn fixtures and an LLM judge, but found that single-reply scoring could not determine safety in a conversation and that averaging one score across a dataset hid localized failures; pre-scripted multi-turn conversations also ignored what Nu actually said in earlier turns. Nu is now a LangGraph graph in which a supervisor model hands off to domain-specialist sub-agents. For behavior decidable from a single reply, the team runs themed datasets covering coaching quality, safety, medication queries, routing and summarisation, scores each category separately with an LLM judge, and limits each rubric to what a single reply can show so that coach-like pacing is not penalized.