https://mouhssine.rifaki.me/2026-08-211.0 https://mouhssine.rifaki.me/og/home.pngexploring how autonomy and coordination keep us safe https://mouhssine.rifaki.me/notes/2026-08-210.9 https://mouhssine.rifaki.me/about/2026-08-210.5 https://mouhssine.rifaki.me/contact/2026-08-210.4 https://mouhssine.rifaki.me/notes/diffusion-models/2026-08-210.8 https://mouhssine.rifaki.me/notes/img/diffusion-score-field.pngSliced score matching loss trajectories on a toy problem: the unconstrained estimator diverges to large negative values while the noise-conditioned variant stays bounded over train https://mouhssine.rifaki.me/notes/img/diffusion-sde-ode-map.pngFigure 1 of Song et al. 2020. Forward SDE turning data into noise (top) and the score-driven reverse SDE turning noise back into data (bottom), with intermediate sample crops at in https://mouhssine.rifaki.me/notes/img/diffusion-forward-reverse.pngFigure 2 of Ho, Jain, Abbeel 2020 (DDPM). Directed graphical model of the forward and reverse chains between x_T and x_0. https://mouhssine.rifaki.me/notes/img/diffusion-design-space.pngTable 1 of Karras et al. 2022 (EDM). Explicit design-space tabulation of noise schedules, prediction targets, preconditioning, and samplers across DDPM, NCSN, EDM, and related vari https://mouhssine.rifaki.me/notes/adversarial-examples/2026-08-210.8 https://mouhssine.rifaki.me/notes/img/adversarial-fgsm.pngFigure 1 of Goodfellow, Shlens, Szegedy 2015. Panda image plus epsilon times sign of the gradient produces an imperceptible perturbation that flips the classifier's prediction to g https://mouhssine.rifaki.me/notes/img/adversarial-minmax-loop.pngFigure 1 of Madry et al. 2018. PGD attack-loss curves over inner-maximization iterations on standard- and adversarially-trained MNIST and CIFAR10 networks: the standard models reac https://mouhssine.rifaki.me/notes/img/adversarial-feature-taxonomy.pngFigure 1 of Ilyas et al. 2019. Robust versus non-robust feature decomposition: standard models exploit non-robust features that are predictive but human-imperceptible. https://mouhssine.rifaki.me/notes/img/adversarial-robust-optimization.pngAdversarial-training loss curves of Madry et al. 2018: PGD-adversarial training loss decays from the initial saddle-point value over 100k MNIST iterations and 75k CIFAR10 iteration https://mouhssine.rifaki.me/notes/mode-connectivity/2026-08-210.8 https://mouhssine.rifaki.me/notes/img/mode-connectivity-paths.pngFigure 1 of Garipov et al. 2018. Loss-landscape view of a low-loss curve connecting two independently trained SGD minima, while the straight segment between them rises through a hi https://mouhssine.rifaki.me/notes/img/mode-connectivity-spawning.pngFigure 3 of Frankle, Dziugaite, Roy, Carbin 2020. Linear interpolation curves for spawn-then-fork SGD pairs at varying late-rewinding points: late enough forks remain linearly conn https://mouhssine.rifaki.me/notes/img/mode-connectivity-simplex.pngFigure 1 of Ainsworth, Hayase, Srinivasa 2023 (Git Re-Basin). Linear-interpolation barrier between two independently trained networks before and after permutation alignment. https://mouhssine.rifaki.me/notes/img/mode-connectivity-rebasin.pngFigure 2 of Ainsworth, Hayase, Srinivasa 2023. Linear interpolation barriers between two independently trained networks across MNIST/CIFAR-10/ImageNet under naive, activation-match https://mouhssine.rifaki.me/notes/neural-tangent-kernel/2026-08-210.8 https://mouhssine.rifaki.me/notes/img/ntk-linearization.pngFigure 2 of Lee et al. 2019. Predictions from the linearized infinite-width model match the trajectory of the actual wide finite network during gradient-descent training. https://mouhssine.rifaki.me/notes/img/ntk-spectrum.pngFigure 1 of Cao et al. 2019 (spectral bias of deep learning). Projection lengths along the lowest few eigenmodes of the NTK as a function of training step: low-frequency (small k) https://mouhssine.rifaki.me/notes/img/ntk-spectrum-timeline.pngFigure 2 of Bordelon, Canatar, Pehlevan 2020. Spectrum-dependent generalization-error scaling: per-mode learning curves $E_k(p)/E_k(0)$ versus number of training samples for varyin https://mouhssine.rifaki.me/notes/img/ntk-lazy-vs-feature-learning.pngFigure 1 of Chizat, Oyallon, Bach 2019. Lazy regime versus feature-learning regime trajectories on a 2D classification problem: the lazy regime stays near initialization while feat https://mouhssine.rifaki.me/notes/implicit-bias/2026-08-210.8 https://mouhssine.rifaki.me/notes/img/implicit-bias-margin.pngFigure 1 of Soudry et al. 2018. Five-panel layout: (A) 2D separable data with the converged separator, (B) normalized weight norm growing logarithmically, (C) logistic loss decayin https://mouhssine.rifaki.me/notes/img/implicit-bias-margin-flow.pngFigure 2 of Soudry et al. 2018. Three panels on a real classification dataset: training/validation objective loss, classification error, and L2 norm of the final layer growing as t https://mouhssine.rifaki.me/notes/img/implicit-bias-optimizer-geometry.pngThree-panel figure from Gunasekar, Lee, Soudry, Srebro 2018: (a) mirror descent with primal momentum, (b) natural gradient descent at varying step sizes, (c) steepest descent under https://mouhssine.rifaki.me/notes/img/implicit-bias-dynamics.pngFigure 1 of Lyu and Li 2020. Training loss and normalized margin trajectories for homogeneous networks under fixed and loss-based learning rates: the loss collapses while the norma https://mouhssine.rifaki.me/notes/edge-of-stability/2026-08-210.8 https://mouhssine.rifaki.me/notes/img/cohen-fig1.pngFigure 1 of Cohen et al. 2021. Train loss (top row) and Hessian sharpness (bottom row) over training steps for a fully-connected net on a CIFAR-10 5k subset, VGG on CIFAR-10, and R https://mouhssine.rifaki.me/notes/img/cohen-fig3.pngFigure 3 of Cohen et al. 2103.00065. Progressive sharpening isolated; sharpness rises before reaching the 2/eta threshold. https://mouhssine.rifaki.me/notes/img/arora-fig1.pngFigure 1 of Arora et al. 2205.09745. Smoothed-loss analysis of the edge-of-stability oscillations. https://mouhssine.rifaki.me/notes/neural-scaling-laws/2026-08-210.8 https://mouhssine.rifaki.me/notes/img/kaplan-fig1.pngFigure 1 of arxiv 2001.08361. Test loss plotted against compute on log-log axes, showing a power-law fit over many orders of magnitude. https://mouhssine.rifaki.me/notes/img/hoffmann-fig3.pngFigure 3 of Hoffmann et al. 2022 (Chinchilla). Left: training loss versus parameter count for fixed FLOP budgets from 6e18 up to 3e21, each forming a U-shape with a clear minimum. https://mouhssine.rifaki.me/notes/img/caballero-fig1.pngFigure 1 of Caballero et al. 2022. Annotated example of a Broken Neural Scaling Law (BNSL) functional form, marking three break points and four slope regimes between them as the pe https://mouhssine.rifaki.me/notes/img/caballero-fig2.pngFigure 2 of Caballero et al. 2022. Two real-task BNSL fits: top panel ImageNet 25-shot test error versus training-dataset size; bottom panel TriviaQA few-shot test accuracy versus https://mouhssine.rifaki.me/notes/lottery-tickets/2026-08-210.8 https://mouhssine.rifaki.me/notes/img/frankle-fig3.pngFigure 3 of Frankle and Carbin 2019. Test accuracy versus training iterations on Lenet-MNIST for lottery tickets at sparsity levels 100%, 51.3%, 21.1%, 7.0%, 3.6%, 1.9% remaining w https://mouhssine.rifaki.me/notes/img/liu-fig2.pngFigure 2 of Liu et al. 2019. Schematic distinguishing predefined pruning (uniform x% per layer) from automatic pruning (per-layer percentages a%, b%, c%, d% chosen by the algorithm https://mouhssine.rifaki.me/notes/img/frankle-lmc-fig3.pngFigure 3 of Frankle, Dziugaite, Roy, Carbin 2020. Linear-interpolation instability versus fork step k across LeNet (MNIST), ResNet-20 (CIFAR-10), VGG-16 (CIFAR-10), ResNet-50 (Imag https://mouhssine.rifaki.me/notes/double-descent/2026-08-210.8 https://mouhssine.rifaki.me/notes/img/belkin-fig1.pngFigure 1 of Belkin et al. 2018. Schematic of test risk as a function of model capacity: the classical U-shape to the left of the interpolation threshold and a second descending bra https://mouhssine.rifaki.me/notes/img/nakkiran-fig1.pngFigure 1 of Nakkiran et al. 2019. Test error and train error versus ResNet18 width parameter under varying label-noise levels (0%, 5%, 10%, 15%, 20%). Test error peaks near the int https://mouhssine.rifaki.me/notes/img/nakkiran-fig4.pngFigure 4 of Nakkiran et al. 2019. Left panel labels the classical (under-parameterized) and modern (over-parameterized) regimes around the interpolation threshold; right panel over https://mouhssine.rifaki.me/notes/information-bottleneck/2026-08-210.8 https://mouhssine.rifaki.me/notes/img/tishby-fig2.pngFigure 2 of Tishby and Zaslavsky 2015. Qualitative information plane: optimal IB limit (black), suboptimal bifurcations (blue), finite-sample distortion bound (red), and a possible https://mouhssine.rifaki.me/notes/img/saxe-fig1.pngFigure 1 of Saxe et al. ICLR 2018. Four information-plane panels (A, B, C, D). Top row uses a small toy network with the binning estimator: (A) tanh nonlinearity reproduces the Shw https://mouhssine.rifaki.me/notes/img/goldfeld-fig1.pngFigure 1 of Goldfeld et al. 2019. Estimated $I(X; \mathrm{Bin}(T_\ell))$ over training epochs for layers 1-5 at four binning resolutions (bin size 0.0001, 0.001, 0.01, 0.1). The ap https://mouhssine.rifaki.me/notes/img/goldfeld-fig2.pngFigure 2 of Goldfeld et al. 2019. Architectural diagram of the noisy DNN: $T_{\ell-1}$ feeds through $\sigma(W_\ell^{(k)} T_{\ell-1} + b_\ell^{(k)})$ to produce a pre-noise hidden https://mouhssine.rifaki.me/notes/flat-minima/2026-08-210.8 https://mouhssine.rifaki.me/notes/img/keskar-fig1.pngFigure 1 of Keskar et al. [1]. A 1D schematic of a wide basin around one minimum and a narrow basin around another. https://mouhssine.rifaki.me/notes/img/dinh-fig1.pngFigure 1 of Dinh et al. 2017. Schematic of an $\epsilon$-flat minimum: a parabolic loss curve in $(\theta, L)$ with a horizontal cutoff at level $\epsilon$ above the minimum, shadi https://mouhssine.rifaki.me/notes/img/foret-fig1.pngFigure 1 of Foret et al. 2020 (SAM). Left: percent error reduction from SAM across CIFAR10, CIFAR100, ImageNet, finetuning, SVHN, F-MNIST, and noisy CIFAR. Right: 3D loss landscape https://mouhssine.rifaki.me/notes/grokking/2026-08-210.8 https://mouhssine.rifaki.me/notes/img/grokking-fig1.pngFigure 1 of arxiv 2201.02177. Training and validation accuracy on modular arithmetic as a function of optimization step on a log scale. The validation curve stays at chance while t https://mouhssine.rifaki.me/notes/img/nanda-fig2.pngFigure 2 of Nanda et al. 2023. Left: histogram of fraction-of-variance-explained by degree-2 polynomials over neurons. Right: heatmap of components of $W_L$ corresponding to freque https://mouhssine.rifaki.me/notes/img/nanda-fig5.pngFigure 5 of arxiv 2301.05217. The Discrete Fourier Transform of the grokked network's input embeddings, showing concentration on a small set of frequencies. https://mouhssine.rifaki.me/notes/img/nanda-fig3.pngFigure 3 of Nanda et al. 2023. Average train accuracy (saturates near 1.0 within ~1k epochs), average test accuracy (stays at chance for ~5k epochs then jumps), and corresponding a