Beyond Short-term Memory: The 3 Types of Long-term Memory AI Agents Need
In this article, you will learn why short-term context isn’t enough for autonomous agents and…
Category: AI
Pretraining a Llama Model on Your Local GPU
import dataclasses import os import datasets import tqdm import tokenizers import torch import torch.nn…
3 Smart Ways to Encode Categorical Features for Machine Learning
In this article, you will learn three reliable techniques — ordinal encoding, one-hot encoding, and…
Evaluating Perplexity on Language Models
A language model is a probability distribution over sequences of tokens. When you train a…
Practical Agentic Coding with Google Jules
Practical Agentic Coding with Google JulesImage by Editor Introducing Google Jules If you have an…
Training a Model on Multiple GPUs with Data Parallelism
import dataclasses import os import datasets import tqdm import tokenizers import torch import torch.distributed…
Train a Model Faster with torch.compile and Gradient Accumulation
Training a language model with a deep transformer architecture is time-consuming. However, there are techniques…
Training a Model with Limited Memory using Mixed Precision and Gradient Checkpointing
Training a language model is memory-intensive, not only because the model itself is large but…
Rotary Position Embeddings for Long Context Length
Rotary Position Embeddings (RoPE) is a technique for encoding token positions in a sequence. It…
The Machine Learning Engineer’s Checklist: Best Practices for Reliable Models
The Machine Learning Engineer’s Checklist: Best Practices for Reliable ModelsImage by Editor Introduction Building newly…