postgres 源码解析29 内存管理--1_serendipity_shy的博客-爱代码爱编程
背景
在绝大多数据库系统中,其存储管理涉及的问题本质上是一致的:尽可能减少IO操作次数。因此,尽可能让最近访问过的文件块停留在内存中,有效减小IO的代价。合理有效的内存管理对整个数据库系统来说是非常最要的。postgreSQL中的内存管理包括共享内存和本地内存的管理(如图所示)接下里几小节将从源码角度进行学习。
内存上下文
** 1 简介**
内存上下文借鉴操作系统中进程环境的概念。在程序运行的时,操作系统会为每个进程分配执行环境,进程环境之间不会互相影响,由操作系统进行上下文切换,进程可以在其环境调用库函数进行内存分配与回收等。postgreSQL采用类似的方法通过内存上下文来实现内存管理。
** 2 MemoryContext 结构**
PostgreSQL中的每一个backend process都有自己的私有上下文,且该内存上下文为树形结构(如下图),根节点为 TopMemoryContext,其下包含多个子节点,每个子结点负责不同的功能:CacheMemoery用于管理Cache、ErrorContext用于错误处理等,每个子结点根据实际工作划分为多个子节点,
申请新的内存上下文时只需将其添加至某个内存上下文的子节点。释放时需要从根节点遍历找到指定待删除的上下文。
2.1 关键数据结构
2.1.1 MemoryContextData
MemoryContextData结构体包含上下文类型,内存上下文操作函数以及该内存上下文父节点、兄弟节点等信息。
typedef struct MemoryContextData
{
NodeTag type; /* identifies exact kind of context */
/* these two fields are placed here to minimize alignment wastage: */
bool isReset; /* T = no space alloced since last reset */
bool allowInCritSection; /* allow palloc in critical section */
Size mem_allocated; /* track memory allocated for this context */
const MemoryContextMethods *methods; /* virtual function table */
MemoryContext parent; /* NULL if no parent (toplevel context) */
MemoryContext firstchild; /* head of linked list of children */
MemoryContext prevchild; /* previous child of same parent */
MemoryContext nextchild; /* next child of same parent */
const char *name; /* context name (just for debugging) */
const char *ident; /* context ID if any (just for debugging) */
MemoryContextCallback *reset_cbs; /* list of reset/delete callbacks */
} MemoryContextData;
typedef struct MemoryContextData *MemoryContext;
2.1.2 MemoryContextMethods
MemoryContextMethods 结构体由一系列函数指针组成,包括分配/释放删除等操作。
typedef struct MemoryContextMethods
{
void *(*alloc) (MemoryContext context, Size size);
/* call this free_p in case someone #define's free() */
void (*free_p) (MemoryContext context, void *pointer);
void *(*realloc) (MemoryContext context, void *pointer, Size size);
void (*reset) (MemoryContext context);
void (*delete_context) (MemoryContext context);
Size (*get_chunk_space) (MemoryContext context, void *pointer);
bool (*is_empty) (MemoryContext context);
void (*stats) (MemoryContext context,
MemoryStatsPrintFunc printfunc, void *passthru,
MemoryContextCounters *totals,
bool print_to_stderr);
void (*check) (MemoryContext context);
} MemoryContextMethods;
在PostgreSQL中使用全局变量CurrentMemoryContext表示当前工作内存上下文,在需要切换上下文,调用 MemoryContextSwitchTo 函数切换至目标上下文
2.1.3 AllocSetContext
AllocSetContext结构体包含 header信息、该内存上下文所有内存块链表头、空闲内存片数组、块大小等信息
typedef struct AllocSetContext
{
MemoryContextData header; /* Standard memory-context fields */
/* Info about storage allocated in this context: */
AllocBlock blocks; /* head of list of blocks in this set */
AllocChunk freelist[ALLOCSET_NUM_FREELISTS]; /* free chunk lists */
/* Allocation parameters for this context: */
Size initBlockSize; /* initial block size */
Size maxBlockSize; /* maximum block size */
Size nextBlockSize; /* next block size to allocate */
Size allocChunkLimit; /* effective chunk size limit */
AllocBlock keeper; /* keep this block over resets */
/* freelist this context could be put in, or -1 if not a candidate: */
int freeListIndex; /* index in context_freelists[], or -1 */
} AllocSetContext;
typedef AllocSetContext *AllocSet;
2.1.4 AllocBlockData
AllocBlockData结构体记录了该内存块所在上下文、该块前后块信息、该块空闲区域首地址和该块的结束地址
typedef struct AllocBlockData
{
AllocSet aset; /* aset that owns this block */
AllocBlock prev; /* prev block in aset's blocks list, if any */
AllocBlock next; /* next block in aset's blocks list, if any */
char *freeptr; /* start of free space in this block */
char *endptr; /* end of space in this block */
}AllocBlockData;
typedef struct AllocBlockData *AllocBlock;
2.1.5 AllocChunk
typedef struct AllocChunkData
{
/* size is always the size of the usable space in the chunk */
Size size; // 内存片可使用空间
Size requested_size; // 请求内存大小
/* aset is the owning aset if allocated, or the freelist link if free */
void *aset; // 该内存片所在 AllocSet
/* there must not be any padding to reach a MAXALIGN boundary here! */
} AllocChunkData;
为方便理解,将上述数据数据结构衔接,可得如下图【转载PostgreSQL数据库内核分析书籍】
2.1.6 FreeList数组
Freelist数组中的每一个元素指向一个特定"大小"空闲内存片组成的链表,这个“大小”与该元素所在数组中的顺序有关。比如,Freelist数组中第K个元素所指向链表的每个空闲数据块大小的2 <<(k + 2)字节,空闲内存片大小为8B,最大内存片大小为 8K.
/*--------------------
* Chunk freelist k holds chunks of size 1 << (k + ALLOC_MINBITS),
* for k = 0 .. ALLOCSET_NUM_FREELISTS-1.
*
* Note that all chunks in the freelists have power-of-2 sizes. This
* improves recyclability: we may waste some space, but the wasted space
* should stay pretty constant as requests are made and released.
*
* A request too large for the last freelist is handled by allocating a
* dedicated block from malloc(). The block still has a block header and
* chunk header, but when the chunk is freed we'll return the whole block
* to malloc(), not put it on our freelists.
*
* CAUTION: ALLOC_MINBITS must be large enough so that
* 1<<ALLOC_MINBITS is at least MAXALIGN,
* or we may fail to align the smallest chunks adequately.
* 8-byte alignment is enough on all currently known machines.
*
* With the current parameters, request sizes up to 8K are treated as chunks,
* larger requests go into dedicated blocks. Change ALLOCSET_NUM_FREELISTS
* to adjust the boundary point; and adjust ALLOCSET_SEPARATE_THRESHOLD in
* memutils.h to agree. (Note: in contexts with small maxBlockSize, we may
* set the allocChunkLimit to less than 8K, so as to avoid space wastage.)
*--------------------
*/
#define ALLOC_MINBITS 3 /* smallest chunk size is 8 bytes */
#define ALLOCSET_NUM_FREELISTS 11
#define ALLOC_CHUNK_LIMIT (1 << (ALLOCSET_NUM_FREELISTS-1+ALLOC_MINBITS))
/* Size of largest chunk that we use a fixed size for */
#define ALLOC_CHUNK_FRACTION 4
/* We allow chunks to be at most 1/4 of maxBlockSize (less overhead) */
参考:PostgreSQL数据库内核分析