Using the upper 32MB of memory

Using the upper 32MB of memory can be useful if you need to store more than 32MB of data or if you want to make use of the hardware blitter.

Accessing this upper 32MB can simply be done by mmaping /dev/mem. map_upper_mem() will return 0 when the mapping failed.

void* map_upper_mem()
{
    int fd = open("/dev/mem", O_RDWR);
    if (fd == -1) return 0;
    void *upper_mem = mmap(0, 32 * 1024 * 1024, PROT_READ | PROT_WRITE,
        MAP_SHARED, fd, 0x02000000);
    close(fd);
    if (upper_mem == MAP_FAILED)
        return 0;
    return upper_mem;
}

void unmap_upper_mem(void *mem)
{
    munmap(mem, 32 * 1024 * 1024);
}

The file descriptor can be closed after mmap since mmap keeps a reference. To unmap the upper memory a call to unmap_upper_mem is enough.

This will give you 32MB of memory in the UpperMem variable. Sort of equal to:

UpperMem = malloc(0x2000000);

If you also want to test under windows/linux then you can put "#ifdef GP2x/#else/#endif" around it, so you end up with malloced mem on windows/linux and the upper memory on the GP2x.

There you have 32MB of memory accessable by a pointer. So if you want to acces the upper memory at 0x3000000 now. You can simply use UpperMem[0x1000000].

Upper memory used by other parts

Not all the memory in the upper 32MB is free to use. Some parts are used as followed:

Address	Length	Usage
0x03000000	342812 bytes (1)	Video decoding firmware.
0x03101000	153600 bytes	Primary frame buffer
0x03381000	153600 bytes	Secondary frame buffer
0x03600000	16384 bytes (2)	Sound buffer
0x03D00000	up to 0x03FFFFFF	Reserved for internal buffers of MPEG H/W decoder

(1) currently 342812 bytes, but may change in size with various firmware releases)

(2) Size depends on multiple factors, mostly on how much you or your library writes to /dev/dsp. Applies for any sound output method.

The kernel source code defines the upper memory map as:

Region	Name	Notes
0x03000000-0x03100000	PA_DUALCPU	Video decoding binary for ARM940 is located at very beginning of the region. If you overwrite that area, the video player will hang instead of starting playing. Can be used if you don't care about the video player.
0x03100000-0x03380000	PA_FB0	Primary framebuffer. Your program is free to use any other location for the framebuffer and use this location for anything else.
0x03380000-0x03600000	PA_FB1	Secondary framebuffer. Same as above.
0x03600000-0x03680000	PA_SOUND_DMA	This is the only location I actually found unusable. 0x03600000 is the hardcoded address where sound samples are copied to (via DMA). Whatever you write to /dev/dsp (directly or indirectly using some high level lib like SDL) will end up there. So if you store anything at the beginning of this region, it will get overwritten. This was causing glitches for some games in PicoDrive <= 0.32, because it was carelessly using this region.
0x03700000-0x03a00000	PA_CAM	This area is used by some leftover code in the kernel, which was apparently written for OV9640 Image Sensor. As GP2X has no such sensor, it should be safe to use this region.

0x02000000-0x03000000 and 0x03a00000-0x04000000 regions should be safe to use, haven't noticed any kernel code using them.

Speed issue

Normally the upper 32MB is uncached. This means that reads/writes on the memory are always done via the physical memory modules rather than the much faster memory built into the processor(called 'cache'). Access to the upper 32MB can be sped up by Squidge's MMU hack. The easiest way to use the MMU hack is to add and load the MMU hack kernel module into your program:

Download the module: http://archive.gp2x.de/cgi-bin/cfiles.cgi?0,0,0,0,46,1690

And some example code on how to apply it:

int mmuhack(void)
{
    int mmufd;
    system("/sbin/rmmod mmuhack");
    system("/sbin/insmod mmuhack.o");
    mmufd = open("/dev/mmuhack", O_RDWR);
    if(mmufd < 0) return 0;

    close(mmufd);
    return 1;
}

void mmuunhack(void)
{
    system("/sbin/rmmod mmuhack");
}

Note: if you downloaded the module before 25 May 2007 and used it without modification, you need to redownload it, because the older one was only hacking 0x02000000-0x02010000 region. The newer one can hack the entire upper memory. It may be also useful to read the newly included documentation. If the above call succeeded, you are all done.

Do not forget to unload the module when your program exits, because the other program may want to load a different mmuhack.o and may fail, because you left your mmuhack.o loaded (it does not get unloaded automatically on exit).

// apply mmuhack
mmuhack();

// make sure it is released on exit
atexit(mmuunhack);

There is also a RAM Hack which may improve performance.

Using the upper memory like malloc/free

This code isn't super or anything. But if you need an example on how to use the upper memory as an extra memory pool, then this could help.

It cuts the upper memory in BLOCKSIZE parts, and allows you to use the UpperMalloc and UpperFree functions like normal Malloc and Free functions.

Call the InitMemPool() at the start of your program, and the DestroyMemPool() at the end of your program.

#include <stdlib.h>
#include <stdio.h>
#include <fcntl.h>
#include <sys/mman.h>
#include <linux/fb.h>
#include <unistd.h>
#include <stropts.h>

int Uppermemfd;
void *UpperMem;

#define BLOCKSIZE 1024

int TakenSize[0x2000000 / BLOCKSIZE];

#define SetTaken(Start, Size) TakenSize[(Start - 0x2000000) / BLOCKSIZE] = (Size - 1) / BLOCKSIZE + 1

void * UpperMalloc(size_t size, int Type) {
  int i = 0;
ReDo:
  for (; TakenSize[i]; i += TakenSize[i]);
  if (i >= 0x2000000 / BLOCKSIZE) {
    printf("UpperMalloc out of mem!");
    return NULL;
  }
  int BSize = (size - 1) / BLOCKSIZE + 1;
  for(int j = 1; j < BSize; j++) {
    if (TakenSize[i + j]) {
      i += j;
      goto ReDo; //OMG Goto, kill me.
    }
  }
  
  TakenSize[i] = BSize;
  void* mem = ((char*)UpperMem) + i * BLOCKSIZE;
  memset(mem, 0, size);
  return mem;
}

//Releases UpperMalloced memory
void UpperFree(void* mem) {
  int i = (((int)mem) - ((int)UpperMem));
  if (i < 0 || i >= 0x2000000) {
    fprintf(stderr, "UpperFree of not UpperMalloced mem: %p\n", mem);
  } else {
    if (i % BLOCKSIZE)
      fprintf(stderr, "delete error: %p\n", mem);
    TakenSize[i / BLOCKSIZE] = 0;
  }
}

//Returns the size of a UpperMalloced block.
int GetUpperSize(void* mem) {
  int i = (((int)mem) - ((int)UpperMem));
  if (i < 0 || i >= 0x2000000) {
    fprintf(stderr, "GetUpperSize of not UpperMalloced mem: %p\n", mem);
    return -1;
  }
  return TakenSize[i / BLOCKSIZE] * BLOCKSIZE;
}

void InitMemPool() {
#ifndef GP2X
  UpperMem = malloc(0x2000000);
#else
  //Try to apply MMU hack.
  int mmufd = open("/dev/mmuhack", O_RDWR);
  if(mmufd < 0) {
    system("/sbin/insmod mmuhack.o");
    mmufd = open("/dev/mmuhack", O_RDWR);
  }
  if(mmufd < 0) {
    Log("MMU hack failed");
  } else {
    close(mmufd);
  }

  Uppermemfd = open("/dev/mem", O_RDWR);
  UpperMem = mmap(0, 0x2000000, PROT_READ | PROT_WRITE, MAP_SHARED, Uppermemfd, 0x2000000);
#endif
  memset(TakenSize, 0, sizeof(TakenSize));

  SetTaken(0x3000000, 0x80000); // Video decoder (you could overwrite this, but if you
                                // don't need the memory then be nice and don't)
  SetTaken(0x3101000, 153600);  // Primary frame buffer
  SetTaken(0x3381000, 153600);  // Secondary frame buffer (if you don't use it, uncomment)
  SetTaken(0x3600000, 0x8000);  // Sound buffer
}

void DestroyMemPool() {
#ifndef GP2X
  free(UpperMem);
#else
  close (Uppermemfd);
#endif
  UpperMem = NULL;
}