http://linuxmafia.com/faq/VALinux-kb/2gb-filesize-limit.html
Q: Is there any way around the 2GB file-size limit in Linux?
Are there any stable patches to fix it?
A: Short answer: In a practical sense, no. The 2GB limit is deeply
embedded in the versions of Linux for 32-bit CPUs: in GNU libc,
in the Linux kernel's filesystem drivers and VFS layer, in the
fundamental design of some filesystems, and in the function
calls used in the compiled applications and utilities furnished
in (and for) typical Linux systems. There is thus currently no
commonly available configuration.
Long answer: All Linux ports for 32-bit CPUs (i.e., all
architectures other than Alpha and IA-64) use 32-bit integers
for file access and locking, yielding a maximum size of 2^31 -
1 = 2GB. Creating a workaround for this situation without
breaking existing code is an obstacle for all 32-bit Unixes[1],
whose creators met to design a standard 64-bit file-access
programming interface, at a meeting sponsored by X/Open (later
renamed The Open Group), called the Large File Summit (LFS)[2].
The resulting large-file standard and related system calls are
likewise called LFS.
To implement LFS (and remove the 2GB limit) on a Linux
system, you would have to:
1. Install glibc v. 2.2 or later, compiled against kernel
headers from kernel 2.4.0test7 or later. At this date, current
distributions are compiled using glibc 2.1.x and kernel 2.2.x.
glibc 2.2 is considered experimental.
2. Install kernel 2.4.0test7 or later, or recent 2.2.x
kernels with unofficial LFS patches, or Red Hat Software's
"enterprise" kernel. Earlier versions omitted the necessary
routines in the middle kernel layers (e.g., VFS) and
improvements to the filesystem drivers. A number of build
problems have been reported with experimental kernel 2.2.x LFS
efforts, including incompatibility with the much-requested NFS
v. 3 and ext3 patches.
3. Use suitable filesystems. These include ext2/ext3,
ReiserFS v. 3.6.x and up, IBM's JFS, SGI's XFS, and recent
NFSv3 client drivers (http://nfs.sourceforge.net).
Problematic filesystems include NFS v. 2, ISO9660, earlier ReiserFS,
AFS, coda, Intermezzo, Minix, UFS, SCO/SysV, msdos/umsdos/vfat,
smbfs, NCPfs, jffs, jffs2, and romfs.
The new maximum size of an individual file is 8 terabytes
(2^63) on ext2 or ReiserFS filesystems, on LFS-compatible
hosts. For other suitable filesystems, a different limit may
apply in each, but reported values are all quite large.
4. Revise and recompile all programs and utilities that will
manipulate large files, to use 64-bit file-access and
file-locking system calls, instead of 32-bit ones. Since you
will not have access to source code for proprietary
applications you may use (such as Oracle), this fix may be
effectively impossible for them, and very daunting to carry out
on the large number of relevant open-source userspace
programs.
Please be aware that, even if you implement these fixes
without error, and, even if the new 64-bit functions work
perfectly, you are likely to encounter performance losses and
greater memory usage, when using the revised and recompiled
code.
Since implementing the above-described LFS regime isn't very
practical in most cases, sites having problems with excessively
large files should generally find other ways around the
problem: Large ongoing records can be written to multiple files
branched off periodically (e.g., daily logfiles), or can be
back-ended into one of the SQL databases, such as Oracle, that
handle (work around) filesystem limits within their
table-storage mechanisms.
To test for basic LFS support in your system:
dd if=/dev/zero of=bigfile bs=1024 count=3145728
dd if=/dev/zero of=bigfile bs=1024 count=5145728
dd if=/dev/zero bs=1024 count=5145728 > bigfile
dd if=/dev/zero of=bigfile bs=1024 count=5145728
dd if=/dev/zero bs=1024 count=5145728 > bigfile
Further information at Andreas Jaeger's page: http://www.suse.de/~aj/linux_lfs.html
[1] I have heard it alleged that some 32-bit Unixes
implemented fixes different from those recommended by the Large
File Summit, in some cases earlier.
[2] The X/Open Large File Summit's standards are documented
here: http://ftp.sas.com/standards/large.file/