how can i get R to release and stop holding memory even after all objects have been removed and cleared?

i am trying to get some code working on computers with less than 4GB of RAM. i am using the 32-bit version of R to enforce that memory ceiling. i'm hitting a wall near the end of the script when i'm trying to run a memory-hogging command and everything breaks. but the memory-hogging task by itself requires less than 4GB. i've narrowed down the problem to the fact that - despite clearing all objects from memory in the current session - the R console is still holding 1.9GB of RAM. the screenshot below highlights exactly where i'm hitting the problem: note there are zero objects in memory and yet task manager says this instance of R has 1.8578GB of RAM held.

if i clear all objects from memory and then run gc() that still does not clear all memory held (as you can see in my screenshot).

is it possible to clear this memory held somehow?

if it's of any use, you can reproduce this up to the point of the crash by running this script

thank you!

edit: at the end of the script i get

[1] "current designing ./2011/bst.rda"
Error: cannot allocate vector of size 434.7 Mb
In addition: There were 50 or more warnings (use warnings() to see the first 50)
> gc(verbose=T)
Garbage collection 27232 = 15350+4362+7520 (level 2) ... 
31.5 Mbytes of cons cells used (49%)
450.6 Mbytes of vectors used (21%)
           used  (Mb) gc trigger   (Mb)  max used   (Mb)
Ncells  1175911  31.5    2421436   64.7   1770749   47.3
Vcells 59048650 450.6  278146328 2122.1 461815808 3523.4
> rm(list=ls(all=TRUE))
> gc(verbose=T)
Garbage collection 27233 = 15350+4362+7521 (level 2) ... 
11.1 Mbytes of cons cells used (21%)
7.1 Mbytes of vectors used (0%)
         used (Mb) gc trigger   (Mb)  max used   (Mb)
Ncells 414283 11.1    1937148   51.8   1770749   47.3
Vcells 920035  7.1  222517062 1697.7 461815808 3523.4
> 

Answers


This is not only for R but for Windows in general. Normally, if you have removed a variable/ object in R. The process does release the memory to OS, but due to working of Windows. That memory is not released totally, it's kept in case the process request memory again so you will see as if R is still holding all that memory.

So please don't worry, that is kept for reuse :)


Is the memory released after you quit R?

Maybe you have some data read in disk (stored in a temporary file) rather than load to R. So gc() will not capture that.

Or do mem_change(your command here) from the very beginning to see what leads to change in memory.


Try using commands

memory.limit()
# [1] 1934
memory.size()
# you can increase memory limit for that particular session in Windows machine
memory.limit(10000)  ## Size in Mbs
memory.limit
# [1] 10000   

As yours is 32-bit, your maximum limit would be 4095 MB. To see more on memory.size() and memory.limit(), use the below commands on your R console and read about it.

?memory.limit()
?memory.size()

Hope you got little help from this. You can increase your memory limit at least for that particular session.


Need Your Help

How to ask Selenium2 to finish loading a page on .Click before doing any assertions?

c# selenium selenium-webdriver

I would like to use the selenium2 c# webdriver to test our login mechanism. The following test passes if I run through it in debug mode and give the Click method some time. If I run it normally tho...

Question about this xaml markup extension

c# wpf xaml markup-extensions

I'm trying to understand what does the markup extension for the x:Key attribute below do and what kind of markup extension is it?