Dotmad.net

There are limits most application developers never encounter. One of them is the 2GB memory limit for applications running on a 32bit operating system.

But what happens if your application is a memory consumer (and most GIS applications are)?

You may find out your application tends to crash and burn once reaching a memory signature of a little over 1GB.

It’s a common knowledge for developers (I hope) that a useful way to measure application’s true memory consumption is using the Task manager’s “VM Size” column (as the “Mem Usage” column only indicates the size of the application’s Working Set). So why does the application crash when supposedly there is still plenty of free memory?

There are 2 possible answers:

1. Although a certain size of memory is allocated to the application, fragmentation to the memory may cause the application to actually consume more than that. This may also be related to the GC behavior in relation to 2nd generation objects, which I’ll discuss later.
2. The VM Size column does not represent the actual Virtual address space allocated to the application, which is limited to 2-3GB on a 32bit OS. In addition to the allocated memory the OS may have designated addition shared memory (shared with other applications, but still counts towards the 2GB limit). To view the actual size of memory allocated by the OS to the application, use the Perfmon.exe tool’s Virtual Size counter:

The .Net Garbage Collector and 2nd generation (gen2) collections

The GC algorithm divides the allocated objects to 3 generations (0-2), when generation 0 objects are retrieved most frequently and “live” for the shortest amount of time. Generation 1 objects are collected less frequently and generation 2 collections are truely a rare occasion.

But when does the GC collects generation 2 allocations (”full” collection)? What triggers such a collection?

Surprisingly I found very little information of this subject, but the information I did find, combined with experimentation led to the conclusion such collection occurs when the overall system is low on physical memory.

Now, let’s consider the implications for a 4GB computer running a 32bit OS. While the gen2 heap may inflate over time (especially considering large objects are also collected only in gen2 collections), the GC will never trigger as a single application can never consume the entire amount of memory on the computer (being limited to 2GB). However, this may lead to a memory leak and eventually crashing the application (once it’s virtual address space reaches 2GB).

The solution? Call the GC.Collect() method yourself to trigger a full collection.

Ray Ozzie gave the PDC08 opening keynote, focusing (today) on the future backend platform from Microsoft, with tomorrow’s keynote focusing on front end (client) applications and services such as Windows Live.
Ozzie began the keynote by replying to the criticism regarding his cloud computing vision, agreeing it’s not a new concept, but that now it’s more relevant than ever before.
According to Ozzie The web is critical to a bussiness today in various issues.
Turns out that for the past two years Microsoft has been developing Windows Azure - a new infrastructure for cloud computing and service management.

The Azure™ Services Platform (Azure) is an internet-scale cloud services platform hosted in Microsoft data centers, which provides an operating system and a set of developer services that can be used individually or together. Azure’s flexible and interoperable platform can be used to build new applications to run from the cloud or enhance existing applications with cloud-based capabilities. Its open architecture gives developers the choice to build web applications, applications running on connected devices, PCs, servers, or hybrid solutions offering the best of online and on-premises.

Azure reduces the need for up-front technology purchases, and it enables developers to quickly and easily create applications running in the cloud by using their existing skills with the Microsoft Visual Studio development environment and the Microsoft .NET Framework. In addition to managed code languages supported by .NET, Azure will support more programming languages and development environments in the near future. Azure simplifies maintaining and operating applications by providing on-demand compute and storage to host, scale, and manage web and connected applications. Infrastructure management is automated with a platform that is designed for high availability and dynamic scaling to match usage needs with the option of a pay-as-you-go pricing model. Azure provides an open, standards-based and interoperable environment with support for multiple internet protocols, including HTTP, REST, SOAP, and XML.

However, Azure platform usage is not free:

The Azure™ Services Platform business model is aligned around four basic principles. With a business model based on these principles, customers will have the power of choice with the Azure Services Platform. You may choose to pay upfront for your usage, or opt to post-pay, or even decide to reserve capacity.

During the Community Technology Preview (CTP), all developers can download both the Windows Azure SDK and the Windows Azure Tools for Microsoft Visual Studio, (the links may take a while to become operational) and simulate a hosted environment on their local system. Developers will also have access to a suite of readiness resources including virtual hands-on labs, webcasts, and documentation such as white papers. Support will be provided through developer-to-developer blogs and forums.

Additional resources are available to developers either through the Azure web site, Azure Services Platform Developer Center, or the Cloud Computing Tools Team blog.

Joel Spolsky wrote a post dealing with Microsoft’s strategy, adding a little “PS” at the end of the post:
PS: in researching this article, I tried to open some of my notes which were written in an old version of Word for Windows. Word 2007 refused to open them for “security” reasons and pointed me on a wild-goose chase of knowledge base articles describing obscure registry settings I would have to set to open old files. It is extremely frustrating how much you have to run in place just to keep where you were before with Microsoft’s products, where every recent release requires hacks, workarounds, and patches just to get to where you were before.
I have started recommending to my friends that they stick with Windows XP, even on new computers, because the few new features on Vista just don’t justify the compatibility problems.

Some time ago I had a problem causing Office to start an installation dialog every time I tried running one of the application. I tried re-installing, which solved the issue, but caused the database connection window in Visual studio to fail. Fixing that negated the use of Window’s search feature.
Finally I managed to get everything to work (Word occasionally opens the installation dialog when I try to open a document from a link), but I learned I can’t trust 3 applications made by the same firm (Office, VS, Windows) to run together in harmony, something that should be obvious in an ideal world.

I read a lot on Vista and Office 2007, enough to decide I have no reason to upgrade just to say I have the “latest cool release” - I will only upgrade if the new release has a feature (or features) I need.
And as Joel, I’ll probably recommend Windows XP to people buying a new computer.

We all know the security pitfalls that exist in Windows XP’s default configuration.
One solution which is widely recommended is to set a new account which is not the default administrator account, and use it for day to day activities.
As a developer I like the freedom I get from administrator privileges, and I don’t want to sacrifice comfort for security.

However, surfing the web may still expose me to dangers, so I decided to set up a limited account for IE.

First I created a new user under the “users” group.
After I created a new shortcut for IE, I went into “properties” and modified the “target” text box:
C:\WINDOWS\system32\runas.exe /user:NewUser /savecred “C:\Program Files\Internet Explorer\IEXPLORE.EXE”

The only downside of this is that you lose all existing configuration data for IE.
And I can still run IE in my admin account whenever I like to.

A crude but simple way of detecting memory leaks is start running the application, looking at the memory it consumes in the task manager, leaving the application to run for a while, and comparing the memory consumption to the earlier one.
And a common mistake is looking at the default “Mem usage” column for the data.
This column doesn’t show you the actual memory reserved for the application, but the memory currently assigned to it by the operating system. You may have also noticed it tends to shrink when you minimize the application.
Instead you should enable the display of the “VM Size” column (using the “view” menu) and use the data displayed in it.