The .NET Preprocessor - Part 2

NTSTATUS: STATUS_DOTNET_MACRO

The .NET Pre-Processor may be downloaded here.

The first debate was which language it should be written in. The candidates were C/C++, C# and Python. I really wanted to write it in C/C++, but the conveniences of C# and the .NET platform was just scoring too many points. No, I will not write managed C++ in Visual C++. I do not like the syntax. As for Python, the code would probably have been shorter, but it’s not a common install, and not as commonly used, I think. Related to that, Python got a boost when an actual Python came to my house a week ago. No, seriously. It’s the second one to come by in 6 months. I told my good friend Walter (Developer Security MVP) that it might have been a sign. But alas, it was C#.

DOTNETPP is a command-line tool that is largely designed to work with Visual Studio’s pre and post-build facilities. The settings are expected to be in an XML file, a sample of which is included in the download. Following is the usage:

Usage: DOTNETPP /p:"<config file path>" [/process|/cleanup]

/process: Process macro expansion

/cleanup: Cleanup expanded macro files

The config file path is specified by the /p: parameter. /process and /cleanup are actually mutually exclusive. If they both appear, /process will take priority. So basically, you will use /process in your Visual Studio .NET Project’s pre-build event, and /cleanup in your post-build event. Of course you can also manually run it to process and cleanup.

Macro Definition File

The macro definition file is basically a text file where you define your C Preprocessor macros. I will post some examples subsequent posts. DOTNETPP supports the following macro/preprocessor features:

- Non-parameterized macros

- Parameterized macros

- Token concatenation (##)

- Stringizing Operator (#)

- C++ style comments (//)

Since the macro definitions are in its own file, the line-continuation backslash ‘\’ is not required at the end of each line of the token list.

Source Files

In the source files which you want to use macros and have DOTNETPP process them, include the following syntax in said source file:

#include DOTNETPP

Since this syntax is not processed, it must be commented or excluded from compilation for the build to succeed. In the case of C#, you may use the following:

//#include DOTNETPP

Or

#if false

#include DOTNETPP

#endif

The Catch

There is a catch. This is because this tool is NOT integrated into Visual Studio. This is what the problem is… the way DOTNETPP works is that it (obviously) expands the macros, and then writes it back into the source file. So the catch is, if any of the source files that are processed by DOTNETPP are open when the build runs (with DOTNETPP running in the pre/post-build events), the build WILL FAIL.

Why? This is because Visual Studio will ask if you want to reload the file as it has been edited outside of the source editor. The compilation would have started way before you’re able to say you want to reload the files. So if you reload and save the changed files, you will LOSE all your macros, and have the expanded code instead. This would be equivalent to setting <RestoreMacrosInSource> to FALSE in the config file.

Here’s the scenario:

1. Build project/solution

2. Build fails

3. Visual Studio asks if you want to reload the files

4. Reload the files and save

5. Build project/solution (this time it will succeed assuming you did not screw up the macro definitions)

This is a clearly a serious inconvenience to have to close all the source files with the macros before doing a build every time. This is a major con. Well, it’s not part of Visual Studio, tough. So maybe there is a way to work around this if the tool is integrated into Visual Studio. That would be something to look into for the future.

However, having thought about it, this problem probably isn’t necessarily a bad thing. I probably don’t actually want the macros to remain in the source as it can be cryptic. So, as a potentially fortunate consequence of the way Visual Studio works, DOTNETPP could work as a one-time-no-return macro expander. After all, one of the config settings is <RestoreMacrosInSource>true/false</RestoreMacrosInSource>. OK, so maybe this is a less ‘evil’ way. Expand the macros so they become the source code, and no more macros in the source.

Next I will write about the config file, but the elements should be quite obvious.

Here’s one of the main reasons I started this project:

#define FOR(a) for (int i = 0; i < a.Count; i++)

The .NET Preprocessor - Part 1

NTSTATUS: STATUS_DOTNET_MACRO

This is my only technical blog, so I’ll just put it here for now…

My favourite programming language is still C. One of the things I liked a lot about it is the C Preprocessor. There are people who love it, and then there are those who hate it. Some say that it is evil. Like they say, with C, you shoot yourself in the foot. I guess the C Preprocessor (CPP) is the same. It can be used for good, or evil. If used appropriately, boy, can it save you a load of tedious coding/keystrokes. If you screw it up, you shoot yourself in the foot.

So nowadays, I don’t use C/C++ as much as there is not all that much need for it, and better languages/platforms for certain jobs… Like web apps, and general Windows apps… Yes, that makes up most of the needs nowadays. I’ve always been Windows kinda guy, so naturally I’m using C# on .NET. Man, C# is sweet, and the .NET libraries are super sweet. OK, so at least C# has some basic compiler-directives, like symbol definition and conditional compilation. From a C background, that is very welcome indeed.

However, there is NO macro expansion!!! [Insert expletive here]. If you’ve used the CPP, you’ll soon start to miss it. I swear, man, I am so sick of typing tedious repetitive code. If you are one of those who would say to substitute the macros with functions, then forget it because I don’t want to create more overhead where it can be avoided. Besides, it just isn’t the same concept or in the same spirit.
So about a month ago, I started moaning about this again because I am so, so, so, did I say I was sooo sick of writing for-loops (for instance)? So I kindly asked someone (anyone) to add the CPP/Precompiler to C# on Facebook. OK, I know it was not included in C# for reasons unknown to me. But I believe Hejlsberg and co. had good reasons. But I know what I am doing, and I WANT IT!!! So how now brown cow?

So, having toyed with the idea for a couple of weeks, I spent another week or two actually thinking it out in my head how to build it, and which features it would include. Of course, I thought about how it could work with Visual Studio. Not surprisingly, after a few weeks of laying it all out in my head, I finally took some ‘time off’, sat down, and went about building it. Surprisingly, it took me just over half a day to finish the first complete app. So next time your boss catches you seemingly doing nothing, just tell ‘em that you’re laying it all out in your head first so the execution will be quicker and smoother.

Anyway, I will continue to write about how it works in subsequent posts and put the links up. Oh, I beg your pardon, I haven’t introduced it. I call it the ‘.NET Pre-Processor’, and it is DOTNETPP.EXE.

Dissassemble Me

NTSTATUS: STATUS_OBJECT_DUMPED

A must have tool is PEBrowse Pro by SmidgeonSoft written by the great Russell Osterlund. Use it to disassemble Win32/Win64 and .NET (assemblies) PE binaries (EXE/DLL).

I have been using this great tool as a lightweight disassembler for a decade. It is updated fairly often improving features. It's really nice tool to use to see a Win32 binary's imports and exports. Much nicer than using DUMPBIN, unless you prefer a command line app. They also have a debugger as well that works on Win32 apps and .NET apps.

Process Session

NTSTATUS: STATUS_JAM_SESSION

I think there are a fair number of people who are not aware of the concept of 'sessions' in Windows, and some are aware of it, but only that there is such a thing. We're going to go through the basics of it, and see how it might be relevant.

A session in Windows NT is basically a user login session. It is managed by the Session Manager Subsystem process (SMSS.EXE). This includes Terminal Services/Remote Desktop sessions. Windows NT servers allow multiple concurrent Terminal Services login sessions whereas the client versions Windows NT does not.

If I recall correctly, in the initial version of Windows XP, you could be logged in to Windows, and have a Remote Desktop session active with another user being logged in. So, you could actually have two simultaneous user sessions. Then it came to pass, in, I think it was SP2 that Terminal Services was changed such that you could not run a Remote Desktop session without having the current user logged off. I think the story was that the change was made because the way Terminal Services was working was in violation of the EULA which allowed only one user of that copy of Windows XP at any one time.

Now, moving along, sessions weren't terribly relevant prior to Windows Vista with the exception of the case of Windows XP prior to the Service Pack with changed Terminal Services. This is because prior to Vista, you could not actually have more than one user logged in to Windows at the same time. You'd remember this because if you wanted to login as another user, you had to logout of your current session. So in that sense, it wasn't actually relevant for your apps to be concerned about which session it was running in.

However, since Vista, you could login multiple users in your machine which I really appreciated because you didn't have to lose your session. I hated having to have my session restarted and to get things back to the way they were, and it took an amount of time I wasn't interested in waiting on.

So now, in Vista and Windows 7, the session in which your process runs becomes relevant. There are two basic scenarios here:

1. There should only be one instance of your app running in Windows
2. There should be only one instance of your app running in each user session

In the first case, it is pretty straightforward. You would enumerate all processes and see if there is already in instance running.

In the second scenario, it is commonly the case where an instance of your app is required to deal with one (instance of a) user specifically. Therefore, if there was another user logged in, you needed another instance of your app to run in that user's context as well. In most cases, this would be a UI app. Also, it is an app where you only want only a single instance running (per user session). So in the old days before multiple concurrent user sessions, you would just check to see if the app is already running, and not start up a second process. If that was done now, you will end up with only one instance of the app running regardless of the number of user sessions.

So now we get to the gist of it, which is: ensuring only one instance of your app runs per user session.

To do this, we need to enumerate all instances of the app in question, and check the session in which it belongs to. As an aside, you can use Process Explorer from Sysinternals to look at the process tree to see all the session running, and their associated processes. And obviously, you are not able to access the processes that do not belong to your current session.

The Session ID of a process can be obtained from the Process Environment Block (PEB) Win32 structure. This one of those magic data structures that are largely undocumented, and in most instances, for good reason. Following is the struct from the Windows SDK version 7.0:

typedef struct _PEB {
BYTE Reserved1[2];
BYTE BeingDebugged;
BYTE Reserved2[1];
PVOID Reserved3[2];
PPEB_LDR_DATA Ldr;
PRTL_USER_PROCESS_PARAMETERS ProcessParameters;
BYTE Reserved4[104];
PVOID Reserved5[52];
PPS_POST_PROCESS_INIT_ROUTINE PostProcessInitRoutine;
BYTE Reserved6[128];
PVOID Reserved7[1];
ULONG SessionId;
} PEB, *PPEB;

You can see the SessionId field in the struct, which is what we're interested in. Another field of potential interest is the Ldr field which contains a doubly-linked list of the modules loaded by the process.

The information of this struct is obtained via the Win32 API function:

NTSTATUS WINAPI NtQueryInformationProcess(
__in       HANDLE ProcessHandle,
__in       PROCESSINFOCLASS ProcessInformationClass,
__out      PVOID ProcessInformation,
__in       ULONG ProcessInformationLength,
__out_opt  PULONG ReturnLength
);

Pass in ProcessBasicInformation in the 
ProcessInformationClass parameter, and the function will return a pointer to a PEB structure.

So there you have it, the session in which the process is running.