Convert FILETIME to Unix timestamp

Yes I know, C/C++ is not trendy these days. I don’t care.

So if you are trying to convert a FILETIME date that comes for example from the FindFirstFile/FindNextFile Win32 API you may find it’s not completely straightforward. Don’t try to accomplish this with some date function conversion because you will probably just waste a lot of time –I know because I did.

A UNIX timestamp contains the number of seconds from Jan 1, 1970, while the FILETIME documentation says: Contains a 64-bit value representing the number of 100-nanosecond intervals since January 1, 1601 (UTC).

Between Jan 1, 1601 and Jan 1, 1970 there are 11644473600 seconds, so we will just subtract that value:

LONGLONG FileTime_to_POSIX(FILETIME ft)
{
	// takes the last modified date
	LARGE_INTEGER date, adjust;
	date.HighPart = ft.dwHighDateTime;
	date.LowPart = ft.dwLowDateTime;

	// 100-nanoseconds = milliseconds * 10000
	adjust.QuadPart = 11644473600000 * 10000;

	// removes the diff between 1970 and 1601
	date.QuadPart -= adjust.QuadPart;

	// converts back from 100-nanoseconds to seconds
	return date.QuadPart / 10000000;
}

And some code to show its usage (with various checks omitted for the sake of simplicity):

#include "stdafx.h"
#include "windows.h"

LONGLONG FileTime_to_POSIX(FILETIME ft);

int _tmain(int argc, _TCHAR* argv[])
{
	char d;
	WIN32_FIND_DATA FindFileData;
	HANDLE hFind;

	hFind = FindFirstFile(TEXT("C:\\test.txt"), &FindFileData);
	LONGLONG posix = FileTime_to_POSIX(FindFileData.ftLastWriteTime);
	FindClose(hFind);

	printf("UNIX timestamp: %ld\n", posix);
	scanf("%c", &d);

	return 0;
}

The Old New Thing* – a theme for Thunderbird 3

Yesterday I downloaded Thunderbird 3, the latest incarnation of the popular email client. While I like the new features and appreciate the work they put in this release, I don’t like the icons.
They are coherent with the Windows 7 default theme, but I still think they are a bit too bright and distracting. On the other hand the icons in the Thunderbird 2 default theme were perfect on this regard.

Faster done than said – I packaged the old icons in the default TB3 theme. To make things clear, I haven’t created anything, I just took the TB2 icons and put them in the TB3 default theme.
You can download it from here until it gets approved at mozilla addons.

A couple of screenshots: default TB3 theme:

screenshot_1

The Old New Thing in action:

screenshot_2

*not related to Raymond Chen’s excellent weblog – I’m not very creative in chosing names, sorry.

update: now marked to work with any 3.* version of Thunderbird.

WCF/Silverlight – some “benchmarks”

I took some very simple measurements from my recent experiments with Silverlight and WCF web services. These are so simple and unscientifical that I suggest you take them only as a general indication.
Please test your scenario to get an accurate picture!

That said, some differences are so large that it already gives you a general idea. These are the http bindings I tested:

1) text formatter (default), i.e. SOAP XML
2) binary formatter, i.e. binary XML
3) text formatter with http gzip compression (see my previous post)
4) binary formatter with http gzip compression

Here are the results.

Response time

time

You can see that text formatter is incredibly slower than binary XML.
One interesting thing I noticed is that this extreme slowness of the text formatter happens only with Silverlight (3). That is, if you use a Windows client (console app or wpf), text formatter is still slower than binary formatter but not that much (compare the red bars).

The Silverlight runtime is probably slower than the Windows runtime and I guess that deserializing a huge xml message is one of the things that clearly expose this difference.

Another observation is that with gzip compression the response time is slightly slower. Keep in mind that these numbers come from a connection on a single machine. In real world, with a large message, size will be so much smaller with compression that the compression overhead is probably largely compensated.

(Side note: these tests were quick and dirty, but I still did some “warmup” calls and measured over multiple runs, so these timings are pretty stable.)

Message size

size

The message I used was quite large and included a pseudo-dataTable. This means that XML serialization results in a lot of string repetitions: a particularly good target for zip compression. Other cases may not benefit so greatly from gzip compression.

Conclusion

This is clearly not a conclusive test -it may not even be enough to be called a test- but one thing is clear: it is worth it spending some time to play with the different binding options as the benefits you could reap may be huge.

GZIP Compression – WCF+Silverlight

In a previous post I wrote a method to enable GZIP compression on a self-hosted WCF service that communicates with Silverlight. Unfortunately that method does not work. I still haven’t understood if it stopped working at some point for some reason, or if I was fooled into thinking that it worked but it didn’t.
Whatever, what I’ll describe here takes slightly more work but gives the desired result.

In brief, this is what we’ll need.

On the client side: still nothing. The browser’s http layer handles it all nicely. When an http response has a Content-Encoding: gzip header it decompresses the message body before feeding Silverlight.
This is also true if you use the SL3 built-in stack.

On the server side: instead of using a basicHttpBinding, we’ll use a custom binding that will handle these steps:

1. Read the request’s Accept-Encoding HTTP header
2. Depending on that header, compress the response body
3. If the response was compressed, add a Content-Encoding: gzip HTTP header

Steps 1. and 3. are managed by a MessageInspector. Step 2. is managed by a MessageEncoder –we will base this on the Microsoft’s Compression Encoder sample. Please download it as I’ll only tell you what to modify there.
It may  be a good idea to study it a bit before starting.

Step 1. and 3. – Managing the HTTP headers

We have to create a MessageInspector that performs the actual work and a Behavior that tells the service endpoint to use the inspector.
The inspector is what we are most interested in: look at AfterReceiveRequest() and BeforeSendReply(). In AfterReceiveRequest we’ll look for “gzip” inside the Accept-Encoding header. If we find it, then we’ll add an extension to OperationContext so that later we will know if we can compress the response (or if we have to return it uncompressed).

public class GzipInspector : IDispatchMessageInspector
{
    public object AfterReceiveRequest(ref Message request, IClientChannel channel, System.ServiceModel.InstanceContext instanceContext)
    {
        try
        {
            var prop = request.Properties[HttpRequestMessageProperty.Name] as HttpRequestMessageProperty;
            var accept = prop.Headers[HttpRequestHeader.AcceptEncoding];

            if (!string.IsNullOrEmpty(accept) && accept.Contains("gzip"))
                OperationContext.Current.Extensions.Add(new DoCompressExtension());
        }
        catch { }

        return null;
    }

    public void BeforeSendReply(ref System.ServiceModel.Channels.Message reply, object correlationState)
    {
        if (OperationContext.Current.Extensions.OfType().Count() > 0)
        {
            HttpResponseMessageProperty httpResponseProperty = new HttpResponseMessageProperty();
            httpResponseProperty.Headers.Add(HttpRequestHeader.ContentEncoding, "gzip");
            reply.Properties[HttpResponseMessageProperty.Name] = httpResponseProperty;
        }
    }
}

And this is the Extension. There is nothing inside it, it’s just a way to store in OperationContext the information “The response of this message will need to be compressed”. We will use this information later in the compression encoder.

public class DoCompressExtension : IExtension
{
 public void Attach(OperationContext owner) { }
 public void Detach(OperationContext owner) { }
}

Finally, we have to provide a behavior that adds our MessageInspector to the service endpoint. His goal is just to tell the endpoint to inspect incoming and outgoing messages with GZipInspector.

public class GZipBehavior : IEndpointBehavior
{
    public void AddBindingParameters(ServiceEndpoint endpoint, BindingParameterCollection bindingParameters)
    { }

    public void ApplyClientBehavior(ServiceEndpoint endpoint, ClientRuntime clientRuntime)
    {
        throw new Exception("Behavior not supported on the client side");
    }

    public void ApplyDispatchBehavior(ServiceEndpoint endpoint, System.ServiceModel.Dispatcher.EndpointDispatcher endpointDispatcher)
    {
        endpointDispatcher.DispatchRuntime.MessageInspectors.Add(new GzipInspector());
    }

    public void Validate(ServiceEndpoint endpoint)
    { }
}

public class GzipBehaviorExtensionElement : BehaviorExtensionElement
{
    public GzipBehaviorExtensionElement()
    { }

    public override Type BehaviorType
    {
        get { return typeof(GZipBehavior); }
    }

    protected override object CreateBehavior()
    {
        return new GZipBehavior();
    }
}

Step 2. – Compressing the response body

As mentioned, we will modify the MS Compression Channel sample (WCF/Extensibility/MessageEncoder/Compression). We need the files inside the GZipEncoder project and we will made some minor changes to the GZipMessageEncoder class (GZipMessageEncoderFactory.cs) and GZipMessageEncodingElement (GZipMessageEncodingBindingElement.cs).

These are the changes to GZipMessageEncoder. First, the sample uses content-type to mark the message as compressed. Since we are using http headers to achieve this, we can leave content-type and media-type intact (we must in fact). Change them to look like this:

public override string ContentType
{
    get { return innerEncoder.ContentType; }
}

public override string MediaType
{
    get { return innerEncoder.ContentType; }
}

Second, we have to skip decompression when reading the message since our requests are always uncompressed. This simplifies the two ReadMessage methods:

public override Message ReadMessage(ArraySegment buffer, BufferManager bufferManager, string contentType)
{
    return innerEncoder.ReadMessage(buffer, bufferManager, contentType);
}

public override Message ReadMessage(System.IO.Stream stream, int maxSizeOfHeaders, string contentType)
{
    return innerEncoder.ReadMessage(stream, maxSizeOfHeaders, contentType);
}

Third, we have to add a condition to WriteMessage as we have to compress the response only when our MessageInspector told us to do so.

public override ArraySegment WriteMessage(Message message, int maxMessageSize, BufferManager bufferManager, int messageOffset)
{
    if (OperationContext.Current.Extensions.OfType().Count() > 0)
    {
        ArraySegment buffer = innerEncoder.WriteMessage(message, maxMessageSize, bufferManager, messageOffset);
        return CompressBuffer(buffer, bufferManager, messageOffset);
    }
    else
        return innerEncoder.WriteMessage(message, maxMessageSize, bufferManager, messageOffset);
}

public override void WriteMessage(Message message, System.IO.Stream stream)
{
    if (OperationContext.Current.Extensions.OfType().Count() > 0)
    {
        using (GZipStream gzStream = new GZipStream(stream, CompressionMode.Compress, true))
        {
            innerEncoder.WriteMessage(message, gzStream);
        }
        stream.Flush();
    }
    else
        innerEncoder.WriteMessage(message, stream);
}

One last details: if you want to create the binding from app.config you may want to make a small change to the GZipMessageEncodingElement class: by default it creates a TextMessageEncodingBindingElement without specifying anything. However since we are trying to replicate a basicHttpBinding, we have to specify Soap11 as message version, and also UTF8 encoding:

public override void ApplyConfiguration(BindingElement bindingElement)
{
    GZipMessageEncodingBindingElement binding = (GZipMessageEncodingBindingElement)bindingElement;
    PropertyInformationCollection propertyInfo = this.ElementInformation.Properties;
    if (propertyInfo["innerMessageEncoding"].ValueOrigin != PropertyValueOrigin.Default)
    {
        switch (this.InnerMessageEncoding)
        {
            case "textMessageEncoding":
                binding.InnerMessageEncodingBindingElement = new TextMessageEncodingBindingElement()
                {
                    MessageVersion = MessageVersion.Soap11,
                    WriteEncoding = Encoding.UTF8
                };
                break;
            case "binaryMessageEncoding":
                binding.InnerMessageEncodingBindingElement = new BinaryMessageEncodingBindingElement();
                break;
        }
    }
}

Putting it all together

Now we have a Behavior/MessageInspector that handle HTTP headers and we have a MessageEncoder that compresses the response body. We only have to tell our service to use them.

The binding: instead of a basicHttpBinding we will use a custom binding. From config:

<customBinding>
  <binding name="BufferedHttpSampleServer">
    <gzipMessageEncoding innerMessageEncoding="textMessageEncoding" />
    <httpTransport transferMode="Buffered"/>
  </binding>
</customBinding>

From code:

var encoding = new GZipMessageEncodingBindingElement(new TextMessageEncodingBindingElement(MessageVersion.Soap11, Encoding.UTF8));
var transport = new HttpTransportBindingElement();
var b = new CustomBinding(encoding, transport);

If you don’t need interoperability you can use binary XML instead of plain XML:

var encoding = new GZipMessageEncodingBindingElement(new BinaryMessageEncodingBindingElement());

It’s possible that on the Silverlight side if you add your service with “Add Service Reference,” the app.config is not created correctly. In that case just modify it to use a basicHttpBinding (or a custom binding with http transport and binaryEncoding if you are using binary XML).

Registering the behavior:

using (ServiceHost myServer = new ServiceHost(typeof(IMyServer)))
 {

myServer.Description.Endpoints[0].Behaviors.Add(new GZipBehavior());
myServer.Open();

Of course this can be also specified in config.

Performance Considerations

With gzip compression you’ll introduce a small overhead and on very fast networks (or on localhost) you may notice slightly slower response times. However the benefit in real world is well worth it in particular if you are transferring large messages using plain XML.

For example I had a 1Mb message that got down to 65Kb with compression. This message included a pseudo-dataset (a lot of repetitions in xml) and was an ideal situation for gzip compression. However I suppose that in most situations you will improve greatly your service response times.