Andre's Blog

Personal blog of Andre Perusse

Dates and Time Zones in Javascript, C#, and SQL Server

An issue that always seems to plague developers who write software that is used in multiple geographically disperse locations is that of dates and time zones. If a user in the Eastern time zone updates a record that contains a date and time field (say, 1/15/2014 4:35 PM), and then a user in the Pacific time zone views that record, what date and time are they supposed to see? The time as it was entered in Eastern time, or should they see their local time instead (1/15/2014 1:35 PM)?

Generally speaking (there are, of course, exceptions) date and time information should be stored in a database as a UTC (Coordinated Universal Time) time, and automatically converted and displayed in the user's current time zone in the client application. But that is often not as easy at it seems to implement. Below, I'll outline a process that has worked well for me in several apps.

Step 1 - Getting data from the client to the server


In the past, web apps collected data in an HTML <form> element that was submitted to the server causing a full page refresh. Today, I prefer to use single-page application frameworks where almost all client-server communication is done using the Javascript XmlHttpRequest (xhr) object. For me, I find jQuery nicely abstracts away the details of xhr using the $.ajax() api method. But how do we wrap up user-entered date-time data into an xhr request so that time zone data is preserved and can be interpreted by our server?

The Javascript date object does have time zone support, but it's not immediately obvious how we can make that work properly with the server. When sending data in an xhr object, it's usually JSON formatted, meaning just about everything is a string - there are no data types. Javascript date objects have a handy toString() method that outputs a date like this:

Sun Feb 23 2014 15:33:09 GMT-0400 (Atlantic Standard Time)

This would seem to contain all the information we require, but alas, the default ASP.NET WebAPI configuration can not translate this to a C# DateTime instance. But there is a solution - the ISO-8601 date format, which looks like this:

2014-02-23T15:33:09-04:00

In fact, most modern browsers have the ability to output this format using the toISOString() method, but there are a couple of drawbacks. First, the toISOString() method is not supported on all browsers and second, this built-in method deletes the time zone information and simply outputs it as UTC (such as 2014-02-23T19:33:09Z). Sometimes it is desirable to preserve the originating time zone data.

Thus, I prefer to use the excellent moment.js library for formatting date and time values. It is incredibly flexible and powerful and you'll find yourself using it everywhere in your Javascript code when dealing with dates. The default format() method of moment.js outputs dates in ISO format.

Step 2 - Processing on the Server


Actually, with WebAPI and its default use of Newtonsoft's Json.NET library, there is nothing else you have to do on the server. Using the default model binder, any date that is submitted in ISO-8601 format will be automatically converted to local server time and, most importantly, the Kind property of the DateTime instance will be set properly to DateTimeKind.Local.

Step 3 - Saving in SQL Server


Now here's one step I see a lot of people miss - saving the datetime properly in SQL Server. We want the date to be stored as a UTC value so that it can be converted to any time zone for display purposes. In order to do this, you must convert the C# datetime instance to UTC when adding it to the parameters of your stored procedure, like so:

parameters.Add("myDateField", myDateTime.ToUniversalTime());

Step 4 - Retrieving from SQL Server


Another step often missed by developers is the requirement to properly stamp the Kind property of the C# DateTime instance when retrieving it from the database. You must mark the value as UTC like this:

myDateField = DateTime.SpecifyKind((DateTime)dbReader("myDateField"), DateTimeKind.Utc);

Step 5 - Displaying on the Web Page


Actually, there is no step 5. The Json.NET serializer will format the date value in ISO-8601 format which is automatically converted to local time in the client Javascript code. If you output this value to the web page, it will be displayed in client local time.

By following this process you can ensure that all date and time data in your database is consistently represented as UTC, and can be reliably converted to the time zone being used by the client application. One gotcha to be careful of, however, is when filtering date values. Just like storing a date value in the database, the client specified date filter must be converted to UTC time before the comparison against SQL data is performed.

Happy coding!

Single Page Application Frameworks

Several months ago I started work on a new project that required a web-based user-interface. I had been out of the HTML game for several years, spending time instead on Windows Forms and Flex-based applications. For this new web project, however, I wanted to retain some of the user-experience and development features that I came to enjoy working with traditional "thick-client" technologies.

My investigations led me to discover a technique now called the JavaScript "Single Page Application," or SPA for short. I am no stranger to some of the paradigms of a SPA having spent the first ten years of my career developing web apps with rich user-interfaces that used AJAX-like techniques before AJAX was even a term. But SPAs have evolved way beyond simple AJAX calls and the now clunky-feeling ASP.NET "Update Panel." The architecture of a SPA more closely resembles that of full-blown client-server apps written in traditional technologies like Windows Forms, bringing with it a more developer-friendly environment and a better end-user experience. For that I think we can thank Google Chrome for stepping up the browser wars and resulting in a doubling of JavaScript performance every nine months over the past five years or so.

After much research (including much material from John Papa), I began to develop my own SPA framework to serve the needs of my project. Keep in mind that a SPA relies on a single HTML page as the application container and the browser makes no further full-page HTTP requests once that page is loaded. Instead, the SPA loads and renders content on-demand as it is required. To make this happen, the SPA has to perform much of the work that has traditionally been performed by the web server, such as view routing and data-binding. While SPAs as an all-inclusive framework are fairly new, many of the functions it must perform are available in the form of individual JavaScript libraries, some of which have been around for years.

And so, I set out to assemble my own framework using several of these individual libraries. In the meantime, and months after I began my own efforts, a wholy-inclusive SPA platform has become available called Durandal. Though I had my own framework mostly developed by this point I wanted to see what Durandal offered and how it compared to the features I had put in my own framework. To my surprise, the two frameworks were amazingly similar. This is most likely because the problem domain was identical, and many of the libraries Durandal uses are the same ones I chose:

  • jQuery for DOM manipulation. This is a no-brainer for me, though there are reasons to choose other DOM frameworks.
  • SammyJS for in-page "hash"-style routing. I have also implemented routing using Path.js.
  • RequireJS for dynamic on-demand loading of resources, whether it's HTML, JavaScript, CSS, or localization resources.
  • Knockout for data-binding. This is turning out to be a pretty core piece of the developer experience since it abstracts away a lot of old-world direct DOM manipulation for form input values and label display.

While all these libraries (and many more ancillary libraries I haven't mentioned) are important in assembling a SPA framework, they don't solve the entire problem. While a routing library helps with determining the view the user wants to see, it doesn't actually help with view management per-se. So I developed a view framework where each view has an event lifecycle: initialize(), beforeShow(), show(), hide(), afterHide(). Not surprisingly, Durandal offers a similar view lifecycle. What appears to be missing from the Durandal framework (though, honestly, I haven't spent more than a few hours examining it) is an integrated user-authentication mechanism, as well as global error handling and the ubiquitous "404" handling for undefined view routes.

If I had to do it over again, I would definitely pick the Durandal framework, however. Their JavaScript-foo is at least an order-of-magnitude better than mine and Durandal is already beginning to accumulate a large community of developers who are using it. This can only mean the product will continue to evolve and grow with a large user base reporting and fixing bugs, making it far more robust than any feeble framework I could manage to tack together. Still, it was gratifying to see that I, a self-proclaimed "hack," was able to assemble a SPA framework that performed many of the tasks Durandal does (though undoubtedly not nearly as well), and maybe even one or two that it doesn't. And it sure is a blast to be back in the web world again. :-)

Visual Studio 2012, Extension Updates, and (417) Expectation failed

While running Visual Studio 2012 behind a proxy, I received the following error message when checking for extension updates in the Visual Studio Gallery:

The remote server returned an unexpected response: (417) Expectation failed

A look around Google finally revealed a solution here:

http://www.jlpaonline.com/?p=176

Basically, he's saying the fix is to edit your devenv.exe.config file and change this:

<settings>
     <ipv6 enabled="true"/>
</settings>

to this:

<settings>
     <ipv6 enabled="true"/>
     <servicePointManager expect100Continue="false"/>
</settings>

Thanks, Jeff!

JavaScript: Rise of the Single Page Application

Last month I wrote about the current state of HTML application development and how it has changed drastically over the past five years. In that post I mentioned a lot of technologies that have evolved, but one area in particular that is in the midst of an amazing sea change is where the user-interface logic is executed.

Historically, most web applications have used a server-templating model where an application server such as Microsoft's ASP.NET processes data and emits it as HTML which is rendered on the client. If the user clicks a button or link on that web page, an HTTP POST or GET operation is performed on the application server and a completely brand-new HTML page is returned to the client browser to be rendered again. The problem with the server-templating model is the annoying screen refresh that occurs between postbacks and page re-draws, and the amount of time required to re-render the new page, often when very little actual data on the page has changed. AJAX techniques can aid immensely with this problem, only updating the areas on the page that have changed. However, this is only half of the solution and a further step must be taken to bring a truly rich user experience to a web-based application.

Enter the JavaScript "single-page application" (SPA). This is a relatively new technique (well, Google's GMail has been using it since 2004 but it's only now starting to catch on in the mainstream) where only a single HTML page is downloaded to the client browser and all further updates are performed by JavaScript code. In fact, the single page that is downloaded is often no more than a shell used to bootstrap the JavaScript application. Once initialized, the application runs completely in the browser with no further HTML pages generated by the server. Instead, the JavaScript code is responsible for creating the HTML DOM elements and binding data to them. This reflects a truer image of the client-server architecture that has existed for decades and is the model most frequently used in Flex and Silverlight applications that run in a browser plug-in (no HTML).

This is not to say that the application server no longer serves any data, it simply no longer wraps that data in HTML. That is the job of the SPA running in JavaScript code on the browser. JavaScript will issue a remoting call to the server for data, which is most often returned in JSON format. Using a client-side databinding framework (such as Knockout.js) this data is then bound to an HTML construct. Any changes are sent back to the application server, once again in JSON format, where the server then performs operations on the data, such as updating a database. The result is a more responsive user-interface that works much like any other rich application developed in more traditional client-side technologies.

From where I'm sitting, 2012 is shaping up to the be the year of the SPA. There has been a tremendous explosion in the number of JavaScript libraries and frameworks designed to provide more power to the client-side developer. Which is truly fantastic - front-end developers have more capabilities at their fingertips than ever before, and it just keeps expanding every month. However, the exploding popularily of JavaScript components is also part of the problem. Many, like me, have no idea where to start. There certainly isn't a clear winner in the race to establish the de-facto framework, and in fact many existing frameworks are happy to just provide what they believe is necessary to fill a given need. Assembling all the pieces to create a complete framework for a web application project is still a task left to the developer.

Which is not so much a bad thing, but compare this with the RIA frameworks of Flex and Silverlight. Using these rather completely-featured frameworks, you may wish to incorporate the odd component or pattern and maybe some additional UI controls, but JavaScript SPA developers have many more choices than that to make. One of the big problems with JavaScript in the browser is that it was never designed to house large applications. Modularizing your JavaScript code involves a peculiar set of work-arounds to address the fact that JavaScript is more object-based than object-oriented (though some may disagree), and requires the use of closures to achieve some degree of modularity. Among other things, this often leads to what can best be called namespace-collisions since all variables defined outside of a function in any script file are invoked in the global namespace. So when you bring all these pieces together in your application you will often find conflicts that you must resolve yourself.

Despite these challenges, the SPA is emerging as the preferred methodology for developing rich and responsive applications in a browser (in no small part because Flex and Silverlight aren't supported on increasingly popular mobile devices). User-interface control vendors are now beginning to offer commercial JavaScript control libraries that have no particular server requirement at all - they consist only of client-side JavaScript code. Developer productivity is still a challenge in this environment as IDEs still have a difficult time with the dynamic nature of JavaScript. But advances are being made and the SPA space is absolutely electric with increasing developer interest and daily news on technique improvements. Indeed, it will be very interesting to see and participate in the SPA revolution over the next couple of years.

Web Development in 2012: This Ain't Your Daddy's HTML

For the first 11 years of my professional career I designed and built applications using HTML. When I first started out, I mostly just specified the application design, having other people much smarter and talented than me do the coding and graphic design, and I would write the HTML myself (back in the day web "programming" was done in C in the form of CGI scripts which was nothing I wanted anything to do with). With Microsoft's Active Server Pages I graduated to performing the coding, too, which was a boatload of fun.

Over the years I considered myself pretty darn clever at web application design (truthfully, I was just a hack but I "thought" I was clever). In 1998 I was using Internet Explorer 4 and Microsoft's Remote Scripting feature to refresh data on a web page without doing a full server refresh. This was years before AJAX became a known term much less a popular feature. A few years later I was using the XMLHttpRequest object to do the same thing. Naturally this required a firm grasp of JavaScript and the browser Document Object Model (DOM), though fortunately I wrote internal corporate applications and cross-browser compatibility wasn't much of a concern (and Microsoft had soundly destroyed Netscape by this time anyway).

I continued honing my JavaScript and DOM knowledge and skills up until about 2007. You remember 2007, don't you? Well, let me refresh your memory anyway. There was no Google Chrome. Mozilla Firefox was barely an up-and-comer. Internet Explorer had only reached version 7 three months ago (October, 2006). Hell, even Facebook had just opened itself to public registrations in September, 2006. AJAX was pretty much a solidified term, but its use wasn't very wide-spread yet. Web services were all SOAP-based (no REST for you!) and jQuery was just barely starting to gather some traction as the de-facto JavaScript abstraction library. HTML and the browser DOM had stagnated for so long that developers were clamoring for a better future.

Enter the era of the Rich Internet Application (RIA). Adobe Flex 3 was released in June, 2007 with a much improved developer feature-set and no longer carried a server licensing requirement. It was way more powerful than the state-of-the-art HTML being used then and many, many web development projects were transitioned from HTML to Flex. Flex's popularity grew for several years and inspired Microsoft's own RIA entry, Silverlight. It seemed as though the future of web applications would be written for browser plug-ins. My own career took a slightly different turn at this point, and for two or three years I worked on .NET WinForm applications but have now found myself on a Flex project for the past little while.

Recently, I've had the opportunity to return to my roots and begin working on a brand new web application. Ah yes, HTML - my old friend. I cracked my knuckles and sat down to dig into everything that was new and wonderful in the world of web development. Even though I had made it a point over the years to stay up-to-date on the terms and technologies being used and advanced in the HTML world, I had no reason to actually learn about them in any depth. So I started to bootstrap my new application, and I have been awash in the staggering number of current techniques that are employed in modern web application design. This ain't your daddy's HTML.

Well, actually, HTML itself hasn't changed too terribly much. The latest version, HTML5, has some new tags, sure, some to help with semantic markup and the Canvas element itself seems poised to open a whole new world of capabilities. It also has its own Video tag to replace Flash videos and adds local isolated storage. But the real changes are with CSS3, JavaScript libraries and frameworks, and the server programming model itself. CSS3 has just about turned into its own programming language, no longer satisfied with simply defining static layout and appearance. While it's unfortunate that complex CSS must still make use of the benefits provided by pre-processors (like LESS), CSS attributes now support complex 3D transformations and even the rule selectors now have a dizzying array of capabilities. Some animation requirements can now be completely defined in CSS without the need for any JavaScript at all. And I thought having rounded corners without images was cool!

And speaking of JavaScript, wow - has it ever come a long way since 2007. Today, only a fool writes JavaScript to directly manipulate the DOM and update HTML elements. jQuery is the de-facto standard library for not only abstracting away the confusing and aggravating differences in DOM implementations across all the various browsers and versions, but also for adding some truly amazing capabilities to JavaScript. And its "fluent" API construction provides the ability to perform multiple operations with a single line of code. It is completely AJAX-aware, making rich client-side interfaces a breeze to implement. jQuery UI, while somewhat less popular that jQuery itself, builds on top of jQuery to provide some user-interface goodies I could only dream of back in 2005.

But jQuery isn't the only JavaScript change in town. JSON (JavaScript Object Notation) has seemingly out of nowhere replaced XML as the preferred inter-platform data exchange format. Like XML it is "human-readable" but it comes without all the heavy tag "weight" of XML and is immediately consumable by JavaScript code without the need for any kind of parser. Marrying data to HTML elements is a new class of JavaScript frameworks that implement the MVC or MVVM patterns, like Knockout.js. These frameworks simulate the automatic model-binding architecture available in other UI frameworks, such as Windows Forms, Silverlight, and Flex.

So, the world of web application development on the client has become much more complex. While there are now a lot of tools out there to help put together a truly advanced UI experience, they continue to be disparate and must be implemented individually. There is yet another class of "utilities" out there that aim to alleviate this problem - the web application template. HTML5Boilerplate calls itself "A rock-solid default for HTML5 awesome" and includes many of the above JavaScript frameworks and a set of cross-platform CSS rules to give developers a strong starting point for new apps. While it brings a lot under one single download, there is still a ton of technology to learn.

And it's not over yet. The server programming model that I came to know and love, ASP.NET, is completely unrecognizable these days. While the old WebForms model still exists and is still getting some love from Microsoft in the latest .NET 4.5 release, the current wonder-child is ASP.NET MVC (which stands for Model-View-Controller). Today, you would be ruthlessly ridiculed if you started a new project using WebForms. ASP.NET MVC does away with the old server control and viewstate style of development and developers now (one could say "again" since the old Active Server Pages framework had the same benefit) have complete control over the HTML output rendered by the server. And the MVC system aims to provide a better "separation of concerns" among the various classes on the server, ostensibly making automated unit-testing a much easier task. Unfortunately, all the skills I learned about handling the WebForms Page object and event model are now completely useless, and so are all the third-party server control libraries I used. But hey, I wouldn't be in this business if there wasn't something new and exciting every other week.

Speaking of new and exciting, what about tooling? Visual Studio 2012 is now out in release-candidate form and this release is all about web development. It seems with Apple's refusal to allow Adobe Flash to run on iOS, and Microsoft's similar stance on eliminating browser plug-ins in the Windows 8 "Metro" version of Internet Explorer (which means no Silverlight, either), we have just experienced the RIApocalyse (a term recently coined by Dan Wahlin) and the whole world is swinging back to HTML development. Visual Studio now ships with jQuery and much work has gone into improving the JavaScript development experience, including vastly improved Intellisense. Add a better CSS editor and a new feature called the Page Inspector and you have a very capable design-time experience. And don't forget the addition of NuGet - a package manager for automatically adding and updating various frameworks, utilities and extensions to your IDE and application projects.

I'm already exhausted just writing this post, and I haven't even mentioned Entity Framework - Microsoft's latest data-access technology du jour. So if you're like me and used to be a web developer back in the glory days of the mid-2000s, then moved into other work and are now back in the HTML world, prepare yourself for a lot of re-learning. The game has certainly changed, but I don't think there's ever been a better time to be a web developer. More capabilities, more utilities, more frameworks, more power!