Category Archives: Development

All things to do with coding

Datatable to XML CSV TAB HTML Json

I have a common functions class (cfc) project i tend to fill with all functions i believe can be applied to anything and will likely be reused.
I was making some adjustments to a few handy functions and I thought it might be useful to publish the following functions which will help convert datatables to various output formats.

Visual Studio references the following libraries within my cfc project, im not entirely sure which ones are required for these functions so i will list them all.

We have 6 functions here
1. ConvertDtToXML
2. ConvertDtToTDF
3. ConvertDtToCSV
4. ConvertDtToJSON
5. ConvertDtToHTML
6. RenderControl

Lets talk about each one briefly.
1. ConvertDtToXML
This function is making use of .NETs Dataset object built in method GetXml, the one simple rule here is that the table must be named in order for the xml to be generated. So we simply declare a dataset, check for a table name, if none then give a standard name, add the table to the dataset and output using GetXml.

2. ConvertDtToTDF
3 things are going on here, the first for loop we get the column headings. Second we begin a for loop on all of the rows, and third we for loop against each column getting the data this time. Each loop builds up the TAB delimited string we end up with.

3. ConvertDtToCSV
A very similar approach to the tab delimited has been taken here with the CSV, its the text used to build the sting that differs.

4. ConvertDtToJSON
We make use of .NETs Json serializer to convert our datatable to a json string

5. ConvertDtToHTML and 6. RenderControl
This final converter function ConvertDtToHTML needs to use last function RenderControl.
Load the datatable into a gridview, bind it, then extract the rendered HTML from it.

ASP.NET Custom Table and Object Caching How To

We have a very complex website, it involves recording everything that is ever rendered and some complex randomising of what we render. Our website is basically a pay per click revenue stream.

The Problem
Ok so the problem is, we rely on caching of a select number of data tables, this may sound silly to do but we are careful and we dont exceed the hardware capacity of the machine, and alas there is no other suitable method for us. The setup we have is IIS running in State Server mode using the ASP.NET state server Windows Service. Some of you will already know this is based on an XML filesystem structure to hold everything that gets placed into the cache.
Now for some reason state server started to break down and cause table and row corruption, we never managed to nail down why this was occuring since ASP.NET Datatables can be serialised and deserialised seemlessly from and to XML we concluded it was to do with asynchronous access to the objects in the cache breaking down. We needed to resolve the problem and we needed to do it now, I decided to write our own caching class that would sit in the Application Object.

The Solution
So a few basic rules here
1. The Class lives in the application object ergo when the website restarts the class disintigrates and re instantiates
2. The Class must be able to handle asynchronous access
3. The Class must offer similar features as ASP.NET cache, ie specifying lifetime of objects
4. The Class must also run maintenance on itself as a result of requirement 3
5. The Class must support Add, Get, Remove of datatables and objects

Ok so now we have some ground rules, it took me about a day to write, and the finished product should be attached to this post.

In your App_Code folder drop these three files (obviously rename them to .vb first).

Now heres how to use it.
1. Instantiate the class in the Global.asax file Application_Start routine (brings everything to life)
2. Attach a new timer to the maintenance event of the class _RunMaintenance (tells itself to cleanup)
3. Start the timer in a new thread .RunAsNewThread (opens the timer in a thread seperate from the application object, we like this seperation)
4. Add the class to the application object (so you can access it from anywhere in the app)

So starting in your global asax

Heres the code snippet you can use as an example of starting the class up, this can been seen inside SiteStart

OK so now the class is active how do i call it i hear you ask, well you have a choice.
If you have a base page then add this to it

If you dont then in your code behind file do this

Once you have done one of the above you can make the calls like so


The reason we start a timer in a seperate thread is because we want to call the maintenance routine periodically and the only way to keep a timer allive from the application_start is to create a seperate thread, forget trying to place it in the application object or cache because it wont stay runtime, it will be deserialised into a static xml structure describing how to rebuild it.

Thats about it. You could attached a sub rountine that writes an information event to the windows event log when it runs, so you can make sure it works. You could also create a page that pulls out the current state of the class, ie how many tables its holding, what the names are etc etc.

Using EXISTS with variable tables

I ran into a need to use EXISTS with a variable table yesterday and thought it might be nice to make it available to others.

As you can see all you need to do is enclose the variable table in [ ].

GridView with Nested GridView in

Have you ever needed to nest controls within gridviews
Its actually pretty simple these days. At first i was a bit stuck but having now successfully nested many types of controls within a gridview im fairly confident its possible to nest anything, and likely down to an infinite nest tree.
So lets keep it simple
What we are going to do is have a primary gridview (grid1), with a few button controls on each row (btnexpand btnselect btndelete), a nested gridview with a couple of buttons (btnselect btndelete) and a nested control within it.
I will show you how to wire up events to handle the nested controls.
You are most likely better served by copying this code out into your favorite editor, visual studio – scite whichever your preference.

Firstly Lets look at the button controls in Grid1. The grid routine rowdatabound wires up the button control command argument to hold the ID of the record like so, this gives us a hook back to the data.

The grid rountine rowcommand is used to handle the button events.

This covers the primary grid button routines, now lets look at the nested grid buttons, how do we handle those. In the same fashion, the nested grid data would likely have been bound when we had clicked on the primary grids select button. We are only concerned at handling the nested grid button wireup and click event handler.

Simply set the “OnCommand” property of the button to our custom subroutine

Note the routine required parameters sender and e.

And that as they say is that. You have now learnt how to wireup control events through nested gridviews.

Run 32 bit Apps in IIS on 64 bit OS

So you want to run your web Apps in 32 bit mode on your 64 bit windows operating system
Ive never needed to do this until now, Cybersouce, the online financial gateway we have recently integrated with dont support 64 bit when you use their .NET API.

OK all you need to do is
1. IIS management >> Local computer node >> Web Server Extensions >> set the “ASP.NET vr xxxx (32 bit)” to Allow
2. Run the following batch command (changing the folder targets if you need to)
cscript c:inetpubadminscriptsadsutil.vbs SET W3SVC/AppPools/Enable32bitAppOnWin64 1
C:WINDOWSMicrosoft.NETFrameworkv2.0.50727aspnet_regiis.exe -i


If things go terribly wrong and you need to revert to 64bit then run the following batch
cscript c:inetpubadminscriptsadsutil.vbs SET W3SVC/AppPools/Enable32bitAppOnWin64 0
C:WINDOWSMicrosoft.NETFramework64v2.0.50727aspnet_regiis.exe -i


Display your latest twitter on your WP Blog

Copy and paste this code anywhere in your website
originally posted by

IIS, ASP.NET, (RSS feed Validation missconfigured server), Content-Length, Compression

An interesting problem was elevated to my desk recently.

An RSS feed written in was failing validation at
The reason given was a missconfigured server.

After looking into this a little further i discovered the root of the problem was not a miss configured server it was a content length related issue.
IIS has two forms of compression, dynamic and static. Static content can be a jpg, a static html file or other such content that does not change and has a defined “size and shape”. where as pages and other such content are considrered dynamic.
Now, IIS when compressing static content knows the size of the object and hence passes the content length to the client in the header of the response stream, so the client knows how much is left to recieve.
If the content length is not passed down to the client then it is possible the client may get confused when it starts receiving all these chunks from IIS and doesnt when they are gong to stop inevitably leading to a splat.

This is also indicative of a response where .ContentLength = -1 so in other words the webserver is chunking the response.

Another important note is that IIS will only compress content to the client if the client has notified IIS it can accept compressed content, this is a header value buried in the header of the request.

However in this particuar instance it would seem that even though the validation engine was compatible with compression, it didnt like dynamic compression. Im unsure if this was a problem with the validation website or if this was an RSS compatibility requirement.

So in other words we have to provide IIS with the content length in order that it can send this information down to the client.

This is simple enough if you have ever needed to stream any type of content across any medium in .NET.

The current way the rss was being transmitted to the client was via the response.write method of the output response stream. This way may appear adequate in your development test browser (typically IE 7 or 8 ) and until you look deeper and apply lifecycle testing across all media you wont see that IIS isnt passing certain required information to the client (ie content length)

So a simple application of binary streaming should fix the problem.

To begin with the original code is correct to a point starting with setting the response type parameters like so

so following on with the original code we create our TextWriter and pass it into our XmlTextWriter to populate with our XML feed

now we can write our xml feed.
Once the xml feed is written we are ready to deposit the feed to the client. We are going to use the Response.BinaryWrite method of the response stream

Essentially what we have done is pull the xml text into a buffer of a specific size based on the xml length, and passed it directly to the BinaryWrite response method which in turn writes directly to the current httpcontext response buffer without character conversion. IIS knows then content length deduced from the binary buffer size and passes this information down to the client followed on with however many chunks of data until the transafer is complete.

this is the reflector of the binaryWrite method

This type of response can also be applied to passing files down to a client request.
Using this approach removes the required “browse directory” permissions one needs to apply to the normal style virtual directory containing your static content such as pdf’s that you want made available for download.
This approach also allows you to hold all of your content in the same location (DB or file system) and you can apply rights to users accessing these files via .net rather than being restricted to NT authentication on the virtual directory simply by removing the need for teh virtual directory.

ASP.NET IIS 7 Visual Studio 2008


This is a strange post but i decided a long time ago i should really start collating all my knowledge in one place.
I have so many “mht” files and random txt documents with bug fixes, solutions, how to’s, best practices etc etc i should get round to putting it in one place so its easy to find.

Anyway so here is the latest bug i hjave discovered.

I was re organising our companies .NET solutions NameSpaces to align them in more of a meaning fashion. I also reorganised the folder structure of the projects so that common projects (projects used in more than one solution) are accessible from relative referencing in the various solutions.

Long story short i discovered after much work that i could not run a website in debug mode working from the locahost IIS. VS 2008 >> web >> settings >> startup options >> under the server settings select use custom server and http://localhost  or whatever you have bound to the website on the local machine

This is the error i was recieving

Microsoft Visual Studio
Unable to start debugging on the web server. The debugger cannot connect to the remote computer. This may be because the remote computer does not exist or a firewall may be preventing communication to the remote computer. Please see Help for assistance.
OK   Help �

Now after much research and trial and error i discovered that of the miriad of possible reasons it could be (none of the documented ones applied to me) i stumbled on the answer by experimentation. If the root folder, or any folder within the path of the website is named with the same name as the base namespace you use, this is the error you get.

so simple, just make sure you dont have a folder named after your namespace