Building high-performance ASP.NET applications


If you are building public facing web sites one of the things you want to achieve at the end of the project is a good performance under the load for your web site. That means, you have to make sure that your product works under a heavy load (e.g. 50 concurrent users, or 200 users per second etc.) even though at the moment you don’t think you would have that much load. Chances are that the web site attracts more and more users over time and then if it’s not a load tolerant web site it will start flaking, leaving you with an unhappy customer and ruined reputation.

There are many articles on the Internet about improving the performance of ASP.NET web sites, which all make sense; however, I think there are some more things you can do to save yourself from facing massive dramas. So what steps can be taken to produce a high-performance ASP.NET or ASP.NET MVC application?

  • Load test your application from early stages

Majority of developers tend to leave performing the load test (if they ever do it) to when the application is developed and has passed the integration and regression tests. Even though performing a load test at the end of the development process is better than not doing it at all, it might be way too late to fix the performance issues once your code has been already been written. A very common example of this issue is that when the application does not respond properly under load, scaling out (adding more servers) is considered. Sometimes this is not possible simply because the code is not suitable for achieving it. Like when the objects that are stored in Session are not serializable, and so adding more web nodes or more worker processes are impossible. If you find out that your application may require to be deployed on more than one server at the early stages of development, you will do your tests in an environment which is close to your final environment in terms of configuration and number of servers etc., then your code will be adapted a lot easier.

  • Use the high-performance libraries

Recently I was diagnosing the performance issues of a web site and I came across a hot spot in the code where JSON messages coming from a third-party web service had to be serialized several times. Those JSON messages were de-serialized by Newtonsoft.Json and tuned out that Newtonsoft.Json was not the fastest library when it came to de-serialization. Then we replaced Json.Net with a faster library (e.g. ServiceStack) and got a much better result.

Again if the load test was done at an early stage when we picked Json.Net as our serialization library we would have find that performance issue a lot sooner and would not have to make so many changes in the code, and would not have to re-test it entirely again.

  • Is your application CPU-intensive or IO-intensive?

Before you start implementing your web site and when the project is designed, one thing you should think about is whether your site is a CPU-intensive or IO-intensive? This is important to know your strategy of scaling your product.

For example if your application is CPU-intensive you may want to use a synchronous pattern, parallel processing and so forth whereas for a product that has many IO-bound operations such as communicating with external web services or network resources (e.g. a database) Task-based asynchronous pattern might be more helpful to scale out your product. Plus you may want to have a centralized caching system in place which will let you create Web Gardens and Web Farms in future, thus spanning the load across multiple worker processes or serves.

  • Use Task-based Asynchronous Model, but with care!

If your product relies many IO-bound operations, or includes long-running operations which may make the expensive IIS threads wait for an operation to complete, you better think of using the Task-based Asynchronous Pattern for your ASP.NET MVC project.

There are many tutorials on the Internet about asynchronous ASP.NET MVC actions (like this one) so in this blog post I refrain from explaining it. However, I just have to point out that traditional synchronous Actions in an ASP.NET (MVC) site keep the IIS threads busy until your operation is done or the request is processed. This means that if the site is waiting for an external resource (e.g. web service) to respond, the thread will be busy. The number of threads in .NET’s thread pool that can be used to process the requests are limited too, therefore, it’s important to release the threads as soon as possible. A task-based asynchronous action or method releases the thread until the request is processed, then grabs a new thread from the thread pool and uses it to return the result of the action. This way, many requests can be processed by few threads, which will lead to better responsiveness for your application.

Although task-based asynchronous pattern can be very handy for the right applications, it must be used with care. There are a few of concerns that you must have when you design or implement a project based on Task-based Asynchronous Pattern (TAP). You can see many of them in here, however, the biggest challenge that developers may face when using async and await keywords is to know that in this context they have to deal with threads slightly differently. For example, you can create a method that returns a Task (e.g. Task<Product>). Normally you can call .Run() method on that task or you can merely call task.Result to force running the task and then fetching the result. In a method or action which is built based on TBP, any of those calls will block your running thread, and will make your program sluggish or even may cause dead-locks.

  • Distribute caching and session state

    It’s very common that developers build a web application on a single development machine and assume that the product will be running on a single server too, whereas it’s not usually the case for big public facing web sites. They often get deployed to more than one server which are behind a load balancer. Even though you can still deploy a web site with In-Proc caching on multiple servers using sticky session (where the load balancer directs all requests that belong to the same session to a single server), you may have to keep multiple copies of session data and cached data. For example if you deploy your product on a web farm made of four servers and you keep the session data in-proc, when a request comes through the chance of hitting a server that already contains a cached data is 1 in 4 or 25%, whereas if you use a centralized caching mechanism in place, the chance of finding a cached item for every request if 100%. This is crucial for web sites that heavily rely on cached data.

    Another advantage of having a centralized caching mechanism (using something like App Fabric or Redis) is the ability to implement a proactive caching system around the actual product. A proactive caching mechanism may be used to pre-load the most popular items into the cache before they are even requested by a client. This may help with massively improving the performance of a big data driven application, if you manage to keep the cache synchronized with the actual data source.

  • Create Web Gardens

As it was mentioned before, in an IO-bound web application that involves quite a few long-running operations (e.g. web service calls) you may want to free up your main thread as much as possible. By default every web site is run under one main thread which is responsible to keep your web site alive, and unfortunately when it’s too busy, your site becomes unresponsive. There is one way of adding more “main threads” to your application which is achievable by adding more worker processes to your site under IIS. Each worker process will include a separate main thread therefore if one is busy there will be another one to process the upcoming processes.

Having more than one worker process will turn your site to a Web Garden, which requires your Session and Application data be persisted out-proc (e.g. on a state server or Sql Server).

  • Use caching and lazy loading in a smart way

    There is no need to emphasize that if you cache a commonly accessed bit of data in memory you will be able to reduce the database and web service calls. This will specifically help with IO-bound applications that as I said before, may cause a lot of grief when the site is under load.

    Another approach for improving the responsiveness of your site is using Lazy Loading. Lazy Loading means that an application does not have a certain piece of data, but it knows that where is that data. For example if there is a drop-down control on your web page which is meant to display list of products, you don’t have to load all products from the database once the page is loaded. You can add a jQuery function to your page which can populate the drop-down list the first time it’s pulled down. You can also apply the same technique in many places in your code, such as when you work with Linq queries and CLR collections.

  • Do not put C# code in your MVC views

    Your ASP.NET MVC views get compiled at run time and not at compile time. Therefore if you include too much C# code in them, your code will not be compiled and placed in DLL files. Not only this will damage the testability of your software but also it will make your site slower because every view will take longer to get display (because they must be compiled). Another down side of adding code to the views is that they cannot be run asynchronously and so if you decide to build your site based on Task-based Asynchronous Pattern (TAP), you won’t be able to take advantage of asynchronous methods and actions in the views.

    For example if there is a method like this in your code:

    public async Task<string> GetName(int code)

    {

    var result = …

    return await result;

    }

This method can be run asynchronously in the context of an asynchronous ASP.NET MVC action like this:

    public Task<ActionResult> Index(CancellationToken ctx)

{

    var name = await GetName(100);

}

But if you call this method in a view, because the view is not asynchronous you will have to run it in a thread-blocking way like this:

var name = GetName(100).Result;

.Result will block the running thread until GetName() processes our request and so the execution of the app will halt for a while, whereas when this code is called using await keyword the thread is not blocked.

  • Use Fire & Forget when applicable

If two or more operations are not forming a single transaction you probably do not have to run them sequentially. For example if users can sign-up and create an account in your web site, and once they register you save their details in the database and then you send them an email, you don’t have to wait for the email to be sent to finalize the operation.

In such a case the best way of doing so is probably starting a new thread and making it send the email to the user and just get back to the main thread. This is called a fire and forgets mechanism which can improve the responsiveness of an application.

  • Build for x64 CPU

32-bit applications are limited to a lower amount of memory and have access to fewer calculation features/instructions of the CPU. To overcome these limitations, if your server is a 64-bit one, make sure your site is running under 64-bit mode (by making sure the option for running a site under 32-bit mode in IIS is not enabled). Then compile and build your code for x64 CPU rather than Any CPU.

One example of x64 being helpful is that to improve the responsiveness and performance of a data-driven application, having a good caching mechanism in place is a must. In-proc caching is a memory consuming option because everything is stored in the memory boundaries of the site’s application pool. For a x86 process, the amount of memory that can be allocated is limited to 4 GB and so if loads of data be added to the cache, soon this limit will be met. If the same site is built explicitly for a x64 CPU, this memory limit will be removed and so more items can be added to the cache thus less communication with the database which leads to a better performance.

  • Use monitoring and diagnostic tools on the server

    There might be many performance issues that you never see them by naked eyes because they never appear in error logs. Identifying performance issues are even more daunting when the application is already on the production servers where you have almost no chance of debugging.

    To find out the slow processes, thread blocks, hangs, and errors and so forth it’s highly recommended to install a monitoring and/or diagnostic tool on the server and get them to track and monitor your application constantly. I personally have used NewRelic (which is a SAS) to check the health of our online sites. See HERE for more details and for creating your free account.

  • Profile your running application

    Once you finish the development of your site, deploy it to IIS, and then attach a profiler (e.g. Visual Studio Profiler) and take snapshots of various parts of the application. For example take a snapshot of purchase operation or user sign-up operation etc. Then check and see if there is any slow or blocking code there. Finding those hot spots at early stages might save you a great amount of time, reputation and money.

Advertisements

Creating a Captcha control – Part:2


In this post I will explain how to generate a hard-to-read image out of our Captcha text. The new Captcha with image will look like this:

Final Captcha image

In MyCaptcha control, each letter is different from other ones in three properties:

1- Font

2- Size

3- Distance from the next letter (character spacing)

Therefore, I wrote a class named Letter that each instance of it holds a character along with all its properties like it’s font name and size. The class has a constructor that accepts a character argument and assigns random properties to it:

public class Letter

{

string[] ValidFonts = {“Segoe Script”, “Century”, “Eccentric Std”,“Freestyle Script”,“Viner Hand ITC”};

public Letter(char c)

{

Random rnd = new Random();

font = new Font(ValidFonts[rnd.Next(ValidFonts.Count()-1)], rnd.Next(20)+20, GraphicsUnit.Pixel);

letter = c;

}

public Font font

{

get;

private set;

}

public Size LetterSize

{

get

{

var Bmp = new Bitmap(1, 1);

var Grph = Graphics.FromImage(Bmp);

return Grph.MeasureString(letter.ToString(), font).ToSize();

}

}

public char letter

{

get;

private set;

}

public int space

{

get;

set;

}

}

As you see in the above source code, I pick a random font name from ValidFonts array. The font names in ValidFonts array are Windows Vista fonts. You must keep in mind that you use font names that exist on your Web Server. I also recommend you to use fantasy fonts (like gothic) to make the produced image more hard-to-read.

I also have added a property of type Size to get the width and height of the letter when it is rendered with its own font. To get the character size I use Graphics.MeasureString method.

The ‘space’ property is set when the Captcha text is being rendered. To render the captcha image, I use a generic handler (.ashx) file. Using a generic handler we can render any output type and send it to the output stream. An .ashx file can be treated like an .aspx file but has noticeably fewer overhead. For example we can pass query strings to it and generate the output based on it.

I will send the captcha text as a query string called CaptchaText to GetImgText.ashx generic handler. In the .ashx code, I will make an instance of Letter class for each character.

var CaptchaText = context.Request.QueryString[“CaptchaText”];

if (CaptchaText != null)

{

List<Letter> letter = new List<Letter>();

int TotalWidth = 0;

int MaxHeight = 0;

foreach (char c in CaptchaText)

{

var ltr = new Letter(c);

int space = (new Random()).Next(5) + 1;

ltr.space = space;

letter.Add(ltr);

TotalWidth += ltr.LetterSize.Width+space;

if (MaxHeight < ltr.LetterSize.Height)

MaxHeight = ltr.LetterSize.Height;

System.Threading.Thread.Sleep(1);

}

As the above piece of code shows, all Letter instances are stored in letter generic list. The width of each letter plus its distance with the next letter is summarized in TotalWidth local variable. We also get the height of the biggest letter so that we get sure all letters fit in the captcha image.

I also have two constants for vertical and horizontal margins:

const int HMargin = 5;

const int VMargin = 3;

Thus, our image will have a size of VMargin+MaxHeight and HMargin+TotalWidth:

Bitmap bmp = new Bitmap(TotalWidth + HMargin, MaxHeight + VMargin);

var Grph = Graphics.FromImage(bmp);

At next step, I will draw each letter with it’s own font size and position it according to it’s space property:

int xPos = HMargin;

foreach (var ltr in letter)

{

Grph.DrawString(ltr.letter.ToString(), ltr.font, new SolidBrush(Color.Navy), xPos, VMargin );

xPos += ltr.LetterSize.Width + ltr.space;

}

Now the image is cooked and ready! We should send it to the output stream:

bmp.Save(context.Response.OutputStream, System.Drawing.Imaging.ImageFormat.Jpeg);

Up to now the Captcha text is sketched well but I’d like to smudge it a little more in order to make it more hard-to-read. To do is I will draw a number of circles on random positions of the image. One nice idea is to define an IShape interface with a method named Sketch(). Then define different classes that implements IShape and draws different shapes on the image.

By the way, the code that smudges the image is this:

Color[] Colors = { Color.Gray , Color.Red, Color.Blue, Color.Olive };

for (int i = 0; i < 200; i++)

{

var rnd = new Random();

var grp = Graphics.FromImage(bmp);

grp.DrawEllipse(new Pen(Colors[rnd.Next(3)]), rnd.Next(bmp.Width – 1), rnd.Next(bmp.Height – 1), 5, 5);

System.Threading.Thread.Sleep(1);

}

You can download the source code of this control from HERE. The password of archived file is: aspguy.wordpress.com

There is also a project for this control at CodePlex. You can upload any changes you might make on the source code.

Have fun with Captchaing!

Aref Karmi

9 Apr 2009

Creating a Captcha control – Part:1


In this post and a post after I will explain how to develop a Captcha control and use it in an ASP.NET web site.

As described in Wikipedia,  A CAPTCHA or Captcha (IPA: /ˈkæptʃə/) is a type of challenge-response test used in computing to ensure that the response is not generated by a computer. In a crude term, a captcha control shows some hard-to-read letters on the screen and asks the user to enter the text in a box. Then checks to see whether the entered text is correct or not.

There are a lot of Captcha controls for ASP.NET that you can download and use in your project but, none is as interesting as the one that you write yourself and know exactly how it works!

I will explain the techniques of writing a simple but powerful Captcha control in two posts because it is made of two parts:

  1. The captcha control that displays a text and asks users to enter text in a box then validates it.
  2. The class that renders the Captcha text as a hard-to-read image.

Because the 2nd part can be reused for different purposes, for example in your own Captcha control, I will describe it in a different part.

OK let’s get started. The Captcha control that we are going to write has the following specs:

  1. Is developed as an .ascx control so it is reusable in every ASP.NET website.
  2. Saves nothing on your hard-disk so if you open a same page in two different windows (or tabs) there will be no conflict between two Captcha controls.
  3. Is very easy to use and needs no complicated concept like understanding and using http handlers.

The only drawback that I can count about this control is that it stores the generated text in ViewState thus, you must always encrypt it.

To create this control first I new a website. Then, I add a .ascx (web user control) to it called MyCaptcha. It looks like this:

Captcha xhtml script

In the above code,  lbltext displays the captcha text. txtCpatcha is a text box in which the user must enter the code. There is also a button named btnTryNewWords that re-generates the code if user has difficulties in reading it.

The control looks like this in design mode:

c2

The control has a property named LetterCount that specifies the number of letters in the code:

public

int LetterCount { get; set; }

It also has a private property that holds the generated key in ViewState.

private

string GeneratedText{

get{

return ViewState[this.ClientID + “text”] != null ?ViewState[

this.ClientID + “text”].ToString() : null;}

set

{

// Encrypt the value before storing it in viewstate.

ViewState[this.ClientID + “text”] = value;

As commented in the above code, you are highly recommended to encrypt the key before you put it in ViewState.

To generate the captcha text I wrote a public method called TryNew() that picks random letters from a list of characters and combine them to each other:

public void TryNew()
{
char[] Valichars = {‘1′,’2′,’3′,’4′,’5′,’6′,’7′,’8′,’9′,’0′,’a’,’b’,’c’,’d’,’e’,’f’,’g’,’h’,’i’,
‘j’,’k’,’l’,’m’,’n’,’o’,’p’,’q’,’r’,’s’,’t’,’u’,’v’,’w’,’x’,’y’,’z’ };
string Captcha = “”;
int LetterCount = MaxLetterCount >5 ? MaxLetterCount : 5;
for (int i = 0; i < LetterCount; i++)
{
int index = new  Random(DateTime.Now.Millisecond).Next(Valichars.Count()-1);
Captcha += Valichars[index].ToString().ToUpper();
Thread.Sleep(1);
}
GeneratedText = Captcha;
lbltext.Text = Captcha;
}

Because the captcha control won’t be case-sensitive, there is no capital letter in ValidChars array.

You also may have noticed the Thread.Sleep(1) statement in the above code! Why should we stop the running thread for one millisecond?!

The answer is that Random class uses DateTime.Now.Millisecond as it’s default seed. If the computer is too fast and the loop that selects and combines the letters is too short (for example 5 loops only) , the loop begins and ends in a millisecond or even a shorter period of time. Therefore, you will get equal letters only. To see and feel what I mean (!) remove Thread.Sleep(1) line and you will get a text like AAAAAA or 333333.

Anyway, there is also another public property called IsValid that indicates if the entered text is equal to captcha text:

public

bool IsValid{

get

{

bool result = GeneratedText.ToUpper() == TxtCpatcha.Text.Trim().ToUpper();

if (!result)TryNew();

return result;

}

}

That’s it. Now to test the control , drag MyCaptcha.ascx and drop it on Default.aspx page.  Your .aspx page will be like this:

<

uc1:MyCaptcha ID=”MyCaptcha1″ runat=”server” />

<br />

<asp:Label ID=”lblCheckResult” runat=”server” Text=”?”></asp:Label>

<br />

<asp:Button ID=”btnCheck” runat=”server” onclick=”btnCheck_Click”

Text=”Check it!” />

in btnCehck_Click event handler, write the following piece of code:

if

(MyCaptcha1.IsValid)lblCheckResult.Text =

“It is ok” ;

else

lblCheckResult.Text =

“oops!, invalid text was entered.” ;

After running the website, you will have a captcha control like the image bellow:

c3

In the next post I will show you how to scramble the text in a way that only a human with a high eye-sight can read it 🙂

I also might add some extra codes so that it can work along with other Validation controls on the page.

Creating a Log Visit Report


oops.. today I’d decided to post a new article but surprisingly noticed that the blog was suspended! Thanks to Matt, it is alive and kicking now!

Anyway, lets see how we can generate a log visit report for our ASP.NET web site. By the term of Log Visit, I mean the statistics of a website which shows at which time, which page (or file) has been viewed by who!

The good news is that IIS can record the above information for us. It can log all activities done by your web site in the following ways and formats:

W3C Extended log file format

World Wide Web Consortium (W3C) Extended format is customizable and is saved in an ASCII text file. You can select your desired fields among a lot of choices.

IIS log file format

IIS log file format has  fixed columns and is stored as an ASCII text file. This file includes more information than W3C format.

NCSA Common log file format

National Center for Supercomputing Applications (NCSA) Common log file format is a fixed ASCII format that is available for Web sites, but not for FTP sites

ODBC logging

Open Database Connectivity (ODBC) logging format is a record of a fixed set of data properties in a database.

Logging into text files is quicker than logging into a database. It does not have any performance pitfall but when it comes to reporting, you probably should import the text file into a database.

Logging by ODBC will slow down your website to some extent  because it records very much of (sometimes unnecessary) information. For example, in W3C format, only information about pages who have been loaded successfully is stored. While in ODBC method, all referred objects, like .js or image files, are recorded, no metter if the resource have been loaded successfully or not.

By the way, since ODBC will make this post shorter, I will choose ODBC method for my sample program.

To enable logging with ODBC, first you must create a table for storing log information and then create an ODBC data source thorough which IIS can store logs.

To create the log table, which is usually called INETLOG, go to System32\InetSrv folder and find LogTemp.Sql file. This file includes a script that creates the required table into your database. This script has been written for Sql Server, but you can modify it to be executed on other databases like MS Access or even My Sql.

In the next step, create a SYSTEM DNS that points to the above table. You must create a System DNS because IIS will use the DNS with a user different from the one you have logged into your computer (usually NetWork Service account).

To create a new System DSN, go to Control Panel-> Administrative Tools and then double-click Data Sources (ODBC) . Then, go to System DNS tab and click Add. Select SQL Server, enter the name of your server and database and specify a user/password. You would better enter a SQL SERVER login (do not choose Windows Authentication) and specify a password that will not be changed.

After doing so, change the default database of this DSN to the database that logging table resides on. Click OK and close the dialog box.

Now go to Control Panel -> Administrative Tools  and double-click on Internet Information Services Manager.

Open Web Sites, and then find your Virtual Directory or Web Site. Right-click on it, choose properties and under the Home Directory tab, find Enable Logging check box. Enable the check box and In the Active log format list box, select ODBC. Afterwards, click on Configure button to view the ODBC settings. In this dialog box, enter the name of ODBC DSN you created in the previous step, name of the table (inetlog), user name and password of logging data base.

For example:

ODBC: WordPressLogDSN

Table: InetLog

Login: sa

Password: 123

In the above sample I used “SA” SQL SERVER’s default login name. This is a risky job and I do not recommend you do it! You would better use a user name with no administrative permission which only has access to logging table or database.

From now on, any visit to a page will be recorded!

NOTICE: The logging table will be filled with tons of records, therefore, I strongly recommend you to store InetLog table on a seperate FileGroup. Also, You would better to schedule a script that removes unnecessary records from inetlog table.

Writing The Code

In the rest of the post I am going to write a sample page that displays how many times each .aspx or .html page has been visited. I will show the stats in numeric and bar chart format.

As I mentioned, data about any referred resources will be stored in InetLog table. Thus, we should be able to determine if a record belongs to a Page or not. Only pages whose ServiceStatus is equal to 200 have been loaded or access with no error. So, I need a Sql function that determines if a resource is a page (i.e. .aspx) and if it has been accessed successfully.

I will call the function IsPage. The code is very simple so I just bring the script body here:

create function IsPage(   @PageName varchar(200) )
returns int
begin
Declare @len int
set @len = len(@PageName)

if Substring(@PageName, @Len, 1) = ‘/’
set @PageName = Substring(@PageName,1,@Len-1)

Declare @Extension  Varchar(4)
Set @Extension = Upper(Substring(@PageName,@Len – 3 , 4))

if @Extension = ‘ASPX’ or @Extension = ‘HTML’
return 1
return  0
end

I will use this function to create my stats and also to remove records of non-page resources like .js files (if I don’t need them).

At the next step I will write a stored procedure that determines how many times each page has been visited. To let the script run on older versions of Sql Server, I have written it as simple as possible:

create procedure rep_LogVisit
@BeginDate smalldatetime,
@EndDate smalldatetime
as
select Count(1) as FileCount, target as PageName
from inetlog
where ServiceStatus = 200 and dbo.isPage(target) =1
and LogTime>=@BeginDate and LogTime<=@EndDate
group by dbo.GetFileName(target)
go

As the above code indicates, rep_LogVisit counts the records who’s ServiceStatus equals to 200 (successful) , the resource is a page and the log has been recorded between a given date.

In order to display the result, I use a GridView control, having two columns. One for the page name, and one for the visit count:

<asp:GridView ID=”GridView1″ runat=”server” AutoGenerateColumns=”False”

CellPadding=”4″ ForeColor=”#333333″

GridLines=”None”>

<Columns>

<asp:BoundField DataField=”target” HeaderText=”Page Name” />

<asp:TemplateField HeaderText=”Visit Count”>

<ItemTemplate>

<asp:Panel runat=”server” ID=”divBar” style=”background-color:Red” Width='<%#GetLength((int)Eval(“TotalCount”)) %>’>

<asp:Label runat=”server” ID=”lblCount” text='<%#Eval(“TotalCount”) %>’ ForeColor=”White”></asp:Label>

</asp:Panel>

</ItemTemplate>

</asp:TemplateField>

</Columns>

</asp:GridView>

Well, I will call  rep_LogVisit stored procedure, bind the grid to it’s result. I also have to write a function called GetLength in order to calculate the lenth of each bar:

int MaxCount = 0;

protected void Page_Load(object sender, EventArgs e)

{

if (!IsPostBack)

{

SqlConnection conn = new SqlConnection(“Your Connection String Here”);

SqlCommand cmd = conn.CreateCommand();

cmd.CommandType = CommandType.StoredProcedure;

cmd.CommandText = “rep_LogVIsit”;

// Add date parameters here. skipped to simplify the code.

DataTable table = new DataTable();

SqlDataAdapter adapter = new SqlDataAdapter(cmd);

adapter.Fill(table);

// Get the bigger visit count

MaxCount = (int)(from C in table.AsEnumerable()

select C).Max(c=>c[“TotalCount”]);

GridView1.DataSource = table;

GridView1.DataBind();

}

}

// Calculate the length of each bar

internal int GetLength(int TotalCount)

{

if (MaxCount == 0)

return 0;

else

return Convert.ToInt32(Math.Round(((double)TotalCount / MaxCount)*300));

}

At run-time, you will see something like the image bellow:

untitled1

Reading and writing images from/to database


Hi,
This post will show you how to save/load images to/from a database. This approach lets you do it without having to save the image on to disk. On a real hosting computer, you probably will not have write permissions. Therefore, it is sometimes vital to do image saving/loading on the fly.

In this post we will work on a database named Personnel, in which there is a table called Personnel:

create table Personnel
(
id int identity(1,1) not null primary key,
Fillname varchar(100) not null,
Picture image null
)

We are going to store out personnel’s name and picture into this table. On the application side, create a .aspx page with the following four controls:
1-a TextBox
2-a RequiredFieldValidator
3-a FileUpload control
4-a Button
5-a DataList control

Your page will look like this:

As the above image indicates, we will save each person’s full name and picture to database and the personnel information will be viewd in a DetailList control.

Behind the AddRecord button, we read the image into a Byte[] object. Then transmit the bytes to database. I strongly recommend you to use a stored procedure (if you are working with ADO.NET) because you won’t face any problems with converting bytes to string.

In Asp.NET there is a class claeed HttpPostedFile. This class lets us get full information about the file that is about to upload.  The code bellow show how we read the file:

if (FileUpload1.HasFile)
{
HttpPostedFile myFile = FileUpload1.PostedFile;
int Length = myFile.ContentLength;
string ContentType = myFile.ContentType.ToUpper();

if (Length == 0)
throw new Exception(“File size must be greater than zero!”);
if (ContentType.CompareTo(“IMAGE/PJPEG”) != 0 && ContentType.CompareTo(“IMAGE/JPEG”) != 0)
throw new Exception(“Only JPEG files are welcome!”);

Byte[] myFileBytes = new byte[Length];
myFile.InputStream.Read(myFileBytes, 0, Length);

In the above code I just allow JPEG files to be uploaded. Actually, I wanted to show you how to control the image type and infact, you can upload any file type.

As the last line of this code shows, HttpPostedFile class contains an inner stream from which we can read the image bytes. After reading the image file, we simply save it into db. I have created a stored procedure for doing this and I call this procedure from my C# code:

ALTER PROCEDURE dbo.SavePersonnel
@FullName Varchar(100),
@Picture Image

AS
Insert into Personnel (FullName,Picture) Values (@FullName,@Picture )
Return

C#:

string cnStr = ConfigurationManager.ConnectionStrings[“ConnectionString”].ToString();
using (SqlConnection connection = new SqlConnection(cnStr))
{
SqlCommand cmd = connection.CreateCommand();
cmd.CommandText = “dbo.SavePersonnel”;
cmd.CommandType = CommandType.StoredProcedure;
cmd.Parameters.AddWithValue(“FullName”, txtFullName.Text.Trim());
cmd.Parameters.AddWithValue(“Picture”, myFileBytes);
connection.Open();
cmd.ExecuteNonQuery();
}

Well, now how to read the image and show it using a System.Web.UI.WebControls.Image control. Unfortunately, despite System.Drawing.Image class, ASP.NET Image class does not provide a public stream. Therefore we can not create the image content withouth an image url.

Thanks to ASP.NET Generic Handlers, we can overcome this problem easily. We may develop a .aspx or .ashx (generic handler) file and use it to read the file from database and write the image bytes into the response output stream. A generic handler (.ashx) file is much more light weight than a .aspx file. Therefore, I will develop a .ashx file. We will pass the ID of each person to this .ashx file. The generic handler then finds the person at DB, reads the image and writes it to the current HttpContext instance :

Here is the full source code of the .ashx file:

<%@ WebHandler Language=”C#” Class=”GetPersonnelImage” %>

using System;
using System.Web;
using System.Data.SqlClient;

public class GetPersonnelImage : IHttpHandler {

public void ProcessRequest (HttpContext context) {
context.Response.ContentType = “image/jpeg”;
if (context.Request.QueryString[“id”] == null || context.Request.QueryString[“id”] == “”)
return;
else
{
string cnStr = System.Configuration.ConfigurationManager.ConnectionStrings[“ConnectionString”].ToString();
using (SqlConnection connection = new SqlConnection(cnStr))
{
SqlCommand cmd = connection.CreateCommand();
cmd.CommandText = “Select Picture from dbo.Personnel Where Id=” + context.Request.QueryString[“id”];
connection.Open();
SqlDataReader reader = cmd.ExecuteReader();
if (reader.HasRows)
{
reader.Read();
byte[] image = reader.GetValue(0) as byte[];
System.IO.MemoryStream ms = new System.IO.MemoryStream(image);
ms.WriteTo(context.Response.OutputStream);
ms.Flush();
context.Response.OutputStream.Flush();
}
}
}
}

public bool IsReusable {
get {
return true;
}
}
}

As is clear, we read the person’s record with a SqlDataReader and then get the image bytes using it’s GetValue method.

To let a Image control show the image we have to set it’s ImageUrl’s property in ths way:

Image img = new Image();

img.ImageUrl = “~/GetPersonnelImage.ashx?id=10″;

Inside a GridView or DataList, we can create a Template field with an Image control and bind the Image control’s ImageUrl property to the mentioned generic handler. For example:

<asp:Image ID=”Image1″ runat=”server” ImageUrl='<%#Eval(“id”,”GetPersonnelImage.ashx?id={0}”)%>’

You can download a full sample program from here. To run the project, you must have Sql Server 2005 Express edition on your machine. If you don’t have SQL Server 2005 Express, change the connection string existing in Web.config file to your own database.

Dealing with REST Web Services


At the time of writing this post, .NET has no specific facility to work with REST services. Though, some services over the web provide information in REST format. For example, http://www.geonames.org (that provides gegraphic information) has a lot of free web services to get geographical information. The other day, I needed to use these services to let the end user choose his/her country and then state from two drop-down lists. So I had to figure out how to consume such services. In this post we will use the following service as an case-study:

http://ws.geonames.org/countryInfo?lang=it&country=DE

If you navigate to this link you will get the following result:

As is seen, the resulting XML is pretty straight forward.  With knowing that REST web services work with GET and POST methods, it is nearly nothing except working with xml!

For example, to query a REST service, like the one I mentioned above, we can use LINQ’s XML functionality.  The XElement class (declared in System.Linq.XML) provides a Load method that can retrieve an XML data from a url with GET  method:

XElement rootXml = XElement.Load(“http://ws.geonames.org/countryInfo?lang=it&country=DE&#8221;);

var Countries = from C in rootXml.Elements()
select new { Code = (string)C.Element(“countryCode”),
Name = (string)C.Element(“countryName”) };
foreach (var x in Countries)
Console.WriteLine(“{0}  {1}”, x.Code, x.Name);
Console.ReadKey();

After retrieveing the full xml, you can extract any kind of information you want. no matter how complex is it!

Adding or modifying information using POST method is a bit more complex. First we have to obtain a channel by which we may POST the information. Then we have to create an instance of data entity in XML format. Finally we have to POST data through the channel.

To get access to REST service we use a HttpWebRequest since it lets use the POST method and let us specify the type of information we are posting.

HttpWebRequest channel = (HttpWebRequest)WebRequest.Create(“http://ws.geonames.org/countryInfo&#8221;);
channel.Method = “POST”;
channel.ContentType = “text/xml”;

HttpWebRequest class provides a stream to which we can write information :

StreamWriter sw = new StreamWriter(channel.GetRequestStream());

Then we can create a concerete entity to post:

XElement country = new XElement(“country” ,

new XElement(“countryCode”, “AUS”),

new XElement(“countryName” , “Australia”),

….

);

Finally we just need to save this XElement object to the stream we obtained from HttpWebRequest:

country.Save(sw);

This post shows how to deal with REST web services, or any other resource that let us work it in a POST/GET manner. Thanks to Linq to XML working with such services is now too easy.

Querying a database using Linq and Reflection


Hi,

With Insert, Update and Delete methods but without a Select method, LinqDataSource control looks just like a chair with only three legs! Such a powerful and flexible control is used to get connected to a DataBound control like GridView, while it could excel in many scenarios if it had a Select method! For example, it was great if we could write such a code:

LinqDataSource1.TableName=’students’;

LinqDatasource1.Where = ‘Grade<5’;

var x= LinqDataSource1.Select();

Anyway, in order to develop a special component, that I am gonna describe it in a separate post later, I needed to query a Table using Linq to Sql and the table’s string name. In ADO.NET territory, we may create a dynamic string query and execute it over the database:

const string TableName = ‘students’;

string Query = String.Format(‘Select * from {0} ‘, TableName);

But, how to query the Student table with Linq? Better to say, how does LinqDataSource probably work? The question is fairly easy, but before explaining it, I invite you to be informed that Linq has come to let us query in a strongly typed fashion! Thus, only in rare situations you might need to query a database in an untyped manner.

Anyway, first we have to get to know which Assembly is already running. In a Console or Windows application we may simply find this by calling Assembly.GetExecutingAssembly() method. But, in a Web application, things are different. In an ASP.NET application classes are compiled into App_config.dll assembly. ASP.NET uses shadow copy mechanism to let an ASP.NET application get updated while it is executing. In fact, ASP.NET copies every .DLL file in the bin directory to ” Temporary ASP.NET Files “ folder and name them with strange , long and unreadable (!) names. For example App_Code.Dll file might be named “App_Code.7vkubplh.dll” ! Therefore, we have to search all loaded assemblies for the required type.

As a result, we need the following code to get all assemblies loaded inside the current Application Domain:

List<Assembly> asmList =  AppDomain.CurrentDomain.GetAssemblies().ToList();

Since DataContext class provides required methods, we actually don’t need to find concrete DataContext classes. But if for any reason you need to find them, the best way is to find for classes that have been decorated with DatabaseAttribute attribute:

var AllDataContext = from C in asmList

from P in C.GetTypes()

from J in P.GetCustomAttributes(true)

where J.GetType().Name == “DatabaseAttribute”

select P;

Well, In order to query a table, you have to Add a DataContext to your application and drag your desired table on to  it. Luckily, since each Linq To Sql entity is a class, we can not have two different tables with the same name. Therefore,  the only job to do is to query a type with your table name:

Type table = (from C in asmList

from P in C.GetTypes()

where P.Name == “user”

select P).Single();

This code looks up all loaded assemblies and checks to see which one is named “user”. “user” is actually the table name. Since we are sure only one “user” class exists, we have used Single extension method.

Well, now that we have found the class, we should create an instance of DataContext class and call it’s non-generic GetTable method:

DataContext dc = new DataContext(” a connection string comes here”);

List<object> Records = dc.GetTable(table).Cast<object>().ToList();

The above code converts the resulting table to List<object> but it is not necessary indeed.

Well, the records are ready. You can assign “Records” variable to a GridView to see the results :

GridView1.DataSource = Records;

GridView1.DataBind();

If you would like to get the field values individually, you should obtain all public property names, loop over them and call GetValue on each one:

Table tbl = new Table();

// Loop over records

foreach (var s in dc.GetTable(table))

{

TableRow row = new TableRow();

// Retrieve all instance (non-static) and public properties.

PropertyInfo[] pList = s.GetType().GetProperties(BindingFlags.Instance | BindingFlags.Public);

// Loop over properties

foreach (PropertyInfo p in pList)

{

TableCell cell = new TableCell();

object temp = p.GetValue(s, null);

cell.Text = temp != null ? p.GetValue(s, null).ToString() : “-“;

row.Cells.Add(cell);

}

tbl.Rows.Add(row);

}

this.Controls.Add(tbl);

Well, in the above code I have converted all the values to String but be reminded that this method won’t work for Boolean fields.

Hope this article be a bit of help 🙂

Aspguy