General

10 reasons why I chose MySQL over MSSQL2005 Express

Stefan Holmberg — Sun, 06 Jan 2008 19:41:52 GMT

as the preferred database engine for my web sites.
(price not being one of the reasons)

I currently have around 100 public websites spread over five servers. For ease of management I have started to look them over and standardize their technology.
I had to decide which database engine to use. These are my OWN highly personal opinions and arguments which made me take the decision based on the needs I have. I am not trying to start a database war here :)
But as the title suggests, I have chosen to use MySQL. Here are the 10 top reasons why:

- LIMIT. In a typical webapplication paging is needed and the LIMIT clause is a simple way of just retrieving the needed records from the database. The traffic between
the database and web application is kept at minimum, as well as the memory required for the web app.

- SQL_CALC_FOUND_ROWS. That select modifier works in conjunction with LIMIT and lets you retrieve the total number of rows you would get if we weren't LIMITing. I have a full C# code example
on this at ASP.NET, MySQL and efficiant paging

- memory requirements etc. Let me just say this: MySQL is so lightweight compared to MSSQL. It might not scale as well, it might not handle the same amount of transactions as fast as MSSQL,
but I don't have those requirements. With a download of 22 MB compared to 50 (or even 250 if you want fulltext) MySQL wins this round in my eyes.

- read performance. I agree: when it comes to transaction intense applications I still recommend MSSQL. but for web apps I have chosen MySQL. A web app is typically mostly reading data, not many
updates. In MySQL you can use the MyISAM storage engine, which uses an extremly fast table locking mechanism, allowing for multiple reader/single writer.

- management. Never thought I'd say this: I have learned to master and really like SQL Enterprise manager, I am also beginning to like SQL Server Management Studio, but, for a remote webserver
nothing beats phpMyAdmin against MySQL. On my development box I run SqlYog.

- backup/restore/moving data. MySqldump is a fantastic tool for moving data from one server to another. Spits out CRERATE TABLE/INSERT statements into a textfile.
So genious, Microsoft recently developed their own version, Database Publishing Wizard.

- MySQL is a full version. Since I said I woudn't talk about price I need to compare MySQL against MSSQL 2005 Express. And that version is crippled. Uses max one CPU,
uses max 1 gigabyte (GB) of RAM, 4 GB maximum database size.

- it's widespread. I have done quite a lot work lately against Wordpress. Having my own stuff in the same database as the Wordpress data opens up a lot of possibilities

- fulltext engine. For MySQL it's inside the 22 MB. Nothing to download/install/configure.

- multi OS support. I now have a Linux box in my machine park as well, since Wordpress is needed. So, being able to run MySQL on Linux as well as Windows sure
is a plus for me.

Last - consider I had all my experience in MSSQL (been using and coding against it daily for 10 years = since SQL 6.5), but still decided to go the MySQL road.

Messages for Microsoft:
- make it easy to upgrade an existing SQL 2005 Express installation to include full text engine as well. A 250 MB download is a joke.
- LIMIT and SQL_CALC_FOUND_ROWS. Something like that. Database support for paging is crucial for web apps, tricking it with TOP (and two separate sort /asc/desc) is not comparable. I know it's not SQL standard, but
I don't care. I need the best functionalty for my solutions, not the "most standard".

These are the two most important reasons I "left you" with regards to my web sites.

Messages for MySQL AB:
- do something about your own admin tools. The Query browser for example is horrible to work with. Luckily there are a lot of third party tools.

Switching to MySQL in 2008

Stefan Holmberg — Tue, 01 Jan 2008 21:48:56 GMT

My directions for 2008

I will do some major changes when it comes to technology in 2008. Being an independent consultant (contractor) is not
all about being totally independent, of course. As in any business you need to adapt and be able to offer what your clients need. Not what you might consider the best/optimal (you can of course always come with suggestions), but in the end the client makes the decision about which platform and solution to choose, and things like existing ardware/software/knowledge often plays an important role in it. Therefore I have been doing a lot of ASP.NET, C# against MSSQL Server. However, if it was just up to me, and not taking any old luggage into consideration, I'd actually switch the MS SQL backend to MySQL!

Which is the direction I will try to take for this year. Lately I've been doing a image gallery system for a client, and since I had the freedom to choose the architechture I went for ASP.NET, C# and MySQL. Even though I live and bread MS SQL server and sps I can't say I made the wrong choice. Yes, some initial problems, no doubt, but, I must say there are certain things making MySQL the optimal database engine for a web solution.

+ price of course. Nothing more to say. The SQL 2005 Express limitations needs to be removed ASAP in my opinion, cause to me it's very much a developers database...

+ built in full text index. I have never needed full text index until a project I entered a few months ago. I looked at getting it up and running on my SQL 2005 Express development box. However, an extra download of 250 MB was needed. I also *think*

+ paging. My favorite! This is SOOO typical a web application. Implementing paging in MSSQL is not fun at all. Getting acceptable performance out of it is possible, but coding hard to generlize and reuse. In MySQL it's a matter of adding LIMIT X,Y to the end of the select statement. And not to talk about SQL_CALC_FOUND_ROWS. Cause typically you need to get the page of rows but also the total number of rows, as if we were not using the limiting. Typically you do
two queries, one select count(*)... and one select * limit x,y. But with SQL_CALC_FOUND_ROWS you get that anyway.
Yes, might not be SQL standard, but I don't care. I don't get paid by creating the most standardized solution, I get paid by delivering a solution with good performance and preberable a low footprint. I have published an article on this, ASP.NET, MySQL and efficient paging.

On the minus side:
- Microsoft default medium trust configuration. Medium trust is a good thing, it's not about that, I just feel that a lot of webhosts has no idea about what it's about. They take the default configuration (recommended by Microsoft) and uses that. And the default medium trust configuration only allows for MSSQL database access...I've had numerous of conversations with webhosts about it and many, I say many, still doesn't understand what it's about. Sure they advertise on their website of supporting asp.net, mssql, mysql etc, but still when firing up your ASP.NET 2.0 app you get the
ecurity exception cause they are not allowing MySQL from medium trust...A lot of respectable webhosts has indeed made the changes needed, GoDaddy is one of them. Hat off for them!

AdMentor PRO will always have a SQL Server driver and I will still do a LOT of SQL server gigs, since I have a lot of clients using it, but for myself and my own servers I will make the transition.

- I might be wrong here - SQL2005 installs some extra services and stuff. In MySQL it was just a matter of specifying it in the create table statement and start querying...You do need a decent stemmer though, making words in normal form.

Dealing with large datasets in browser

Stefan Holmberg — Fri, 21 Sep 2007 15:42:38 GMT

It's always hard working with large datasets, I have written quite a few desktop apps of that kind - and this week I have been trying to educate myself using large datavolumes in a web scenario.

I have no trouble optimizing the actual dataaccess ( I more or less live and breed indexes and sp:s ), but the trouble is how to design the actual interface. I mean you can't just throw in 100 000 <tr> records and then expect the user to stay waiting for it to first come up at all, and then (try to) navigate it.

The virtual listview is cool for Windows Forms and desktop apps, but to be honest I have a hard time with the existing
asp.net grid controls available - while the controls themseleves might be great - we have the problem of trying to be accessible. Meaning people without Javascript (and not to say search engines) should be able to get all our pages, so some sort of cool Ajax solution is (in my eyes) out of the question.

So - what can you do? I have been experimenting a little lately and have a few tips:

1. Use CSS as much as you can. Minimize the HTML markup.
Not
<li style="color:black;....">Hello</li>

but just
<li>Hello</li>

If you have, say some hundred or more records it CAN make a difference.

2. Are there "natural" ways of reducing (filtering) the dataset?
I have one example:
http://www.usradiostations.info/

a site listing US radio stations.

Here the first choice is state - then you select the city - and not until now you get a list of the actual stations.

If I was to do such an application for myself - for it to be really useful, I would probably have one dropdown list for states - and when selecting a state all cities pops up in a linked dropdown - which in turn lists all radio stations in a third. Or something like that. But I think there is more to it than just creating the easiest to use application. As I said accessibility
of that site has a value too.

You can even reduce it further - another example is this site on aircraft models http://www.aircraftdata.net. To reduce the dataset I took the liberty to split it into different subdomains - that being step 1. Then you get listings for that subset.

I will do some more investigations in this matter - cause it is very interesting and the task of creating an accessible, yet easily navigatable - yet again USEFUL - site might not be as simple as it sounds when dealing with larger data volumes.

Source and version control

Stefan Holmberg — Wed, 12 Sep 2007 10:18:37 GMT

When setting up new box I had some vague ideas of trying the opensource road. The very example this post is about is source control. I have earlied talked about Sourcegear but I see alot of buzz around Subversion so hey, why not give it a shot.

While Sourcegear Vault is a fantastic product, what scared me from using it (it's free for single user - which is what I needed for my websites sourcecode - so price was not the reason) is the rather hefty system requirements for the server end. 2 GB RAM is recommended. Since it's implemented as a webservice it's also integrated with IIS - and I have an idea of NOT running iis at all on my development box (but rather in a virtual VMWare box), considering I run´the built in web server when debugging my VS2005 projects.

So - off I went. First searching for info on *WHAT* and *HOW* to install Subversion. Then trying to do it. I finally got it running as a service as I wanted. And it is indeed slick - really fast and low on resources as far as I can tell. However there were some things I didn't like about it:

1. can't check out a single file. You work on directories. That's a problem with the server end - I read somewhere they are gonna implement it in the future, but I need it now

2. can't say I found a GUI I likes. Tried out TurtoiseSVN as well as RapidSVN. While RapidSVN is pretty close to VSS interface (which I fell like I was breastfed with, more or less) - still not close enough.

So - back to reality and Sourcegear Vault. What I didn't want was to clutter my development box with the server part - so I took a shortcut and installed it on one of my public webservers. I also get the benefit of having the code at a second location.

At least i tried Vista

Stefan Holmberg — Mon, 10 Sep 2007 08:16:35 GMT

Got a new box almost a month ago. Havn't wanted to start configuring it cause I know the time (and pain) it takes to get it configured. Let me tell you a little about how I would like my box to be

1. SQL 2005
2. VS 2005
3. Virtual box with AMP (apache, Mysql, php)

I was also keen on trying out Vista, since it came preinstalled on the box I thought, why not,

Now that is how it looks in my world - however when considering my customer projects that I still need to be able to support/debug/compile etc I need
4. VS 2003
5. SQL 2000

(it was about a year ago I got rid of the last VB6 project so at least I wont need old Visual Studio 97).

And to add to it - one clients VPN only works with Windows XP - meaning Vista was out of the question. But I decided to
still give it a try. So the plan was to use use Vista as host and install XP as a virtual machine.

Vista (host OS)
-SQL 2005
-VS 2005
-code directory (shared)

XP (in virtual box)
-VPN client
- VS 2003
- SQL 2000
- code taken from shared directory on host

I was really happy when I had spent the hours needed to install VMWare server (cause I wanna run Ubuntu as well)
and installed XP inside, installed the VPN and got it working - and installed VS2003. Last thing was to really try to compile -
and here the BIG problems started. First I had big trouble getting the boxes to talk to each other at all, but finally
I got a grip on how things should be configured in Vista to allow disk sharing. However - no matter how I tried the performance between the two were painfully slow. On a GB network I made around 20 KB/s. Yes I didn't mistype - 20 KILOBIT... Coping a VS 2005 simple project of around 30 source files whould take me 34 minutes or so...

lso:
From XP to another (real) XP box on the network performance was normal
From another(real) XP to Vista operformance was normal

Now it was sunday afternoon - I had spent friday and saturday night on this and really wanted to be done by monday morning. So I gave up.I wiped the Vista box and installed XP. Now I got the problem of the OS not finding drivers for network, sound card etc (think the motherboard's too new). Found an old USB network interface to plugin to connect to Internet and start searching for drivers. And finally I got most parts working...

Other from the host OS I have the same configuration as I wanted. Not it works like a charm - and actually the overall performance IS better in XP than in Vista - at least that's what I think.

To end it up - this post is not a flame on Vista, it might very well be me being not clever enough, for example there are a some reports on VMWare Server trouble with XP - but the truth is I shouldn't have to care. The most important thing for me is to get my system working according to what I and my customers need and XP - not Vista - was the solution once again.

Hopefully I will get my Vista install sometime - but the reality is that's not MY decision - but rather my clients in a way.

More about code structure

Stefan Holmberg — Thu, 30 Aug 2007 08:20:30 GMT

Long time, no blogging. Sorry about that, but I have had a LONG summer vacation. Really nice.

Lately I have actually been doing some Wordpress stuff. What started out with some simple customizations for
a client quickly turned into some deep WP hacking. I can say this, I am really impressed with the product. As a
pure publishing tool I doubt there is nothing beating it. If you can live with using Apache (well, technically IIS is possible, but when doing some url rewrites etc, which you most certainly want in Wordpress, it's a lot easier with Apache) . You also have to live with being tied to PHP and MySQL, of course.

My first reflection after a few weeks of PHP hacking: I miss the ASP coding days in a way. If you don't know it,
coding in ASP and PHP are very much alike. You typically insert some PHP/ASP tags inside your HTML and just upload the page. Done... It simply allows for quick coding and quick results. While you *could* program the same way with ASP.NET there is almost like an unwritten law saying you should use code behind (what moron wrote that - oh, right, it was me...), you should create multitiered (three layers are an absolute minimum, you know) applications etc.

And I agree, myself I *almost always* do use code behind, I almost always separate business layer/database layer/GUI in different dll:s etc and I really do believe in that model. But lets face it - we must question ourselves - WHY should
we separate our layers? And is it possible to achieve in some other way?

Take code reuse for one example. If you implement your FooBar business class inside a dll it means your ASP.NET app as well as your Windows app as well as your console batch routine etc can use it. Fine! But that could also be done with ASP and PHP, not involving any dll:s, but simply including a the FooBar.php/asp file.

Cause, back to Wordpress now, I am amazed over how well organized that code project is. Using a "simple" untyped language they have implemented a model which still allows for event handling etc, third party plugin registration etc.
Yes, it's all string based, meaning you register a hook for say "wp_footer" (in this example your function will be called each time the page footer is being rendered) - but it's still really easy because of the structure. The "correct" ASP.NET equivalent would be having the server app define an interface and the plugin implementing it.

To finish this ranting off - I will not leave my "structured way" of ASP.NET coding, it works really well for me,
but the Wordpress work I have done has kind of opened my eyes a little. I kind of have this idea "best practice is not what someone else says is best practice - best practice is what best for me" (maybe a luxury of being self employed being able to define my own models) so therefore I think it's important to think always try out new and different things.

SQL Server columns never updated from SP

Stefan Holmberg — Wed, 27 Jun 2007 06:27:52 GMT

Have a look at this SP:

CREATE proc [dbo].[KB_UpdateArticle](@id int, @header varchar(255), @txt text, @active varchar(1),
@allowcomment varchar(1), @extracontent text, @sortorder int, @metakeywords varchar(255), @metadescription varchar(512), @publishdate datetime )
as
If @id = -1
BEGIN
insert into kb_article (header, txt, created, lastmodified, viewcount,votecount,
  totalrate, active, allowcomment, allowrating, extracontent, sortorder, metakeywords , metadescription, publishdate )
select @header, @txt, getdate(), getdate(), 0,0,
  0, @active, @allowcomment, '', @extracontent, @sortorder, @metakeywords , @metadescription, @publishdate
select @@identity
END
ELSE
BEGIN
update kb_article set header=@header, txt=@txt, lastmodified=getdate(),
  active=@active, allowcomment=@allowcomment, extracontent=@extracontent,
  sortorder=@sortorder, @metakeywords =metakeywords , @metadescription=metadescription, @publishdate=publishdate
  where id=@id
select @id

END
GO

Spot the error? It is an "authentic" stored procedure from a CMS system I am developing - and the lesson to learn is to always test out all test scenarios - even after the smallest additions. I added some new columns in the table, metakeywords and metadescription, and had to update the stored procedure to update it accordingly.

And in the "update" case

@metakeywords =metakeywords , @metadescription=metadescription, @publishdate=publishdate

should be

metakeywords = @metakeywords, metadescription= @metadescription, publishdate= @publishdate

of course. My biggest error was not this, but not testing that case out. I just created a NEW article and saw that the information made it into the database - then I was happy with it. I can tell you I had some trouble finding the cause later, at first I looked in the code of course.

New stuff on my sites

Stefan Holmberg — Tue, 19 Jun 2007 10:55:28 GMT

Just a "new stuff" article. I've been working really hard lately which hasn't given me much time to publish stuff. But today I made some progress:

The two articles are worth reading I'd say. In short I needed some performance improvements from my laptop, thounsands of sequential calls to do DNS lookups were dead slow because of network latency over the wireless network. So I created a small ASP.NET page, uploaded it to my webserver (connected to Internet with 100MB fiber line), and sent a batch to it from my client. All DNS calls were then made from the server and the result sent back as a singe batch. Really improved the overall performance!

Posting to blogs with C# code

Stefan Holmberg — Tue, 12 Jun 2007 14:44:03 GMT

If you have a couple of blogs you soon get tired of logging in (oh wait, first find the right userid and password) and then clicking some menu buttons to get to the post page.

So I have started to look into a Windows client (create my own of course, heck I'm a developer, am I not) for that. It has been some "fun" trying to get the posting API:s to work and today I added a new one to the list - LiveJournal. I never even knew about it (don't know much more than wordpress and blogspot to be honest) and it was not too easy to get the posting right from the C# client.

However, a quick and dirty solution, turning into concatenating XML parts into a long string, hehe - well not big biggest moment of pride - but who cares about cred and "slick" solutions...I prefer working solutions.

The CSS and tables problem

Stefan Holmberg — Fri, 11 May 2007 09:56:11 GMT

While I am really starting to appriciate the power of CSS there are some certain drawbacks which drives me nuts. Beside the obvious about CSS - the possiblity to separate layout and content - the power lies in the CSS - where C stands for cascading.

However you gotta be really careful cause soon enough you will have trouble knowing which styles will be inherited or not. One thing I miss (correct me if I am wrong) is however a possibility to - for a certain item - break the inheritance chain, this element should NOT inherit any style but rather start from the "fresh" default style.

One example of this I have been struggling lately is tables. For AdMentor PRO I use some heavy table formatting:

Live demo

Now, what if I want to include, say a datepicker Javascript component for one of the fields. Since most Javascript datepickers are implemented as tables, all styles from the outer table WILL be cascaded down to the datepicker - no matter what. Since the new table is contained inside...

What I am left to do is try to reverse all settings from the outer table,th,tr,td etc for the datepicker component...That was not so fun at all, and I didn't even get an result good enough. Somehow some cellspacing or margin or something got carried over so it didn't look ok. I simply guess I have more to learn about that.