Since the release of LINQ in .NET 3.5 I had been a little skeptical of it. I’ve never been a fan of code that does things automagically (yes, I said automagically). When I took my latest job I started working on a project which made use of LINQ-2-SQL very highly, and so I was forced to embrace it and begin working with it. My overall impression of it is that it’s not quite as bad as I originally thought. Selecting objects from a list with SQL-like syntax actually seems very natural, and the selection can replace a lot of lines of messy code when you’re searching for a certain subset of items within a larger collection. On the other hand, I still feel a foreach loop iterating over a collection makes your code more readable in the long run.

There are also some nasty pitfalls when using LINQ-2-SQL which most developers won’t ever notice unless they’re using large datasets, which we just so happen to be doing. The system I was working on as mentioned earlier has a database which is between 10-20GB at any point in time. This is by no means extremely large, but it was big enough for us to start seeing major performance issues with sloppy LINQ-2-SQL code.

No matter what the queries LINQ generated for selections were fairly efficient; I have no complaints about that at all. Where it breaks down, and horribly so if you don’t know what you’re doing, is when doing insert, update and delete operations. The default behavior of LINQ-2-SQL is to open the connection to the data store execute one data modification query for one record, close the connection, and then move to the next record. Like I said before, unless you’re working with big datasets this probably isn’t an issue. On the other hand our system was working with datasets on the size of hundreds of thousands to millions of rows at a time, working one row at a time. Can you imagine how much overhead was being generated for data modifications?? It was utter ridiculousness.

The bright side of this whole situation is you can make LINQ-2-SQL more efficient in this regard, but you do need to be explicit about it. You must make sure to open your connection to the database before you begin your updates. If you want your whole update wrapped in a transaction you must tell LINQ this or each update will be its own transaction. Now, and not before, you ask LINQ to update your database. (There may be some inconsistencies here, I’m writing this from memory and I haven’t touched the offending project in months.)

How does being explicit with LINQ fare compared to trusting the default behavior? Quite well I must say. Our modifications which were taking hours upon hours before could be done in as 15 minutes or less. We’re talking an order of magnitude of difference here. Personally I couldn’t believe how inefficient it was off the bat and how well it performed before. A couple little steps before any data modifications saved me the trouble of having to rip out the guts of the data access layer for this project and rewrite it.

Anyway, I hope this helps someone else in the future. I’ll update with some code examples later when I get back to work.

Now, as you can guess from the title of this post we’re making use of the MVP pattern to build this project, the passive view pattern to be exact. My first impressions of this pattern were a little fuzzy, I was trying to figure out why this was better than using MVC ( As I side note, I tried MS’ MVC framework a few months back and was pleasantly surprised — It reminded me a lot of the Zend framework ). When I started implementing the pattern and began writing my unit tests I started to see where this pattern really began to shine. As a side effect though, I have a lot of interfaces and I hope those who touch the code later don’t get entirely confused.

In the passive view pattern, all of your views are dumb. They contain absolutely no logic (or at least as little as possible). The views are only containers for your data, and what happens to that data is the responsibility of your presenter. Obviously when using event driven GUI’s you need a little bit of code in your view, but other than deciding which events to handle there’s basically nothing there. I’ve made a personal call by putting some validation into my views, but that was because I didn’t want to throw away the usefulness of the ASP.NET validation controls. I also continue to do validation in the presenter before anything gets persisted.

The beauty of the pattern, as I stated already, is the unit testing. Since my views are as dumb as they can be the bulk of my testing is done at the presenter and data access layer. I think this is absolutely wonderful since automatic testing of GUIs is a horrible chore! While there are many tools which can help you along the way, it just never seems to be worth the effort. With the passive view pattern I can literally test the same set of events which would be raised through the GUI easily with unit tests.

As an aside I’d like to mention that unit testing, while a great boon to the development community, doesn’t replace the task of testing by yourself. I still test every method I write with as much care and pride as possible. Assuming your code works correctly just because your unit test passed doesn’t mean it works right! I like to view unit tests more as an aid to help me pinpoint when I break things later.

And now back to the downside of the MVP pattern… The complexity! I’m sure this is partially due to the way we’re tackling this particular project, but at the moment we have a lot of assemblies being generated in order to support the pattern:

• Data Objects
• Data Access Layer
  • Data Access Interfaces
  • Data Factory
• Presentation Layer
  • View Interfaces
  • Concrete Presenters
• UI Layer
  • ASPX Pages
  • User Controls (implementing view interfaces)
• WCF Service

That’s the basics of it anyway… And I’m still not sure I’m doing it entirely right. I made the design decision to communicate from presenter to view using only primitives (string, int, etc ). I’ve seen other people use another layer of interfaces for their data objects to facilitate the views using these, and the presenter translating between view data objects and actual data objects. In my opinion this just adds another layer of unnecessary complexity, but there could be a good reason for it as well.

I do think the testability of this pattern adds significant value. The complexity is a side effect, but one which I believe is manageable as long as I keep things clear and concise. All told, I do think the MVP/Passive View pattern is adding value. If I didn’t think so I would have abandoned it early on in the process of development.


******:zenwerx.com/pi/cache$ ls | grep -c png
6544
******:zenwerx.com/pi/cache$ du . -h
2.0G .


6500 images totalling 2GB! Oh, and I accidentally deleted the browsing script when I cleaned up too. Whoops! Thank goodness it was just a symbolic link, but it had me worried for a bit!

Update – June 24th
Apparently the Pi picture script has been broken for a few months (ever since the server was upgraded). The permissions were set incorrectly on the cache folder and the script couldn’t save any images there. It’s all fixed now, but I wonder how big the folder would have been if there hadn’t been an issue…

I’ve put off the learning of it for some time because of several factors. First and foremost, I had previously rolled my own framework before the Zend framework ever existed. While my framework isn’t as good as the Zend framework, it was specifically tailored towards my own needs. Second, most of the other projects I work on I always thought were too small to pick up the framework for. I always said to myself “I could do this quick and dirty, or spend a couple days learning ZF.” Quick and dirty always won out. Finally I’ve got a project which I figured was big enough in scope to dig into the framework for, and I really like it.

From having (no so) simple things like a working class autoloader, which includes namespace support, to the routing of requests to controllers, or even using PHP itself as the templating language… It’s wonderful. As I said before the learning curve was a little much, and some of the documentation leaves you wondering what’s going on, but once you pick it up things are pretty good.

At first I wasn’t entirely impressed with using PHP as the templating language. I had used the Smarty templating engine in my own framework and liked the way it worked. I thought it kept my templates nice and clean and unpolluted with code. Then I started thinking about how much work I used to put in to Smarty every time I wanted to create a new drop down list, or other data bound control. Sure, I had helpers set up… but I ended up writing a lot of code like this:


public static function wrap_some_new_tag($params, &$smarty ) {
    $newtag = new DataComboBox();

    // parse params common for all combo boxes
    self::$SMARTYMGR->parseComboParams($ts, $params);

    $newtag->setDefault("0", "(Select a XXXX)");
    $newtag->setDataSource( DataSourceFactory::getXXXXXX() );
    $newtag->setDisplayField("NewTagDisplayField");
    $newtag->setValueField("NewTagID");

    return $newtag->buildControl();
}


And in my templates was something like this:

{newtag id='id' param1='param1' ... }

I’m still polluting my view logic with a lot of data, as well as writing a wrapper every time I want a new form element which deals with parameters/etc. And I really started thinking, did I want to keep doing this?  With the Zend framework I can design forms in XML (or PHP code), have them filled with data… and no need to worry about how to hack Smarty to make it do what I like. I can also subclass Zend_Form_Element_XXXX to be data aware. Who knows, maybe I’m thinking about this whole form thing all wrong; I am new… but I think this has potential.

Anyway, this post is getting a little long and I have work to do… More updates at a later date.

Even with a decent amount of design with my manager and I have already done I ran into a slight issue today. We are using ADO.Net and manually built data objects ( entity framework was investigated but passed on for reasons I won’t get in to ). I wanted to have many properties on my data objects lazily loaded and here is where I made my mistake.

In order to facilitate the lazy loading I placed the data objects in the same project as the data access layer and ended up coupling them tightly to the access layer. While thinking about the WCF service today I realized my issues. The other project has to know about my data objects, but it shouldn’t know about my data access layer.

Ok, well at this point we just separate the data objects out… Right? Well the data objects are coupled to the data access layer. A very bad idea I now have to remedy. This is an issue which will take a significant amount of refactoring.

Despite days of design we still have an issue which is going to take development time out of our schedule. The moral here is to always think ahead, even when you believe you have a design worked out!

Enjoy!

Also… There’s a new section under oddities. Something which has been laying dormant for about 4 years and I figured should see the light of day.

And since I have all this free time now, I think I’m going to start on a new project I’ve been meaning to get around to for a while now. Should have more to post in the near future.