Strike Two Entity Framework

This weekend will be my last try with trying to do something limited with the Entity Framework (EF) that is both reproducible and maintainable.

Working with the EF means a dependency on an Entity Data Model (EDM). No matter how nifty the EDM visual tool is, it seems to fall apart during evolutions of change in the underlying physical database schema during its incessant and overkill "updates" against said schema.

Posts like this from the ADO.NET Team give me the distinct impression that using the EF is going to be, over the long run, a heavy dose XML editing to get it to work productively. Honestly, I'm not interested in coding XML files in the least, which seems to be a trend in lots of areas of software frameworks these days, not just the EF.

As I try to apply the EF towards a practical purpose, I realize (again, for the umpteenth million time) why I have historically embraced the pattern of segregating the physical database schema and application code.

The only reason I haven't already abandoned the EF is because I'm keen to push myself to try something new. By Monday the proof will be in the pudding, or the pudding will be in the trash along with all the other ORM-like clap-trap I've previously suffered through.

Hmmmmm

Even though I've soured almost entirely on the .NET Entity Framework as a comprehensive design and application component - I still see it useful as a highly focused and limited tool.

Eliminating the unworkable process of generating an application's all-inclusive entity model, has seemed to be useful so far. Although, I'm still stuck in SQL Server World with the EF. Or am I?

.NET 3.5 (SP1 Beta) Entity Framework Test Drive

I'm giving the Entity Framework (EF) a test drive on the shortBet project. My primary reason for giving it a go was centered around the fact that I work primarily with Oracle database server on the backend and LINQ to SQL (L2S) is not a good fit for me. shortBet will be using SQL Server, so it serves as an excellent platform for this test drive.

Despite LINQ's greatness, I haven't been all that interested in L2S which has several nasty weaknesses. First, typical LINQ coding elegance falls down on its face when faced with joining more than a few queryables that are closely bound to relational views and tables. Second it generates SQL command text automagically, and I'm not intrinsically happy with that. Finally, and most significantly, use of L2S is bound to the SQL Server platform for the forseeable future.

The EF is an interesting tidbit I had heard was a better deal than the L2S way from a variety of .NET and Oracle resources I trust for leading edge information. After my test drive so far, my assertion is that it is and it isn't a better deal than L2S. It is, because queryables in the EF have a greater potential for expressing complexity and thus more elegant LINQ statements than using L2S queryables. It is, because the tools for expressing Entity Data Models (EDM) are fairly slick and effective. It isn't, because as far as I can fathom from my little dip in the pool, EF is utterly dependent upon L2S - unless you want to write your own custom Data Access Layer, which eliminating is one of the pillars of the EF. By using the EF, at best I'm out of the logical fire, but I'm still in the physical frying pan. It isn't, because even though I have a highly functional relational model for my Shortbet application - I can't seem to see how I would effectively express it with an EDM.

Part of that last failing is clearly because I'm a relational guy, and I'm not very good at OOP modeling. Partly it's because I believe (in my heart of hearts) that entity modeling is simply inferior.

So, at this point - although I'm still continuing with the EF test drive (thankfully I have some help) - I'm of the mindset right now that rich and meaningful entities are great - but a unified model of more than a child-sized handful of them is an irrational endeavor.

ping.fm begging

I was interested in examining the ping.fm site features, especially tying this blog, my facebook and twitter together. Well, you need one of these beta codes to get in on the fun. The best part is that they encourage you to beg for one, which I thought was genius. So here is my begging effort:

I could write a long diatribe and divert all writing effort away from my overdue blog posting efforts, but I won't do that. I wouldn't waste your time with a long plea for a beta code. I mean, life is too short and we just don't know when the end will come, and I know I don't have time to sit around and wait for you to send me some sort of beta code like I'm a plebian and you are my centurion overlord. Who do you think you are, telling me that I can't use ping.fm? What makes you so special and me so pathetic? Listen, I'm the customer and the customer is always right, right? Give me a break already, this is going nowhere. I watch Adult Swim and I use Tweeter; isn't that enough basis to know that I'm part of the cool crowd? I'm going to The Cure concert tonight in Dallas, and if you need more proof than that to make you know how much I don't need or want one of your precious beta codes, then you just don't *get it*. Anyway, if you would send me one of those beta codes, I'll see what I can do to help you out.

I'll update with any results I acheive.

UPDATE 6/15/2008 Success!

I Get Tired of OOP Sometimes

I'm learning the ADO.NET Entity Framework particulars and I found this highly instructional ADO.NET Entity Framework Overview document => http://msdn.microsoft.com/en-us/library/aa697427(vs.80).aspx

In it I found a somewhat contentious paragraph describing the value of the client-views infrastructure required to stylize the consumption of the logical data model, using .NET code rather than database views.

An obvious question at this point would be why not just use traditional database views for this. While database views can abstract many of the mappings, often that solution won't work for several process and functional reasons: (a) many of the views are simply too complex to be generated and maintained by developers in a cost-effective way, even for some simple conceptual to logical mappings, (b) the classes of views that have the property of being automatically updatable at the store are limited, and (c) databases for core-systems in medium and large companies are used by many central and departmental applications, and having each individual application create several views in the database would pollute the database schema and create significant maintenance workload for the database administrators. In addition, database views are limited to the expressivity of the relational model, and typically lack some of the more real-world concepts of the Entity Data Model, such as inheritance and complex types.

I'd like to address this paragraph point by point since it is an obvious developer-helpless-against-the-database snippet of half truths that I think is troublesome too read, coming from my I-am-king-of-my-database point of view.

An obvious question at this point would be why not just use traditional database views for this.
This is a great question that was percolating in my mind the second I started reading about the client-view infrastructure. I love it when that happens.

While database views can abstract many of the mappings, often that solution won't work for several process and functional reasons: (a) many of the views are simply too complex to be generated and maintained by developers in a cost-effective way, even for some simple conceptual to logical mappings,
This has never happened to me even once. I've mapped dozens of domains to normalized database objects and all the necessary stylized views required from the nature of the conceptual model. Never once has it been too complicated or expensive to do. What I have experienced are conceptual models that radically change in structure because they never really represented whatever they were intended to represent in the first place. Such radical overhauls are problematic during development and possibly catastrophic post-production. However that is not a function of the shortcomings of the database tier to cooperate with whatever is asked from it. Also, I've seen conceptually illogical models that were overly complex and irrational to cover up the tracks of the modeling wing of the development team's incompetency - illogical irrational models are indeed too complex to be generated and maintained in a cost effective way. Perhaps, I've just never run into these simple conceptual models that can't be mapped to logical underpinnings in the database. My intuition says, show me such a model and my guess is that the model is ill-serving the application.

(b) the classes of views that have the property of being automatically updatable at the store are limited, and
This is true, sort of. A view can easily be constructed that provides an ambiguity on update, rendering this "automatic" feature unusable. However with Oracle, it's a simple matter to put an INSTEAD OF trigger to eliminate the ambiguity.

(c) databases for core-systems in medium and large companies are used by many central and departmental applications, and having each individual application create several views in the database would pollute the database schema
Huh? Every database platform has means to cleanly segregate database objects to eliminate this so-called "pollution". Calling something useful and perhaps necessary isn't really pollution, is it? It sounds like something the convenience store guy in Clerks would say, "You know the Customers are the only problem with this job." Also, when you have views in a database, the greatest asset of the database comes into play, physical integrity. For instance, if you have on of those core-systems, and the development team creates a new application that necessitates creating some views in the database there is no better means of knowing that changes to the core-systems physical structure have impacted this new application than when the view becomes invalid, obviously known from the data dictionary. This coupling is essential in my opinion.

and create significant maintenance workload for the database administrators,
If your database administrator is complaining about the maintenance workload for adding views into the database, please fire him immediately.

In addition, database views are limited to the expressivity of the relational model, and typically lack some of the more real-world concepts of the Entity Data Model, such as inheritance and complex types.
This is perhaps the most irritating statement. Listen up Mister and Missus Object Modeler: there isn't a single concept that you can model that the relational model can't persist. Period. If you believe otherwise, shed your belief immediately. Although you may not be able to express relationally your Masterpiece object model, that is reflection on you, not the massively stable and proven underpinnings of relational modeling. Just as a relational data guy like me struggles to create rich and useful conceptual object models realizes and admits where the rubber isn't meeting the road, me, so should you.

SyntaxHighlighter test

I am tired of not having syntax highlighting on my blog, and it is probably one of the reasons I neglect posting code here more often. I'm trying out SyntaxHighlighter that I found on Yet Another Coding Blog.

Here's some C#

public class Foo
{
 public Foo()
 {
 }

    public void Bar()
    {
    }
}

Here's some SQL

select e.name, d.name
  from employee e
  join department on (e.dept = d.id)
 where e.title = 'Manager'

If it works, I'll update with the details on how I did it, because there were some twists and turns.

---

Update: It worked! Here are the steps, as concisely as I can write them.

1. Download the latest SyntaxHighlighter. The file release was a compressed RAR file. I hate the RAR, so I leave it to you to figure out how to decompress it.
   
2. Put the Scripts and Style files on a Internet accessible web server (I used the free Google Pages). Unfortunately, you cannot put these files on blogger.com itself. You must bring some external web resource to the table to implement these steps.


3. In your Blogger HTML template, find the </div> that closes <div id='outer-wrapper'>. This is likely the last </div> before the </body>. Your mileage may vary depending on what hacking you've done to your template. After that but before the </body> insert the following code:
   
   

<link href='http://YourWebserver/SyntaxHighlighter.css' rel='stylesheet' type='text/css'/>

<script language='javascript' src='http://YourWebserver/shCore.js'/>

<script language='javascript' src='http://YourWebserver/shBrushCsharp.js'/>
<script language='javascript' src='http://YourWebserver/shBrushSql.js'/>


<script language='javascript'>
dp.SyntaxHighlighter.ClipboardSwf = 'http://YourWebserver/clipboard.swf' dp.SyntaxHighlighter.BloggerMode();
dp.SyntaxHighlighter.HighlightAll('code');
</script> 

Replace YourWebserver appropriately, depending on where you choose to host the necessary files.

P.S. Blogger is a miserable platform to post HTML instructions, and I will never ever do it again.

Michael Michael Bicycle

Thanks to the blood sucking NTTA, I finally have a bridge I can use to get to work by bicycle. Once I figured out a way to navigate the 7.5 mile (12km) trip with the least contact with traffic, I started bicycling to work as frequently as I've been able to will myself.

The first few days were hard on my body, very tired when I got to the office and subsequently depressed in the afternoon on the prospect of having to face about 40 minutes of effort to get home.

Two weeks later, I no longer even care about the effort - it's actually easy. And, now it's closer to 30 minutes because I'm hauling ass as best I can.

My goal is to use my bicycle to eliminate at least two commutes by car per week. Coupled with working 4x10 instead of 5x8 hours at the office, that's a 60% reduction in my gasoline consumption via commute. Screw you Exxon.

I'm logging my miles per day, and I think 50 miles (80km) per week - overall not just on commuting - is a great goal for me. Also, I've added my logging to my FriendFeed, to beat down all the idiots who think I'm interesting that follow me that way.

Giving Disqus a Try (redux)

Giving Disqus a try (too). I think the only reason I've avoided a deluge of comment spam or stalkers is because my blog is utterly irrelevant and uninteresting. At least that's what my AdSense account tells me...

The Sum Total of My StumbleUpon Experience

Not exactly awesome.

Two New Projects

Because just one project isn't enough

The time has come to stop playing around with the new .NET 3.5 features and put them to good use in real development projects. I have two projects in mind, MiTunes and BigCheese. Each represents software I've long desired and been unable to fulfill externally. Here's a little background on each project:

MiTunes
I require companion software for Apple's iTunes. Describing the built-in sharing features of iTunes as crippled is an understatement. I believe the crippled nature is intentional, so there's no sense in waiting around for Apple to improve iTunes in this area. Here are the proposed feature highlights:

• Expose entire iTunes library via web interface, for sharing content among my family
  
• Optionally expose "clipped" media, for sharing playlists with the public
• Operate as a self-sustained real-time Windows service
• Function in a very lightweight manner
  

There is software already in the marketplace that somewhat implements what I'm looking to do, but they fall short for various reasons. If I've overlooked any, please drop a comment with a link and I'd gladly take a look.

• Apple's iMix (iTunes Store)
  upside: free, Apple foots the bandwith bill entirely, easy to use, integrated seamlessly with iTunes, easy to publish references on blogs, etc
  downside: songs only, songs must exist in the iTunes Store, limited to 100 songs
• DotTunes
  upside: free version is usable, exposes iTunes library via adequate web interface
  downside: not open source, registration process is goofy, somewhat buggy, memory/processor hog, plugins to make it real-time and self-sustaining are expensive, DotTunes branding is pervasive and not removable, interface is not extendable
• KavaTunes
  upside: very impressive iTunes-like browser interface
  downside: not open source, not free, requires a Mac

BigCheese
Fantasy (NFL) football these days is awful. My first foray into fantasy football was in 1985, when you had to do everything with paper and pencil and you were lucky to scrape six guys together. It was wonderful. I've only played two fantasy football seasons since ubiquitous integration with web resources, and both experiences were absent of joy. With the web, there's simply an overwhelming inflation dynamic of available players and available league entry points. The quality of owner skill and commitment is at an all time low. Fantasy football could be so great if it were only implemented better. BigCheese will do it better.

The goal of BigCheese is to increase owner interest and commitment by

• Rich and interesting new revolutionary complexities and modeling of the real NFL
  
• Simple interfaces to affect those complexities in less than ten minutes a day
  
• Meaningful financial rewards
  

Rich and interesting revolutionary complexities will come from modeling the actual NFL owner perch as closely as feasibly possible. That means there's both year-round and year-to-year aspects. Rosters will be same size as real NFL teams; all positions matter. There will only EVER be a single BigCheese league, with a maximum of 32 owners. I imagine the league will start with 4-8 the first year. To some extent, where feasible, the league's engine of complexity will be obscured to discourage copy cat BigCheese leagues - but the value of the league will not be inherently protected through obscurity.

Simple interfaces to affect those complexities will come from a browser interface that reflects the ideals of Ten-Minutes-A-Day to participate in a league that is persistent from year to year.

Meaningful financial rewards will stem from the revolutionary goal of creating a league with external interest. This external interest will generate revenue for the owners through web space advertising (on the public website), league growth (franchise fees) and franchise sales (eventual franchise scarcity). For this reason, owner selectivity will be essential and enforced by all existing owners' self interest. However, the most unique aspect of BigCheese is that it will attempt to do something not seen in today's fantasy sports - generate third party interest. There will only be, ever, one BigCheese league. That league will only, ever, have a maximum 32 owners (unless the actual NFL expands). Franchise owners will switch hands, fold, etc. just like the real world of a startup sport leagues.

This goal cannot be understated: financial rewards for the owners will be primarily sourced externally. I plan on recruiting an initial seed of owners this summer.