Using Mathmatical Concepts to Enhance Your Site / Application

I was playing an RPG the other day, and I reflected back the idea that rewarding users with "levels" is a very powerful mechanism to draw people in to the "game" of participating in your web site.

Stage Select has had user "XP" (Experience Points) for several years, and has even had a leveling system with 14 levels.  I found that people really enjoyed working toward a goal, even if that goal wasn't tangible - it turned into a great way to show your "senority" on the site, even if you were a relative newcomer.  A user scoring system helps to establish a person's contribution and participation relative to others.  On a video game web site, it just "makes sense" - it's a meme that everyone is familar with.  User points are scored through participation - people get 1 point for participating in the forum, 5 points for posting a news story, and so on.   Points in turn determine your level.

Unfortunately, the levels implemented at Stage Select for a long time had two problems:  1)  It only went to 14 levels, so it would be difficult to measure progress between levels, and 2)  It wasn't really modeled on any well known RPG or anything like that, so it was very linear.

When you play an RPG, you'll notice that the first few levels you reach come quite easy.  Later levels are much much harder to achieve.  Eventually, you run into a level cap, but gaining those last two levels becomes an extreme challenge.

Sound like a familiar mathmatical situation?  "Numbers infinitely approaching X?"

No?  Give up?

Well, mathletes, to me, it sure looks like the pattern that logarithms follow.  A quick ramp up followed by a slow burn to get to the top - yup, logs. I suppose you could do this exponentially, since it's essentially the "same difference", but I don't really want to explain how my mind works, only that on occasion, my mind does in fact, work!

So once you come to that realization, you just pick a level cap, throw together the formula, and go to town, right?  Well, almost.  I wanted a level cap of around 30,000 points (with 100 levels).  In applying that formula, I discovered that the first few levels were much harder to attain than I expected them to be.  I solved that dilemma by moving the level cap down to 20,000 for the first few levels, making the multiplier a bit more friendly.  Even then, the first couple of levels were quite the ramp up, so I lowered the cap to 10,000 for just a few levels!  The trick is to pick a point at which moving from the small level cap to the larger level cap makes sense, and doesn't overburden the "player".

But I get ahead of myself.  Here's how a leveling system works (followed by my implementation below):

A LOG with a "base" of 100 (against # of levels "above" you) yields a multiplier.  This multiplier represents the percentage of the level cap (say, 10,000 points) which you DON'T have to earn to reach a level.  So, the formula looks a bit like this:

Level Cap - (Multiplier * Level Cap) = Points Needed to Reach a Level

In Excel, this formula looks like this:  =10000-(10000*LOG(99,100)) = 22 XP Needed
(for Level 2, because Level 1 is 0XP - 21XP for my purposes...  This could be Level 1 if you're so inclined)

And 22 seems about right for the first level that a person can get to on my site.  It then goes to 44, 66, 89, 111, 134, 158...  Then I broke to increase the level cap to 20,000, which means the next level is attainable at 362 points.  So, that's a bit of a jump, but it seems to make sense at that point, because it's just a bit over 2X the previous amount.  Here's what the final function looked like in VB.NET / ASP.NET:

                    Dim x As Integer = 99
                    Dim CalcLevel As Integer = 0
                    Do Until CalcLevel >= Score
                        If Score < 160 Then
                            CalcLevel = CInt(10000 - (System.Math.Log(x, 100) * 10000))
                        ElseIf Score > 160 And Score < 9999 Then
                            CalcLevel = CInt(20000 - (System.Math.Log(x, 100) * 20000))
                       Else
                            CalcLevel = CInt(30000 - (System.Math.Log(x, 100) * 30000))
                       End If

                        x = x - 1
                        If x = 0 Then
                            Exit Do
                        End If

                    Loop

And here's what the leveling tree looks like when you graph it out (click for more detail):

The level progression means that you'll be in the early 20's by the time you hit just 1000 points, giving you a quick win, but by the time you hit the early 90's, you're working your tail off to hit those last few levels.  And for Stage Select, I think that works quite well - but I can imagine that for your purposes, you may want to smooth out the transition between level 90-91 by moving the level cap to 30,000 at an earlier point.

Mentoring in Tech; also, SQLAgentMan's 1st Computer as a Chastity Belt

I have to take a minute to share my joy at finding one of my friend's recent blog posts: My First Computer (Or Another Reason Why I Did Not Have a Girlfriend Until I Was 16)

My first computer was the TI-99/4a, and unlike Tim's first computer, it had no geek cache' whatsoever. My parents purchased the computer from JC Penny's when the store decided it was never going to carry electronics again (a promise that they've kept for the most part, excluding the junk).



So, my parents exchanged $125 for any chance I had at normalcy. I am, of course, grateful for the experience, as I learned how to program in BASIC on the device (Basic programming is apparently a bad habit I will never break). Although I chronicle my experience growing up with the TI-99/4a here, I should mention that my big brother also had quite a bit of experience with the computer. He was the one that taught me several of the commands I would later use to create little programs that used the Speech Synthesizer, the joystick, etc

So, the point here isn't necessarily a trip down memory lane... I'd like to make a different point in this particular post:

It's hard to overstate the importance of having a mentor when you're a programmer or other technician.

I've seen programmers and network techs struggle throughout my career, and the MO is always the same: they have no one to look up to, they work by themselves, and they frequently are working in an area where they're trying to gain experience. It's so easy to hop down a bunny trail when it comes to tech - and when you do that, you might never be seen again.

Knowing others - networking - is important in a tech's career, and most techs get this pretty quickly.

But getting a tech that knows when to ask for help is another story altogether. I'm not talking about a person that'll never learn - one that asks the same questions all the time, but rather one that is constantly expanding their knowledge base. These are the types of techs that are rare: they're not looking for spec sheets, they're looking to understand and improve the business they're involved in. They take things personally.

These techs typically have the concept of mentoring in common; either they've acted as a mentor, or they've been mentored. I've done a bit of both, and can tell you, it's added an element of satisfaction to my career that's difficult to quantify.

In the early days, without my brother showing me what was possible on that little TI, I probably wouldn't have thought much about how things got onto the BIG GLOWING BOX. Later, as I was starting my career, I remember working with a guy named Kevin Zion. He took me under his wing and because of that, I learned a ton about servers / server administration. He was also a pretty fun guy - although I was running cables through walls (NOT fun), I remember the lunches and learning experiences there... Good stuff.

As you teach others, you'll learn pretty quickly what you really know and what you don't. If you sound confusing, chances are you need to know more yourself. If you're able to relay an idea quickly and clearly (relative to the complexity of the topic), then you know the topic.

Anyway - I'm not going to mention all of my mentors here, but I will say this - Thanks. If you've considered taking some time after work, or explained things to others, Thanks. If you're thinking that 19 year old intern needs some help understanding OO or Servers, or whatever, help the guy / gal out. I know I appreciated it, and we need all the good techs we can get.

Scripting a SQL Server Database is Easy with sp_generate_inserts

One of the most useful SQL Server development tools I've stumbled upon recently is a system stored procedure that this guy created called "sp_generate_inserts".


It does precisely what you think it'd do - if you have text data to insert in-masse from a development database, the stored proc creates a data table which contains the INSERT commands you'll eventually need to run against your production database system. I like handling data this way - in my opinion, it's probably best to script changes out so that you can store that script into your usual source code repository.

The best part is that this particular stored procedure has a myriad of options, here's some pointers on usage, from the provider's web site:

Usage:
Example 1: To generate INSERT statements for table 'titles':
EXEC sp_generate_inserts 'titles'

Example 2: To ommit the column list in the INSERT statement: (Column list is included by default)
NOTE: If you have too many columns, you are advised to ommit column list, as shown below, to avoid erroneous results
EXEC sp_generate_inserts 'titles', @Include_Column_List = 0

Example 3: To generate INSERT statements for 'titlesCopy' table from 'titles' table:
EXEC sp_generate_inserts 'titles', 'titlesCopy'

Example 4: To generate INSERT statements for 'titles' table for only those titles which contain the word 'Computer' in them:
EXEC sp_generate_inserts 'titles', @From = "from titles where title like '%Computer%'"

Example 5: To specify that you want to include TIMESTAMP column's data as well in the INSERT statement:
NOTE: By default TIMESTAMP column's data is not scripted
EXEC sp_generate_inserts 'titles', @Include_Timestamp = 1

Example 6: To print the debug information:
EXEC sp_generate_inserts 'titles', @debug_mode = 1

Example 7: If you are not the owner of the table, use @owner parameter to specify the owner name:
NOTE: To use this option, you must have SELECT permissions on that table
EXEC sp_generate_inserts Nickstable, @owner = 'Nick'

Example 8: To generate INSERT statements for the rest of the columns excluding images:
NOTE: When using this otion, DO NOT set @include_column_list parameter to 0
EXEC sp_generate_inserts imgtable, @ommit_images = 1

Example 9: To generate INSERT statements for the rest of the columns excluding IDENTITY column:
EXEC sp_generate_inserts mytable, @ommit_identity = 1

Example 10: To generate INSERT statements for the top 10 rows in the table:
EXEC sp_generate_inserts mytable, @top = 10

Example 11: To generate INSERT statements only with the columns you want:
EXEC sp_generate_inserts titles, @cols_to_include = "'title','title_id','au_id'"

Example 12: To generate INSERT statements by ommitting some columns:
EXEC sp_generate_inserts titles, @cols_to_exclude = "'title','title_id','au_id'"

Example 13: To avoid checking the foreign key constraints while loading data with INSERT statements:
NOTE: The @disable_constraints option will disable foreign key constraints, by assuming that the source data is valid and referentially sound
EXEC sp_generate_inserts titles, @disable_constraints = 1

Example 14: To avoid scripting data from computed columns:
EXEC sp_generate_inserts MyTable, @ommit_computed_cols = 1

The stored procedure is treated as a system stored procedure, so it installs to your "master" database. It's then available to all of your databases on the server, but the downside is that you probably shouldn't install this on your production box (keep tight control of those systems, kids!).

When you're dealing with a database which has thousands and thousands of games listed, this stored procedure is really useful for syncing your development environment with your production environment. Less time inserting data, and more time playing the games I love!

Follow the link above to get a copy of the stored procedure for either SQL 2000 or SQL 2005.

HTML 5 Provides Offline DB, Offline App Caching

One of my friends (Joel Anderson, Twitter here) shared the following video from Barcelona (via SlashGear):



So, one of the more important features of HTML 5 is the offline database. An Outlook-like application is shown in the above video, and it looks the same on two different devices. Powerful stuff for developers - which is almost a garauntee that offline databases will be used inappropriately at some point.

Thinking briefly about some scenarios where we might benefit from an offline DB for web pages, I've come up with the following apps I think would work well:

1. Medical Applications - One of the most common offline applications in the medical field is a prescription drug database, one which allows a doctor or pharmacist to look up items in their formularies for interactions or indications. I think HTML 5's ability to provide an offline datastore would be a great way to provide this functionality in an industry standard manner. Whether or not the companies producing the software would agree that is another question - after all, you're talking about their company's know-how being packaged in such a way as to be repurposed by anyone with the motivation.
2. Car mechanic database applications - Car shops are in a difficult position, as most of their software comes in big 50+ GB packages, listing every single car ever manufactured. The software is helpful in that it details nearly every procedure, but it's also problematic. Since it's so big, it's got to sit on a computer somewhere... Since it's in a shop, that computer sits well out of harms way in the back office. If a small, rugged device could sync up one car at a time (let's pretend it's the diagnostic computer or a handheld device), you could have more information at the mechanic's finger tips.
3. Having game collection data available at any time - what can I say, I'm a fan of providing my site's data anywhere my "clients" go!
   

And here are some that may not:

1. CRM - It'll be awfully tempting for people to develop a Customer Relationship Management site which allows for offline browsing of contact info. It seems to make a lot of sense on the surface, and if done right, it could be the "killer app" for HTML 5. However, I worry that an offline cache such as this won't offer any manner of protection of company data. What happens when someone copies the offline DB? What happens when a laptop is stolen? Many CRM programs out there now offer at least some protection in these scenarios.
   
2. Medical Applications -Anything involving personal patient information should stay the heck away from this until it is WELL known what the security risks are, and that those security risks can be mitigated.

Anyway, there's my two cents - I've given this all of 1/2 hour's thought. What do you think - what kind of applications would you like to see? What are you most likely to develop?

ASP.NET Global Variables - Context, Application, Session, Cache, and more

Here's a quick refresher about how to make global variables available in ASP.NET...

Global variables, while traditionally frowned upon in favor of tightly scoped variables, have their place. Globals make for great data transport mechanisms between classes, and can reduce trips to the database server if you're looking for the same information in several places.

The problem is that ASP.NET is a stateless environment. Unlike Windows Forms programming, you don't have a big pool of memory that you can go to for any variable that you need, wherever you need it. Once a page is done executing, the memory is dumped, and you can't access anything that you've done. Similarly, where do you temporarily store things you're working on during page execution, especially if you're skipping into and out of classes?

The solution ASP.NET offers is some memory spaces which are manifested as if they were dictionaries / hash tables / whatever. Basically, you'll need to explicitly create an entry in a particular memory space to store your variable. One quick example here: Session("bob") = "billy" -- this will store your variable just fine.

I needed a variable available to me to share data with other business classes, but I wanted to dispose of it once the page was done executing. I didn't want the same user to get the same variable, likewise I didn't want to persist the variable across the entire Application. All the articles I was reading online ALMOST had it right, but veered off course in one direction or the other, hence this post.

Here are some questions and answers about where you can store your variables given different scenarios:

• Where do I store a variable that I only need during the page's execution, but I need it across different classes / controls?
• In a class: HttpContext.Current.Items("Item") = "My Item" (can be an instantiated instance of a class, or nearly anything else).
• In a page: Current.Items("Item") = "My Item" (same memory space as above, this is the shortcut)
  

• Where do I store a constant or a function that I need in several places?
• Public Shared - Function, Sub, and Property allow you to set a variable / procedure once, and have it available without setting an instance before using it. (example, you name your class "bob" and your method "tony" and you can say in any page or class "bob.tony"). This is the same concept as a "static" function. There are issues with this method: You can't use "Private" variables in your functions, as you'll never have an instance of your class. While this makes your variables / functions quickly available, it doesn't allow you to directly modify their contents, and you'll execute a function fresh every time. There may be a way around this last caveat, but I don't know what that way may be - if you happen to know, please post a comment.
  
• Module - if you code "Public Module" instead of "Public Class", it's the same exact thing as making a bunch of Public Shared functions within your Class.
  
• You can also put constants into various config files, see below.
  

• Where do I store a variable that I need through multiple page executions, for one user, on the same ASPX?
• Use the ViewState.
• Use a Literal control with Visible = False

• Where do I store a variable that I need for one user only, possibly across multiple pages?
• This is where you'd use Session or Cookies. Cookies scale a bit better than Session variables, because you're not asking the server to hold onto data. However, you can scale Sessions using SQL Server, so keep that in mind, especially if security is an issue.
• In a class: Conext.Current.Session("Item") = "My Item"
• Where do I store a variable that I need for every user of the application?
• The Application space is one place. Application("Item") is still available, for backwards compatibility with ASP.
• A better construct is Cache. Cache allows you to set up a lot of little triggers that will expire the variable, from strict time to keep it alive, to weird conditions, like the existence of a file!
  
• Where do I store constants to be available throughout the application, like connection strings?
• Web.config

• Where do I store constants to be available throughout the entire SITE, including subordinate applications, like connection strings?
• Web.config - web.config files cascade, so your application's web.config can override your site's web.config, BUT that doesn't mean that your work in the site's web.config has gone to waste.

• Where do I store constants to be available across an entire machine? Or - I'm getting errors when I cluster web servers, because the ViewState becomes invalid when I switch to a different machine in the cluster... why?
• machine.config sets constants that are global across an entire machine. One of the more common problems in creating web clusters is as above - the encryption key is different between machines, and because of this, one machine's ViewState can't be used on another machine in the cluster. machine.config is where you set your machinekey, and solve this problem.
  

Here's a diagram of the relationships you'll see between different spaces you can place variables and constants:

Here's a sample implementation of a couple of global variables:


Public Class Globals

#Region "Page Scope Variables - HttpContext"

'''

''' Gets the current myInfo object. If there is none, then create a new one.
'''

'''
'''
'''
Public Shared Property CurrentInfo() As myInfo
Get
If Not IsNothing(HttpContext.Current.Items("Info")) Then
Return CType(HttpContext.Current.Items("Info"), myInfo)
Else
Dim myInfo As New myInfo()
myInfo.SubIWantToRunWhichSetsExtraThingsUpInMyClass()
HttpContext.Current.Items("Info") = myInfo
HttpContext.Current.Items("InfoRefreshedAt") = Now.ToString
Return CType(HttpContext.Current.Items("Info"), myInfo)
End If
End Get
Set(ByVal value As myInfo)
HttpContext.Current.Items("Info") = value
End Set
End Property


#End Region

#Region "Session Scope Variables - Session"

'''

''' Gets the current ClientInfo object
'''

'''
'''
'''
Public Shared Property CurrentClient() As ClientInfo
Get
If Not IsNothing(HttpContext.Current.Session("Client")) Then
Return CType(HttpContext.Current.Session("Client"), ClientInfo)
Else
Dim theClient As New ClientInfo()
HttpContext.Current.Session("Client") = theClient
HttpContext.Current.Session("ClientPostedAt") = Now.ToString
Return CType(HttpContext.Current.Session("Client"), ClientInfo)
End If
End Get
Set(ByVal value As ClientInfo)
HttpContext.Current.Session("Client") = value
End Set
End Property
#End Region
End Class


And here in a page, I'm testing for a value which exists in each of the classes, and am also checking WHEN that variable was last populated. In this example, you'll see Literal2 repopulate with each refresh, and Literal4 won't refresh until you begin a new session.

Protected Sub Page_Load(ByVal sender As Object, ByVal e As System.EventArgs) Handles Me.Load
Dim thisInfo As myInfo = Globals.CurrentInfo
Literal1.Text = thisInfo.Variable1
Literal2.Text = Context.Items("InfoRefreshedAt").ToString
Dim thisClient As ClientInfo = Globals.CurrentClient
Literal3.Text = thisClient.IPAddress
Literal4.Text = Session("ClientPostedAt").ToString
End Sub

When you're done messing around with code, check out some information about NES Classics. There's over 800 NES games listed.

Starting a Personal Blog

I know, I should have done this a long time ago, but the wait is over. Woo.

In this blog, I'll likely be covering a variety of coding / tech topics, which is pretty nerdy of me, but I may sneak some other personal information in there on occasion.

You're just as likely to see hardware specs for the Xbox 360 as you are information about programming using Windows Media Encoder - and maybe even a healthy portion of dog/cat/kid/wife pictures, who knows.