Jan Gombert: Blog

The Studio

I work for a hospital, adding to and supporting a medical documentation system which records everything from patient age and weight to heart rate during an operation. A lot of what my team does is support various kinds of hardware and software breakdowns, and with a thousand users and a body of software that goes well beyond 100,000 lines of code, there are a lot of them.

When I started this job, one of my first projects was to write a web-based incident tracking system. I did the first version in PHP and MySQL, because this is what I knew, and I needed to get something running quickly. But in order to put the system on the hospital�s intranet, I needed to convert it to IIS, C#, and SQL.

It was a switch, but I gotta tell you, I love Visual Studio. I edit everything in one place, and I can test changes to the site in about five seconds. The Studio doesn�t know too much about JavaScript, but it knows everything about CSS, HTML, and C#, and the IntelliSense cues are extremely useful. Still, there were some irritants. They offer a million widgets for your pages (from the toolbox), but they all use this mysterious runat=�server�, and generate HTML that really didn�t look like what I was used to. Since I had already decided that I didn�t want to lock myself into Microsoft, I adopted a couple of rules:

Let JavaScript control changes to the display. In the time management site I wrote about earlier, parts of the page were written by JavaScript, part by PHP, and sometimes it was the same part, with redundant coding.

Limit the server side code to providing raw data (see rule one). Almost all of the C# code is called via Ajax � I send out a request for data, and get some JSON back. If at some point I have to move the site from C#/SQL back to PHP/MySQL, it won�t be a major undertaking.

Keep the formatting separate from the code. This really has nothing to do with the Studio, but it�s worth remembering. If I want to change how the page looks, I change the CSS.

In the examples section of the site, I�ve included the code for one page from the tracking system, to show how this works. The example is primarily about the JavaScript Ajax/C# interaction for retrieving data. I�ve become a big fan of Ajax, primarily because it lets me do so much more without constantly interrupting the user with a page refresh. It works almost (we�re not there yet) as smoothly as a desktop application.

There are a few other tidbits in the example, like how to convert a Word document to a pdf file using C#.

The Dark Side

I�ve been an IBM/Windows user my whole life, but back in January, when I needed a new laptop, I bought a Macintosh. This was simply the next step in a gradual withdrawal from Vader�s realm; I had switched from Explorer and Outlook a year earlier. Part of this was practical -- we used Macs at the school where I was teaching, and I wanted to be able to read my email and have the same bookmarks � but mostly it was the result of an indefinable disaffection with Windows. And when the second versions of Parallels was released, so that in a desperate moment I could run my favorite Windows programs, I jumped.

Surprisingly, with the exception of Microsoft Access, I found that there wasn�t a single program I needed to bring over from Windows. I got Fetch for FTP and switched over to BBEdit (a wonderful editor), but I use the Mac version of everything else � Office, Dreamweaver, Flash, Firefox, Perl, etc. I�m still learning things about the Mac and Unix (obviously), but I found the transition far less painful than I expected. Mostly I just need to remember to press the Apple key instead of the Ctrl-key. On those rare occasions when I need Explorer or Access, I use my old Dell.

At the beginning of this month, as it happened, Vader pulled me back. I accepted a full-time job working with SQL Server, C# and Visual Basic. I�m an old fan of Visual Basic � ten years ago, nothing beat it for rapid prototyping � and I was eager to learn C#. What I expect to find is that it combines the speed of VB with the hard edges of C.
So. I will be writing less. Up through December, in addition to the database job, I will still be teaching. When I do return to the blog, in addition to C#, here�s what I�ll be writing about.

I finished the time tracking software. I�ve been using it every day now for about a month and I find it very helpful for finding out where my day goes, but it also taught me a lot about JavaScript and PHP. I have some complaints.

I finished version two of my Google math gadget. With the right idea, I think there may be a minor business in gadget making.

Stealing

September 23, 2007: Stealing

In 1987, the SkillsBank software occupied about 45 floppy disks, and our customers could either buy a single user license (you can�t copy the disks) or a site license (make as many copies as you need). At first we managed the license on trust, but when we learned that one of our dealers was making his own copies and selling them, I decided it was time to add some copy protection.

Most copy protection has two components. The original disks need some kind of marker that can�t be duplicated, and the software that reads the disk needs to check that the marker is there. If someone wants to break the protection, they either need to duplicate the marker or bypass the checking software.

I felt pretty certain that the software wasn�t going to be hacked. We used the Aztec C compiler for the Apple II that would either output native 6502 code or a custom pseudo-code (like Java�s bytecode). We used the pseudo-code because it was smaller (size was almost everything then), but it also made it very hard to read. The code was in overlays to boot.

Creating a good marker was the problem. CopyIIPlus was the program of the day for making copies of disks. Although it was marketed simply to let you make backups, people routinely used it to steal software, and it had a million tricks up its sleeve. One of the issues was that the Apple II firmware would let you read literally every bit on a track. In theory, all CopyIIPlus had to do was rewrite the bits it had read.

There were a lot of bits. The disks contained 35 tracks x 16 sectors per track x 256 bytes per sector x 8 bits per byte, or about 1.1 megabits (140k bytes). On a given track, in addition to the 32,768 bits that made up the data, there was identifying information for each sector plus some extra bits for filler.

Any disk had to contain the 35 x 16 x 256, but the filler was variable. We used a $12,000 disk duplicator to manufacture our disks, and I found that every track the duplicator laid down was very consistent, plus or minus a small number of bits. An Apple II drive, by contrast, wrote down tracks with wildly varying amounts of filler. It could read everything, but it couldn�t write it, even when CopyIIPlus told it to. So I didn�t put any markers on the disks at all. I just counted the bits. Close enough to our number? Then it had been copied by us. (Or another high-end duplicator. I wasn�t going to fret about that.)

Twenty years later, stealing is more complicated. What�s worse, the bad guys don�t necessarily even want to take your software; they just want to mess with you.

I�m in the process of writing time tracking software (where does my day go?), and it involves a login and a MySQL database. Both of these aspects, I am slowly learning, offer numerous cracks for the bad guys to squeeze though. Instead of counting bits, here is what I am doing in 2007.

I encrypt your login password in my database, using the PHP crypt function. This means, for starters, that not even I can read your password; If you forget it, I can�t send it back to you. Sites that mail you back your password must be storing it in plain text. If you�re serious about encryption, you should follow the small details: us2.php.net/crypt.

I tuck away most of the PHP code outside of the public_html directory. About six month�s ago, I discovered that someone had hacked into the visitor counter on my son�s website. (The guy bragged about it on the web. What is wrong with these people?) Stupid me, I had called it counter.txt and left it in the primary directory. I still don�t know how he did it, but after that, I move what I can out of html space.

I need to protect against SQL injection. You can get the whole story in this Wikipedia article, but here�s an example. When someone tries to login, I get their ID in the variable $userID and use the following query to find their record in the database:

SELECT password FROM Users WHERE userID = '$userID';

Unfortunately, it�s possible to type anything into the user ID field, including

a�; DELETE FROM Users WHERE 1 OR userID = '

Bye, bye users! To protect against this, I use the PHP function mysql_real_escape_string prior to invoking the query, which effectively nullifies any SQL syntax:

$userID = mysql_real_escape_string($_POST['userID']);

Problem of the Week

If two chickens lay a total of two eggs in two days, how long will it take 100 chickens to lay 100 eggs?

In ten years of teaching, I have given this problem to every class on our first meeting, and something like eight students have ever gotten it right. If you solve it (answer next week), I have another problem for you: invent a new gradebook. In the past, I have used Easy Grade Pro (probably the best), PowerSchool (tries to solve all the problems of the world, imperfectly), and Blackboard (twelve clicks to even find the gradebook), and I have always needed a spreadsheet for additional analysis. Here�s one example. I assign and check homework every class, but if I added each of these small grades into the book, the sheer number of entries would obscure more important information, like tests.

So this fall, when I am teaching two classes at the local community college, I decided to simplify: my gradebook would be completely in Excel. I had to enter formulas for weighting grade components and so forth, but it took me less than two hours to set it up. Not rocket science.

This allowed me to enter and compute grades easily enough, but then I had the problem of communicating those grades to the students. I could have exported it to some other format (a MySQL database, for example), but it seemed simpler to just upload the spreadsheet each week. Fortunately, Excel will now save spreadsheets in XML format, which you can actually read, and PHP 5 provides a SimpleXML extension, which loads the spreadsheet into a single object. Here, for example, is what it takes to read the file and access the data at (row, column):

$xml = simplexml_load_file('../data/Grades.xml');

$grade = $xml->Worksheet[0]->Table[0]->Row[row]->Cell[column]->Data;

There is a security issue, of course. The grades are in a plain text file on the web server. I want students to see their information, but not anyone else�s. So I put the database outside the public_html directory. The login page (which can work outside the box) finds the row in the spreadsheet that corresponds to the student�s ID, then displays the student�s grades. There�s probably still a security problem, and I would be grateful to hear from anybody who could tell me how to hack into this. The PHP source code and a sample of the gradebook are on the examples page.

I must tell you, I love PHP. It works on the server side, so the details are hidden away, and it greatly simplifies repetitive HTML coding. For example, each of the files on the examples page has two hyperlinks: one to view the file in your browser and one to download it. There�s a gross amount of HTML to code each file (about four lines), and there are currently eleven such files on the page (translation: 44 highly redundant lines of HTML). So I wrote a PHP function to output the HTML, and one short PHP snippet generates those four lines:

Footnotes

I know I promised last week that I would write about Flash, but Flash was not the problem of the week. So I�m going to stop making predictions.

In the new world of computing, as I noted earlier, help is everywhere. One side effect of this blog, which I did not anticipate, is that people would contribute ideas right here. George Anderson�s comments gave us three different web sites as good sources of JavaScript. Keep it coming.

Lord Google

A little over a decade ago, in Dave Barry in Cyberspace, Dave wrote, "THERE IS NO PERSONAL FINANCE PROGRAM OTHER THAN QUICKEN. THERE IS ONLY QUICKEN. YOU LOVE QUICKEN. QUICKEN IS GOD. YOU WILL BUY QUICKEN." This was accurate enough at the time, but if he were writing today, he would replace QUICKEN with GOOGLE. Of course, there's nothing to buy with Google; you just have to let them inhabit your body.

I let iGoogle move in about six months ago, mostly for the calendar and Gmail, and then I began to notice how popular the gadgets were. 1.3 million people have added Quotes of the Day, 500 thousand people subscribe to Current Moon Phase (huh?), and 14 million people are using Weather. To get a sense of these numbers, 3.5 million people use Gmail.

So I decided that my first project, post high school teaching, would be to write a gadget. The Google developer's guide indicated that the tools were JavaScript and XML, both of which I wanted to learn more about. Of course, this also seemed like a way of paying homage to our new master.

Prior to the gadget, I had assumed that JavaScript was a minor player in the HTML world, a way to add small decorations to your web page. For my son's web site, for example, I had written a few lines of code that displayed his current age on the main page. This became, in fact, the genesis for the gadget - Age Gauge - which would display a picture of your child (or grandchild, spouse) and their current age, in a variety of formats.

But JavaScript turns out to be a complete programming language, almost object oriented, that gives you powerful tools for manipulating the web display on the client side. Translation: it happens right in your browser, so you don't have the sluggishness associated with a server conversation. The technical name for JavaScript, by the way, is ECMAScript, which nobody uses. It sounds too much like a skin disease.

I'm not going to get into details of the programming here; you can get the code and a limited amount of documentation on the examples page. Instead, I want to describe a couple of ways in which the development process now differs from what I was doing in the previous century.

In 1998 and before, most of the time I was working with one language, and the languages were compact. C, for example, has something like fourteen words, and I can say anything I want to with those fourteen. Perl, by contrast, not only starts big (I'm not going to say a word about regular expressions), but seems to get bigger every five minutes. Want to (fill in the blank)? There's a package that does just that. In addition, I now use three or four languages daily, and the differences between them often lead me astray. In Perl, for example, "if statements" need braces; nobody else cares. JavaScript and Flash concatenate strings with plus, Perl and PHP with a dot.

The solution to this verbosity, of course, is Lord Google. I usually don't even bother going to the online language references. It takes too many clicks. I just ask Google to find "javascript concatenate strings" and the first or second hit gives me the answer.

Which leads me to the second big change in programming. If I didn't know the answer to a problem in 1998, and the guy down the hall didn't either, then I dug in and figured it out. The one book on my shelf was The C Programming Language. Now help is everywhere. When I was writing my second gadget, I hit an obscure snag related to communication between Flash and JavaScript. When I posted this question on a message board, someone was kind enough to write back the same day.

Even more than the message boards, examples abound. Before I wrote a line of code for Age Gauge, or even thought about it very much, I downloaded and dissected the source to Google's Todo gadget. The examples not only outline the general landscape of the problem, but they also give you the gritty details. In Todo, for example, I learned how to detect that an Enter key was pressed in various browsers.

Next week: Flash.

Overdrive

In The World of Mathematics, James Newman writes about Truman Henry Safford, who, at the age of ten, was able to square an 18-digit number in his head in less than a minute. While he was performing this incredible feat, however, he �flew around the room like a top, pulled his pantaloons over the tops of his boots, bit his hands, rolled his eyes in their sockets, sometimes smiling and talking.�

There are many days when I feel like that boy. Give me a minute to explain.

In 1965, I learned to program on an IBM 1620 (pictured). It was the size of a desk, but much heavier, owing to the fact that the 20k of memory in the base consisted of little hunks of iron (several years later, when I was teaching at the Woodberry Forest School, I broke my car jack trying to move the school�s 1620 a few inches.)

I learned Fortran and assembly language on the 1620, which was pretty much what there was at the time. In 1977, when I left high school teaching to work as a programmer, the computer landscape was still very familiar. I worked for a couple of consulting companies in Washington on a wide range of projects, and while the problems were harder, the tools were essentially the same (mostly Fortran, with sprinkles of 360 assembly language and SAS).

In 1982, I started one company, in which I created Dr. Halo and other graphics tools, and later helped start (with Garry McDaniels) a second company, which produced the SkillsBank (now SkillsTutor) educational products. A lot of my work focused on the hardware (you try to make an Apple II do something interesting), but bit by bit, the hardware and the software tools evolved. I moved to C++ in 1996, which was a conceptual shift, but a completely natural one. It reminded me of a homology course I had taken in graduate school, in which the ideas of group theory and topology, already fairly abstruse, were abstracted one level further.

When I left SkillsBank in 1998, the web was not a factor in our development efforts. We were selling to the educational market, which had always been a step behind the times. We still produced Apple II versions of our products, for example, until 1994.

After SkillsBank, I went back to college, taught high school for several years, and now, nine years after I left it, have returned to the world of computers. I haven�t completely lost touch with technology � my work in education had led me to create a few web sites and develop some Flash applications � but a whole lot had changed behind my back. XML, Java, DOM, Ajax, PHP, MySQL, C#, Flex, Ruby. Not only are there a lot of acronyms, but I�m learning that if you want to accomplish anything useful, you actually have to know how to use them. On any given day, you�ll find me in my office, channeling Truman Henry Safford, all systems on overdrive.

What do you have to know now to make a living in the software world? This is what I�m going to write about. Along the way I will explain some of the technical problems I have encountered and solved in my work on various projects, and what I have found to be the best tools.

Next week: how to write a Google gadget.